Commit Graph

185 Commits

Author SHA1 Message Date
Brad King ab5baa32ad Merge topic 'ctest-WORKING_DIRECTORY'
a116fd413d ctest: print WORKING_DIRECTORY property with -V

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !7060
2022-03-10 08:39:33 -05:00
Kevin Puetz a116fd413d ctest: print WORKING_DIRECTORY property with -V 2022-03-09 13:00:01 -06:00
Frank Winklmeier 140704d443 ctest: add option for output truncation
Add `--test-output-truncation` to `ctest`. This option can be used to
customize which part of the test output is being truncated. Currently
supported values are `tail`, `middle` and `head`.

Also add equivalent `CTEST_CUSTOM_TEST_OUTPUT_TRUNCATION` variable.

Fixes: #23206
2022-03-08 08:18:02 -05:00
Brad King 4c68a0e71a Merge topic 'envmod-test-modifying-existing'
9c4d6404eb Tests/Environment: also test modifying ambient values
7d52d48a32 cmCTestRunTest: get the default value from the environment

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !6682
2021-11-01 09:08:11 -04:00
Ben Boeckel 7d52d48a32 cmCTestRunTest: get the default value from the environment
This only works due to some assumptions about how the `ENVIRONMENT`
property is processed. Comments have been added to notify anyone
modifying the behavior about where to look.

Fixes: #22819
2021-10-29 10:54:02 -04:00
Brad King 9bb7982a15 Merge topic 'purge-sprintf'
b7e9cd05cd Replace the only non-standard _snprintf with snprintf
5ba6e8ac59 Source: Replace most calls to sprintf with snprintf

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !6649
2021-10-27 09:23:17 -04:00
Ben Boeckel 69f95cf1d9 cmCTestRunTest: fix modifying non-existent envvars
When appending or modifying to a variable that has not been modified
before, the iterator was not valid, but it was used to insert into the
map again. Instead, just use indexing into the map since we know it will
exist by the end of the function anyways.

Fixes: #22796
2021-10-26 12:51:44 -04:00
Sean McBride 5ba6e8ac59 Source: Replace most calls to sprintf with snprintf 2021-10-25 18:23:13 -04:00
Ben Boeckel de4f1f26b0 CTest: add an ENVIRONMENT_MODIFICATION property
This property allows projects to modify environment variables at test
time rather than trying to guess what the state should be based on what
is present at configure time. Of particular interest is the ability to
use a `PATH` present at test time while adding entries known to be
necessary for the test itself.

There are multiple operations provided to modify variables, including:

  - setting and unsetting
  - appending and prepending as:
    - strings
    - path lists
    - CMake lists

Additionally, a `reset` action is provided to cancel any prior
modifications to that particular variable in the case of incremental
additions to the test property.
2021-07-09 08:45:18 -04:00
Zack Galbreath 5489ce74b3 ctest: support <CTestMeasurement> for runtime measurements
Teach CTest to parse output for <CTestMeasurement> in addition to
<DartMeasurement> for measurements defined at runtime.

Use a new class (cmCTestTestMeasurementXMLParser) derived from cmXMLParser
to parse the data and attributes these XML elements. This is an improvement
over our previous approach of using a series of regular expressions.

As part of this commit we also rename some member variables and methods
to make their purpose more clear.

DartStuff                        -> AllTestMeasurementsRegex
DartStuff1                       -> SingleTestMeasurementRegex
DartString                       -> TestMeasurementsOutput
GenerateDartOutput()             -> GenerateCTestXML()
GenerateRegressionImages()       -> RecordCustomTestMeasurements()
cmCTestRunTest::DartProcessing() -> ParseOutputForMeasurements()
2021-07-01 15:16:23 -04:00
Zack Galbreath bd38749fd4 ctest: allow test output to add labels
Parse test output for <CTestLabel>...</CTestLabel>.
If found, add this value to the list of labels associated with this test.
2021-06-17 14:09:01 -04:00
Zack Galbreath 02f1271bdf ctest: allow test output to override the 'details' field
Parse test output for <CTestDetails>...</CTestDetails>.
If found, use this value to override the default 'Details' string reported
to CDash.
2021-06-09 08:31:32 -04:00
Brad King 18b1ccfa51 Merge topic 'ctest-progress-output-fixes'
39cdc8afb9 ctest: Make verbose mode override progress behavior
1bcd9b06b9 ctest: Don't treat skipped test as failed tests in progess report

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !5663
2021-01-07 08:14:48 -05:00
Oleksandr Koval 209daa20b2 Code style: add missed explicit 'this->'
CMake uses explicit 'this->' style. Using custom clang-tidy check we can
detect and fix places where 'this->' was missed.
2021-01-05 14:32:36 +02:00
Sylvain Joubert 1bcd9b06b9 ctest: Don't treat skipped test as failed tests in progess report
Skipped tests should not stand out on their own lines
Fixes #21143
2021-01-05 10:32:33 +01:00
Vitaly Stakhovsky 825b04c67a cmCTest: Members accept std::string parameters 2020-07-27 10:58:39 -04:00
Brad King 0a74d7c2b8 Merge branch 'backport-ctest-repeat-notrun' into ctest-repeat-notrun 2020-05-27 06:45:39 -04:00
Robert Maynard bbb62dcc72 CTest: Make sure NOT_RUN tests show up in the failed test log
Issue: #20543
2020-05-27 06:43:01 -04:00
Kyle Edwards a1612af749 CTest: Log environment variables as a test measurement 2020-05-19 10:26:57 -04:00
Ben Boeckel 24a9fed5d9 IWYU: mark <cstddef> as needed
Newer IWYU is not seeing them as needed for `size_t`.
2020-04-29 11:16:44 -04:00
Marc Chevrier f964739ead cmCTestRunTest: modernize memory management 2020-03-11 17:46:11 +01:00
Kyle Edwards f1c34443b7 CTest: Improve error reporting with bad working directory for tests 2020-02-24 15:44:01 -05:00
Kyle Edwards 1dec359422 Refactor: Require detail when calling cmCTestRunTest::StartFailure() 2020-02-24 15:44:01 -05:00
Brad King ed65b3e984 CTest: Rename internal APIs for --repeat options
Replace use of the term "rerun" with "repeat" to match the public names.
2019-11-07 14:14:28 -05:00
Brad King 016601e5e6 Merge branch 'backport-ctest-resource-groups' 2019-11-05 12:59:16 -05:00
Craig Scott c544cb6698 CTest: Rename hardware -> resources for source code 2019-11-05 12:08:35 -05:00
Brad King af9ed543b0 CTest: Rename PROCESSES test property to RESOURCE_GROUPS
The `PROCESSES` test property name added for CMake 3.16 is too close to
the existing `PROCESSORS` test property.  Furthermore, the property in
principle specifies groups of resources organized in a way that is
meaningful to a particular test.  The groups may often correspond to
processes but they could have other meanings.  Since the property name
`PROCESSES` has not been in a final 3.16 release yet, simply rename it
to `RESOURCE_GROUPS`.

Fixes: #19914
2019-11-05 12:08:01 -05:00
Brad King 39ac8b4eb5 ctest: Add --repeat-after-timeout option
Add an option to re-run tests if they timeout.  This will help tolerate
sporadic timeouts on busy machines.
2019-10-29 15:14:36 -04:00
Brad King 80c2c9d14c ctest: Add --repeat-until-pass option
Add an option to re-run tests if they fail.  This will help tolerate
sporadic failures.

Issue: #17010
Co-Author: Ben Boeckel <ben.boeckel@kitware.com>
Co-Author: Chuck Atkins <chuck.atkins@kitware.com>
2019-10-29 15:10:12 -04:00
Brad King 0187e52244 cmCTestRunTest: Use inline member initializers 2019-10-29 13:47:23 -04:00
Kyle Edwards e34de0691b CTest: Allocate hardware to tests 2019-10-02 09:33:54 -04:00
Kitware Robot ed98209ddc Revise include order using clang-format-6.0
Run the `clang-format.bash` script to update our C and C++ code to a new
include order `.clang-format`.  Use `clang-format` version 6.0.
2019-10-01 12:26:36 -04:00
Marc Chevrier c688b401d3 cmstd: Modernize CMake system headers
Provide a standardized way to handle the C++ "standard" headers
customized to be used with current CMake C++ standard constraints.
Offer under directory `cm` headers which can be used as direct
replacements of the standard ones.  For example:

    #include <cm/string_view>

can be used safely for CMake development in place of the `<string_view>`
standard header.

Fixes: #19491
2019-09-20 10:01:37 -04:00
Regina Pfeifer f30523d090 clang-tidy: modernize-deprecated-headers 2019-09-16 10:11:13 -04:00
Regina Pfeifer d25a5a7ec9 clang-tidy: modernize-use-auto
Set the MinTypeNameLength option to an impossibly high value in order
to limit the diagnostics to iterators.  Leave new expressions and cast
expressions for later.
2019-09-10 22:21:41 +02:00
Brad King 89479bde94 Merge topic 'pvs-cleanup'
7fe3e874d5 cmCPackLog: Fix support for multiple log message tags
74f2c0ea56 cmCTestTestHandler: Remove extra layer of parentheses
7c2767ef3b cmCTestMultiProcessHandler: Explain testRun ownership in comments
303e813438 CTest: Simplify some boolean conditions
51565abe79 cmMessageCommand: Remove extra layer of parentheses
b1cfaf7b91 cmVSSetupHelper: Remove unused SmartBSTR copy operations
3f4c4e7afe cmVSSetupHelper: Fix SmartBSTR copy operations
a8ca5aea94 cmMakefileTargetGenerator: Check for null before using a pointer
...

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Daniel Pfeifer <daniel@pfeifer-mail.de>
Acked-by: Artalus <artalus-mail@yandex.ru>
Merge-request: !3715
2019-08-26 10:51:11 -04:00
Brad King 303e813438 CTest: Simplify some boolean conditions
Directly compare two boolean values instead of spelling out accepted
combinations.

Issue: #19610
2019-08-26 09:43:46 -04:00
Sebastian Holtermann 9b334397f5 Source sweep: Use cmStrCat for string concatenation
This patch is generated by a python script that uses regular expressions to
search for string concatenation patterns of the kind

```
std::string str = <ARG0>;
str += <ARG1>;
str += <ARG2>;
...
```

and replaces them with a single `cmStrCat` call

```
std::string str = cmStrCat(<ARG0>, <ARG1>, <ARG2>, ...);
```

If any `<ARGX>` is itself a concatenated string of the kind

```
a + b + c + ...;
```

then `<ARGX>` is split into multiple arguments for the `cmStrCat` call.

If there's a sequence of literals in the `<ARGX>`, then all literals in the
sequence are concatenated and merged into a single literal argument for
the `cmStrCat` call.

Single character strings are converted to single char arguments for
the `cmStrCat` call.

`std::to_string(...)` wrappings are removed from `cmStrCat` arguments,
because it supports numeric types as well as string types.

`arg.substr(x)` arguments to `cmStrCat` are replaced with
`cm::string_view(arg).substr(x)`
2019-08-22 16:38:10 +02:00
Kitware Robot 54e9d38c28 Refactor: Convert all instances of CMAKE_BUILD_WITH_CMAKE to CMAKE_BOOTSTRAP 2019-08-09 10:41:44 -04:00
Gabor Bencze 4f1dec86a5 CTest: Add SKIP_REGULAR_EXPRESSION test property 2019-07-18 11:52:05 -04:00
Marc Chevrier d9b2c7dae2 Introduce memory management helper: cm_memory.hxx 2019-07-14 15:37:30 +02:00
Stephen Manz adcbdb143c CTest: Clean up stdout and logfile output for Pass/Fail regex
For pass regex, display only the element that was found, rather
than all elements
Rename loop variable for fail regex, from pass to fail
For consistency, add space in output for pass
Add tests that find and don't find PASS_REGULAR_EXPRESSION, and a test
that finds FAIL_REGULAR_EXPRESSION, whose LastTest.log files are checked
using *-check.cmake.
2019-07-09 09:01:04 -07:00
Regina Pfeifer 670d27da64 cmCTest: Remove friend declarations 2019-03-18 22:25:50 +01:00
Gregor Jasny 1166aa5ce7 ctest: refactor some code 2019-03-05 19:20:29 +01:00
Brad King c0ba467d2d Merge topic 'cmCTestRunTest-compress'
948c55857e cmCTestRunTest: Remove duplicated compression logic

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2946
2019-02-12 08:28:24 -05:00
Regina Pfeifer 948c55857e cmCTestRunTest: Remove duplicated compression logic 2019-02-10 15:47:15 +01:00
Zsolt Parragi 440b08e4f0 CTest: Represent process exit codes as 64-bit signed integer
Exit code constants on Windows, such as `STATUS_NO_MEMORY` do not fit in
a 32-bit signed integer type.  They do fit in an unsigned 32-bit type,
but for compatibility with UNIX semantics we treat exit codes as signed.
Use a 64-bit signed integer to handle both.
2019-02-08 14:49:20 -05:00
Michael Wegner b3d5b8b3fb ctest: Add option for live progress summary in terminal 2018-09-26 11:47:49 -04:00
Michael Wegner 62fbe5002a cmCTestRunTest: Thread number of completed tests through start APIs 2018-09-25 08:20:57 -04:00
Michael Wegner 6a285bb737 cmCTestRunTest: Buffer test result output before printing 2018-09-25 08:20:57 -04:00