Update commit 712cb8cb14 (ctest: Restore support for -T Test without
dashboard configuration, 2025-03-06, v4.0.0-rc4~11^2) to avoid
conflating coverage of CMake itself with the coverage mode of ctest
being tested.
When `--schedule-random` is used in automated CI jobs, failures may
occur due to test order. We now log the seed. Provide a way for
developers to re-run the same order by specifying the seed.
Fixes: #26760
Co-authored-by: Brad King <brad.king@kitware.com>
Since commit 774fcbe49c (CTest: Base command line mode on top of
scripting commands, 2024-10-05, v4.0.0-rc1~653^2) we drive dashboard
client steps through `cmCTestHandlerCommand::ExecuteHandlerCommand`
instead of calling `ProcessHandler` directly. This requires the
`BuildDirectory` to be known to establish a work directory.
Fixes: #26743
Co-authored-by: Brad King <brad.king@kitware.com>
In a complex test environment, there may be a desire to add a test
property label to indicate context. Here is one such example:
set_property(TEST mygoofytest PROPERTY LABELS might-fail-on-config-change)
Upon looking at the failed test summary, one can quickly decide whether
the failure might be expected or not.
198 - mygoofytest (Failed) might-fail-on-config-change
Add a dedicated `TLSVerify` ctest option and a `CTEST_TLS_VERIFY`
variable to control it. Deprecate `CurlOptions` because it exposes
internal implementation details.
Under job server integration, added by commit 80fe56c481 (ctest: Add
support for running under a make job server on POSIX systems,
2023-11-15, v3.29.0-rc1~324^2), use a very high default so that
parallelism is effectively limited only by available job server tokens.
Otherwise, choose a default limit based on the number of processors.
Also allow passing `0` to specify unbounded parallelism.
Fixes: #25739
If the options added by
* commit 022f20f663 (ctest: add command line option to run the tests listed
in a given file, 2023-11-29, v3.29.0-rc1~66^2~2)
* commit dbacc1d5a8 (ctest: add command line option to exclude tests listed
in a given file, 2023-11-30, v3.29.0-rc1~66^2~1)
* commit 701029726f (ctest_test: add options INCLUDE_FROM_FILE and
EXCLUDE_FROM_FILE, 2023-12-03, v3.29.0-rc1~66^2)
are given a missing file, fail instead of ignoring it.
Fixes: #25740
Add `--tests-from-file <filename>` to run only the tests listed in the
given file. The test names must match exactly, no regexps or something.
The listed tests can still be filtered with a regexp using -R.
Issue: #25455
CTest normally terminates test processes on timeout using `SIGKILL`.
Offer tests a chance to exit gracefully, on platforms supporting POSIX
signals, by setting `TIMEOUT_SIGNAL_{NAME,GRACE_PERIOD}` properties.
Fixes: #17288
Some RunCMake tests unset some environment variables that affect ctest's
behavior before running it. Any test using the RunCTest helper needs to
do this, so do it in the helper itself.
Issue: #24153
The `ctest_test` and `CTestCommandLine` truncation tests had multiple problems:
- escape expected result string to avoid regex matching
- specify the truncation size
- pass the truncation mode correctly into the test definition
- use unique test names
Issue: #23868
Add `--test-output-truncation` to `ctest`. This option can be used to
customize which part of the test output is being truncated. Currently
supported values are `tail`, `middle` and `head`.
Also add equivalent `CTEST_CUSTOM_TEST_OUTPUT_TRUNCATION` variable.
Fixes: #23206
Check for the existence of the Binary directory string before checking
for coverage files. If the directory is not specified, write an error
message and do not do any checking.
Fixes: #22102
In commit 49948f7221 (ctest_memcheck: Add support for ThreadSanitizer,
2014-07-07, v3.1.0-rc1~322^2~1) and commit 1e005eadbc (CTest: Fix
MemoryCheckType from 'ctest -T MemCheck', 2014-07-15, v3.1.0-rc1~298^2),
the `CMakeCommand` internal setting was left set only when using `ctest
-S` scripts. Instead simply use CTest's corresponding CMake directly
without passing it through an internal setting.
Fixes: #20584
If no tests were found, the default behavior of CTest is to always log an
error message but to return an error code in script mode only. This option
unifies the behavior of CTest by either returning an error code if no tests
were found or by ignoring it.
Signed-off-by: Stefan Dinkelacker <s.dinkelacker@dkfz-heidelberg.de>
