Update commit 712cb8cb14 (ctest: Restore support for -T Test without
dashboard configuration, 2025-03-06, v4.0.0-rc4~11^2) to avoid
conflating coverage of CMake itself with the coverage mode of ctest
being tested.
When `--schedule-random` is used in automated CI jobs, failures may
occur due to test order. We now log the seed. Provide a way for
developers to re-run the same order by specifying the seed.
Fixes: #26760
Co-authored-by: Brad King <brad.king@kitware.com>
Since commit 774fcbe49c (CTest: Base command line mode on top of
scripting commands, 2024-10-05, v4.0.0-rc1~653^2) we drive dashboard
client steps through `cmCTestHandlerCommand::ExecuteHandlerCommand`
instead of calling `ProcessHandler` directly. This requires the
`BuildDirectory` to be known to establish a work directory.
Fixes: #26743
Co-authored-by: Brad King <brad.king@kitware.com>
Make sure that all CMake variables that are translated into
CTest options in `cmCTest*Command` implementations are translated
from CTest options into CMake variables before the functions are
called. This back-and-forth translation should be temporary.
It is a necessary prerequisite for refactoring `cmCTest*Handler`
implementations to operate on CMake variables directly rather
than CTest options.
In a complex test environment, there may be a desire to add a test
property label to indicate context. Here is one such example:
set_property(TEST mygoofytest PROPERTY LABELS might-fail-on-config-change)
Upon looking at the failed test summary, one can quickly decide whether
the failure might be expected or not.
198 - mygoofytest (Failed) might-fail-on-config-change
Add a dedicated `TLSVerify` ctest option and a `CTEST_TLS_VERIFY`
variable to control it. Deprecate `CurlOptions` because it exposes
internal implementation details.
Under job server integration, added by commit 80fe56c481 (ctest: Add
support for running under a make job server on POSIX systems,
2023-11-15, v3.29.0-rc1~324^2), use a very high default so that
parallelism is effectively limited only by available job server tokens.
Otherwise, choose a default limit based on the number of processors.
Also allow passing `0` to specify unbounded parallelism.
Fixes: #25739
Test names can contain `#`. Since we ignore lines that do not match any
test names anyway, "commenting" can still work without explicit syntax.
Also drop whitespace trimming for similar reasons.
Fixes: #25741
If the options added by
* commit 022f20f663 (ctest: add command line option to run the tests listed
in a given file, 2023-11-29, v3.29.0-rc1~66^2~2)
* commit dbacc1d5a8 (ctest: add command line option to exclude tests listed
in a given file, 2023-11-30, v3.29.0-rc1~66^2~1)
* commit 701029726f (ctest_test: add options INCLUDE_FROM_FILE and
EXCLUDE_FROM_FILE, 2023-12-03, v3.29.0-rc1~66^2)
are given a missing file, fail instead of ignoring it.
Fixes: #25740
Add `--tests-from-file <filename>` to run only the tests listed in the
given file. The test names must match exactly, no regexps or something.
The listed tests can still be filtered with a regexp using -R.
Issue: #25455
CTest normally terminates test processes on timeout using `SIGKILL`.
Offer tests a chance to exit gracefully, on platforms supporting POSIX
signals, by setting `TIMEOUT_SIGNAL_{NAME,GRACE_PERIOD}` properties.
Fixes: #17288
For policy-specific tests, use the version before the policy was
introduced. Otherwise, use 3.5 where possible.
Also, remove `cmake_minimum_required()` and `project()` calls from
individual cases where they are handled by `CMakeLists.txt`.
Some RunCMake tests unset some environment variables that affect ctest's
behavior before running it. Any test using the RunCTest helper needs to
do this, so do it in the helper itself.
Issue: #24153
The `ctest_test` and `CTestCommandLine` truncation tests had multiple problems:
- escape expected result string to avoid regex matching
- specify the truncation size
- pass the truncation mode correctly into the test definition
- use unique test names
Issue: #23868
