Combine `--repeat-until-fail`, `--repeat-until-pass`, and
`--repeat-after-timeout` to create a single `--repeat <mode>:<n>`
option. Retain `--repeat-until-fail` too because that has been
available in previous releases.
The `PROCESSES` test property name added for CMake 3.16 is too close to
the existing `PROCESSORS` test property. Furthermore, the property in
principle specifies groups of resources organized in a way that is
meaningful to a particular test. The groups may often correspond to
processes but they could have other meanings. Since the property name
`PROCESSES` has not been in a final 3.16 release yet, simply rename it
to `RESOURCE_GROUPS`.
Fixes: #19914
A case added by commit 39ac8b4eb5 (ctest: Add --repeat-after-timeout
option, 2019-10-29) includes a success-case timeout of 2 seconds. That
is too short when running under valgrind, so extend it to 5 seconds.
For pass regex, display only the element that was found, rather
than all elements
Rename loop variable for fail regex, from pass to fail
For consistency, add space in output for pass
Add tests that find and don't find PASS_REGULAR_EXPRESSION, and a test
that finds FAIL_REGULAR_EXPRESSION, whose LastTest.log files are checked
using *-check.cmake.
Avoid exposing the internal special value that we use to track tests not
available in the tested configuration. This also prevents clients from
having to do the filtering themselves.
Since commit v3.11.0-rc1~117^2 (CTest: Re-implement test process
handling using libuv, 2017-12-10) we do not give the child test
processes any stdin. Prior to that change we let the child test
processes inherit stdin from ctest itself. Tests that run serially
might be able to use the real stdin meaningfully, so restore that
behavior and add a test case.
Fixes: #18591
The `ctest --test-load` option is implemented in `StartNextTests` by not
starting any tests when the load is too high and instead sleeping and
then returning. Prior to commit v3.11.0-rc1~117^2 (CTest: Re-implement
test process handling using libuv, 2017-12-10) our outer loop in
`RunTests` would immediately call `StartNextTests` again. However, now
the `uv_run` loop may simply terminate if there are no tests running
because no events are left pending.
Fix this by converting the sleep in `StartNextTests` into a libuv timer
that it starts instead. This avoids leaving `uv_run` with no pending
events. In the case that there are other running tests this also allows
CTest to detect when they finish even if it during the wait period where
we previously slept.
This regression was not caught by the test suite because it only
verified that we do not start new tests when the load was too high and
not that we proceed to start tests when the load drops. Revise the test
suite to cover both.
Fixes: #18338
In commit v2.8.0~170 (ENH: Added ctest test options PROCESSORS and
RUN_SERIAL, 2009-09-07) CTest learned to track the number of processors
allocated to running tests in order to balance it against the desired
level of parallelism. Extend this idea by introducing a new
`PROCESSOR_AFFINITY` test property to ask that CTest run a test
with the CPU affinity mask set. This will allow a set of tests
that are running concurrently to use disjoint CPU resources.
This commit changes the times reported by labels and subprojects to be
weighted by the PROCESSORS property. It is reported with `sec*proc`
instead of just `sec`.
Refactoring in commit v3.4.0-rc1~390^2~1 (cmCTestMultiProcessHandler:
Refactor RUN_SERIAL implementation, 2015-06-01) forgot to update a code
path for cleaning up after a failed RUN_SERIAL test. This causes an
infinite loop after a RUN_SERIAL test fails. Fix it and add a test.
Add ctest command-line options:
--test-output-size-passed <n>
--test-output-size-failed <n>
to set the amount of test output to store in Test.xml as a command-line
dashboard client.
Add a TestLoad setting to CTest that can be set via a new --test-load
command-line option, CTEST_TEST_LOAD variable, or TEST_LOAD option to
the ctest_test command. Teach cmCTestMultiProcessHandler to measure
the CPU load and avoid starting tests that may take more than the
spare load currently available. The expression
<current_load> + <test_processors> <= <max-load>
must be true to start a new test.
Co-Author: Zack Galbreath <zack.galbreath@kitware.com>
Use the KWSys Process "MergeOutput" option to give each test child
process the same pipe for both stdout and stderr. This allows natural
merging of stdout and stderr together instead of merging on arbitrary
buffered read boundaries as before.
When matching ctest command-line output, we must account for the
formatting of times that take 10 seconds or more. The values are
right-justified, so use " +" to match any amount of space before them.
