Commit Graph

1830 Commits

Author SHA1 Message Date
Brad King
80fe56c481 ctest: Add support for running under a make job server on POSIX systems
Share job slots with the job server by acquiring a token before running
each test, and releasing the token when the test finishes.
2023-12-03 08:30:07 -05:00
Brad King
5396f4a9a3 cmUVJobServerClient: Add libuv-based job server integration client 2023-12-03 08:30:06 -05:00
Brad King
0432b921ae cmCTestMultiProcessHandler: Inline removal of pending test as it starts
Avoid searching the entire list of ordered pending tests to remove one
when we already know where it is.
2023-11-22 07:49:04 -05:00
Brad King
b17c732e88 cmCTestMultiProcessHandler: Clarify role of StartTestProcess
Focus its role on actually running the test process.
Move administrative tasks to the call site.
2023-11-22 07:49:04 -05:00
Brad King
0950acb337 cmCTestMultiProcessHandler: Manage concurrency slots with other resources 2023-11-22 07:49:04 -05:00
Brad King
697900da29 cmCTestMultiProcessHandler: Manage affinity assignments with other resources 2023-11-22 07:49:04 -05:00
Brad King
086a41c0f3 cmCTestMultiProcessHandler: Simplify test startup batching
Once a test is ready to run, count it against the number of tests to
start in the current batch whether or not the test process actually
starts successfully.  If a test process does fail to start, simply
schedule a new startup batch on the next loop iteration.
2023-11-21 12:40:13 -05:00
Brad King
e528cd795f cmCTestMultiProcessHandler: Start new tests asynchronously
When a test finishes, defer starting new tests until the next loop
iteration.  That way, if multiple tests finish in a single loop
iteration, we can free all of their resources first, and then start
a new batch of tests.
2023-11-21 12:37:53 -05:00
Brad King
9d8415c17b cmCTestMultiProcessHandler: Clarify test-load retry timer infrastructure 2023-11-21 12:37:41 -05:00
Brad King
61e98ca33b cmCTestMultiProcessHandler: Factor out loop startup and teardown 2023-11-21 12:37:31 -05:00
Brad King
5ff0b4ed57 cmCTestMultiProcessHandler: Consolidate test readiness checks 2023-11-21 12:36:31 -05:00
Brad King
ad3df3ce4d cmCTestMultiProcessHandler: Exclude dependent tests earlier
Tests with unfinished dependencies should not be considered at all when
looking for tests that fit within the load and concurrency limits.
2023-11-21 08:29:17 -05:00
Brad King
3c4767f467 cmCTestTestHandler: Clarify name of member storing RESOURCE_LOCK property
The property represents project-defined resources.
2023-11-21 08:29:17 -05:00
Brad King
6818925b9a Merge topic 'ctest-cleanup'
5d2e93f9e8 cmCTestMultiProcessHandler: Simplify logic on unavailable resources
a4b061a035 cmCTestMultiProcessHandler: Clarify resource availability error member names
1487e540aa cmCTestMultiProcessHandler: Reduce repeat test property map lookups
b02b628ad9 cmCTestMultiProcessHandler: Simplify loop termination on serial test
8f1e8af0cc cmCTestMultiProcessHandler: Stop searching for tests when limit is reached
bd0b4ca867 cmCTestMultiProcessHandler: Invert spare load condition
9b548139fd cmCTestMultiProcessHandler: Clarify search for tests <= concurrency limit
ee321dc85f cmCTestMultiProcessHandler: Clarify search for tests <= spare load
...

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !8981
2023-11-20 08:16:19 -05:00
Brad King
4c1a6f2e23 Merge topic 'ci-fedora-39'
cdd741ebf9 Merge branch 'backport-ci-fedora-39' into ci-fedora-39
9283b20659 ci: Suppress CPack/RPM tests pending fix for Fedora 39
18145e8745 ci: Update FindMPI test environment for mpich on Fedora 39
a8be80ccf2 ci: Drop now-unnecessary Clang rules for CXXModules tests
99238b23e9 ci: use Fedora 39 images and environments
57eadec617 ci: update Linux image to Fedora 39
653262162c clang-tidy module: Update to build against LLVM/Clang 17
2cf9a65835 clang-tidy: ignore warnings new in version 17
...

Acked-by: Kitware Robot <kwrobot@kitware.com>
Tested-by: buildbot <buildbot@kitware.com>
Merge-request: !8983
2023-11-20 08:10:37 -05:00
Brad King
5d2e93f9e8 cmCTestMultiProcessHandler: Simplify logic on unavailable resources
Deallocation is not necessary after allocation fails.
2023-11-17 18:43:01 -05:00
Brad King
a4b061a035 cmCTestMultiProcessHandler: Clarify resource availability error member names
The members are about the availability of sufficient resources,
not allocation of them.
2023-11-17 18:43:01 -05:00
Brad King
1487e540aa cmCTestMultiProcessHandler: Reduce repeat test property map lookups 2023-11-17 18:43:01 -05:00
Brad King
b02b628ad9 cmCTestMultiProcessHandler: Simplify loop termination on serial test 2023-11-17 18:43:01 -05:00
Brad King
8f1e8af0cc cmCTestMultiProcessHandler: Stop searching for tests when limit is reached
Avoid an extra loop iteration if we have no room for more tests.
2023-11-17 18:42:17 -05:00
Brad King
bd0b4ca867 cmCTestMultiProcessHandler: Invert spare load condition
Switch then/else blocks to reduce indentation.
2023-11-17 18:41:42 -05:00
Brad King
9b548139fd cmCTestMultiProcessHandler: Clarify search for tests <= concurrency limit 2023-11-17 18:41:42 -05:00
Brad King
ee321dc85f cmCTestMultiProcessHandler: Clarify search for tests <= spare load
Move code into conditions where it is needed and comment its purpose.
2023-11-17 18:41:42 -05:00
Brad King
afd185881f cmCTestMultiProcessHandler: Make loops over all pending tests more consistent
Some loops were using the ordered list unnecessarily while others used
the main map of pending test.  Update all to use the latter.
2023-11-17 18:41:41 -05:00
Brad King
3bc3f6cbb5 cmCTestMultiProcessHandler: Clarify representation of pending test order
Also avoid copying the entire list each time we start a batch of tests.
2023-11-17 18:41:41 -05:00
Brad King
b27969c89d cmCTestMultiProcessHandler: Clarify representation of pending tests 2023-11-17 18:41:41 -05:00
Brad King
7bca3f8c80 cmCTestMultiProcessHandler: Avoid extra copy of test maps 2023-11-17 18:41:41 -05:00
Brad King
d6d114f3e8 cmCTestMultiProcessHandler: Remove unused members
The `TestRunningMap` and `TestFinishMap` members have not been used
since commit 44017a4767 (CTest: handle dependent and non dependent test
requirements equally, 2013-10-17, v3.0.0-rc1~451^2~7).
2023-11-17 18:41:41 -05:00
Brad King
451429e19c cmCTestMultiProcessHandler: Use cm::uv_loop_ptr abstraction
This ensures all loop resources are released.
2023-11-17 18:41:41 -05:00
John Parent
419443f68f cmCTestMultiProcessHandler: Factor out helper to check for completion 2023-11-17 18:41:41 -05:00
Chris Mahoney
8c307ab567 cmCTestMultiProcessHandler: Replace false condition with opposite assert 2023-11-17 18:41:41 -05:00
Brad King
1f8f270f62 cmCTestRunTest: Consolidate initialization in constructor 2023-11-17 18:41:40 -05:00
Brad King
cdd741ebf9 Merge branch 'backport-ci-fedora-39' into ci-fedora-39 2023-11-17 11:35:41 -05:00
Brad King
7517923418 codespell: Match valgrind output without repeating a spelling error 2023-11-17 09:58:21 -05:00
Brad King
6ef03ca03e cmUVSignalHackRAII: Drop outdated and unused libuv SA_RESTART workaround
It is only needed for libuv < 1.19, but since commit c050d6a01e
(string(TIMESTAMP): add %f specifier for microseconds, 2022-01-27,
v3.23.0-rc1~59^2) we require libuv >= 1.28.
2023-11-17 08:49:21 -05:00
Brad King
07edab8ef2 ctest_empty_binary_directory: Report more detail in failure message 2023-11-09 09:11:55 -05:00
Brad King
85a9fe056f Merge topic 'cxx23'
a0fabc4769 cmGlobalGeneratorFactory: Provide complete cmGlobalGenerator to deleter
85627a93c9 cmCTestBuildCommand: Avoid requiring complete cmGlobalGenerator type publicly
641c02a3ce cmList: Avoid using operator-> on input iterator
e4483b8871 Tests: Avoid compiling call to dap::optional<dap::string>(nullptr)

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Acked-by: scivision <michael@scivision.dev>
Merge-request: !8950
2023-11-08 10:32:04 -05:00
Brad King
85627a93c9 cmCTestBuildCommand: Avoid requiring complete cmGlobalGenerator type publicly 2023-11-07 09:13:25 -05:00
Rose
9ca6dfc280 Source: Reduce vector entry allocations and copies
Prefer `emplace_back` over `push_back`.
2023-10-23 11:18:32 -04:00
Brad King
37c80628ba Merge topic 'ctest-timeout-flag'
d267c128a2 ctest: Restore support for --timeout values higher than default test timeout
dd779a4bc2 Tests: Clarify RunCMake.CTestTimeout case name

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !8851
2023-10-02 08:57:36 -04:00
Brad King
d267c128a2 ctest: Restore support for --timeout values higher than default test timeout
Since refactoring in commit 0a5aeaf302 (cmCTestRunTest: Consolidate test
timeout selection logic, 2023-05-04, v3.27.0-rc1~120^2) we accidentally
truncate `--timeout` values to ctest's default `TimeOut`.  Fix the
logic to prefer the flag whenever the `TIMEOUT` property is not set.

In combination with the prior refactoring, this also fixes a bug that
caused `--timeout` values of 10000000 seconds or more to be ignored.

Fixes: #23979
2023-09-29 09:25:01 -04:00
scivision
d49ea4a4d9 Source: Remove redundant FileIsDirectory checks 2023-09-18 23:11:28 -04:00
Brad King
825e8c3d09 Merge topic 'ctest_submit_follow_redirects'
26ce8dc290 ctest: Restore support for http redirects during Submit step

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !8782
2023-09-08 08:58:56 -04:00
Zack Galbreath
26ce8dc290 ctest: Restore support for http redirects during Submit step
After CDash PR 1519 introduced HTTP status codes greater than 200 for
various error cases, CMake commit b7c871f745 (ctest: Update ctest_submit
for CDash behavior change, 2023-07-24, v3.27.1~3^2) modified CTest's
submit handler to check the status returned by CDash and throw an error
when this status is not equal to 200.

That change had the unintended side effect of causing CTest submissions
to fail when uploading results to a URL that returns a redirect status
code (3xx).  Fix this by configuring cURL to follow the redirect.  The
status cURL reports to CTest is now 200 instead of 3xx when CDash is
located behind a redirect.

Fixes: #25159
2023-09-07 15:37:27 -04:00
Kyle Edwards
0712e3cfea cmCTestCoverageHandler: Replace cmsysProcess with cmUVProcessChain 2023-08-29 10:51:30 -04:00
Kyle Edwards
96b3dd329e cmCTestLaunchReporter: Replace cmsysProcess with cmUVProcessChain
And convert cmCTestLaunch and cmCTestBuildHandler too.
2023-08-29 10:51:30 -04:00
Kyle Edwards
b15ad7ebb6 cmCTest: Replace cmsysProcess with cmUVProcessChain 2023-08-29 10:51:30 -04:00
Kyle Edwards
49a37d5a97 cmCTestScriptHandler: Replace cmsysProcess with cmUVProcessChain
And update cmSystemTools::WaitForLine() to use cmUVProcessChain.
2023-08-29 10:51:30 -04:00
Kyle Edwards
ec124582ac cmProcessTools::RunProcess(): Replace cmsysProcess with cmUVProcessChain
And convert the VCS code to std::vector<std::string>.
2023-08-29 10:51:30 -04:00
Kyle Edwards
0d95b68bd8 Merge topic 'ctest-generate-resource-spec-file'
c8c1dd0d95 CTest: Add ability to dynamically generate resource spec file
3f5a5a5856 cmCTestRunTest::StartFailure(): Add total argument

Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !8699
2023-08-14 11:18:03 -04:00