Commit Graph

113 Commits

Author SHA1 Message Date
Rolf Eike Beer
dc21177461 remove pointless return value from cmCTestTestHandler::CleanTestOutput() 2020-03-23 19:47:47 +01:00
Kyle Edwards
a5be3916ee CTest: Provide more detailed information on resource allocation error 2020-02-24 15:44:01 -05:00
Brad King
28994115e8 ctest_test: Add option to REPEAT tests 2019-11-07 14:21:18 -05:00
Craig Scott
c544cb6698 CTest: Rename hardware -> resources for source code 2019-11-05 12:08:35 -05:00
Brad King
af9ed543b0 CTest: Rename PROCESSES test property to RESOURCE_GROUPS
The `PROCESSES` test property name added for CMake 3.16 is too close to
the existing `PROCESSORS` test property.  Furthermore, the property in
principle specifies groups of resources organized in a way that is
meaningful to a particular test.  The groups may often correspond to
processes but they could have other meanings.  Since the property name
`PROCESSES` has not been in a final 3.16 release yet, simply rename it
to `RESOURCE_GROUPS`.

Fixes: #19914
2019-11-05 12:08:01 -05:00
Kyle Edwards
e34de0691b CTest: Allocate hardware to tests 2019-10-02 09:33:54 -04:00
Kyle Edwards
bb4a141059 CTest: Add lexer for PROCESSES property 2019-10-02 09:33:54 -04:00
Kitware Robot
ed98209ddc Revise include order using clang-format-6.0
Run the `clang-format.bash` script to update our C and C++ code to a new
include order `.clang-format`.  Use `clang-format` version 6.0.
2019-10-01 12:26:36 -04:00
Regina Pfeifer
a1ddf2d0ba clang-tidy: Replace typedef with using
Automate the conversion with

  perl -i -0pe 's/typedef ([^;]*) ([^ ]+);/using $2 = $1;/g'

then manually fix a few places.
2019-09-04 18:03:01 +02:00
Kyle Edwards
ae2cfc4bd8 Refactor: Break cmCTestTestHandler::ProcessHandler() into smaller functions 2019-07-19 14:13:56 -04:00
Gabor Bencze
4f1dec86a5 CTest: Add SKIP_REGULAR_EXPRESSION test property 2019-07-18 11:52:05 -04:00
Sebastian Holtermann
735c6f39d9 Fix invalid ///! doxygen comment line starts
In various places `///!` was used to start a comment line.  This is not valid
Doygen syntax.  This patch replaces `///!` comment starts with `//!`.
2019-03-31 11:27:12 +02:00
Zsolt Parragi
440b08e4f0 CTest: Represent process exit codes as 64-bit signed integer
Exit code constants on Windows, such as `STATUS_NO_MEMORY` do not fit in
a 32-bit signed integer type.  They do fit in an unsigned 32-bit type,
but for compatibility with UNIX semantics we treat exit codes as signed.
Use a 64-bit signed integer to handle both.
2019-02-08 14:49:20 -05:00
Justin Goshi
7b81d8c21e TestGenerator: Record support file and line where test was added
Add internal test properties that ctest can use to report where
the test was added in CMake code.
2019-01-08 11:22:11 -05:00
Sylvain Joubert
ffdec37a19 CTest: Add colored output on tests summary where supported
- Number of passed/failed tests is colored according to the whole outcome
- Individual listed tested are colored according to their completion status:
  * Disabled: blue
  * Failed: red
  * Not Run: yellow
2018-11-09 18:57:41 +01:00
Brad King
6be53c6695 CTest: Add options to control test process affinity to CPUs
In commit v2.8.0~170 (ENH: Added ctest test options PROCESSORS and
RUN_SERIAL, 2009-09-07) CTest learned to track the number of processors
allocated to running tests in order to balance it against the desired
level of parallelism.  Extend this idea by introducing a new
`PROCESSOR_AFFINITY` test property to ask that CTest run a test
with the CPU affinity mask set.  This will allow a set of tests
that are running concurrently to use disjoint CPU resources.
2018-03-05 09:21:32 -05:00
Wouter Klouwen
695951bc46 CTest: introduce cmDuration
This commit introduces cmDuration as a typedef for
std::chrono::duration<double, std::ratio<1>>. It is less verbose and
provides for a point to put future common functionality for durations.

No functional change intended.
2018-01-23 10:04:38 -05:00
Brad King
f984296ba8 CTest: Remove unfinished batch test mode
This was partially implemented by commit v2.8.0~154 (Added some ctest
batch capabilities, 2009-09-10) but never finished.
2017-12-08 11:23:33 -05:00
Wouter Klouwen
66419bc046 CTest: convert timeouts to std::chrono::duration
This commit continues the refactoring of CTest to adopt std::chrono.
After the last sets of changes that introduced std::chrono::steady_clock
and std::chrono::system_clock respectively, it makes sense to have all
the timeouts be stored as std::chrono::duration.

No functional change intended.
2017-12-04 10:43:14 -05:00
Wouter Klouwen
5fd979a8a3 CTest: adopt std::chrono::system_clock
After the refactor to make CTest use std::chrono::steady_clock for the
keeping of time for test duration, there are still references to
cmSystemTools::GetTime() left.
To further adopt std::chrono for time related activities, this commit
changes those remaining references to std::chrono::system_clock::now()
calls and alters the storage from either unsigned int or double to
std::chrono::system_clock::time_point.

For ease of conversion, a converter method is added to cmXMLWriter that
converts from a std::chrono::system_clock::time_point to the number of
seconds since the UN*X epoch as that is expected behaviour. This means
no more casts as required.

Functionally should be no difference as the system_clock is implemented
in the same terms.
2017-11-17 15:22:55 +00:00
Wouter Klouwen
e8a4036e96 CTest: use std::chrono::steady_clock for time keeping
It was reported in issue #17345 that CTest does not use monotonic time
to report test duration. Monotonic clocks are not affected by large NTP
adjustments or things like daylight savings time.

As CMake 3.10 requires C++11, which introduced std::chrono, this commit
moves the time keeping in CTest from cmSystemTools::GetTime() to
std::chrono::steady_clock.

Fixes: #17345
2017-11-14 13:30:14 +00:00
luzpaz
777e87ae00 Fix trivial typos in text
Some are user facing.

Found using

    codespell -q 3 --skip="./Utilities" -I .cmake-whitelist.txt`

whereby the whitelist contained:

    ans
    dum
    helpfull
    emmited
    emmitted
    buil
    iff
    isnt
    nto
    ot
    pathes
    substract
    te
    todays
    upto
    whitespaces
2017-11-03 10:25:05 -04:00
Brad King
0b33aee48b Use C++11 override instead of CM_OVERRIDE
We now require C++11 support including `override`.  Drop use of
the old compatibility macro.  Convert references as follows:

    git grep -l CM_OVERRIDE -- '*.h' '*.hxx' '*.cxx' |
      xargs sed -i 's/CM_OVERRIDE/override/g'
2017-09-15 10:06:41 -04:00
Pavel Solodovnikov
63f6fd144e Meta: modernize old-fashioned loops to range-based for (CTest).
Changes done via `clang-tidy` with some manual fine-tuning
for the variable naming and `auto` type deduction
where appropriate.
2017-09-14 16:11:58 +03:00
Bill Hoffman
a6e32eb0c2 CTest: Split out labels and subproject labels in summary output
This commit splits out the reporting of labels and labels used for sub
projects. If a label is a sub project label it will not be included in
the label summary.  To implement this the commit creates
PrintLabelOrSubprojectSummary which is able to do the work of both
PrintLabelSummary and PrintSubprojectSummary avoiding code duplication.
2017-09-08 10:02:39 -04:00
Daniel Pfeifer
2b4c32c95f clang-format: format all code as Cpp11 2017-08-30 11:07:05 -04:00
Daniel Pfeifer
ca2233e31f IWYU: Mark cmConfigure.h with pragma: keep
Also remove `#include "cmConfigure.h"` from most source files.
2017-08-26 07:41:04 +02:00
Brad King
47d9772618 Merge topic 'better-exception-reporting'
0c650f39 CTest: Report more detail about system exceptions in tests

Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !1046
2017-07-13 07:51:05 -04:00
Bill Hoffman
0c650f3965 CTest: Report more detail about system exceptions in tests
This passes the system exception string up to CDash and to the command line
instead of just printing "other".
2017-07-12 13:29:21 -04:00
Betsy McPhail
47b3a57c9b Display subproject timing summary
Use the '--no-subproject-summary' option to disable timing summary.
2017-07-10 16:25:18 -04:00
Betsy McPhail
d385962419 Add directory property 'LABELS' and CMAKE_DIRECTORY_LABELS variable
The specified LABELS will be passed down to subdirectories as well as
any targets or tests in the directory.
2017-07-10 16:25:18 -04:00
Craig Scott
c1b2b7c03c Add ctest options for limiting which tests fixtures add
The new options allow the user to restrict the setup and cleanup tests
automatically added for fixtures.
2017-05-02 01:54:13 +10:00
Daniel Pfeifer
1d829c862c Use quotes for non-system includes
Automate with:

git grep -l '#include <cm_' -- Source \
  | xargs sed -i 's/#include <\(cm_.*\)>/#include "\1"/g'

git grep -l '#include <cmsys/' -- Source \
  | xargs sed -i 's/#include <\(cmsys\/.*\)>/#include "\1"/g'

git grep -l '#include <cm[A-Z]' -- Source \
  | xargs sed -i 's/#include <\(cm[A-Z].*\)>/#include "\1"/g'
2017-04-11 22:35:21 +02:00
Betsy McPhail
b070947dff Add 'DISABLED' test property
When this property is set, the test is skipped and its status is
automatically set to 'Not Run'.  A disabled test will not be counted in
the total number of tests and its completion status will be 'Disabled'.
2017-03-29 10:29:57 -04:00
Daniel Pfeifer
15aa814b85 Remove cmObject.h and cmTypeMacro.h 2016-10-24 21:49:28 +02:00
Daniel Pfeifer
fb0b087c4d Make cmCTestGenericHandler not inherit from cmObject 2016-10-23 00:59:55 +02:00
Brad King
86578eccf2 Simplify CMake per-source license notices
Per-source copyright/license notice headers that spell out copyright holder
names and years are hard to maintain and often out-of-date or plain wrong.
Precise contributor information is already maintained automatically by the
version control tool.  Ultimately it is the receiver of a file who is
responsible for determining its licensing status, and per-source notices are
merely a convenience.  Therefore it is simpler and more accurate for
each source to have a generic notice of the license name and references to
more detailed information on copyright holders and full license terms.

Our `Copyright.txt` file now contains a list of Contributors whose names
appeared source-level copyright notices.  It also references version control
history for more precise information.  Therefore we no longer need to spell
out the list of Contributors in each source file notice.

Replace CMake per-source copyright/license notice headers with a short
description of the license and links to `Copyright.txt` and online information
available from "https://cmake.org/licensing".  The online URL also handles
cases of modules being copied out of our source into other projects, so we
can drop our notices about replacing links with full license text.

Run the `Utilities/Scripts/filter-notices.bash` script to perform the majority
of the replacements mechanically.  Manually fix up shebang lines and trailing
newlines in a few files.  Manually update the notices in a few files that the
script does not handle.
2016-09-27 15:14:44 -04:00
Craig Scott
73f47c9e46 CTest: Add support for test fixtures
Add new test properties:

* FIXTURES_SETUP
* FIXTURES_CLEANUP
* FIXTURES_REQUIRED

to specify the roles and dependencies of tests providing/using
test fixtures.
2016-09-20 14:37:38 -04:00
Daniel Pfeifer
3849164454 CTest: fix include-what-you-use violations 2016-08-25 09:34:37 -04:00
Daniel Pfeifer
a7a9239096 mark functions with CM_OVERRIDE 2016-06-27 23:24:38 +02:00
Kitware Robot
d9fd2f5402 Revise C++ coding style using clang-format
Run the `Utilities/Scripts/clang-format.bash` script to update
all our C++ code to a new style defined by `.clang-format`.
Use `clang-format` version 3.8.

* If you reached this commit for a line in `git blame`, re-run the blame
  operation starting at the parent of this commit to see older history
  for the content.

* See the parent commit for instructions to rebase a change across this
  style transition commit.
2016-05-16 16:05:19 -04:00
Brad King
180538c706 Source: Stabilize include order
Each source file has a logical first include file.  Include it in an
isolated block so that tools that sort includes do not move them.
2016-04-29 13:58:31 -04:00
Zack Galbreath
993e48d045 CTest: Optionally use a secondary test timeout after matching output
Allow a test N seconds to complete after we detect a matching line in
its output.  Activate this behavior with a new TIMEOUT_AFTER_MATCH test
property.
2016-03-22 11:17:55 -04:00
Roman Wüger
fd47df4503 CTest: Add options to limit output of passed and failed tests
Add ctest command-line options:

  --test-output-size-passed <n>
  --test-output-size-failed <n>

to set the amount of test output to store in Test.xml as a command-line
dashboard client.
2015-09-18 11:32:06 -04:00
Daniel Pfeifer
b04500a725 cmCTest{Test,MemCheck}Handler: Port to cmXMLWriter 2015-05-26 11:09:22 -04:00
Daniel Pfeifer
b26e5b552a CTest: Generate NamedMeasurement elements directly into xml file
Re-order the logic to avoid use of a temporary std::ostringstream.
2015-05-26 10:40:00 -04:00
Ben Boeckel
270eb96df0 strings: Remove cmStdString references
Casts from std::string -> cmStdString were high on the list of things
taking up time. Avoid such implicit casts across function calls by just
using std::string everywhere.

The comment that the symbol name is too long is no longer relevant since
modern debuggers alias the templates anyways and the size is a
non-issue since the underlying methods are generated since it's
inherited.
2014-03-08 13:05:35 -05:00
Rolf Eike Beer
3a0d164bb2 allow to mark a test as "Not Run" with a specific return code (#8466) 2014-01-14 23:57:40 +01:00
Clinton Stimpson
5730710c86 Use cmsys::[io]fstream instead of cmsys_ios::[io]fstream.
Also use SystemTools::Fopen() instead of fopen().
This is to eventually support utf-8 filenames.
2014-01-07 09:27:44 -05:00
Zack Galbreath
eb2decc02d ctest: Add --rerun-failed option
Add a new command line argument to ctest.  This allows users to
rerun tests that failed during the previous call to ctest.  This
is accomplished by analyzing the most recently modified file named
"^LastTestsFailed*" in the Testing/Temporary subdirectory of the
project's binary directory.
2013-10-08 09:18:13 -04:00