mirror of
https://github.com/Kitware/CMake.git
synced 2026-04-20 21:28:23 -05:00
Ryan Thornton
1ba4cb565e
Now, the unit tests are ran twice -- once with POST_BUILD (i.e. default mode)
and again with PRE_TEST (i.e. new discovery mode).
Both modes of setting gtest discovery mode are also tested:
1. Using the global override (i.e. CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE)
2. Explicitly passing DISCOVERY_MODE in calls to gtest_discover_tests (in GoogleTestDiscoveryTimeout.cmake)
The goal is to show that the new PRE_TEST discovery mode does not break existing behavior
(i.e. should not break POST_BUILD mode) and should also pass the same tests
in the same way.
A few non trivial implementation details worth noting:
1. Refactoring discovery_timeout_test into own project
Originally, I tried doing:
```
run_GoogleTest(POST_BUILD)
run_GoogleTest(PRE_TEST)
```
Without changing the internal structure of run_GoogleTest.
But since discovery_timeout_test is part of the same project as the other tests,
and CTest include files always get evaluated and that's where test discovery occurs,
this means every other test now notices the timeout problem when running in PRE_TEST mode.
As a result, keeping the existing test structure meant that each existing test
(and any new test) would need to have its own PRE_TEST / POST_BUILD variant for stderr and stdout
in order to handle the case where discovery_timeout_test timed out.
This exponential increase in test output files introduced unnecessary complexity
and made it more cumbersome to work on test cases.
Why should an unrelated test case care about discovery_timeout_test?
So, to fix that issue, the tests were broken apart into two main groups:
1. run_GoogleTest_discovery_timeout (the test dealing with discovery_timeout_test)
2. run_GoogleTest (everything else)
This isolates the PRE_TEST / POST_BUILD timeout variants to a single test case.
And the other test cases remain unchanged -- further driving home the point that
DISCOVERY_MODE shouldn't change existing behavior.
2. Different number of PRE_TEST / POST_BUILD file variants
On the PRE_TEST path, different build systems / compilers (i.e. MSBuild and ninja/gcc)
produces different build output when building discovery_timeout_test,
but we don't actually care what it is, just as long as it builds
successfully.
This the fundamental difference in behavior between POST_BUILD (which would have failed)
and PRE_TEST (which doesn't) and is the reason why we don't need
a GoogleTest-discovery-build-result.txt or GoogleTest-discovery-build-stdout.txt
3. Fix flaky discovery timeout test
The test expects to see:
> Output:
> timeout
> case.
But sometimes, the test would only produce:
> Output:
> timout
In certain environments, specifically when built with OpenWatcom 1.4,
and while the build server was under heavy load (i.e. running many tests in parallel),
std::endl behaves inconsistently and doesn't completely
flush std::cout when the program is terminated due to timeout.
This results in inconsistent test failures because the actual output
doesn't fully match what's expected.
At first we tried adding an additional:
std::cout << std::flush
That didn't work. But using C-style printf() and fflush() appears to do
the trick:
> This time I managed to get on the machine while it was still busy doing other nightly builds
> and could reproduce the problem reliably. With that I was finally able to find a fix.
> It turns out my earlier hypothesis that C++ stream flushing was not working on the old compiler was correct,
> but even .flush() is not enough.
> I changed it to use C-style printf() and fflush() and now the test passes on that build.
> -- Brad King <brad.king@kitware.com>
Co-authored-by: Ryan Thornton <ThorntonRyan@JohnDeere.com>
Co-authored-by: Kevin Puetz <PuetzKevinA@JohnDeere.com>
CMake
*****
Introduction
============
CMake is a cross-platform, open-source build system generator.
For full documentation visit the `CMake Home Page`_ and the
`CMake Documentation Page`_. The `CMake Community Wiki`_ also
references useful guides and recipes.
.. _`CMake Home Page`: https://cmake.org
.. _`CMake Documentation Page`: https://cmake.org/documentation
.. _`CMake Community Wiki`: https://gitlab.kitware.com/cmake/community/wikis/home
CMake is maintained and supported by `Kitware`_ and developed in
collaboration with a productive community of contributors.
.. _`Kitware`: http://www.kitware.com/cmake
License
=======
CMake is distributed under the OSI-approved BSD 3-clause License.
See `Copyright.txt`_ for details.
.. _`Copyright.txt`: Copyright.txt
Building CMake
==============
Supported Platforms
-------------------
* Microsoft Windows
* Apple macOS
* Linux
* FreeBSD
* OpenBSD
* Solaris
* AIX
Other UNIX-like operating systems may work too out of the box, if not
it should not be a major problem to port CMake to this platform.
Please post to the `CMake Discourse Forum`_ to ask if others have
had experience with the platform.
.. _`CMake Discourse Forum`: https://discourse.cmake.org
Building CMake from Scratch
---------------------------
UNIX/Mac OSX/MinGW/MSYS/Cygwin
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You need to have a C++ compiler (supporting C++11) and a ``make`` installed.
Run the ``bootstrap`` script you find in the source directory of CMake.
You can use the ``--help`` option to see the supported options.
You may use the ``--prefix=<install_prefix>`` option to specify a custom
installation directory for CMake. Once this has finished successfully,
run ``make`` and ``make install``.
For example, if you simply want to build and install CMake from source,
you can build directly in the source tree::
$ ./bootstrap && make && sudo make install
Or, if you plan to develop CMake or otherwise run the test suite, create
a separate build tree::
$ mkdir cmake-build && cd cmake-build
$ ../cmake-source/bootstrap && make
Windows
^^^^^^^
There are two ways for building CMake under Windows:
1. Compile with MSVC from VS 2015 or later.
You need to download and install a binary release of CMake. You can get
these releases from the `CMake Download Page`_. Then proceed with the
instructions below for `Building CMake with CMake`_.
2. Bootstrap with MinGW under MSYS2.
Download and install `MSYS2`_. Then install the required build tools::
$ pacman -S --needed git base-devel mingw-w64-x86_64-gcc
and bootstrap as above.
.. _`CMake Download Page`: https://cmake.org/download
.. _`MSYS2`: https://www.msys2.org/
Building CMake with CMake
-------------------------
You can build CMake as any other project with a CMake-based build system:
run the installed CMake on the sources of this CMake with your preferred
options and generators. Then build it and install it.
For instructions how to do this, see documentation on `Running CMake`_.
.. _`Running CMake`: https://cmake.org/runningcmake
To build the documentation, install `Sphinx`_ and configure CMake with
``-DSPHINX_HTML=ON`` and/or ``-DSPHINX_MAN=ON`` to enable the "html" or
"man" builder. Add ``-DSPHINX_EXECUTABLE=/path/to/sphinx-build`` if the
tool is not found automatically.
.. _`Sphinx`: http://sphinx-doc.org
Reporting Bugs
==============
If you have found a bug:
1. If you have a patch, please read the `CONTRIBUTING.rst`_ document.
2. Otherwise, please post to the `CMake Discourse Forum`_ and ask about
the expected and observed behaviors to determine if it is really
a bug.
3. Finally, if the issue is not resolved by the above steps, open
an entry in the `CMake Issue Tracker`_.
.. _`CMake Issue Tracker`: https://gitlab.kitware.com/cmake/cmake/issues
Contributing
============
See `CONTRIBUTING.rst`_ for instructions to contribute.
.. _`CONTRIBUTING.rst`: CONTRIBUTING.rst