Files
CMake/Tests/RunCMake
Chuck Cranor 5990ecb741 Compute implicit include directories from compiler output
- CMakeParseImplicitIncludeInfo.cmake: new parser that extracts the
   compiler's include path from verbose output.  If the parser cannot
   parse the output, we fall back to the old behavior.  On osx we skip
   over framework directories (handled elsewhere).

 - CMakeDetermineCompilerABI.cmake:
     - use verbose flag in try_compile for ${src}
     - use new cmake_parse_implicit_include_info() to attempt extract
       implicit include directory path and if successful set
        CMAKE_${LANG}_IMPLICIT_INCLUDE_DIRECTORIES

 - CMakeCCompiler.cmake.in and CMakeCXXCompiler.cmake.in - preserve
   CMAKE_${LANG}_IMPLICIT_INCLUDE_DIRECTORIES value between runs in
   the same way CMAKE_${LANG}_IMPLICIT_LINK_DIRECTORIES is preserved

 - Tests/RunCMake/ParseImplicitIncludeInfo: tests for parse
   based on the older Tests/CMakeTests/ImplicitLinkInfoTest.cmake.in.
   The test runs a set of verbose compiler outputs collected from
   various machines through the parser and checks the results.  New
   compiler files can be added by dropping input/output files in the
   ParseImplicitIncludeInfo/data subdirectory and then adding the new set
   of files to the ${targets} list in ParseImplicitIncludeInfo.cmake.
   There is a helper CMakeLists.txt in ParseImplicitIncludeInfo/data
   that can help with the generation of test input files.
   NOTE: the standard cmake pre-commit hook rejects verbose compiler
   output with trailing spaces... you have to manually edit them out.
   This shouldn't impact the test.

Note that both the parser and the test code can use CMAKE_${LANG}_COMPILER_*
variables such as ${CMAKE_CXX_COMPILER_ID} to decide how to parse
verbose compiler output.  For the test code, this requires us to
save the variables values in the test input files.

Fixes: #16291
2019-01-21 11:14:08 -05:00
..
2018-08-28 09:27:26 -04:00
2017-11-03 10:25:05 -04:00

This directory contains tests that run CMake to configure a project
but do not actually build anything.  To add a test:

1. Add a subdirectory named for the test, say ``<Test>/``.

2. In ``./CMakeLists.txt`` call ``add_RunCMake_test`` and pass the
   test directory name ``<Test>``.

3. Create script ``<Test>/RunCMakeTest.cmake`` in the directory containing::

    include(RunCMake)
    run_cmake(SubTest1)
    ...
    run_cmake(SubTestN)

   where ``SubTest1`` through ``SubTestN`` are sub-test names each
   corresponding to an independent CMake run and project configuration.

   One may also add calls of the form::

    run_cmake_command(SubTestI ${CMAKE_COMMAND} ...)

   to fully customize the test case command-line.

   Alternatively, if the test is to cover running ``ctest -S`` then use::

    include(RunCTest)
    run_ctest(SubTest1)
    ...
    run_ctest(SubTestN)

   and create ``test.cmake.in``, ``CTestConfig.cmake.in``, and
   ``CMakeLists.txt.in`` files to be configured for each case.

4. Create file ``<Test>/CMakeLists.txt`` in the directory containing::

    cmake_minimum_required(...)
    project(${RunCMake_TEST} NONE) # or languages needed
    include(${RunCMake_TEST}.cmake)

   where ``${RunCMake_TEST}`` is literal.  A value for ``RunCMake_TEST``
   will be passed to CMake by the ``run_cmake`` macro when running each
   sub-test.

5. Create a ``<Test>/<SubTest>.cmake`` file for each sub-test named
   above containing the actual test code.  Optionally create files
   containing expected test results:

   ``<SubTest>-result.txt``
    Regex matching expected process result, if not ``0``
   ``<SubTest>-stdout.txt``
    Regex matching expected stdout content
   ``<SubTest>-stderr.txt``
    Regex matching expected stderr content, if not ``^$``
   ``<SubTest>-check.cmake``
    Custom result check.

   Note that trailing newlines will be stripped from actual and expected
   test output before matching against the stdout and stderr expressions.
   The code in ``<SubTest>-check.cmake`` may use variables

   ``RunCMake_TEST_SOURCE_DIR``
    Top of test source tree
   ``RunCMake_TEST_BINARY_DIR``
    Top of test binary tree

   and an failure must store a message in ``RunCMake_TEST_FAILED``.

To speed up local testing, you can choose to run only a subset of
``run_cmake()`` tests in a ``RunCMakeTest.cmake`` script by using the
``RunCMake_TEST_FILTER`` environment variable. If this variable is set,
it is treated as a regular expression, and any tests whose names don't
match the regular expression are not run. For example::

  $ RunCMake_TEST_FILTER="^example" ctest -R '^RunCMake\.Example$'

This will only run subtests in ``RunCMake.Example`` that start with
``example``.