mirror of
https://github.com/Kitware/CMake.git
synced 2026-01-05 21:31:08 -06:00
Tests: CPack test documentation facelift
This commit is contained in:
Domen Vrankar
@@ -1,41 +1,75 @@
|
||||
RunCMake.CPack is a test module that is intended for testing of package
|
||||
generators that can be validated from command line.
|
||||
|
||||
TODO: all tests should cover all packaging types (single package, grouped and
|
||||
component packaging)
|
||||
|
||||
-------------
|
||||
Adding a test
|
||||
-------------
|
||||
|
||||
CPack test root directory: 'Tests/RunCMake/CPack'.
|
||||
CPack test root directory: 'Tests/RunCMake/CPack/tests'.
|
||||
|
||||
All phases are executed separately for each generator that is bound to a test.
|
||||
Tests for each generator are subtests of test 'RunCMake.CPack_<generator_name>'.
|
||||
|
||||
Each test must also be added to 'RunCMakeTest.cmake' script located in CPack
|
||||
test root directory.
|
||||
Line that adds a test is:
|
||||
run_cpack_test(<test_name> "<generator_name>")
|
||||
|
||||
<generator_name> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB" and
|
||||
will be run for all listed generators.
|
||||
Line that adds a test is:
|
||||
run_cpack_test(<test_name> "<generator_name_list>" <compile_stage>
|
||||
"<packaging_type_list>")
|
||||
|
||||
<generator_name_list> may be one generator e.g. "RPM" or multiple e.g. "RPM;DEB"
|
||||
and will be run for all listed generators. In test and verification scripts
|
||||
current generator can be accessed through GENERATOR_TYPE variable.
|
||||
|
||||
<compile_stage> is a boolean variable that enables or disables compile stage -
|
||||
most tests don't require compilation as a non binary file can be used for
|
||||
package content but sometimes an executable or a library has to be created
|
||||
before the packaging stage.
|
||||
|
||||
<packaging_type_list> can be a list of one or more packaging types: MONOLITHIC,
|
||||
COMPONENT or GROUP - each type sets per generator default variables which can
|
||||
be overwritten in the test if needed
|
||||
(see <generator_type>/packaging_<packaging_type>_default.cmake for the variables
|
||||
that are set by default for each packaging type).
|
||||
Alternatively CUSTOM value can be set which means that default values will not
|
||||
be set and that values will be set manually in the test itself.
|
||||
|
||||
Alternatively a test with subtests can be added:
|
||||
run_cpack_test_subtests(<test_name> "<subtests_list>" "<generator_name_list>"
|
||||
<compile_stage> "<packaging_type_list>")
|
||||
|
||||
<subtests_list> is the only new parameter and it is a list of names that will
|
||||
be used for subtests. In test and verification scripts subtest name can be
|
||||
accessed through RunCMake_SUBTEST_SUFFIX variable.
|
||||
|
||||
Also source package tests can be added:
|
||||
run_cpack_source_test(<test_name> "<generator_name_list>" true)
|
||||
|
||||
Test consists of
|
||||
- test prerequirements phase (optional)
|
||||
- CMake execution phase
|
||||
- build phase (optional and not available for source package tests)
|
||||
- CPack execution phase
|
||||
- verification of generated files
|
||||
|
||||
The phases are executed once per specified generator, packaging type and subtest
|
||||
combinatiion.
|
||||
|
||||
test prerequirements phase (optional):
|
||||
--------------------------------------
|
||||
|
||||
In some cases individual tests for certain generator need additional
|
||||
prerequirements met.
|
||||
|
||||
In such cases '<generator_name>/<test_name>-Prerequirements.cmake' file
|
||||
containing 'function(get_test_prerequirements found_var)' should be created.
|
||||
Function should return true if all prerequirements are met.
|
||||
In such cases '<test_name>/<generator_name>-Prerequirements.cmake' file
|
||||
containing 'function(get_test_prerequirements found_var config_file)' should be
|
||||
created. Function should return true in found_var if all prerequirements are
|
||||
met. config_file variable contains the location of config file generated by the
|
||||
generator so that this function can check if prerequired variable is set in the
|
||||
file.
|
||||
NOTE: All required programs should be searched for in generator prerequirements
|
||||
function and only checked for the variable in configure file in per test
|
||||
function.
|
||||
|
||||
If prerequirements are not met test will be skipped outputting
|
||||
'<test_name> - SKIPPED' string. Note that this doesn't fail the entire test
|
||||
@@ -48,15 +82,21 @@ TODO: skipped tests should provide expected error string so test should fail
|
||||
CMake execution phase:
|
||||
----------------------
|
||||
|
||||
To add a new CPack test we first create a <test_name>.cmake script that contains
|
||||
CMake commands that should be used as a preparation script for generation of
|
||||
different types of packages. This script is placed into CPack test root
|
||||
directory even if it will be used for only one of the generators.
|
||||
To add a new CPack test we first create a '<test_name>/test.cmake' script that
|
||||
contains CMake commands that should be used as a preparation script for
|
||||
generation of different types of packages. This script is placed into CPack
|
||||
test root directory.
|
||||
|
||||
If test will be used for multiple generators but some of them require some
|
||||
generator specific commands then those commands should be added to a separate
|
||||
file that should be located in '<generator_name>/<test_name>-specifics.cmake'
|
||||
in CPack test root directory.
|
||||
generator specific commands then those commands should be added to 'test.cmake'
|
||||
script wrapped with 'if(GENERATOR_TYPE STREQUAL <name_of_the_generator>)'.
|
||||
|
||||
build phase (optional and not available for source package tests)
|
||||
-----------------------------------------------------------------
|
||||
|
||||
This phase only runs make command.
|
||||
|
||||
NOTE: By default all tests have CMAKE_BUILD_TYPE variable set to Debug.
|
||||
|
||||
CPack execution phase:
|
||||
----------------------
|
||||
@@ -64,6 +104,8 @@ CPack execution phase:
|
||||
Only executes CPack for content that was generated during CMake execution
|
||||
phase.
|
||||
|
||||
NOTE: By default CPACK_PACKAGE_NAME variable is set to lower case test name.
|
||||
|
||||
Verification of generated files:
|
||||
--------------------------------
|
||||
|
||||
@@ -76,24 +118,49 @@ contain expected files.
|
||||
Mandatory verification phase also checks that no other unexpected package files
|
||||
were generated (this is executed even if EXPECTED_FILES_COUNT contains 0 in
|
||||
order to verify that no files were generated).
|
||||
CMake script '<generator_name>/<test_name>-ExpectedFiles.cmake' is required by
|
||||
CMake script '<test_name>/ExpectedFiles.cmake' is required by
|
||||
this step and must contain
|
||||
- EXPECTED_FILES_COUNT variable that contains the number of expected files that
|
||||
will be generated (0 or more)
|
||||
|
||||
- EXPECTED_FILE_<file_number_starting_with_1> that contains globing expression
|
||||
that uniquely defines expected file name (will be used to find expected file)
|
||||
and should be present once for each expected file
|
||||
and should be present once for each expected file.
|
||||
NOTE: This variable should be used only as last resort as it sets generator
|
||||
specific globbing expression. Each generator can recreate file name from
|
||||
parts that are already populated by default but can always be
|
||||
overwritten if needed:
|
||||
- EXPECTED_FILE_<file_number_starting_with_1>_NAME is the name component of
|
||||
the file (by default it is set to package name that is same as test name
|
||||
in lowercase)
|
||||
- EXPECTED_FILE_<file_number_starting_with_1>_VERSION is the version of the
|
||||
package (by default it is set to '0.1.1')
|
||||
- EXPECTED_FILE_<file_number_starting_with_1>_REVISION is the revision of the
|
||||
package (by default it is set to '1')
|
||||
|
||||
- EXPECTED_FILE_CONTENT_<file_number_starting_with_1> that contains regular
|
||||
expression of files that should be present in generated file and should be
|
||||
present once for each expected file
|
||||
NOTE: This variable should be used only as last resort as it sets generator
|
||||
specific regular expression.
|
||||
EXPECTED_FILE_CONTENT_<file_number_starting_with_1>_LIST should be
|
||||
prefered as it requires a list of expected files and directories that
|
||||
is later changed automatically depending on the generator so expected
|
||||
package content can be written only once per test for all generators.
|
||||
|
||||
Optional verification phase is generator specific and is optionaly executed.
|
||||
This phase is executed if '<generator_name>/<test_name>-VerifyResult.cmake'
|
||||
script exists.
|
||||
In case that the script doesn't exist VerifyResult.cmake script automatically
|
||||
prints out standard output and standard error from CPack execution phase that
|
||||
is compared with '<generator_name>/<test_name>-stdout.txt' regular expression
|
||||
and '<generator_name>/<test_name>-stderr.txt' regular expresson respectively.
|
||||
This phase is executed if '<test_name>/VerifyResult.cmake' script exists.
|
||||
|
||||
VerifyResult.cmake script also automatically prints out standard output and
|
||||
standard error from CPack execution phase that is compared with
|
||||
'<test_name>/<generator_name>-stdout.txt' regular expression and
|
||||
and '<test_name>/<generator_name>-stderr.txt' regular expresson respectively.
|
||||
NOTE: For subtests generator name can also be suffixed with subtest name and in
|
||||
such case subtest file is preferred.
|
||||
File name format: '<generator_name>-<subtest_name>-std<type>.txt' where
|
||||
<type> can either be 'out' or 'err'.
|
||||
NOTE: If none of the comparison files are present then the default generator
|
||||
file is used if present.
|
||||
|
||||
----------------------
|
||||
Adding a new generator
|
||||
@@ -101,13 +168,44 @@ Adding a new generator
|
||||
|
||||
To add a new generator we must
|
||||
- add new generator directory (e.g. RPM for RPM generator) to CPack test root
|
||||
directory that contains 'Helpers.cmake' script. In this script a function
|
||||
named 'getPackageContent' must exist. This function should list files that
|
||||
are contained in a package. Function should accept two parameters
|
||||
+ FILE variable that will contain path to file for which the content should be
|
||||
listed
|
||||
+ RESULT_VAR that will tell the function which variable in parent scope should
|
||||
contain the result (list of files inside package file)
|
||||
directory that contains 'Helpers.cmake' script.
|
||||
- In this script some functions must exist:
|
||||
- getPackageContent: This function should list files that are contained in
|
||||
a package.
|
||||
Function parameters:
|
||||
+ FILE variable that will contain path to file for which the content
|
||||
should be listed
|
||||
+ RESULT_VAR that will tell the function which variable in parent scope
|
||||
should contain the result (list of files inside package file)
|
||||
- getPackageNameGlobexpr: This function should generate package name
|
||||
globbing expression.
|
||||
Function parameters:
|
||||
+ NAME that will contain the expected package name
|
||||
+ COMPONENT that will contain the expected package component
|
||||
+ VERSION that will contain the expected package version
|
||||
+ REVISION that will contain the expected package revision number
|
||||
+ FILE_NO that will contain the file number for which the globbing
|
||||
expression is generated
|
||||
+ RESULT_VAR that will tell the function which variable in parent scope
|
||||
should contain the result (file name globbing expression)
|
||||
- getPackageContentList: This function should return a list of files and
|
||||
directories that are in the provided package.
|
||||
Function parameters:
|
||||
+ FILE that will contain the package file for which the package content
|
||||
should be returned.
|
||||
+ RESULT_VAR that will tell the function which variable in parent scope
|
||||
should contain the result (list of pacakge content)
|
||||
- toExpectedContentList: This function should convert an expected package
|
||||
content list into one that is expected for the
|
||||
generator (e.g. rpm packages have install/relocate
|
||||
path prefixes which aren't part of the package so
|
||||
those paths have to be removed from the expected
|
||||
content list).
|
||||
Function parameters:
|
||||
+ FILE_NO that will contain the file number for which the conversion
|
||||
should be performed
|
||||
+ CONTENT_VAR that will contain the input list and is also the variable
|
||||
in parent scope which should contain the result (converted content list)
|
||||
- add 'Prerequirements.cmake' script to generator directory. In this script a
|
||||
function named 'get_test_prerequirements' must exist. This function should
|
||||
set a variable in parent scope (name of the variable is the first parameter)
|
||||
@@ -119,3 +217,9 @@ To add a new generator we must
|
||||
- add tests the same way as described above
|
||||
- add generator to 'add_RunCMake_test_group' function call that is located in
|
||||
RunCMake CMakeLists.txt file
|
||||
- if needed add 'packaging_<packaging_type>_default.cmake' scripts that define
|
||||
default variables that will be set for each packaging type (MONOLITHIC,
|
||||
COMPONENT and GROUP)
|
||||
- if needed add 'default_expected_std<type>.txt' files where <type> is either
|
||||
'out' or 'err' which will contain default expected output of package
|
||||
generation regular expression.
|
||||
|
||||
Reference in New Issue
Block a user