The behavior when system-level failures occur was not previously defined.
Clarify the behavior and provide an example for how to work around them.
Affected test properties:
- PASS_REGULAR_EXPRESSION
- FAIL_REGULAR_EXPRESSION
- SKIP_REGULAR_EXPRESSION
- SKIP_RETURN_CODE
Also update the existing WILL_FAIL test property docs to using the same
consistent wording.
This makes the WILL_FAIL behavior explicit. Users could be confused
because some other build systems treat equivalent test properties
as allowing a failure, but still pass on a passing test, and may
catch SIGABRT unlike CMake.
We show an example of how to wrap a test so that sigabrt, segfault,
etc. are also handled by WILL_FAIL.
CTest normally terminates test processes on timeout using `SIGKILL`.
Offer tests a chance to exit gracefully, on platforms supporting POSIX
signals, by setting `TIMEOUT_SIGNAL_{NAME,GRACE_PERIOD}` properties.
Fixes: #17288
* enable_testing must be called to enable testing
* can only set properties on tests created in same directory
* WILL_FAIL still fails on segfault and similar
* compact wording
8abd714176 Help: Clarify that ENVIRONMENT test properties take ;-separated lists
02cf404ace Help: Add advice for dealing with semicolons in lists
c4117d9116 ExternalProject: Document that LIST_SEPARATOR works for CMAKE_ARGS too
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: Ben Boeckel <ben.boeckel@kitware.com>
Merge-request: !7066
Follow up commit 79f64cc773 (Help: ENVIRONMENT_MODIFICATION clarify only
supports singular values, 2022-02-23) to specifically call out the list
format.
Issue: #23315
Run the script:
Utilities/Sphinx/update_versions.py --since v3.21.0 --overwrite
Manually select updates that really belong to the 3.22 release, as
against adding documentation for previously-existing entities.
This property allows projects to modify environment variables at test
time rather than trying to guess what the state should be based on what
is present at configure time. Of particular interest is the ability to
use a `PATH` present at test time while adding entries known to be
necessary for the test itself.
There are multiple operations provided to modify variables, including:
- setting and unsetting
- appending and prepending as:
- strings
- path lists
- CMake lists
Additionally, a `reset` action is provided to cancel any prior
modifications to that particular variable in the case of incremental
additions to the test property.
The "restored" bit is an implementation detail as it could also be
implemented by passing a crafted environment to `execve` or
`CreateProcess` arguments. Instead, state that the environment changes
only affects the test with the property set.
The `PROCESSES` test property name added for CMake 3.16 is too close to
the existing `PROCESSORS` test property. Furthermore, the property in
principle specifies groups of resources organized in a way that is
meaningful to a particular test. The groups may often correspond to
processes but they could have other meanings. Since the property name
`PROCESSES` has not been in a final 3.16 release yet, simply rename it
to `RESOURCE_GROUPS`.
Fixes: #19914
In commit v2.8.0~170 (ENH: Added ctest test options PROCESSORS and
RUN_SERIAL, 2009-09-07) CTest learned to track the number of processors
allocated to running tests in order to balance it against the desired
level of parallelism. Extend this idea by introducing a new
`PROCESSOR_AFFINITY` test property to ask that CTest run a test
with the CPU affinity mask set. This will allow a set of tests
that are running concurrently to use disjoint CPU resources.
* Make it clearer that fixture names are distinct from test case names.
* Mention -FS, -FC and -FA ctest command line options for overriding
fixture behavior.
Fixes: #17141
When this property is set, the test is skipped and its status is
automatically set to 'Not Run'. A disabled test will not be counted in
the total number of tests and its completion status will be 'Disabled'.
Add new test properties:
* FIXTURES_SETUP
* FIXTURES_CLEANUP
* FIXTURES_REQUIRED
to specify the roles and dependencies of tests providing/using
test fixtures.
