Add an `INSTALL_JOB_SERVER_AWARE` option to `ExternalProject_Add`.
When using an explicit `INSTALL_COMMAND`, the generated commands
won't use `$(MAKE)` thus failing to connect to the outer make's
job server. Add an option enable explicit job server integration.
This is the install step's equivalent to the build step's
`BUILD_JOB_SERVER_AWARE` option added by commit bc43398e72
(ExternalProject: Enable Make Job Server with Explicit Build Command,
2023-08-09, v3.28.0-rc1~217^2). It is useful when the external
project's installation is driven by its build system. Note that with
Makefile generators, our default install command does use `$(MAKE)` to
connect to the outer make's job server.
Issue: #26398
The can_fetch_default variable needs to be set before the call to
_ep_write_gitupdate_script(). Move it to before that call and always
set the variable, even if arg_SCRIPT_FILE is not set. This makes the
generated git update script file easier to follow if someone tries to
follow what it is doing or wants to investigate problems and run it
manually.
Fixes: #26164
7e80415e79 ExternalProject: Reduce number of local variables for svn logic
e16f65be15 ExternalProject: Don't add SVN auth args if not set
Acked-by: Kitware Robot <kwrobot@kitware.com>
Acked-by: buildbot <buildbot@kitware.com>
Merge-request: !9678
In 462e583267 (ExternalProject: Switch download, update and patch to
use _EP_ vars, 2024-05-13), the svn_username and svn_password variables
were only updated appropriately in the svn download implementation, but
not in the svn update implementation. This was easily missed in part
because the old implementation was relying on behavior that differed
from the documentation for get_property(), and the new implementation
tried to keep the intermediate variables from the old implementation but
missed this subtle aspect.
Issue: #26152Fixes: #26148
The file(MAKE_DIRECTORY) implementation checks whether a path is
allowed to be written to before it checks if it already exists. For the
scenario where a SOURCE_DIR is an existing directory within the main
project's source directory, this triggers a fatal error if
CMAKE_DISABLE_SOURCE_CHANGES is set to true for ExternalProject,
and some FetchContent scenarios. Therefore, add an explicit check for
existence first to avoid making such error-triggering calls.
Fixes: #21872
When re-running the build with the git download method,
and where GIT_TAG is set to a tag name that we already have
locally from the first run, the second run will skip the update
step if it is already at the requested tag. The message in this
case was previously printing the commit hash instead of the
tag name, which was wrong. This wasn't noticed before
because that output wasn't usually shown, but we now do
show that message after making message output log levels
more consistent between the steps.
The commands moved to shared_internal_commands.cmake
will soon be used directly by FetchContent, which cannot
always include the full ExternalProject.cmake file (e.g. it may
be used in CMake script mode).
Issue: #21703
Add a `TLS_VERSION` option and honor `CMAKE_TLS_VERSION` variables.
Also map the version to Git options as we already do for `TLS_VERIFY`.
Issue: #25701
If a URL contains special characters like parentheses and a few others,
they would previously have caused a foreach() call that iterates over the
URLs to parse those special characters as separate, unquoted arguments.
They would then have effectively split the list of URLs at unexpected places.
Prepare the arguments for the foreach() call by using use bracket syntax
to robustly handle any URLs that do have unescaped special characters.
Issue: #25148
If we are given an empty string for URL, or we have a logic error that leads
to the file we download to being an empty string, we will now catch this at
CMake configure time instead of whenever the download is attempted at
build time.
Git config options can be passed to git clone before or after the
"clone" keyword. If specified before, the config setting is only
applied to that command invocation. It acts to override the value
in the global or project defaults (the latter doesn't exist for clone).
When the config setting is passed after the "clone" keyword, it is
saved into the cloned repository's config and will persist for later
git operations. The existing implementation expected the latter
behavior, but put the config setting before the "clone" keyword
and therefore the setting was not persisting to the git update
step. Move it to after the "clone" keyword so that it will persist.
The submodule handling is different. There is no support for
doing a "git submodule update" with a "sticky" config setting.
Instead, you have to pass the setting with all such calls. The
existing implementation was doing this for the clone step, but
not the git update step. Add the config setting there as well
so that submodules also effectively have the sslVerify setting
persist to the update step too.
Fixes: #18948
When UPDATE_DISCONNECTED is true, create separate update_disconnected
and patch_disconnected targets which have ALWAYS set to false.
Make the configure step depend on patch_disconnected in this case too.
This ensures the configure, build, install and test steps are not
executed unnecessarily when updates are disconnected.
Make the update and patch commands depend on the details of those
steps. This ensures they are re-executed when any of those details
change, even if updates are disconnected.
Allow updates to occur even if UPDATE_DISCONNECTED is true, but don't
contact the remote in that case. If asked to update to a ref that isn't known
locally, that is now detected and causes a fatal error when updates are
disconnected. Previously, the build would have silently and erroneously
continued to use the old ref.
Fixes: #16419, #19703, #21146
Teach `ExternalProject_Add` and `FetchContent_Declare` to resolve
relative remote URLs provided via `GIT_REPOSITORY`. Add policy
CMP0150 to maintain compatibility.
Fixes: #24211
Co-Authored-By: Craig Scott <craig.scott@crascit.com>
Since commit f24e34975a (ExternalProject: retry download on recoverable
errors, 2020-08-08, v3.19.0-rc1~298^2~1) ExternalProject has a list of
curl errors which trigger a retry today: 6, 7, 8 and 15.
Add curl error 28. It is a "timeout error", and should also trigger a
retry attempt at a download of an external project. The complete list
of curl errors is https://curl.se/libcurl/c/libcurl-errors.html
The change to `_ep_add_mkdir_command` in commit 5fbac2bb24
(ExternalProject: Move inline scripts to separate files, 2022-01-22,
v3.23.0-rc1~101^2) did not account for the possibility that
`CMAKE_CFG_INTDIR` is `$(CONFIGURATION)$(EFFECTIVE_PLATFORM_NAME)`
instead of just the configuration name. Pass the value into the helper
script on the command line so that the native buildsystem placeholders
are evaluated.
Fixes: #23645
Add the option to keep the current filestamps when extracting an
archive in ExternalProject_Add.
Enabling this option makes the behavior consistent with how
ExternalProject_Add is used when checking out code from revision
control instead of an archive.
Fixes: #22746
The --format=... argument to git for-each-ref was using a stray pair
of single quotes around the format string. These were then part of
the output, which caused it to never match the checkout_name.
The end result was that a REBASE_CHECKOUT always became a
CHECKOUT in all scenarios.
Fixes: #23216
This makes the scripts easier to work on, since the separate files
don't require the extra level of escaping that the inlined code did.
This also means the scripts can be rendered with appropriate
syntax highlighting in IDEs, etc.
This is in preparation for moving more inline content to separate
scripts as well. Having them in a separate subdirectory is consistent
with other modules and will make them easier to find and work with.
Refactoring of the ExternalProject and FetchContent modules moved
the commands into CMake scripts. This broke custom commands that
used shell redirection or special build tool variables of the form
$(MakeVar). Undo the sequence of commits that performed this
refactoring and follow-up fixes associated with it.
The following commits are reverted by this change:
4f3d1abbb4 (ExternalProject: Refactor pre-configure steps to support
no-target uses, 2021-02-05)
17e5516e60 (FetchContent: Invoke steps directly and avoid a separate
sub-build, 2021-01-29)
bd876f3849 (FetchContent: Restore patch command support,
2021-02-18)
404cddb7bb (ExternalProject: Fix misuse of IS_NEWER_THAN in
timestamp checks, 2021-02-21)
b0da671243 (FetchContent: Don't update timestamps if files don't
change, 2021-02-18)
Fixes: #21892
When using a file system which only has second resolution timestamps,
there is a reasonably high likelihood of timestamps being the same.
The IS_NEWER_THAN test returns true when timestamps are the same,
so don't redo downloads when they match exactly.
The cost of setting up and executing a separate sub-build to do the
download, update and patch steps required for FetchContent population
can be significant with some platforms and CMake generators. Avoid the
sub-build altogether by invoking the step scripts directly.
Previously, if no generator was set (e.g. population was being done in
script mode), a generator needed to be available on the default PATH.
Since we no longer use a sub-build, this restriction is also now gone.
Fixes: #21703
The mkdir, download, update and patch steps are used by
FetchContent during the configure phase of the main build. Because
these steps need a target, this has so far required a sub-build to be
set up. The changes here factor out the preparation of the scripts
from the creation of the targets, allowing future work to leverage these
steps without a sub-build (see #21703).
As part of the refactoring, some rationalisation of the stamp files,
repository info files and script names was done to make things more
consistent between download methods and step implementations.
Every download method now records its own specific repository info
in a file and that file is a dependency of the download step. The source
directory is also written to that file, so if the SOURCE_DIR changes, the
download will be retriggered (the existing implementation fails in this
scenario). Each download method now also has just one driver script
that implements the whole step (it may pull in other scripts to do its
task though). The patch step gained support for USES_TERMINAL as
a result of generalising the implementation for custom commands.
Fixes: #21748
