Provide a way for custom commands to inform the ninja build tool about
their implicit dependencies. For now simply make use of the option an
error on other generators.
Closes: #15479
Custom command dependencies are followed for each target's source files
and add their transitive closure to the corresponding target. This
means that when a custom command in one target has a dependency on a
custom command in another target, both will appear in the dependent
target's sources. For the Makefile, VS IDE, and Xcode generators this
is not a problem because each target gets its own independent build
system that is evaluated in target dependency order. By the time the
dependent target is built the custom command that belongs to one of its
dependencies will already have been brought up to date.
For the Ninja generator we need to generate a monolithic build system
covering all targets so we can have only one copy of a custom command.
This means that we need to reconcile the target-level ordering
dependencies from its appearance in multiple targets to include only the
least-dependent common set. This is done by computing the set
intersection of the dependencies of all the targets containing a custom
command. However, we previously included only the direct dependencies
so any target-level dependency not directly added to all targets into
which a custom command propagates was discarded.
Fix this by computing the transitive closure of dependencies for each
target and then intersecting those sets. That will get the common set
of dependencies. Also add a test to cover a case in which the
incorrectly dropped target ordering dependencies would fail.
5784747d Improve string find: prefer character overloads.
5cec953e Use std::replace for replacing chars in strings.
2a1a2033 cmExtraEclipseCDT4Generator: use std::replace.
34bc6e1f cmCTestScriptHandler: don't call find repeatedly.
All paths generated on Ninja-invoked command lines should be passed
through ConvertToNinjaPath. Fix ConvertToLinkReference to call this
instead of partially duplicating its implementation.
Run the `Utilities/Scripts/clang-format.bash` script to update
all our C++ code to a new style defined by `.clang-format`.
Use `clang-format` version 3.8.
* If you reached this commit for a line in `git blame`, re-run the blame
operation starting at the parent of this commit to see older history
for the content.
* See the parent commit for instructions to rebase a change across this
style transition commit.
Sort include directives within each block (separated by a blank line) in
lexicographic order (except to prioritize `sys/types.h` first). First
run `clang-format` with the config file:
---
SortIncludes: false
...
Commit the result temporarily. Then run `clang-format` again with:
---
SortIncludes: true
IncludeCategories:
- Regex: 'sys/types.h'
Priority: -1
...
Commit the result temporarily. Start a new branch and cherry-pick the
second commit. Manually resolve conflicts to preserve indentation of
re-ordered includes. This cleans up the include ordering without
changing any other style.
Use the following command to run `clang-format`:
$ git ls-files -z -- \
'*.c' '*.cc' '*.cpp' '*.cxx' '*.h' '*.hh' '*.hpp' '*.hxx' |
egrep -z -v '(Lexer|Parser|ParserHelper)\.' |
egrep -z -v '^Source/cm_sha2' |
egrep -z -v '^Source/(kwsys|CursesDialog/form)/' |
egrep -z -v '^Utilities/(KW|cm).*/' |
egrep -z -v '^Tests/Module/GenerateExportHeader' |
egrep -z -v '^Tests/RunCMake/CommandLine/cmake_depends/test_UTF-16LE.h' |
xargs -0 clang-format -i
This selects source files that do not come from a third-party.
Inspired-by: Daniel Pfeifer <daniel@pfeifer-mail.de>
The change in commit v3.5.0-rc1~198^2 (Ninja: Always re-run custom
commands that have symbolic dependencies, 2015-11-19) broke the
byproducts feature added by commit v3.2.0-rc1~340^2~2 (Add an option for
explicit BYPRODUCTS of custom commands, 2014-11-13) when SYMBOLIC
outputs also appear. This case occurs with AUTORCC-generated custom
targets because the output is SYMBOLIC (to always run) and the generated
file is a byproduct (for restat so dependents do not run unnecessarily).
The two use cases conflict because Ninja does not support per-output
restat. Favor restat whenever byproducts are present because it is
required for byproducts to work correctly. In use cases where we want
an always-run chain we simply will not be able to also use byproducts.
Fix logic introduced by commit v3.5.0-rc1~198^2 (Ninja: Always re-run
custom commands that have symbolic dependencies, 2015-11-19) to not
consider only the last output. We need to know if any output is
SYMBOLIC, so stop checking as soon as one is found.
Generate custom command build statements in the order we encounter
source files specifying them. Do not depend on pointer values of
internally allocated structures for ordering.
If a custom command has a SYMBOLIC output (that is never actually
created) then do not mark the custom command build statement as
'restat'. Otherwise other custom commands that depend on the symbolic
output may not always re-run because after running the first custom
command Ninja 'restat' will detect that the output timestamp did not
change and skip its dependents.
This was observed with the ExternalProject BUILD_ALWAYS option where
Ninja would not re-run the 'install' step each time 'build' re-runs.
55474e61 cmState: Move GetTargetTypeName from cmTarget.
38df5c36 Remove now-obsolete casts.
4ee2b267 cmGeneratorTarget: Use enum for GetType.
eac15298 cmState: Move TargetType enum from cmTarget.
482b3811 cmTarget: Move link type enum out.
2ee1cb85 cmTarget: Move ImportInfoMap out of internal class.
a48bcabd cmTarget: Move backtrace member out of internal class.
6694d993 cmTarget: Remove unneeded constructors.
983c00f8 Generators: Use GetType from the cmGeneratorTarget.
In the Ninja generator we run all build rules from the top of the build
tree rather than changing into each subdirectory. Therefore we convert
all paths relative to the HOME_OUTPUT directory. However, the Convert
method on cmLocalGenerator restricts relative path conversions to avoid
leaving the build tree with a "../" sequence. Therefore conversions
performed for "subdirectories" that are outside the top of the build
tree always use full paths while conversions performed for
subdirectories that are inside the top of the build tree may use
relative paths to refer to the same files.
Since Ninja always runs rules from the top of the build tree we should
convert them using only the top-level cmLocalGenerator in order to
remain consistent. Also extend the test suite with a case that fails
without this fix.
Refactor the local generator creation API to accept a
cmState::Snapshot. Adjust MakeLocalGenerator to use the 'current'
snapshot in cases where there is no parent. Create the snapshot
for subdirectories in cmMakefile::AddSubdirectory.
This means that snapshots are now created at the point of extending the tree,
as appropriate, and independently of the cmLocalGenerator and cmMakefile they
represent the state for.
Global properties are already global in scope, so remove the
overload for specifying it and port users of the API.
The call from cmMakefile::GetProperty can be simplified because
the scope is only used during chaining, and there is no further
chaining after processing global properties.
A common idiom in CMake-based build systems is to have custom commands
that generate files not listed explicitly as outputs so that these
files do not have to be newer than the inputs. The file modification
times of such "byproducts" are updated only when their content changes.
Then other build rules can depend on the byproducts explicitly so that
their dependents rebuild when the content of the original byproducts
really does change.
This "undeclared byproduct" approach is necessary for Makefile, VS, and
Xcode build tools because if a byproduct were listed as an output of a
rule then the rule would always rerun when the input is newer than the
byproduct but the byproduct may never be updated.
Ninja solves this problem by offering a 'restat' feature to check
whether an output was really modified after running a rule and tracking
the fact that it is up to date separately from its timestamp. However,
Ninja also stats all dependencies up front and will only restat files
that are listed as outputs of rules with the 'restat' option enabled.
Therefore an undeclared byproduct that does not exist at the start of
the build will be considered missing and the build will fail even if
other dependencies would cause the byproduct to be available before its
dependents build.
CMake works around this limitation by adding 'phony' build rules for
custom command dependencies in the build tree that do not have any
explicit specification of what produces them. This is not optimal
because it prevents Ninja from reporting an error when an input to a
rule really is missing. A better approach is to allow projects to
explicitly specify the byproducts of their custom commands so that no
phony rules are needed for them. In order to work with the non-Ninja
generators, the byproducts must be known separately from the outputs.
Add a new "BYPRODUCTS" option to the add_custom_command and
add_custom_target commands to specify byproducts explicitly. Teach the
Ninja generator to specify byproducts as outputs of the custom commands.
In the case of POST_BUILD, PRE_LINK, and PRE_BUILD events on targets
that link, the byproducts must be specified as outputs of the link rule
that runs the commands. Activate 'restat' for such rules so that Ninja
knows it needs to check the byproducts, but not for link rules that have
no byproducts.
Changes in commit b9aa5041 (cmLocalGenerator: Simplify GetIncludeFlags
output formatting, 2014-03-04) caused Windows Resource Compiler include
directories to be computed as relative paths in the Ninja generator.
This breaks the cmcldeps handling of include paths. The reason for the
regression is that several cmLocalGenerator::GetIncludeFlags callers
treated the fourth "bool forResponseFile" argument as if it controlled
whether include directories were a full path. It actually did control
that by accident until the above commit.
Add an explicit "bool forceFullPaths" argument to GetIncludeFlags
and thread the value through ConvertToIncludeReference as needed.
Update GetIncludeFlags call sites that really wanted to control the
forResponseFile setting to be aware of the new argument. Extend the
VSResource test to cover this case.
The pre-defined 'console' pool is a non-buffered pool that runs with a
depth of 1. CMake re-run cannot be run concurrently and it will
eventually output something. A non-buffered pool allows to get it as
soon as possible
Also, generate the minimal required version of Ninja in the build file.
Until now the cmCustomCommandGenerator was used only to compute the
command lines of a custom command. Generalize it to get the comment,
working directory, dependencies, and outputs of custom commands. Update
use in all generators to support this.
Use the clang RemoveCStrCalls tool to automatically migrate the
code. This was only run on linux, so does not have any positive or
negative effect on other platforms.
As an INTERFACE_LIBRARY has no direct link dependencies, we can
short-circuit in cmGeneratorExpressionEvaluator and
in cmGlobalGenerator::CheckLocalGenerators.
As they do not generate any output directly, any generate- or install-
related code acn also be short-circuited. Many of the local generators
already do this.
Because only INTERFACE related properties make sense on INTERFACE_LIBRARY
targets, avoid setting other properties, for example via defaults.
Just enough to reach the BuildMacContentDirectory method and the
NeedRelinkBeforeInstall methods.
In the future, those methods can be moved to cmGeneratorTarget.
Drop the "vsProjectFile" argument from cmTarget::TraceDependencies. It
appears to be the modern equivalent to a hunk added in commit ba68f771
(...added new custom command support, 2003-06-03):
+ name = libName;
+ name += ".dsp.cmake";
+ srcFilesToProcess.push(name);
but was broken by refactoring at some point. The current behavior tries
to trace dependencies on a source file named the same as a target, which
makes no sense. Furthermore, in code of the form
add_executable(foo foo.c)
add_custom_command(OUTPUT "${somewhere}/foo" ... DEPENDS foo)
the "vsProjectFile" value "foo" matches source "${somewhere}/foo.rule"
generated to hold the custom command and causes the command to be added
to the "foo" target incorrectly.
Simply drop the incorrect source file trace and supporting logic.
To properly track the usage of dependencies that are generated at
compile time as the side effect of other build steps we need
to make the WriteBuild and WritePhonyBuild commands non static
eb410e8 Ninja: disable cldeps for bcc32, it's too old, and ninja would also not build
5ead31d Ninja: try work around for bcc32 bug
1333b57 Ninja: build server fixes
9081e3a remove warning about unused parameter
f430bea Ninja: maybe this fixes the bcc32 build
f2c1288 Ninja: msvc6 for-scoping
44b9bbc Ninja: build with old msvc versions
57156a5 Ninja: build server fixes
f1abdce Ninja: some bytes of the rc files couldn't be piped correctly
2de963d Ninja: don't remove space between command and parameters
50b6f33 Ninja: build cmcldeps with mingw
c05653e Ninja: try to make GetProcessId visible
ab245ff Ninja: but cl supports /nologo ...
bf58e9a Ninja: no /nologo option in old rc.exe
2fb07fc Ninja: Eclipse and KDevelop fixes for ninja
518c065 Ninja: don't pollute build dir with preprocessed rc files
...
When linking with cmake and vs_link_* the command line
could be too long for cmd.exe, which needs not to be
called in this case. (was not cached by a test)
Introduce rules which don't use the shell and use this
rule when there are no pre or post step.
For free we get a small speedup, because cmd is then
not called.
Also be more accurate when estimating the
command line length.
Older C++ compilers do not provide a standard std::stringstream.
Use our compatibility interfaces instead.
Also avoid std::stringstream(openmode) signature. Our approximate
stringstream implementation provided when the standard one is not
available does not support the openmode argument.
When MinGW is used slashes are used for dependencies
because ar.exe can't read rsp files with backslashes.
Many thx to Claus Klein for starting working on this.
Implement cmGlobalGenerator::ComputeTargetObjects in the Ninja generator
to pre-compute all the object file names. Use the results during
generation instead of re-computing it later.
Generally these are only required in build statements, as Ninja wants
to be able to chop paths up. But it doesn't hurt to also try to use
them in command line arguments.
