In commit 695f0d0d3a (cmFortranParser: Parse keywords as lexical tokens,
2016-09-05, v3.7.0-rc1~150^2) we created keyword-specific variants of
the original `USE WORD other EOSTMT` production, such as
`MODULE WORD other EOSTMT` and `INTERFACE WORD other EOSTMT`. The same
pattern was used by more keyword-specific productions in commit b5ac8b8aa7
(Fortran: Add support for submodule syntax in dependency scanning,
2016-09-05, v3.7.0-rc1~73^2~1).
The postfix part (`other`) of several keyword-specific productions is
not needed to match Fortran syntax. See the Fortran 2018 standard,
para.4.1.4/1 on p.28, para.14.2.1/2 on pp.293-294. The postfix is
needed only for a case of operator 'use':
use <module-name> [, only : <list-of-vars>]
The unnecessary postfix matching from the keyword-specific productions
such as module, submodule, and interface declarations can cause spurious
module dependencies to be detected, so remove it.
Extend the test suite with examples covering the previously-broken
cases.
Fixes: #18427
We read sources in binary format, so we need to explicitly match CR
(`\r`) characters that occur as part of newlines in CRLF sources.
This is particularly important when line continuation (`&`) occurs
in the middle of module declaration or usage.
Old implementation uses involved Flex input management technique that
requires usage of obsolete YY_INPUT macro. This causes a lot of useless
allocations and byte-by-byte scanning. New implementation avoids those
hacks, it uses yy_scan_string() API to setup Flex input. Also it fixes
reporting of syntax error position and corresponding tests.
Introduce the function cmReadGccDepfile that parses a GCC-style depfile
and returns its content. The implementation uses a lexer that is
modeled after the re2c implementation in Ninja.
The sample files of the autotest have been created with gcc 8.3.0.
This depfile reader is to be used by the Autogen facility to make use
of the depfiles that are generated by Qt's meta object compiler.
The `cmsys/Enconding.h` include had a typo in its surrounding ifdef,
possibly causing a missing function declaration (`cmsysEncoding_DupToWide`).
As this is C code, this resulted in the code compiling, but with a truncated
return value, possibly causing crashes.
clang scan-build warns about YY_CURRENT_BUFFER_LVALUE being zero-
allocated, but the whole point of the macro is to know that it's
not zero. Wrap the entire file in an "#ifndef __clang_analyzer__"
block in order to suppress the warning.
The lexical token expression added by commit v3.7.0-rc1~73^2~1 (Fortran:
Add support for submodule syntax in dependency scanning, 2016-09-05)
has a typo and does not match upper-case `B` in `SUBMODULE`. Fix it.
Fixes: #18595
Prior to commit 7c4c13ffef (math: Reject unexpected expression input
explicitly, 2018-05-18) we ignored unexpected characters in an
expression that otherwise can be parsed. In order to preserve
compatibility with projects that accidentally used this, convert the
error to a warning.
5dbee9d2d0 math: Add options to calculate and format output as hexadecimal
8661e7052c math: Diagnose divide-by-zero
7c4c13ffef math: Reject unexpected expression input explicitly
5b0f73a15a math: Use 64-bit integer type for computation
963b8d9f6c math: Add RunCMake.math test to prepare for error cases
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2056
Extend the fix from commit v3.10.0-rc1~188^2 (ListFileLexer: fix
heap-buffer-overflow on malicious input, 2017-08-26) to apply to all
lexer token matches. Replace all `.` with `[^\0\n]`. Update all
`[^...]` match expressions to not match `\0`.
We cannot safely process null bytes in strings.
Fixes: #18124
Since commit v3.7.0-rc1~73^2~1 (Fortran: Add support for submodule
syntax in dependency scanning, 2016-09-05) we support parsing Fortran
sources that use submodule syntax, but it left addition of `.smod`
dependencies to future work. Add it now.
The syntax
submodule (module_name) submodule_name
means the current source requires `module_name.mod` and provides
`module_name@submodule_name.smod`. The syntax
submodule (module_name:submodule_name) nested_submodule_name
means the current source requires `module_name@submodule_name.smod`
provides `module_name@nested_submodule_name.smod`.
Fixes: #17017
In case a list file contains a null terminated string that is continued
until a later space, the lexer token information got inconsistent:
e.g. an argument "TEST\0FOOBAR" is passed by the lexer as a token
char* = "TEST\0FOOBAR" and length 11
^^ note: ascii 0x00
Using strdup in cmListFileLexer leads lexer->token.text to be allocated
with size 5 and lexer->token.length to be set to 11
A subsequent call to this function with an argument of 5 < length <= 11
wrongly assumed a sufficiently sized buffer and therefore corrupted the
heap buffer. The program might crash due to this corruption.
The case "NullTerminatedArgument" is intentionally using a quite large
'rest' to increase the chance to actually hit the issue. It will reliably
crash with address sanitizer enabled though.
This fix addresses all rules where arbitrary characters are matched to
ignore \0 in order to fall through to the rule that matches an arbitrary
character as BadCharacter.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
flex 2.6.3 introduces symbol remapping through macro definitions. hence the
change appears bigger than one would expect from a minor version upgrade.
In addition some manual cleanup that had to be done previously is now
obsolete. namely:
- the size_t cast of _yybytes_len in yy_scan_bytes
(i is now also defined int and not size_t anymore)
- the redefinition of yyl within yy_find_action
(yyl is now already defined as int)
Line number preprocessor directives (#line) were previously generated into
the c source file. This actually breaks debugging as debuggers have a hard
time finding the original cmFortranLexer.in.l and mapping the current
instruction to a meaningful location within that file.
The prefix "cmFortran_yy" can already be set as %option directly.
For convenience also provide a sed command for all the manual steps that
need to be done after generating.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
flex 2.6.3 introduces symbol remapping through macro definitions. hence the
change appears bigger than one would expect from a minor version upgrade.
In addition some manual cleanup that had to be done previously is now
obsolete. namely:
- the size_t cast of _yybytes_len in yy_scan_bytes
(i is now also defined int and not size_t anymore)
- the redefinition of yyl within yy_find_action
(yyl is now already defined as int)
Line number preprocessor directives (#line) were previously generated into
the c source file. This actually breaks debugging as debuggers have a hard
time finding the original cmExprLexer.in.l and mapping the current
instruction to a meaningful location within that file.
The prefix "cmExpr_yy" can already be set as %option directly.
For convenience also provide a sed command for all the manual steps that
need to be done after generating.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
flex 2.6.3 introduces symbol remapping through macro definitions. hence the
change appears bigger than one would expect from a minor version upgrade.
In addition some manual cleanup that had to be done previously is now
obsolete. namely:
- the size_t cast of _yybytes_len in yy_scan_bytes
(i is now also defined int and not size_t anymore)
- the redefinition of yyl within yy_find_action
(yyl is now already defined as int)
Line number preprocessor directives (#line) were previously generated into
the c source file. This actually breaks debugging as debuggers have a hard
time finding the original cmDependsJavaLexer.in.l and mapping the current
instruction to a meaningful location within that file.
The prefix "cmDependsJava_yy" can already be set as %option directly.
For convenience also provide a sed command for all the manual steps that
need to be done after generating.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
flex 2.6.3 introduces symbol remapping through macro definitions. hence the
change appears bigger than one would expect from a minor version upgrade.
In addition some manual cleanup that had to be done previously is now
obsolete. namely:
- the size_t cast of _yybytes_len in yy_scan_bytes
(i is now also defined int and not size_t anymore)
- the redefinition of yyl within yy_find_action
(yyl is now already defined as int)
Line number preprocessor directives (#line) were previously generated into
the c source file. This actually breaks debugging as debuggers have a hard
time finding the original cmCommandArgumentLexer.in.l and mapping the current
instruction to a meaningful location within that file.
The prefix "cmCommandArgument_yy" can already be set as %option directly.
For convenience also provide a sed command for all the manual steps that
need to be done after generating.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
flex 2.6.3 introduces symbol remapping through macro definitions. hence the
change appears bigger than one would expect from a minor version upgrade.
In addition some manual cleanup that had to be done previously is now
obsolete. namely:
- the size_t cast of _yybytes_len in yy_scan_bytes
(i is now also defined int and not size_t anymore)
- the redefinition of yyl within yy_find_action
(yyl is now already defined as int)
Line number preprocessor directives (#line) were previously generated into
the c source file. This actually breaks debugging as debuggers have a hard
time finding the original cmListFileLexer.in.l and mapping the current
instruction to a meaningful location within that file.
The prefix "cmListFileLexer_yy" can already be set as %option directly.
For convenience also provide a sed command for all the manual steps that
need to be done after generating.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
The commit v3.9.0-rc1~502^2 (cmListFileLexer: bail out on seek-errors,
2017-02-19) for bug # 16607 was only applied to the generated file.
Also apply the fix now to the original.
Signed-off-by: Matthias Maennich <matthias@maennich.net>
