| 1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
| 2 |
------------------ |
------------------ |
| 3 |
|
|
| 4 |
|
Version 8.37 xx-xxx-2015 |
| 5 |
|
------------------------ |
| 6 |
|
|
| 7 |
|
1. When an (*ACCEPT) is triggered inside capturing parentheses, it arranges |
| 8 |
|
for those parentheses to be closed with whatever has been captured so far. |
| 9 |
|
However, it was failing to mark any other groups between the hightest |
| 10 |
|
capture so far and the currrent group as "unset". Thus, the ovector for |
| 11 |
|
those groups contained whatever was previously there. An example is the |
| 12 |
|
pattern /(x)|((*ACCEPT))/ when matched against "abcd". |
| 13 |
|
|
| 14 |
|
2. If an assertion condition was quantified with a minimum of zero (an odd |
| 15 |
|
thing to do, but it happened), SIGSEGV or other misbehaviour could occur. |
| 16 |
|
|
| 17 |
|
3. If a pattern in pcretest input had the P (POSIX) modifier followed by an |
| 18 |
|
unrecognized modifier, a crash could occur. |
| 19 |
|
|
| 20 |
|
4. An attempt to do global matching in pcretest with a zero-length ovector |
| 21 |
|
caused a crash. |
| 22 |
|
|
| 23 |
|
5. Fixed a memory leak during matching that could occur for a subpattern |
| 24 |
|
subroutine call (recursive or otherwise) if the number of captured groups |
| 25 |
|
that had to be saved was greater than ten. |
| 26 |
|
|
| 27 |
|
6. Catch a bad opcode during auto-possessification after compiling a bad UTF |
| 28 |
|
string with NO_UTF_CHECK. This is a tidyup, not a bug fix, as passing bad |
| 29 |
|
UTF with NO_UTF_CHECK is documented as having an undefined outcome. |
| 30 |
|
|
| 31 |
|
7. A UTF pattern containing a "not" match of a non-ASCII character and a |
| 32 |
|
subroutine reference could loop at compile time. Example: /[^\xff]((?1))/. |
| 33 |
|
|
| 34 |
|
8. When a pattern is compiled, it remembers the highest back reference so that |
| 35 |
|
when matching, if the ovector is too small, extra memory can be obtained to |
| 36 |
|
use instead. A conditional subpattern whose condition is a check on a |
| 37 |
|
capture having happened, such as, for example in the pattern |
| 38 |
|
/^(?:(a)|b)(?(1)A|B)/, is another kind of back reference, but it was not |
| 39 |
|
setting the highest backreference number. This mattered only if pcre_exec() |
| 40 |
|
was called with an ovector that was too small to hold the capture, and there |
| 41 |
|
was no other kind of back reference (a situation which is probably quite |
| 42 |
|
rare). The effect of the bug was that the condition was always treated as |
| 43 |
|
FALSE when the capture could not be consulted, leading to a incorrect |
| 44 |
|
behaviour by pcre2_match(). This bug has been fixed. |
| 45 |
|
|
| 46 |
|
9. A reference to a duplicated named group (either a back reference or a test |
| 47 |
|
for being set in a conditional) that occurred in a part of the pattern where |
| 48 |
|
PCRE_DUPNAMES was not set caused the amount of memory needed for the pattern |
| 49 |
|
to be incorrectly calculated, leading to overwriting. |
| 50 |
|
|
| 51 |
|
10. A mutually recursive set of back references such as (\2)(\1) caused a |
| 52 |
|
segfault at study time (while trying to find the minimum matching length). |
| 53 |
|
The infinite loop is now broken (with the minimum length unset, that is, |
| 54 |
|
zero). |
| 55 |
|
|
| 56 |
|
11. If an assertion that was used as a condition was quantified with a minimum |
| 57 |
|
of zero, matching went wrong. In particular, if the whole group had |
| 58 |
|
unlimited repetition and could match an empty string, a segfault was |
| 59 |
|
likely. The pattern (?(?=0)?)+ is an example that caused this. Perl allows |
| 60 |
|
assertions to be quantified, but not if they are being used as conditions, |
| 61 |
|
so the above pattern is faulted by Perl. PCRE has now been changed so that |
| 62 |
|
it also rejects such patterns. |
| 63 |
|
|
| 64 |
|
12. A possessive capturing group such as (a)*+ with a minimum repeat of zero |
| 65 |
|
failed to allow the zero-repeat case if pcre2_exec() was called with an |
| 66 |
|
ovector too small to capture the group. |
| 67 |
|
|
| 68 |
|
13. Fixed two bugs in pcretest that were discovered by fuzzing and reported by |
| 69 |
|
Red Hat Product Security: |
| 70 |
|
|
| 71 |
|
(a) A crash if /K and /F were both set with the option to save the compiled |
| 72 |
|
pattern. |
| 73 |
|
|
| 74 |
|
(b) Another crash if the option to print captured substrings in a callout |
| 75 |
|
was combined with setting a null ovector, for example \O\C+ as a subject |
| 76 |
|
string. |
| 77 |
|
|
| 78 |
|
14. A pattern such as "((?2){0,1999}())?", which has a group containing a |
| 79 |
|
forward reference repeated a large (but limited) number of times within a |
| 80 |
|
repeated outer group that has a zero minimum quantifier, caused incorrect |
| 81 |
|
code to be compiled, leading to the error "internal error: |
| 82 |
|
previously-checked referenced subpattern not found" when an incorrect |
| 83 |
|
memory address was read. This bug was reported as "heap overflow", |
| 84 |
|
discovered by Kai Lu of Fortinet's FortiGuard Labs and given the CVE number |
| 85 |
|
CVE-2015-2325. |
| 86 |
|
|
| 87 |
|
23. A pattern such as "((?+1)(\1))/" containing a forward reference subroutine |
| 88 |
|
call within a group that also contained a recursive back reference caused |
| 89 |
|
incorrect code to be compiled. This bug was reported as "heap overflow", |
| 90 |
|
discovered by Kai Lu of Fortinet's FortiGuard Labs, and given the CVE |
| 91 |
|
number CVE-2015-2326. |
| 92 |
|
|
| 93 |
|
24. Computing the size of the JIT read-only data in advance has been a source |
| 94 |
|
of various issues, and new ones are still appear unfortunately. To fix |
| 95 |
|
existing and future issues, size computation is eliminated from the code, |
| 96 |
|
and replaced by on-demand memory allocation. |
| 97 |
|
|
| 98 |
|
25. A pattern such as /(?i)[A-`]/, where characters in the other case are |
| 99 |
|
adjacent to the end of the range, and the range contained characters with |
| 100 |
|
more than one other case, caused incorrect behaviour when compiled in UTF |
| 101 |
|
mode. In that example, the range a-j was left out of the class. |
| 102 |
|
|
| 103 |
|
26. Fix JIT compilation of conditional blocks, which assertion |
| 104 |
|
is converted to (*FAIL). E.g: /(?(?!))/. |
| 105 |
|
|
| 106 |
|
27. The pattern /(?(?!)^)/ caused references to random memory. This bug was |
| 107 |
|
discovered by the LLVM fuzzer. |
| 108 |
|
|
| 109 |
|
28. The assertion (?!) is optimized to (*FAIL). This was not handled correctly |
| 110 |
|
when this assertion was used as a condition, for example (?(?!)a|b). In |
| 111 |
|
pcre2_match() it worked by luck; in pcre2_dfa_match() it gave an incorrect |
| 112 |
|
error about an unsupported item. |
| 113 |
|
|
| 114 |
|
|
| 115 |
|
Version 8.36 26-September-2014 |
| 116 |
|
------------------------------ |
| 117 |
|
|
| 118 |
|
1. Got rid of some compiler warnings in the C++ modules that were shown up by |
| 119 |
|
-Wmissing-field-initializers and -Wunused-parameter. |
| 120 |
|
|
| 121 |
|
2. The tests for quantifiers being too big (greater than 65535) were being |
| 122 |
|
applied after reading the number, and stupidly assuming that integer |
| 123 |
|
overflow would give a negative number. The tests are now applied as the |
| 124 |
|
numbers are read. |
| 125 |
|
|
| 126 |
|
3. Tidy code in pcre_exec.c where two branches that used to be different are |
| 127 |
|
now the same. |
| 128 |
|
|
| 129 |
|
4. The JIT compiler did not generate match limit checks for certain |
| 130 |
|
bracketed expressions with quantifiers. This may lead to exponential |
| 131 |
|
backtracking, instead of returning with PCRE_ERROR_MATCHLIMIT. This |
| 132 |
|
issue should be resolved now. |
| 133 |
|
|
| 134 |
|
5. Fixed an issue, which occures when nested alternatives are optimized |
| 135 |
|
with table jumps. |
| 136 |
|
|
| 137 |
|
6. Inserted two casts and changed some ints to size_t in the light of some |
| 138 |
|
reported 64-bit compiler warnings (Bugzilla 1477). |
| 139 |
|
|
| 140 |
|
7. Fixed a bug concerned with zero-minimum possessive groups that could match |
| 141 |
|
an empty string, which sometimes were behaving incorrectly in the |
| 142 |
|
interpreter (though correctly in the JIT matcher). This pcretest input is |
| 143 |
|
an example: |
| 144 |
|
|
| 145 |
|
'\A(?:[^"]++|"(?:[^"]*+|"")*+")++' |
| 146 |
|
NON QUOTED "QUOT""ED" AFTER "NOT MATCHED |
| 147 |
|
|
| 148 |
|
the interpreter was reporting a match of 'NON QUOTED ' only, whereas the |
| 149 |
|
JIT matcher and Perl both matched 'NON QUOTED "QUOT""ED" AFTER '. The test |
| 150 |
|
for an empty string was breaking the inner loop and carrying on at a lower |
| 151 |
|
level, when possessive repeated groups should always return to a higher |
| 152 |
|
level as they have no backtrack points in them. The empty string test now |
| 153 |
|
occurs at the outer level. |
| 154 |
|
|
| 155 |
|
8. Fixed a bug that was incorrectly auto-possessifying \w+ in the pattern |
| 156 |
|
^\w+(?>\s*)(?<=\w) which caused it not to match "test test". |
| 157 |
|
|
| 158 |
|
9. Give a compile-time error for \o{} (as Perl does) and for \x{} (which Perl |
| 159 |
|
doesn't). |
| 160 |
|
|
| 161 |
|
10. Change 8.34/15 introduced a bug that caused the amount of memory needed |
| 162 |
|
to hold a pattern to be incorrectly computed (too small) when there were |
| 163 |
|
named back references to duplicated names. This could cause "internal |
| 164 |
|
error: code overflow" or "double free or corruption" or other memory |
| 165 |
|
handling errors. |
| 166 |
|
|
| 167 |
|
11. When named subpatterns had the same prefixes, back references could be |
| 168 |
|
confused. For example, in this pattern: |
| 169 |
|
|
| 170 |
|
/(?P<Name>a)?(?P<Name2>b)?(?(<Name>)c|d)*l/ |
| 171 |
|
|
| 172 |
|
the reference to 'Name' was incorrectly treated as a reference to a |
| 173 |
|
duplicate name. |
| 174 |
|
|
| 175 |
|
12. A pattern such as /^s?c/mi8 where the optional character has more than |
| 176 |
|
one "other case" was incorrectly compiled such that it would only try to |
| 177 |
|
match starting at "c". |
| 178 |
|
|
| 179 |
|
13. When a pattern starting with \s was studied, VT was not included in the |
| 180 |
|
list of possible starting characters; this should have been part of the |
| 181 |
|
8.34/18 patch. |
| 182 |
|
|
| 183 |
|
14. If a character class started [\Qx]... where x is any character, the class |
| 184 |
|
was incorrectly terminated at the ]. |
| 185 |
|
|
| 186 |
|
15. If a pattern that started with a caseless match for a character with more |
| 187 |
|
than one "other case" was studied, PCRE did not set up the starting code |
| 188 |
|
unit bit map for the list of possible characters. Now it does. This is an |
| 189 |
|
optimization improvement, not a bug fix. |
| 190 |
|
|
| 191 |
|
16. The Unicode data tables have been updated to Unicode 7.0.0. |
| 192 |
|
|
| 193 |
|
17. Fixed a number of memory leaks in pcregrep. |
| 194 |
|
|
| 195 |
|
18. Avoid a compiler warning (from some compilers) for a function call with |
| 196 |
|
a cast that removes "const" from an lvalue by using an intermediate |
| 197 |
|
variable (to which the compiler does not object). |
| 198 |
|
|
| 199 |
|
19. Incorrect code was compiled if a group that contained an internal recursive |
| 200 |
|
back reference was optional (had quantifier with a minimum of zero). This |
| 201 |
|
example compiled incorrect code: /(((a\2)|(a*)\g<-1>))*/ and other examples |
| 202 |
|
caused segmentation faults because of stack overflows at compile time. |
| 203 |
|
|
| 204 |
|
20. A pattern such as /((?(R)a|(?1)))+/, which contains a recursion within a |
| 205 |
|
group that is quantified with an indefinite repeat, caused a compile-time |
| 206 |
|
loop which used up all the system stack and provoked a segmentation fault. |
| 207 |
|
This was not the same bug as 19 above. |
| 208 |
|
|
| 209 |
|
21. Add PCRECPP_EXP_DECL declaration to operator<< in pcre_stringpiece.h. |
| 210 |
|
Patch by Mike Frysinger. |
| 211 |
|
|
| 212 |
|
|
| 213 |
Version 8.35 04-April-2014 |
Version 8.35 04-April-2014 |
| 214 |
-------------------------- |
-------------------------- |
| 215 |
|
|
| 236 |
|
|
| 237 |
6. Improve character range checks in JIT. Characters are read by an inprecise |
6. Improve character range checks in JIT. Characters are read by an inprecise |
| 238 |
function now, which returns with an unknown value if the character code is |
function now, which returns with an unknown value if the character code is |
| 239 |
above a certain treshold (e.g: 256). The only limitation is that the value |
above a certain threshold (e.g: 256). The only limitation is that the value |
| 240 |
must be bigger than the treshold as well. This function is useful, when |
must be bigger than the threshold as well. This function is useful when |
| 241 |
the characters above the treshold are handled in the same way. |
the characters above the threshold are handled in the same way. |
| 242 |
|
|
| 243 |
7. The macros whose names start with RAWUCHAR are placeholders for a future |
7. The macros whose names start with RAWUCHAR are placeholders for a future |
| 244 |
mode in which only the bottom 21 bits of 32-bit data items are used. To |
mode in which only the bottom 21 bits of 32-bit data items are used. To |
| 324 |
|
|
| 325 |
24. Experimental support for ARM-64 and MIPS-64 has been added to the JIT |
24. Experimental support for ARM-64 and MIPS-64 has been added to the JIT |
| 326 |
compiler. |
compiler. |
| 327 |
|
|
| 328 |
25. Change all the temporary files used in RunGrepTest to be different to those |
25. Change all the temporary files used in RunGrepTest to be different to those |
| 329 |
used by RunTest so that the tests can be run simultaneously, for example by |
used by RunTest so that the tests can be run simultaneously, for example by |
| 330 |
"make -j check". |
"make -j check". |
| 331 |
|
|
| 332 |
|
|
| 333 |
Version 8.34 15-December-2013 |
Version 8.34 15-December-2013 |
Parent Directory
| 
Revision Log
| 
Patch