1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.34 xx-xxxx-201x |
Version 8.37 28-April-2015 |
5 |
-------------------------- |
-------------------------- |
6 |
|
|
7 |
|
1. When an (*ACCEPT) is triggered inside capturing parentheses, it arranges |
8 |
|
for those parentheses to be closed with whatever has been captured so far. |
9 |
|
However, it was failing to mark any other groups between the hightest |
10 |
|
capture so far and the currrent group as "unset". Thus, the ovector for |
11 |
|
those groups contained whatever was previously there. An example is the |
12 |
|
pattern /(x)|((*ACCEPT))/ when matched against "abcd". |
13 |
|
|
14 |
|
2. If an assertion condition was quantified with a minimum of zero (an odd |
15 |
|
thing to do, but it happened), SIGSEGV or other misbehaviour could occur. |
16 |
|
|
17 |
|
3. If a pattern in pcretest input had the P (POSIX) modifier followed by an |
18 |
|
unrecognized modifier, a crash could occur. |
19 |
|
|
20 |
|
4. An attempt to do global matching in pcretest with a zero-length ovector |
21 |
|
caused a crash. |
22 |
|
|
23 |
|
5. Fixed a memory leak during matching that could occur for a subpattern |
24 |
|
subroutine call (recursive or otherwise) if the number of captured groups |
25 |
|
that had to be saved was greater than ten. |
26 |
|
|
27 |
|
6. Catch a bad opcode during auto-possessification after compiling a bad UTF |
28 |
|
string with NO_UTF_CHECK. This is a tidyup, not a bug fix, as passing bad |
29 |
|
UTF with NO_UTF_CHECK is documented as having an undefined outcome. |
30 |
|
|
31 |
|
7. A UTF pattern containing a "not" match of a non-ASCII character and a |
32 |
|
subroutine reference could loop at compile time. Example: /[^\xff]((?1))/. |
33 |
|
|
34 |
|
8. When a pattern is compiled, it remembers the highest back reference so that |
35 |
|
when matching, if the ovector is too small, extra memory can be obtained to |
36 |
|
use instead. A conditional subpattern whose condition is a check on a |
37 |
|
capture having happened, such as, for example in the pattern |
38 |
|
/^(?:(a)|b)(?(1)A|B)/, is another kind of back reference, but it was not |
39 |
|
setting the highest backreference number. This mattered only if pcre_exec() |
40 |
|
was called with an ovector that was too small to hold the capture, and there |
41 |
|
was no other kind of back reference (a situation which is probably quite |
42 |
|
rare). The effect of the bug was that the condition was always treated as |
43 |
|
FALSE when the capture could not be consulted, leading to a incorrect |
44 |
|
behaviour by pcre_exec(). This bug has been fixed. |
45 |
|
|
46 |
|
9. A reference to a duplicated named group (either a back reference or a test |
47 |
|
for being set in a conditional) that occurred in a part of the pattern where |
48 |
|
PCRE_DUPNAMES was not set caused the amount of memory needed for the pattern |
49 |
|
to be incorrectly calculated, leading to overwriting. |
50 |
|
|
51 |
|
10. A mutually recursive set of back references such as (\2)(\1) caused a |
52 |
|
segfault at study time (while trying to find the minimum matching length). |
53 |
|
The infinite loop is now broken (with the minimum length unset, that is, |
54 |
|
zero). |
55 |
|
|
56 |
|
11. If an assertion that was used as a condition was quantified with a minimum |
57 |
|
of zero, matching went wrong. In particular, if the whole group had |
58 |
|
unlimited repetition and could match an empty string, a segfault was |
59 |
|
likely. The pattern (?(?=0)?)+ is an example that caused this. Perl allows |
60 |
|
assertions to be quantified, but not if they are being used as conditions, |
61 |
|
so the above pattern is faulted by Perl. PCRE has now been changed so that |
62 |
|
it also rejects such patterns. |
63 |
|
|
64 |
|
12. A possessive capturing group such as (a)*+ with a minimum repeat of zero |
65 |
|
failed to allow the zero-repeat case if pcre2_exec() was called with an |
66 |
|
ovector too small to capture the group. |
67 |
|
|
68 |
|
13. Fixed two bugs in pcretest that were discovered by fuzzing and reported by |
69 |
|
Red Hat Product Security: |
70 |
|
|
71 |
|
(a) A crash if /K and /F were both set with the option to save the compiled |
72 |
|
pattern. |
73 |
|
|
74 |
|
(b) Another crash if the option to print captured substrings in a callout |
75 |
|
was combined with setting a null ovector, for example \O\C+ as a subject |
76 |
|
string. |
77 |
|
|
78 |
|
14. A pattern such as "((?2){0,1999}())?", which has a group containing a |
79 |
|
forward reference repeated a large (but limited) number of times within a |
80 |
|
repeated outer group that has a zero minimum quantifier, caused incorrect |
81 |
|
code to be compiled, leading to the error "internal error: |
82 |
|
previously-checked referenced subpattern not found" when an incorrect |
83 |
|
memory address was read. This bug was reported as "heap overflow", |
84 |
|
discovered by Kai Lu of Fortinet's FortiGuard Labs and given the CVE number |
85 |
|
CVE-2015-2325. |
86 |
|
|
87 |
|
23. A pattern such as "((?+1)(\1))/" containing a forward reference subroutine |
88 |
|
call within a group that also contained a recursive back reference caused |
89 |
|
incorrect code to be compiled. This bug was reported as "heap overflow", |
90 |
|
discovered by Kai Lu of Fortinet's FortiGuard Labs, and given the CVE |
91 |
|
number CVE-2015-2326. |
92 |
|
|
93 |
|
24. Computing the size of the JIT read-only data in advance has been a source |
94 |
|
of various issues, and new ones are still appear unfortunately. To fix |
95 |
|
existing and future issues, size computation is eliminated from the code, |
96 |
|
and replaced by on-demand memory allocation. |
97 |
|
|
98 |
|
25. A pattern such as /(?i)[A-`]/, where characters in the other case are |
99 |
|
adjacent to the end of the range, and the range contained characters with |
100 |
|
more than one other case, caused incorrect behaviour when compiled in UTF |
101 |
|
mode. In that example, the range a-j was left out of the class. |
102 |
|
|
103 |
|
26. Fix JIT compilation of conditional blocks, which assertion |
104 |
|
is converted to (*FAIL). E.g: /(?(?!))/. |
105 |
|
|
106 |
|
27. The pattern /(?(?!)^)/ caused references to random memory. This bug was |
107 |
|
discovered by the LLVM fuzzer. |
108 |
|
|
109 |
|
28. The assertion (?!) is optimized to (*FAIL). This was not handled correctly |
110 |
|
when this assertion was used as a condition, for example (?(?!)a|b). In |
111 |
|
pcre2_match() it worked by luck; in pcre2_dfa_match() it gave an incorrect |
112 |
|
error about an unsupported item. |
113 |
|
|
114 |
|
29. For some types of pattern, for example /Z*(|d*){216}/, the auto- |
115 |
|
possessification code could take exponential time to complete. A recursion |
116 |
|
depth limit of 1000 has been imposed to limit the resources used by this |
117 |
|
optimization. |
118 |
|
|
119 |
|
30. A pattern such as /(*UTF)[\S\V\H]/, which contains a negated special class |
120 |
|
such as \S in non-UCP mode, explicit wide characters (> 255) can be ignored |
121 |
|
because \S ensures they are all in the class. The code for doing this was |
122 |
|
interacting badly with the code for computing the amount of space needed to |
123 |
|
compile the pattern, leading to a buffer overflow. This bug was discovered |
124 |
|
by the LLVM fuzzer. |
125 |
|
|
126 |
|
31. A pattern such as /((?2)+)((?1))/ which has mutual recursion nested inside |
127 |
|
other kinds of group caused stack overflow at compile time. This bug was |
128 |
|
discovered by the LLVM fuzzer. |
129 |
|
|
130 |
|
32. A pattern such as /(?1)(?#?'){8}(a)/ which had a parenthesized comment |
131 |
|
between a subroutine call and its quantifier was incorrectly compiled, |
132 |
|
leading to buffer overflow or other errors. This bug was discovered by the |
133 |
|
LLVM fuzzer. |
134 |
|
|
135 |
|
33. The illegal pattern /(?(?<E>.*!.*)?)/ was not being diagnosed as missing an |
136 |
|
assertion after (?(. The code was failing to check the character after |
137 |
|
(?(?< for the ! or = that would indicate a lookbehind assertion. This bug |
138 |
|
was discovered by the LLVM fuzzer. |
139 |
|
|
140 |
|
34. A pattern such as /X((?2)()*+){2}+/ which has a possessive quantifier with |
141 |
|
a fixed maximum following a group that contains a subroutine reference was |
142 |
|
incorrectly compiled and could trigger buffer overflow. This bug was |
143 |
|
discovered by the LLVM fuzzer. |
144 |
|
|
145 |
|
35. A mutual recursion within a lookbehind assertion such as (?<=((?2))((?1))) |
146 |
|
caused a stack overflow instead of the diagnosis of a non-fixed length |
147 |
|
lookbehind assertion. This bug was discovered by the LLVM fuzzer. |
148 |
|
|
149 |
|
36. The use of \K in a positive lookbehind assertion in a non-anchored pattern |
150 |
|
(e.g. /(?<=\Ka)/) could make pcregrep loop. |
151 |
|
|
152 |
|
37. There was a similar problem to 36 in pcretest for global matches. |
153 |
|
|
154 |
|
38. If a greedy quantified \X was preceded by \C in UTF mode (e.g. \C\X*), |
155 |
|
and a subsequent item in the pattern caused a non-match, backtracking over |
156 |
|
the repeated \X did not stop, but carried on past the start of the subject, |
157 |
|
causing reference to random memory and/or a segfault. There were also some |
158 |
|
other cases where backtracking after \C could crash. This set of bugs was |
159 |
|
discovered by the LLVM fuzzer. |
160 |
|
|
161 |
|
39. The function for finding the minimum length of a matching string could take |
162 |
|
a very long time if mutual recursion was present many times in a pattern, |
163 |
|
for example, /((?2){73}(?2))((?1))/. A better mutual recursion detection |
164 |
|
method has been implemented. This infelicity was discovered by the LLVM |
165 |
|
fuzzer. |
166 |
|
|
167 |
|
40. Static linking against the PCRE library using the pkg-config module was |
168 |
|
failing on missing pthread symbols. |
169 |
|
|
170 |
|
|
171 |
|
Version 8.36 26-September-2014 |
172 |
|
------------------------------ |
173 |
|
|
174 |
|
1. Got rid of some compiler warnings in the C++ modules that were shown up by |
175 |
|
-Wmissing-field-initializers and -Wunused-parameter. |
176 |
|
|
177 |
|
2. The tests for quantifiers being too big (greater than 65535) were being |
178 |
|
applied after reading the number, and stupidly assuming that integer |
179 |
|
overflow would give a negative number. The tests are now applied as the |
180 |
|
numbers are read. |
181 |
|
|
182 |
|
3. Tidy code in pcre_exec.c where two branches that used to be different are |
183 |
|
now the same. |
184 |
|
|
185 |
|
4. The JIT compiler did not generate match limit checks for certain |
186 |
|
bracketed expressions with quantifiers. This may lead to exponential |
187 |
|
backtracking, instead of returning with PCRE_ERROR_MATCHLIMIT. This |
188 |
|
issue should be resolved now. |
189 |
|
|
190 |
|
5. Fixed an issue, which occures when nested alternatives are optimized |
191 |
|
with table jumps. |
192 |
|
|
193 |
|
6. Inserted two casts and changed some ints to size_t in the light of some |
194 |
|
reported 64-bit compiler warnings (Bugzilla 1477). |
195 |
|
|
196 |
|
7. Fixed a bug concerned with zero-minimum possessive groups that could match |
197 |
|
an empty string, which sometimes were behaving incorrectly in the |
198 |
|
interpreter (though correctly in the JIT matcher). This pcretest input is |
199 |
|
an example: |
200 |
|
|
201 |
|
'\A(?:[^"]++|"(?:[^"]*+|"")*+")++' |
202 |
|
NON QUOTED "QUOT""ED" AFTER "NOT MATCHED |
203 |
|
|
204 |
|
the interpreter was reporting a match of 'NON QUOTED ' only, whereas the |
205 |
|
JIT matcher and Perl both matched 'NON QUOTED "QUOT""ED" AFTER '. The test |
206 |
|
for an empty string was breaking the inner loop and carrying on at a lower |
207 |
|
level, when possessive repeated groups should always return to a higher |
208 |
|
level as they have no backtrack points in them. The empty string test now |
209 |
|
occurs at the outer level. |
210 |
|
|
211 |
|
8. Fixed a bug that was incorrectly auto-possessifying \w+ in the pattern |
212 |
|
^\w+(?>\s*)(?<=\w) which caused it not to match "test test". |
213 |
|
|
214 |
|
9. Give a compile-time error for \o{} (as Perl does) and for \x{} (which Perl |
215 |
|
doesn't). |
216 |
|
|
217 |
|
10. Change 8.34/15 introduced a bug that caused the amount of memory needed |
218 |
|
to hold a pattern to be incorrectly computed (too small) when there were |
219 |
|
named back references to duplicated names. This could cause "internal |
220 |
|
error: code overflow" or "double free or corruption" or other memory |
221 |
|
handling errors. |
222 |
|
|
223 |
|
11. When named subpatterns had the same prefixes, back references could be |
224 |
|
confused. For example, in this pattern: |
225 |
|
|
226 |
|
/(?P<Name>a)?(?P<Name2>b)?(?(<Name>)c|d)*l/ |
227 |
|
|
228 |
|
the reference to 'Name' was incorrectly treated as a reference to a |
229 |
|
duplicate name. |
230 |
|
|
231 |
|
12. A pattern such as /^s?c/mi8 where the optional character has more than |
232 |
|
one "other case" was incorrectly compiled such that it would only try to |
233 |
|
match starting at "c". |
234 |
|
|
235 |
|
13. When a pattern starting with \s was studied, VT was not included in the |
236 |
|
list of possible starting characters; this should have been part of the |
237 |
|
8.34/18 patch. |
238 |
|
|
239 |
|
14. If a character class started [\Qx]... where x is any character, the class |
240 |
|
was incorrectly terminated at the ]. |
241 |
|
|
242 |
|
15. If a pattern that started with a caseless match for a character with more |
243 |
|
than one "other case" was studied, PCRE did not set up the starting code |
244 |
|
unit bit map for the list of possible characters. Now it does. This is an |
245 |
|
optimization improvement, not a bug fix. |
246 |
|
|
247 |
|
16. The Unicode data tables have been updated to Unicode 7.0.0. |
248 |
|
|
249 |
|
17. Fixed a number of memory leaks in pcregrep. |
250 |
|
|
251 |
|
18. Avoid a compiler warning (from some compilers) for a function call with |
252 |
|
a cast that removes "const" from an lvalue by using an intermediate |
253 |
|
variable (to which the compiler does not object). |
254 |
|
|
255 |
|
19. Incorrect code was compiled if a group that contained an internal recursive |
256 |
|
back reference was optional (had quantifier with a minimum of zero). This |
257 |
|
example compiled incorrect code: /(((a\2)|(a*)\g<-1>))*/ and other examples |
258 |
|
caused segmentation faults because of stack overflows at compile time. |
259 |
|
|
260 |
|
20. A pattern such as /((?(R)a|(?1)))+/, which contains a recursion within a |
261 |
|
group that is quantified with an indefinite repeat, caused a compile-time |
262 |
|
loop which used up all the system stack and provoked a segmentation fault. |
263 |
|
This was not the same bug as 19 above. |
264 |
|
|
265 |
|
21. Add PCRECPP_EXP_DECL declaration to operator<< in pcre_stringpiece.h. |
266 |
|
Patch by Mike Frysinger. |
267 |
|
|
268 |
|
|
269 |
|
Version 8.35 04-April-2014 |
270 |
|
-------------------------- |
271 |
|
|
272 |
|
1. A new flag is set, when property checks are present in an XCLASS. |
273 |
|
When this flag is not set, PCRE can perform certain optimizations |
274 |
|
such as studying these XCLASS-es. |
275 |
|
|
276 |
|
2. The auto-possessification of character sets were improved: a normal |
277 |
|
and an extended character set can be compared now. Furthermore |
278 |
|
the JIT compiler optimizes more character set checks. |
279 |
|
|
280 |
|
3. Got rid of some compiler warnings for potentially uninitialized variables |
281 |
|
that show up only when compiled with -O2. |
282 |
|
|
283 |
|
4. A pattern such as (?=ab\K) that uses \K in an assertion can set the start |
284 |
|
of a match later then the end of the match. The pcretest program was not |
285 |
|
handling the case sensibly - it was outputting from the start to the next |
286 |
|
binary zero. It now reports this situation in a message, and outputs the |
287 |
|
text from the end to the start. |
288 |
|
|
289 |
|
5. Fast forward search is improved in JIT. Instead of the first three |
290 |
|
characters, any three characters with fixed position can be searched. |
291 |
|
Search order: first, last, middle. |
292 |
|
|
293 |
|
6. Improve character range checks in JIT. Characters are read by an inprecise |
294 |
|
function now, which returns with an unknown value if the character code is |
295 |
|
above a certain threshold (e.g: 256). The only limitation is that the value |
296 |
|
must be bigger than the threshold as well. This function is useful when |
297 |
|
the characters above the threshold are handled in the same way. |
298 |
|
|
299 |
|
7. The macros whose names start with RAWUCHAR are placeholders for a future |
300 |
|
mode in which only the bottom 21 bits of 32-bit data items are used. To |
301 |
|
make this more memorable for those maintaining the code, the names have |
302 |
|
been changed to start with UCHAR21, and an extensive comment has been added |
303 |
|
to their definition. |
304 |
|
|
305 |
|
8. Add missing (new) files sljitNativeTILEGX.c and sljitNativeTILEGX-encoder.c |
306 |
|
to the export list in Makefile.am (they were accidentally omitted from the |
307 |
|
8.34 tarball). |
308 |
|
|
309 |
|
9. The informational output from pcretest used the phrase "starting byte set" |
310 |
|
which is inappropriate for the 16-bit and 32-bit libraries. As the output |
311 |
|
for "first char" and "need char" really means "non-UTF-char", I've changed |
312 |
|
"byte" to "char", and slightly reworded the output. The documentation about |
313 |
|
these values has also been (I hope) clarified. |
314 |
|
|
315 |
|
10. Another JIT related optimization: use table jumps for selecting the correct |
316 |
|
backtracking path, when more than four alternatives are present inside a |
317 |
|
bracket. |
318 |
|
|
319 |
|
11. Empty match is not possible, when the minimum length is greater than zero, |
320 |
|
and there is no \K in the pattern. JIT should avoid empty match checks in |
321 |
|
such cases. |
322 |
|
|
323 |
|
12. In a caseless character class with UCP support, when a character with more |
324 |
|
than one alternative case was not the first character of a range, not all |
325 |
|
the alternative cases were added to the class. For example, s and \x{17f} |
326 |
|
are both alternative cases for S: the class [RST] was handled correctly, |
327 |
|
but [R-T] was not. |
328 |
|
|
329 |
|
13. The configure.ac file always checked for pthread support when JIT was |
330 |
|
enabled. This is not used in Windows, so I have put this test inside a |
331 |
|
check for the presence of windows.h (which was already tested for). |
332 |
|
|
333 |
|
14. Improve pattern prefix search by a simplified Boyer-Moore algorithm in JIT. |
334 |
|
The algorithm provides a way to skip certain starting offsets, and usually |
335 |
|
faster than linear prefix searches. |
336 |
|
|
337 |
|
15. Change 13 for 8.20 updated RunTest to check for the 'fr' locale as well |
338 |
|
as for 'fr_FR' and 'french'. For some reason, however, it then used the |
339 |
|
Windows-specific input and output files, which have 'french' screwed in. |
340 |
|
So this could never have worked. One of the problems with locales is that |
341 |
|
they aren't always the same. I have now updated RunTest so that it checks |
342 |
|
the output of the locale test (test 3) against three different output |
343 |
|
files, and it allows the test to pass if any one of them matches. With luck |
344 |
|
this should make the test pass on some versions of Solaris where it was |
345 |
|
failing. Because of the uncertainty, the script did not used to stop if |
346 |
|
test 3 failed; it now does. If further versions of a French locale ever |
347 |
|
come to light, they can now easily be added. |
348 |
|
|
349 |
|
16. If --with-pcregrep-bufsize was given a non-integer value such as "50K", |
350 |
|
there was a message during ./configure, but it did not stop. This now |
351 |
|
provokes an error. The invalid example in README has been corrected. |
352 |
|
If a value less than the minimum is given, the minimum value has always |
353 |
|
been used, but now a warning is given. |
354 |
|
|
355 |
|
17. If --enable-bsr-anycrlf was set, the special 16/32-bit test failed. This |
356 |
|
was a bug in the test system, which is now fixed. Also, the list of various |
357 |
|
configurations that are tested for each release did not have one with both |
358 |
|
16/32 bits and --enable-bar-anycrlf. It now does. |
359 |
|
|
360 |
|
18. pcretest was missing "-C bsr" for displaying the \R default setting. |
361 |
|
|
362 |
|
19. Little endian PowerPC systems are supported now by the JIT compiler. |
363 |
|
|
364 |
|
20. The fast forward newline mechanism could enter to an infinite loop on |
365 |
|
certain invalid UTF-8 input. Although we don't support these cases |
366 |
|
this issue can be fixed by a performance optimization. |
367 |
|
|
368 |
|
21. Change 33 of 8.34 is not sufficient to ensure stack safety because it does |
369 |
|
not take account if existing stack usage. There is now a new global |
370 |
|
variable called pcre_stack_guard that can be set to point to an external |
371 |
|
function to check stack availability. It is called at the start of |
372 |
|
processing every parenthesized group. |
373 |
|
|
374 |
|
22. A typo in the code meant that in ungreedy mode the max/min qualifier |
375 |
|
behaved like a min-possessive qualifier, and, for example, /a{1,3}b/U did |
376 |
|
not match "ab". |
377 |
|
|
378 |
|
23. When UTF was disabled, the JIT program reported some incorrect compile |
379 |
|
errors. These messages are silenced now. |
380 |
|
|
381 |
|
24. Experimental support for ARM-64 and MIPS-64 has been added to the JIT |
382 |
|
compiler. |
383 |
|
|
384 |
|
25. Change all the temporary files used in RunGrepTest to be different to those |
385 |
|
used by RunTest so that the tests can be run simultaneously, for example by |
386 |
|
"make -j check". |
387 |
|
|
388 |
|
|
389 |
|
Version 8.34 15-December-2013 |
390 |
|
----------------------------- |
391 |
|
|
392 |
1. Add pcre[16|32]_jit_free_unused_memory to forcibly free unused JIT |
1. Add pcre[16|32]_jit_free_unused_memory to forcibly free unused JIT |
393 |
executable memory. Patch inspired by Carsten Klein. |
executable memory. Patch inspired by Carsten Klein. |
394 |
|
|
395 |
2. ./configure --enable-coverage defined SUPPORT_GCOV in config.h, although |
2. ./configure --enable-coverage defined SUPPORT_GCOV in config.h, although |
396 |
this macro is never tested and has no effect, because the work to support |
this macro is never tested and has no effect, because the work to support |
397 |
coverage involves only compiling and linking options and special targets in |
coverage involves only compiling and linking options and special targets in |
398 |
the Makefile. The comment in config.h implied that defining the macro would |
the Makefile. The comment in config.h implied that defining the macro would |
399 |
enable coverage support, which is totally false. There was also support for |
enable coverage support, which is totally false. There was also support for |
400 |
setting this macro in the CMake files (my fault, I just copied it from |
setting this macro in the CMake files (my fault, I just copied it from |
401 |
configure). SUPPORT_GCOV has now been removed. |
configure). SUPPORT_GCOV has now been removed. |
402 |
|
|
403 |
3. Make a small performance improvement in strlen16() and strlen32() in |
3. Make a small performance improvement in strlen16() and strlen32() in |
404 |
pcretest. |
pcretest. |
405 |
|
|
406 |
4. Change 36 for 8.33 left some unreachable statements in pcre_exec.c, |
4. Change 36 for 8.33 left some unreachable statements in pcre_exec.c, |
407 |
detected by the Solaris compiler (gcc doesn't seem to be able to diagnose |
detected by the Solaris compiler (gcc doesn't seem to be able to diagnose |
408 |
these cases). There was also one in pcretest.c. |
these cases). There was also one in pcretest.c. |
409 |
|
|
410 |
5. Cleaned up a "may be uninitialized" compiler warning in pcre_exec.c. |
5. Cleaned up a "may be uninitialized" compiler warning in pcre_exec.c. |
411 |
|
|
412 |
6. In UTF mode, the code for checking whether a group could match an empty |
6. In UTF mode, the code for checking whether a group could match an empty |
413 |
string (which is used for indefinitely repeated groups to allow for |
string (which is used for indefinitely repeated groups to allow for |
416 |
data item and had a minimum repetition of zero (for example, [^\x{100}]* in |
data item and had a minimum repetition of zero (for example, [^\x{100}]* in |
417 |
UTF-8 mode). The effect was undefined: the group might or might not be |
UTF-8 mode). The effect was undefined: the group might or might not be |
418 |
deemed as matching an empty string, or the program might have crashed. |
deemed as matching an empty string, or the program might have crashed. |
419 |
|
|
420 |
7. The code for checking whether a group could match an empty string was not |
7. The code for checking whether a group could match an empty string was not |
421 |
recognizing that \h, \H, \v, \V, and \R must match a character. |
recognizing that \h, \H, \v, \V, and \R must match a character. |
422 |
|
|
423 |
8. Implemented PCRE_INFO_MATCH_EMPTY, which yields 1 if the pattern can match |
8. Implemented PCRE_INFO_MATCH_EMPTY, which yields 1 if the pattern can match |
424 |
an empty string. If it can, pcretest shows this in its information output. |
an empty string. If it can, pcretest shows this in its information output. |
425 |
|
|
426 |
9. Fixed two related bugs that applied to Unicode extended grapheme clusters |
9. Fixed two related bugs that applied to Unicode extended grapheme clusters |
427 |
that were repeated with a maximizing qualifier (e.g. \X* or \X{2,5}) when |
that were repeated with a maximizing qualifier (e.g. \X* or \X{2,5}) when |
428 |
matched by pcre_exec() without using JIT: |
matched by pcre_exec() without using JIT: |
429 |
|
|
430 |
(a) If the rest of the pattern did not match after a maximal run of |
(a) If the rest of the pattern did not match after a maximal run of |
431 |
grapheme clusters, the code for backing up to try with fewer of them |
grapheme clusters, the code for backing up to try with fewer of them |
432 |
did not always back up over a full grapheme when characters that do not |
did not always back up over a full grapheme when characters that do not |
433 |
have the modifier quality were involved, e.g. Hangul syllables. |
have the modifier quality were involved, e.g. Hangul syllables. |
434 |
|
|
435 |
(b) If the match point in a subject started with modifier character, and |
(b) If the match point in a subject started with modifier character, and |
436 |
there was no match, the code could incorrectly back up beyond the match |
there was no match, the code could incorrectly back up beyond the match |
437 |
point, and potentially beyond the first character in the subject, |
point, and potentially beyond the first character in the subject, |
438 |
leading to a segfault or an incorrect match result. |
leading to a segfault or an incorrect match result. |
439 |
|
|
440 |
10. A conditional group with an assertion condition could lead to PCRE |
10. A conditional group with an assertion condition could lead to PCRE |
441 |
recording an incorrect first data item for a match if no other first data |
recording an incorrect first data item for a match if no other first data |
442 |
item was recorded. For example, the pattern (?(?=ab)ab) recorded "a" as a |
item was recorded. For example, the pattern (?(?=ab)ab) recorded "a" as a |
443 |
first data item, and therefore matched "ca" after "c" instead of at the |
first data item, and therefore matched "ca" after "c" instead of at the |
444 |
start. |
start. |
445 |
|
|
446 |
11. Change 40 for 8.33 (allowing pcregrep to find empty strings) showed up a |
11. Change 40 for 8.33 (allowing pcregrep to find empty strings) showed up a |
447 |
bug that caused the command "echo a | ./pcregrep -M '|a'" to loop. |
bug that caused the command "echo a | ./pcregrep -M '|a'" to loop. |
448 |
|
|
449 |
12. The source of pcregrep now includes z/OS-specific code so that it can be |
12. The source of pcregrep now includes z/OS-specific code so that it can be |
450 |
compiled for z/OS as part of the special z/OS distribution. |
compiled for z/OS as part of the special z/OS distribution. |
451 |
|
|
452 |
13. Added the -T and -TM options to pcretest. |
13. Added the -T and -TM options to pcretest. |
453 |
|
|
454 |
14. The code in pcre_compile.c for creating the table of named capturing groups |
14. The code in pcre_compile.c for creating the table of named capturing groups |
455 |
has been refactored. Instead of creating the table dynamically during the |
has been refactored. Instead of creating the table dynamically during the |
456 |
actual compiling pass, the information is remembered during the pre-compile |
actual compiling pass, the information is remembered during the pre-compile |
457 |
pass (on the stack unless there are more than 20 named groups, in which |
pass (on the stack unless there are more than 20 named groups, in which |
458 |
case malloc() is used) and the whole table is created before the actual |
case malloc() is used) and the whole table is created before the actual |
459 |
compile happens. This has simplified the code (it is now nearly 150 lines |
compile happens. This has simplified the code (it is now nearly 150 lines |
460 |
shorter) and prepared the way for better handling of references to groups |
shorter) and prepared the way for better handling of references to groups |
461 |
with duplicate names. |
with duplicate names. |
462 |
|
|
463 |
15. A back reference to a named subpattern when there is more than one of the |
15. A back reference to a named subpattern when there is more than one of the |
464 |
same name now checks them in the order in which they appear in the pattern. |
same name now checks them in the order in which they appear in the pattern. |
465 |
The first one that is set is used for the reference. Previously only the |
The first one that is set is used for the reference. Previously only the |
466 |
first one was inspected. This change makes PCRE more compatible with Perl. |
first one was inspected. This change makes PCRE more compatible with Perl. |
467 |
|
|
468 |
16. Unicode character properties were updated from Unicode 6.3.0. |
16. Unicode character properties were updated from Unicode 6.3.0. |
469 |
|
|
470 |
17. The compile-time code for auto-possessification has been refactored, based |
17. The compile-time code for auto-possessification has been refactored, based |
471 |
on a patch by Zoltan Herczeg. It now happens after instead of during |
on a patch by Zoltan Herczeg. It now happens after instead of during |
472 |
compilation. The code is cleaner, and more cases are handled. The option |
compilation. The code is cleaner, and more cases are handled. The option |
473 |
PCRE_NO_AUTO_POSSESSIFY is added for testing purposes, and the -O and /O |
PCRE_NO_AUTO_POSSESS is added for testing purposes, and the -O and /O |
474 |
options in pcretest are provided to set it. |
options in pcretest are provided to set it. It can also be set by |
475 |
|
(*NO_AUTO_POSSESS) at the start of a pattern. |
476 |
18. The character VT has been added to the set of characters that match \s and |
|
477 |
are generally treated as white space, following this same change in Perl |
18. The character VT has been added to the default ("C" locale) set of |
478 |
5.18. There is now no difference between "Perl space" and "POSIX space". |
characters that match \s and are generally treated as white space, |
479 |
|
following this same change in Perl 5.18. There is now no difference between |
480 |
19. The code for checking named groups as conditions, either for being set or |
"Perl space" and "POSIX space". Whether VT is treated as white space in |
481 |
|
other locales depends on the locale. |
482 |
|
|
483 |
|
19. The code for checking named groups as conditions, either for being set or |
484 |
for being recursed, has been refactored (this is related to 14 and 15 |
for being recursed, has been refactored (this is related to 14 and 15 |
485 |
above). Processing unduplicated named groups should now be as fast at |
above). Processing unduplicated named groups should now be as fast at |
486 |
numerical groups, and processing duplicated groups should be faster than |
numerical groups, and processing duplicated groups should be faster than |
487 |
before. |
before. |
488 |
|
|
489 |
20. Two patches to the CMake build system, by Alexander Barkov: |
20. Two patches to the CMake build system, by Alexander Barkov: |
490 |
|
|
491 |
(1) Replace the "source" command by "." in CMakeLists.txt because |
(1) Replace the "source" command by "." in CMakeLists.txt because |
492 |
"source" is a bash-ism. |
"source" is a bash-ism. |
493 |
|
|
494 |
(2) Add missing HAVE_STDINT_H and HAVE_INTTYPES_H to config-cmake.h.in; |
(2) Add missing HAVE_STDINT_H and HAVE_INTTYPES_H to config-cmake.h.in; |
495 |
without these the CMake build does not work on Solaris. |
without these the CMake build does not work on Solaris. |
496 |
|
|
497 |
21. Perl has changed its handling of \8 and \9. If there is no previously |
21. Perl has changed its handling of \8 and \9. If there is no previously |
498 |
encountered capturing group of those numbers, they are treated as the |
encountered capturing group of those numbers, they are treated as the |
499 |
literal characters 8 and 9 instead of a binary zero followed by the |
literal characters 8 and 9 instead of a binary zero followed by the |
500 |
literals. PCRE now does the same. |
literals. PCRE now does the same. |
501 |
|
|
502 |
22. Following Perl, added \o{} to specify codepoints in octal, making it |
22. Following Perl, added \o{} to specify codepoints in octal, making it |
503 |
possible to specify values greater than 0777 and also making them |
possible to specify values greater than 0777 and also making them |
504 |
unambiguous. |
unambiguous. |
505 |
|
|
506 |
23. Perl now gives an error for missing closing braces after \x{... instead of |
23. Perl now gives an error for missing closing braces after \x{... instead of |
507 |
treating the string as literal. PCRE now does the same. |
treating the string as literal. PCRE now does the same. |
508 |
|
|
509 |
24. RunTest used to grumble if an inappropriate test was selected explicitly, |
24. RunTest used to grumble if an inappropriate test was selected explicitly, |
510 |
but just skip it when running all tests. This make it awkward to run ranges |
but just skip it when running all tests. This make it awkward to run ranges |
511 |
of tests when one of them was inappropriate. Now it just skips any |
of tests when one of them was inappropriate. Now it just skips any |
512 |
inappropriate tests, as it always did when running all tests. |
inappropriate tests, as it always did when running all tests. |
513 |
|
|
514 |
25. If PCRE_AUTO_CALLOUT and PCRE_UCP were set for a pattern that contained |
25. If PCRE_AUTO_CALLOUT and PCRE_UCP were set for a pattern that contained |
515 |
character types such as \d or \w, too many callouts were inserted, and the |
character types such as \d or \w, too many callouts were inserted, and the |
516 |
data that they returned was rubbish. |
data that they returned was rubbish. |
517 |
|
|
518 |
26. In UCP mode, \s was not matching two of the characters that Perl matches, |
26. In UCP mode, \s was not matching two of the characters that Perl matches, |
519 |
namely NEL (U+0085) and MONGOLIAN VOWEL SEPARATOR (U+180E), though they |
namely NEL (U+0085) and MONGOLIAN VOWEL SEPARATOR (U+180E), though they |
520 |
were matched by \h. The code has now been refactored so that the lists of |
were matched by \h. The code has now been refactored so that the lists of |
521 |
the horizontal and vertical whitespace characters used for \h and \v (which |
the horizontal and vertical whitespace characters used for \h and \v (which |
522 |
are defined only in one place) are now also used for \s. |
are defined only in one place) are now also used for \s. |
523 |
|
|
524 |
27. Add JIT support for the 64 bit TileGX architecture. |
27. Add JIT support for the 64 bit TileGX architecture. |
525 |
Patch by Jiong Wang (Tilera Corporation). |
Patch by Jiong Wang (Tilera Corporation). |
526 |
|
|
527 |
|
28. Possessive quantifiers for classes (both explicit and automatically |
528 |
|
generated) now use special opcodes instead of wrapping in ONCE brackets. |
529 |
|
|
530 |
|
29. Whereas an item such as A{4}+ ignored the possessivenes of the quantifier |
531 |
|
(because it's meaningless), this was not happening when PCRE_CASELESS was |
532 |
|
set. Not wrong, but inefficient. |
533 |
|
|
534 |
|
30. Updated perltest.pl to add /u (force Unicode mode) when /W (use Unicode |
535 |
|
properties for \w, \d, etc) is present in a test regex. Otherwise if the |
536 |
|
test contains no characters greater than 255, Perl doesn't realise it |
537 |
|
should be using Unicode semantics. |
538 |
|
|
539 |
|
31. Upgraded the handling of the POSIX classes [:graph:], [:print:], and |
540 |
|
[:punct:] when PCRE_UCP is set so as to include the same characters as Perl |
541 |
|
does in Unicode mode. |
542 |
|
|
543 |
|
32. Added the "forbid" facility to pcretest so that putting tests into the |
544 |
|
wrong test files can sometimes be quickly detected. |
545 |
|
|
546 |
|
33. There is now a limit (default 250) on the depth of nesting of parentheses. |
547 |
|
This limit is imposed to control the amount of system stack used at compile |
548 |
|
time. It can be changed at build time by --with-parens-nest-limit=xxx or |
549 |
|
the equivalent in CMake. |
550 |
|
|
551 |
|
34. Character classes such as [A-\d] or [a-[:digit:]] now cause compile-time |
552 |
|
errors. Perl warns for these when in warning mode, but PCRE has no facility |
553 |
|
for giving warnings. |
554 |
|
|
555 |
|
35. Change 34 for 8.13 allowed quantifiers on assertions, because Perl does. |
556 |
|
However, this was not working for (?!) because it is optimized to (*FAIL), |
557 |
|
for which PCRE does not allow quantifiers. The optimization is now disabled |
558 |
|
when a quantifier follows (?!). I can't see any use for this, but it makes |
559 |
|
things uniform. |
560 |
|
|
561 |
|
36. Perl no longer allows group names to start with digits, so I have made this |
562 |
|
change also in PCRE. It simplifies the code a bit. |
563 |
|
|
564 |
|
37. In extended mode, Perl ignores spaces before a + that indicates a |
565 |
|
possessive quantifier. PCRE allowed a space before the quantifier, but not |
566 |
|
before the possessive +. It now does. |
567 |
|
|
568 |
|
38. The use of \K (reset reported match start) within a repeated possessive |
569 |
|
group such as (a\Kb)*+ was not working. |
570 |
|
|
571 |
|
40. Document that the same character tables must be used at compile time and |
572 |
|
run time, and that the facility to pass tables to pcre_exec() and |
573 |
|
pcre_dfa_exec() is for use only with saved/restored patterns. |
574 |
|
|
575 |
|
41. Applied Jeff Trawick's patch CMakeLists.txt, which "provides two new |
576 |
|
features for Builds with MSVC: |
577 |
|
|
578 |
|
1. Support pcre.rc and/or pcreposix.rc (as is already done for MinGW |
579 |
|
builds). The .rc files can be used to set FileDescription and many other |
580 |
|
attributes. |
581 |
|
|
582 |
|
2. Add an option (-DINSTALL_MSVC_PDB) to enable installation of .pdb files. |
583 |
|
This allows higher-level build scripts which want .pdb files to avoid |
584 |
|
hard-coding the exact files needed." |
585 |
|
|
586 |
|
42. Added support for [[:<:]] and [[:>:]] as used in the BSD POSIX library to |
587 |
|
mean "start of word" and "end of word", respectively, as a transition aid. |
588 |
|
|
589 |
|
43. A minimizing repeat of a class containing codepoints greater than 255 in |
590 |
|
non-UTF 16-bit or 32-bit modes caused an internal error when PCRE was |
591 |
|
compiled to use the heap for recursion. |
592 |
|
|
593 |
|
44. Got rid of some compiler warnings for unused variables when UTF but not UCP |
594 |
|
is configured. |
595 |
|
|
596 |
|
|
597 |
Version 8.33 28-May-2013 |
Version 8.33 28-May-2013 |
598 |
------------------------ |
------------------------ |