1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.13 30-Apr-2011 |
Version 8.34 xx-xxxx-201x |
5 |
|
-------------------------- |
6 |
|
|
7 |
|
1. Add pcre[16|32]_jit_free_unused_memory to forcibly free unused JIT |
8 |
|
executable memory. Patch inspired by Carsten Klein. |
9 |
|
|
10 |
|
2. ./configure --enable-coverage defined SUPPORT_GCOV in config.h, although |
11 |
|
this macro is never tested and has no effect, because the work to support |
12 |
|
coverage involves only compiling and linking options and special targets in |
13 |
|
the Makefile. The comment in config.h implied that defining the macro would |
14 |
|
enable coverage support, which is totally false. There was also support for |
15 |
|
setting this macro in the CMake files (my fault, I just copied it from |
16 |
|
configure). SUPPORT_GCOV has now been removed. |
17 |
|
|
18 |
|
3. Make a small performance improvement in strlen16() and strlen32() in |
19 |
|
pcretest. |
20 |
|
|
21 |
|
4. Change 36 for 8.33 left some unreachable statements in pcre_exec.c, |
22 |
|
detected by the Solaris compiler (gcc doesn't seem to be able to diagnose |
23 |
|
these cases). There was also one in pcretest.c. |
24 |
|
|
25 |
|
5. Cleaned up a "may be uninitialized" compiler warning in pcre_exec.c. |
26 |
|
|
27 |
|
6. In UTF mode, the code for checking whether a group could match an empty |
28 |
|
string (which is used for indefinitely repeated groups to allow for |
29 |
|
breaking an infinite loop) was broken when the group contained a repeated |
30 |
|
negated single-character class with a character that occupied more than one |
31 |
|
data item and had a minimum repetition of zero (for example, [^\x{100}]* in |
32 |
|
UTF-8 mode). The effect was undefined: the group might or might not be |
33 |
|
deemed as matching an empty string, or the program might have crashed. |
34 |
|
|
35 |
|
7. The code for checking whether a group could match an empty string was not |
36 |
|
recognizing that \h, \H, \v, \V, and \R must match a character. |
37 |
|
|
38 |
|
8. Implemented PCRE_INFO_MATCH_EMPTY, which yields 1 if the pattern can match |
39 |
|
an empty string. If it can, pcretest shows this in its information output. |
40 |
|
|
41 |
|
9. Fixed two related bugs that applied to Unicode extended grapheme clusters |
42 |
|
that were repeated with a maximizing qualifier (e.g. \X* or \X{2,5}) when |
43 |
|
matched by pcre_exec() without using JIT: |
44 |
|
|
45 |
|
(a) If the rest of the pattern did not match after a maximal run of |
46 |
|
grapheme clusters, the code for backing up to try with fewer of them |
47 |
|
did not always back up over a full grapheme when characters that do not |
48 |
|
have the modifier quality were involved, e.g. Hangul syllables. |
49 |
|
|
50 |
|
(b) If the match point in a subject started with modifier character, and |
51 |
|
there was no match, the code could incorrectly back up beyond the match |
52 |
|
point, and potentially beyond the first character in the subject, |
53 |
|
leading to a segfault or an incorrect match result. |
54 |
|
|
55 |
|
10. A conditional group with an assertion condition could lead to PCRE |
56 |
|
recording an incorrect first data item for a match if no other first data |
57 |
|
item was recorded. For example, the pattern (?(?=ab)ab) recorded "a" as a |
58 |
|
first data item, and therefore matched "ca" after "c" instead of at the |
59 |
|
start. |
60 |
|
|
61 |
|
11. Change 40 for 8.33 (allowing pcregrep to find empty strings) showed up a |
62 |
|
bug that caused the command "echo a | ./pcregrep -M '|a'" to loop. |
63 |
|
|
64 |
|
12. The source of pcregrep now includes z/OS-specific code so that it can be |
65 |
|
compiled for z/OS as part of the special z/OS distribution. |
66 |
|
|
67 |
|
13. Added the -T and -TM options to pcretest. |
68 |
|
|
69 |
|
14. The code in pcre_compile.c for creating the table of named capturing groups |
70 |
|
has been refactored. Instead of creating the table dynamically during the |
71 |
|
actual compiling pass, the information is remembered during the pre-compile |
72 |
|
pass (on the stack unless there are more than 20 named groups, in which |
73 |
|
case malloc() is used) and the whole table is created before the actual |
74 |
|
compile happens. This has simplified the code (it is now nearly 150 lines |
75 |
|
shorter) and prepared the way for better handling of references to groups |
76 |
|
with duplicate names. |
77 |
|
|
78 |
|
15. A back reference to a named subpattern when there is more than one of the |
79 |
|
same name now checks them in the order in which they appear in the pattern. |
80 |
|
The first one that is set is used for the reference. Previously only the |
81 |
|
first one was inspected. This change makes PCRE more compatible with Perl. |
82 |
|
|
83 |
|
16. Unicode character properties were updated from Unicode 6.3.0. |
84 |
|
|
85 |
|
17. The compile-time code for auto-possessification has been refactored, based |
86 |
|
on a patch by Zoltan Herczeg. It now happens after instead of during |
87 |
|
compilation. The code is cleaner, and more cases are handled. The option |
88 |
|
PCRE_NO_AUTO_POSSESSIFY is added for testing purposes, and the -O and /O |
89 |
|
options in pcretest are provided to set it. It can also be set by |
90 |
|
(*NO_AUTO_POSSESS) at the start of a pattern. |
91 |
|
|
92 |
|
18. The character VT has been added to the set of characters that match \s and |
93 |
|
are generally treated as white space, following this same change in Perl |
94 |
|
5.18. There is now no difference between "Perl space" and "POSIX space". |
95 |
|
|
96 |
|
19. The code for checking named groups as conditions, either for being set or |
97 |
|
for being recursed, has been refactored (this is related to 14 and 15 |
98 |
|
above). Processing unduplicated named groups should now be as fast at |
99 |
|
numerical groups, and processing duplicated groups should be faster than |
100 |
|
before. |
101 |
|
|
102 |
|
20. Two patches to the CMake build system, by Alexander Barkov: |
103 |
|
|
104 |
|
(1) Replace the "source" command by "." in CMakeLists.txt because |
105 |
|
"source" is a bash-ism. |
106 |
|
|
107 |
|
(2) Add missing HAVE_STDINT_H and HAVE_INTTYPES_H to config-cmake.h.in; |
108 |
|
without these the CMake build does not work on Solaris. |
109 |
|
|
110 |
|
21. Perl has changed its handling of \8 and \9. If there is no previously |
111 |
|
encountered capturing group of those numbers, they are treated as the |
112 |
|
literal characters 8 and 9 instead of a binary zero followed by the |
113 |
|
literals. PCRE now does the same. |
114 |
|
|
115 |
|
22. Following Perl, added \o{} to specify codepoints in octal, making it |
116 |
|
possible to specify values greater than 0777 and also making them |
117 |
|
unambiguous. |
118 |
|
|
119 |
|
23. Perl now gives an error for missing closing braces after \x{... instead of |
120 |
|
treating the string as literal. PCRE now does the same. |
121 |
|
|
122 |
|
24. RunTest used to grumble if an inappropriate test was selected explicitly, |
123 |
|
but just skip it when running all tests. This make it awkward to run ranges |
124 |
|
of tests when one of them was inappropriate. Now it just skips any |
125 |
|
inappropriate tests, as it always did when running all tests. |
126 |
|
|
127 |
|
25. If PCRE_AUTO_CALLOUT and PCRE_UCP were set for a pattern that contained |
128 |
|
character types such as \d or \w, too many callouts were inserted, and the |
129 |
|
data that they returned was rubbish. |
130 |
|
|
131 |
|
26. In UCP mode, \s was not matching two of the characters that Perl matches, |
132 |
|
namely NEL (U+0085) and MONGOLIAN VOWEL SEPARATOR (U+180E), though they |
133 |
|
were matched by \h. The code has now been refactored so that the lists of |
134 |
|
the horizontal and vertical whitespace characters used for \h and \v (which |
135 |
|
are defined only in one place) are now also used for \s. |
136 |
|
|
137 |
|
27. Add JIT support for the 64 bit TileGX architecture. |
138 |
|
Patch by Jiong Wang (Tilera Corporation). |
139 |
|
|
140 |
|
28. Possessive quantifiers for classes (both explicit and automatically |
141 |
|
generated) now use special opcodes instead of wrapping in ONCE brackets. |
142 |
|
|
143 |
|
29. Whereas an item such as A{4}+ ignored the possessivenes of the quantifier |
144 |
|
(because it's meaningless), this was not happening when PCRE_CASELESS was |
145 |
|
set. Not wrong, but inefficient. |
146 |
|
|
147 |
|
30. Updated perltest.pl to add /u (force Unicode mode) when /W (use Unicode |
148 |
|
properties for \w, \d, etc) is present in a test regex. Otherwise if the |
149 |
|
test contains no characters greater than 255, Perl doesn't realise it |
150 |
|
should be using Unicode semantics. |
151 |
|
|
152 |
|
31. Upgraded the handling of the POSIX classes [:graph:], [:print:], and |
153 |
|
[:punct:] when PCRE_UCP is set so as to include the same characters as Perl |
154 |
|
does in Unicode mode. |
155 |
|
|
156 |
|
32. Added the "forbid" facility to pcretest so that putting tests into the |
157 |
|
wrong test files can sometimes be quickly detected. |
158 |
|
|
159 |
|
33. There is now a limit (default 250) on the depth of nesting of parentheses. |
160 |
|
This limit is imposed to control the amount of system stack used at compile |
161 |
|
time. It can be changed at build time by --with-parens-nest-limit=xxx or |
162 |
|
the equivalent in CMake. |
163 |
|
|
164 |
|
34. Character classes such as [A-\d] or [a-[:digit:]] now cause compile-time |
165 |
|
errors. Perl warns for these when in warning mode, but PCRE has no facility |
166 |
|
for giving warnings. |
167 |
|
|
168 |
|
35. Change 34 for 8.13 allowed quantifiers on assertions, because Perl does. |
169 |
|
However, this was not working for (?!) because it is optimized to (*FAIL), |
170 |
|
for which PCRE does not allow quantifiers. The optimization is now disabled |
171 |
|
when a quantifier follows (?!). I can't see any use for this, but it makes |
172 |
|
things uniform. |
173 |
|
|
174 |
|
36. Perl no longer allows group names to start with digits, so I have made this |
175 |
|
change also in PCRE. It simplifies the code a bit. |
176 |
|
|
177 |
|
|
178 |
|
Version 8.33 28-May-2013 |
179 |
|
------------------------ |
180 |
|
|
181 |
|
1. Added 'U' to some constants that are compared to unsigned integers, to |
182 |
|
avoid compiler signed/unsigned warnings. Added (int) casts to unsigned |
183 |
|
variables that are added to signed variables, to ensure the result is |
184 |
|
signed and can be negated. |
185 |
|
|
186 |
|
2. Applied patch by Daniel Richard G for quashing MSVC warnings to the |
187 |
|
CMake config files. |
188 |
|
|
189 |
|
3. Revise the creation of config.h.generic so that all boolean macros are |
190 |
|
#undefined, whereas non-boolean macros are #ifndef/#endif-ed. This makes |
191 |
|
overriding via -D on the command line possible. |
192 |
|
|
193 |
|
4. Changing the definition of the variable "op" in pcre_exec.c from pcre_uchar |
194 |
|
to unsigned int is reported to make a quite noticeable speed difference in |
195 |
|
a specific Windows environment. Testing on Linux did also appear to show |
196 |
|
some benefit (and it is clearly not harmful). Also fixed the definition of |
197 |
|
Xop which should be unsigned. |
198 |
|
|
199 |
|
5. Related to (4), changing the definition of the intermediate variable cc |
200 |
|
in repeated character loops from pcre_uchar to pcre_uint32 also gave speed |
201 |
|
improvements. |
202 |
|
|
203 |
|
6. Fix forward search in JIT when link size is 3 or greater. Also removed some |
204 |
|
unnecessary spaces. |
205 |
|
|
206 |
|
7. Adjust autogen.sh and configure.ac to lose warnings given by automake 1.12 |
207 |
|
and later. |
208 |
|
|
209 |
|
8. Fix two buffer over read issues in 16 and 32 bit modes. Affects JIT only. |
210 |
|
|
211 |
|
9. Optimizing fast_forward_start_bits in JIT. |
212 |
|
|
213 |
|
10. Adding support for callouts in JIT, and fixing some issues revealed |
214 |
|
during this work. Namely: |
215 |
|
|
216 |
|
(a) Unoptimized capturing brackets incorrectly reset on backtrack. |
217 |
|
|
218 |
|
(b) Minimum length was not checked before the matching is started. |
219 |
|
|
220 |
|
11. The value of capture_last that is passed to callouts was incorrect in some |
221 |
|
cases when there was a capture on one path that was subsequently abandoned |
222 |
|
after a backtrack. Also, the capture_last value is now reset after a |
223 |
|
recursion, since all captures are also reset in this case. |
224 |
|
|
225 |
|
12. The interpreter no longer returns the "too many substrings" error in the |
226 |
|
case when an overflowing capture is in a branch that is subsequently |
227 |
|
abandoned after a backtrack. |
228 |
|
|
229 |
|
13. In the pathological case when an offset vector of size 2 is used, pcretest |
230 |
|
now prints out the matched string after a yield of 0 or 1. |
231 |
|
|
232 |
|
14. Inlining subpatterns in recursions, when certain conditions are fulfilled. |
233 |
|
Only supported by the JIT compiler at the moment. |
234 |
|
|
235 |
|
15. JIT compiler now supports 32 bit Macs thanks to Lawrence Velazquez. |
236 |
|
|
237 |
|
16. Partial matches now set offsets[2] to the "bumpalong" value, that is, the |
238 |
|
offset of the starting point of the matching process, provided the offsets |
239 |
|
vector is large enough. |
240 |
|
|
241 |
|
17. The \A escape now records a lookbehind value of 1, though its execution |
242 |
|
does not actually inspect the previous character. This is to ensure that, |
243 |
|
in partial multi-segment matching, at least one character from the old |
244 |
|
segment is retained when a new segment is processed. Otherwise, if there |
245 |
|
are no lookbehinds in the pattern, \A might match incorrectly at the start |
246 |
|
of a new segment. |
247 |
|
|
248 |
|
18. Added some #ifdef __VMS code into pcretest.c to help VMS implementations. |
249 |
|
|
250 |
|
19. Redefined some pcre_uchar variables in pcre_exec.c as pcre_uint32; this |
251 |
|
gives some modest performance improvement in 8-bit mode. |
252 |
|
|
253 |
|
20. Added the PCRE-specific property \p{Xuc} for matching characters that can |
254 |
|
be expressed in certain programming languages using Universal Character |
255 |
|
Names. |
256 |
|
|
257 |
|
21. Unicode validation has been updated in the light of Unicode Corrigendum #9, |
258 |
|
which points out that "non characters" are not "characters that may not |
259 |
|
appear in Unicode strings" but rather "characters that are reserved for |
260 |
|
internal use and have only local meaning". |
261 |
|
|
262 |
|
22. When a pattern was compiled with automatic callouts (PCRE_AUTO_CALLOUT) and |
263 |
|
there was a conditional group that depended on an assertion, if the |
264 |
|
assertion was false, the callout that immediately followed the alternation |
265 |
|
in the condition was skipped when pcre_exec() was used for matching. |
266 |
|
|
267 |
|
23. Allow an explicit callout to be inserted before an assertion that is the |
268 |
|
condition for a conditional group, for compatibility with automatic |
269 |
|
callouts, which always insert a callout at this point. |
270 |
|
|
271 |
|
24. In 8.31, (*COMMIT) was confined to within a recursive subpattern. Perl also |
272 |
|
confines (*SKIP) and (*PRUNE) in the same way, and this has now been done. |
273 |
|
|
274 |
|
25. (*PRUNE) is now supported by the JIT compiler. |
275 |
|
|
276 |
|
26. Fix infinite loop when /(?<=(*SKIP)ac)a/ is matched against aa. |
277 |
|
|
278 |
|
27. Fix the case where there are two or more SKIPs with arguments that may be |
279 |
|
ignored. |
280 |
|
|
281 |
|
28. (*SKIP) is now supported by the JIT compiler. |
282 |
|
|
283 |
|
29. (*THEN) is now supported by the JIT compiler. |
284 |
|
|
285 |
|
30. Update RunTest with additional test selector options. |
286 |
|
|
287 |
|
31. The way PCRE handles backtracking verbs has been changed in two ways. |
288 |
|
|
289 |
|
(1) Previously, in something like (*COMMIT)(*SKIP), COMMIT would override |
290 |
|
SKIP. Now, PCRE acts on whichever backtracking verb is reached first by |
291 |
|
backtracking. In some cases this makes it more Perl-compatible, but Perl's |
292 |
|
rather obscure rules do not always do the same thing. |
293 |
|
|
294 |
|
(2) Previously, backtracking verbs were confined within assertions. This is |
295 |
|
no longer the case for positive assertions, except for (*ACCEPT). Again, |
296 |
|
this sometimes improves Perl compatibility, and sometimes does not. |
297 |
|
|
298 |
|
32. A number of tests that were in test 2 because Perl did things differently |
299 |
|
have been moved to test 1, because either Perl or PCRE has changed, and |
300 |
|
these tests are now compatible. |
301 |
|
|
302 |
|
32. Backtracking control verbs are now handled in the same way in JIT and |
303 |
|
interpreter. |
304 |
|
|
305 |
|
33. An opening parenthesis in a MARK/PRUNE/SKIP/THEN name in a pattern that |
306 |
|
contained a forward subroutine reference caused a compile error. |
307 |
|
|
308 |
|
34. Auto-detect and optimize limited repetitions in JIT. |
309 |
|
|
310 |
|
35. Implement PCRE_NEVER_UTF to lock out the use of UTF, in particular, |
311 |
|
blocking (*UTF) etc. |
312 |
|
|
313 |
|
36. In the interpreter, maximizing pattern repetitions for characters and |
314 |
|
character types now use tail recursion, which reduces stack usage. |
315 |
|
|
316 |
|
37. The value of the max lookbehind was not correctly preserved if a compiled |
317 |
|
and saved regex was reloaded on a host of different endianness. |
318 |
|
|
319 |
|
38. Implemented (*LIMIT_MATCH) and (*LIMIT_RECURSION). As part of the extension |
320 |
|
of the compiled pattern block, expand the flags field from 16 to 32 bits |
321 |
|
because it was almost full. |
322 |
|
|
323 |
|
39. Try madvise first before posix_madvise. |
324 |
|
|
325 |
|
40. Change 7 for PCRE 7.9 made it impossible for pcregrep to find empty lines |
326 |
|
with a pattern such as ^$. It has taken 4 years for anybody to notice! The |
327 |
|
original change locked out all matches of empty strings. This has been |
328 |
|
changed so that one match of an empty string per line is recognized. |
329 |
|
Subsequent searches on the same line (for colouring or for --only-matching, |
330 |
|
for example) do not recognize empty strings. |
331 |
|
|
332 |
|
41. Applied a user patch to fix a number of spelling mistakes in comments. |
333 |
|
|
334 |
|
42. Data lines longer than 65536 caused pcretest to crash. |
335 |
|
|
336 |
|
43. Clarified the data type for length and startoffset arguments for pcre_exec |
337 |
|
and pcre_dfa_exec in the function-specific man pages, where they were |
338 |
|
explicitly stated to be in bytes, never having been updated. I also added |
339 |
|
some clarification to the pcreapi man page. |
340 |
|
|
341 |
|
44. A call to pcre_dfa_exec() with an output vector size less than 2 caused |
342 |
|
a segmentation fault. |
343 |
|
|
344 |
|
|
345 |
|
Version 8.32 30-November-2012 |
346 |
|
----------------------------- |
347 |
|
|
348 |
|
1. Improved JIT compiler optimizations for first character search and single |
349 |
|
character iterators. |
350 |
|
|
351 |
|
2. Supporting IBM XL C compilers for PPC architectures in the JIT compiler. |
352 |
|
Patch by Daniel Richard G. |
353 |
|
|
354 |
|
3. Single character iterator optimizations in the JIT compiler. |
355 |
|
|
356 |
|
4. Improved JIT compiler optimizations for character ranges. |
357 |
|
|
358 |
|
5. Rename the "leave" variable names to "quit" to improve WinCE compatibility. |
359 |
|
Reported by Giuseppe D'Angelo. |
360 |
|
|
361 |
|
6. The PCRE_STARTLINE bit, indicating that a match can occur only at the start |
362 |
|
of a line, was being set incorrectly in cases where .* appeared inside |
363 |
|
atomic brackets at the start of a pattern, or where there was a subsequent |
364 |
|
*PRUNE or *SKIP. |
365 |
|
|
366 |
|
7. Improved instruction cache flush for POWER/PowerPC. |
367 |
|
Patch by Daniel Richard G. |
368 |
|
|
369 |
|
8. Fixed a number of issues in pcregrep, making it more compatible with GNU |
370 |
|
grep: |
371 |
|
|
372 |
|
(a) There is now no limit to the number of patterns to be matched. |
373 |
|
|
374 |
|
(b) An error is given if a pattern is too long. |
375 |
|
|
376 |
|
(c) Multiple uses of --exclude, --exclude-dir, --include, and --include-dir |
377 |
|
are now supported. |
378 |
|
|
379 |
|
(d) --exclude-from and --include-from (multiple use) have been added. |
380 |
|
|
381 |
|
(e) Exclusions and inclusions now apply to all files and directories, not |
382 |
|
just to those obtained from scanning a directory recursively. |
383 |
|
|
384 |
|
(f) Multiple uses of -f and --file-list are now supported. |
385 |
|
|
386 |
|
(g) In a Windows environment, the default for -d has been changed from |
387 |
|
"read" (the GNU grep default) to "skip", because otherwise the presence |
388 |
|
of a directory in the file list provokes an error. |
389 |
|
|
390 |
|
(h) The documentation has been revised and clarified in places. |
391 |
|
|
392 |
|
9. Improve the matching speed of capturing brackets. |
393 |
|
|
394 |
|
10. Changed the meaning of \X so that it now matches a Unicode extended |
395 |
|
grapheme cluster. |
396 |
|
|
397 |
|
11. Patch by Daniel Richard G to the autoconf files to add a macro for sorting |
398 |
|
out POSIX threads when JIT support is configured. |
399 |
|
|
400 |
|
12. Added support for PCRE_STUDY_EXTRA_NEEDED. |
401 |
|
|
402 |
|
13. In the POSIX wrapper regcomp() function, setting re_nsub field in the preg |
403 |
|
structure could go wrong in environments where size_t is not the same size |
404 |
|
as int. |
405 |
|
|
406 |
|
14. Applied user-supplied patch to pcrecpp.cc to allow PCRE_NO_UTF8_CHECK to be |
407 |
|
set. |
408 |
|
|
409 |
|
15. The EBCDIC support had decayed; later updates to the code had included |
410 |
|
explicit references to (e.g.) \x0a instead of CHAR_LF. There has been a |
411 |
|
general tidy up of EBCDIC-related issues, and the documentation was also |
412 |
|
not quite right. There is now a test that can be run on ASCII systems to |
413 |
|
check some of the EBCDIC-related things (but is it not a full test). |
414 |
|
|
415 |
|
16. The new PCRE_STUDY_EXTRA_NEEDED option is now used by pcregrep, resulting |
416 |
|
in a small tidy to the code. |
417 |
|
|
418 |
|
17. Fix JIT tests when UTF is disabled and both 8 and 16 bit mode are enabled. |
419 |
|
|
420 |
|
18. If the --only-matching (-o) option in pcregrep is specified multiple |
421 |
|
times, each one causes appropriate output. For example, -o1 -o2 outputs the |
422 |
|
substrings matched by the 1st and 2nd capturing parentheses. A separating |
423 |
|
string can be specified by --om-separator (default empty). |
424 |
|
|
425 |
|
19. Improving the first n character searches. |
426 |
|
|
427 |
|
20. Turn case lists for horizontal and vertical white space into macros so that |
428 |
|
they are defined only once. |
429 |
|
|
430 |
|
21. This set of changes together give more compatible Unicode case-folding |
431 |
|
behaviour for characters that have more than one other case when UCP |
432 |
|
support is available. |
433 |
|
|
434 |
|
(a) The Unicode property table now has offsets into a new table of sets of |
435 |
|
three or more characters that are case-equivalent. The MultiStage2.py |
436 |
|
script that generates these tables (the pcre_ucd.c file) now scans |
437 |
|
CaseFolding.txt instead of UnicodeData.txt for character case |
438 |
|
information. |
439 |
|
|
440 |
|
(b) The code for adding characters or ranges of characters to a character |
441 |
|
class has been abstracted into a generalized function that also handles |
442 |
|
case-independence. In UTF-mode with UCP support, this uses the new data |
443 |
|
to handle characters with more than one other case. |
444 |
|
|
445 |
|
(c) A bug that is fixed as a result of (b) is that codepoints less than 256 |
446 |
|
whose other case is greater than 256 are now correctly matched |
447 |
|
caselessly. Previously, the high codepoint matched the low one, but not |
448 |
|
vice versa. |
449 |
|
|
450 |
|
(d) The processing of \h, \H, \v, and \ in character classes now makes use |
451 |
|
of the new class addition function, using character lists defined as |
452 |
|
macros alongside the case definitions of 20 above. |
453 |
|
|
454 |
|
(e) Caseless back references now work with characters that have more than |
455 |
|
one other case. |
456 |
|
|
457 |
|
(f) General caseless matching of characters with more than one other case |
458 |
|
is supported. |
459 |
|
|
460 |
|
22. Unicode character properties were updated from Unicode 6.2.0 |
461 |
|
|
462 |
|
23. Improved CMake support under Windows. Patch by Daniel Richard G. |
463 |
|
|
464 |
|
24. Add support for 32-bit character strings, and UTF-32 |
465 |
|
|
466 |
|
25. Major JIT compiler update (code refactoring and bugfixing). |
467 |
|
Experimental Sparc 32 support is added. |
468 |
|
|
469 |
|
26. Applied a modified version of Daniel Richard G's patch to create |
470 |
|
pcre.h.generic and config.h.generic by "make" instead of in the |
471 |
|
PrepareRelease script. |
472 |
|
|
473 |
|
27. Added a definition for CHAR_NULL (helpful for the z/OS port), and use it in |
474 |
|
pcre_compile.c when checking for a zero character. |
475 |
|
|
476 |
|
28. Introducing a native interface for JIT. Through this interface, the compiled |
477 |
|
machine code can be directly executed. The purpose of this interface is to |
478 |
|
provide fast pattern matching, so several sanity checks are not performed. |
479 |
|
However, feature tests are still performed. The new interface provides |
480 |
|
1.4x speedup compared to the old one. |
481 |
|
|
482 |
|
29. If pcre_exec() or pcre_dfa_exec() was called with a negative value for |
483 |
|
the subject string length, the error given was PCRE_ERROR_BADOFFSET, which |
484 |
|
was confusing. There is now a new error PCRE_ERROR_BADLENGTH for this case. |
485 |
|
|
486 |
|
30. In 8-bit UTF-8 mode, pcretest failed to give an error for data codepoints |
487 |
|
greater than 0x7fffffff (which cannot be represented in UTF-8, even under |
488 |
|
the "old" RFC 2279). Instead, it ended up passing a negative length to |
489 |
|
pcre_exec(). |
490 |
|
|
491 |
|
31. Add support for GCC's visibility feature to hide internal functions. |
492 |
|
|
493 |
|
32. Running "pcretest -C pcre8" or "pcretest -C pcre16" gave a spurious error |
494 |
|
"unknown -C option" after outputting 0 or 1. |
495 |
|
|
496 |
|
33. There is now support for generating a code coverage report for the test |
497 |
|
suite in environments where gcc is the compiler and lcov is installed. This |
498 |
|
is mainly for the benefit of the developers. |
499 |
|
|
500 |
|
34. If PCRE is built with --enable-valgrind, certain memory regions are marked |
501 |
|
unaddressable using valgrind annotations, allowing valgrind to detect |
502 |
|
invalid memory accesses. This is mainly for the benefit of the developers. |
503 |
|
|
504 |
|
25. (*UTF) can now be used to start a pattern in any of the three libraries. |
505 |
|
|
506 |
|
26. Give configure error if --enable-cpp but no C++ compiler found. |
507 |
|
|
508 |
|
|
509 |
|
Version 8.31 06-July-2012 |
510 |
|
------------------------- |
511 |
|
|
512 |
|
1. Fixing a wrong JIT test case and some compiler warnings. |
513 |
|
|
514 |
|
2. Removed a bashism from the RunTest script. |
515 |
|
|
516 |
|
3. Add a cast to pcre_exec.c to fix the warning "unary minus operator applied |
517 |
|
to unsigned type, result still unsigned" that was given by an MS compiler |
518 |
|
on encountering the code "-sizeof(xxx)". |
519 |
|
|
520 |
|
4. Partial matching support is added to the JIT compiler. |
521 |
|
|
522 |
|
5. Fixed several bugs concerned with partial matching of items that consist |
523 |
|
of more than one character: |
524 |
|
|
525 |
|
(a) /^(..)\1/ did not partially match "aba" because checking references was |
526 |
|
done on an "all or nothing" basis. This also applied to repeated |
527 |
|
references. |
528 |
|
|
529 |
|
(b) \R did not give a hard partial match if \r was found at the end of the |
530 |
|
subject. |
531 |
|
|
532 |
|
(c) \X did not give a hard partial match after matching one or more |
533 |
|
characters at the end of the subject. |
534 |
|
|
535 |
|
(d) When newline was set to CRLF, a pattern such as /a$/ did not recognize |
536 |
|
a partial match for the string "\r". |
537 |
|
|
538 |
|
(e) When newline was set to CRLF, the metacharacter "." did not recognize |
539 |
|
a partial match for a CR character at the end of the subject string. |
540 |
|
|
541 |
|
6. If JIT is requested using /S++ or -s++ (instead of just /S+ or -s+) when |
542 |
|
running pcretest, the text "(JIT)" added to the output whenever JIT is |
543 |
|
actually used to run the match. |
544 |
|
|
545 |
|
7. Individual JIT compile options can be set in pcretest by following -s+[+] |
546 |
|
or /S+[+] with a digit between 1 and 7. |
547 |
|
|
548 |
|
8. OP_NOT now supports any UTF character not just single-byte ones. |
549 |
|
|
550 |
|
9. (*MARK) control verb is now supported by the JIT compiler. |
551 |
|
|
552 |
|
10. The command "./RunTest list" lists the available tests without actually |
553 |
|
running any of them. (Because I keep forgetting what they all are.) |
554 |
|
|
555 |
|
11. Add PCRE_INFO_MAXLOOKBEHIND. |
556 |
|
|
557 |
|
12. Applied a (slightly modified) user-supplied patch that improves performance |
558 |
|
when the heap is used for recursion (compiled with --disable-stack-for- |
559 |
|
recursion). Instead of malloc and free for each heap frame each time a |
560 |
|
logical recursion happens, frames are retained on a chain and re-used where |
561 |
|
possible. This sometimes gives as much as 30% improvement. |
562 |
|
|
563 |
|
13. As documented, (*COMMIT) is now confined to within a recursive subpattern |
564 |
|
call. |
565 |
|
|
566 |
|
14. As documented, (*COMMIT) is now confined to within a positive assertion. |
567 |
|
|
568 |
|
15. It is now possible to link pcretest with libedit as an alternative to |
569 |
|
libreadline. |
570 |
|
|
571 |
|
16. (*COMMIT) control verb is now supported by the JIT compiler. |
572 |
|
|
573 |
|
17. The Unicode data tables have been updated to Unicode 6.1.0. |
574 |
|
|
575 |
|
18. Added --file-list option to pcregrep. |
576 |
|
|
577 |
|
19. Added binary file support to pcregrep, including the -a, --binary-files, |
578 |
|
-I, and --text options. |
579 |
|
|
580 |
|
20. The madvise function is renamed for posix_madvise for QNX compatibility |
581 |
|
reasons. Fixed by Giuseppe D'Angelo. |
582 |
|
|
583 |
|
21. Fixed a bug for backward assertions with REVERSE 0 in the JIT compiler. |
584 |
|
|
585 |
|
22. Changed the option for creating symbolic links for 16-bit man pages from |
586 |
|
-s to -sf so that re-installing does not cause issues. |
587 |
|
|
588 |
|
23. Support PCRE_NO_START_OPTIMIZE in JIT as (*MARK) support requires it. |
589 |
|
|
590 |
|
24. Fixed a very old bug in pcretest that caused errors with restarted DFA |
591 |
|
matches in certain environments (the workspace was not being correctly |
592 |
|
retained). Also added to pcre_dfa_exec() a simple plausibility check on |
593 |
|
some of the workspace data at the beginning of a restart. |
594 |
|
|
595 |
|
25. \s*\R was auto-possessifying the \s* when it should not, whereas \S*\R |
596 |
|
was not doing so when it should - probably a typo introduced by SVN 528 |
597 |
|
(change 8.10/14). |
598 |
|
|
599 |
|
26. When PCRE_UCP was not set, \w+\x{c4} was incorrectly auto-possessifying the |
600 |
|
\w+ when the character tables indicated that \x{c4} was a word character. |
601 |
|
There were several related cases, all because the tests for doing a table |
602 |
|
lookup were testing for characters less than 127 instead of 255. |
603 |
|
|
604 |
|
27. If a pattern contains capturing parentheses that are not used in a match, |
605 |
|
their slots in the ovector are set to -1. For those that are higher than |
606 |
|
any matched groups, this happens at the end of processing. In the case when |
607 |
|
there were back references that the ovector was too small to contain |
608 |
|
(causing temporary malloc'd memory to be used during matching), and the |
609 |
|
highest capturing number was not used, memory off the end of the ovector |
610 |
|
was incorrectly being set to -1. (It was using the size of the temporary |
611 |
|
memory instead of the true size.) |
612 |
|
|
613 |
|
28. To catch bugs like 27 using valgrind, when pcretest is asked to specify an |
614 |
|
ovector size, it uses memory at the end of the block that it has got. |
615 |
|
|
616 |
|
29. Check for an overlong MARK name and give an error at compile time. The |
617 |
|
limit is 255 for the 8-bit library and 65535 for the 16-bit library. |
618 |
|
|
619 |
|
30. JIT compiler update. |
620 |
|
|
621 |
|
31. JIT is now supported on jailbroken iOS devices. Thanks for Ruiger |
622 |
|
Rill for the patch. |
623 |
|
|
624 |
|
32. Put spaces around SLJIT_PRINT_D in the JIT compiler. Required by CXX11. |
625 |
|
|
626 |
|
33. Variable renamings in the PCRE-JIT compiler. No functionality change. |
627 |
|
|
628 |
|
34. Fixed typos in pcregrep: in two places there was SUPPORT_LIBZ2 instead of |
629 |
|
SUPPORT_LIBBZ2. This caused a build problem when bzip2 but not gzip (zlib) |
630 |
|
was enabled. |
631 |
|
|
632 |
|
35. Improve JIT code generation for greedy plus quantifier. |
633 |
|
|
634 |
|
36. When /((?:a?)*)*c/ or /((?>a?)*)*c/ was matched against "aac", it set group |
635 |
|
1 to "aa" instead of to an empty string. The bug affected repeated groups |
636 |
|
that could potentially match an empty string. |
637 |
|
|
638 |
|
37. Optimizing single character iterators in JIT. |
639 |
|
|
640 |
|
38. Wide characters specified with \uxxxx in JavaScript mode are now subject to |
641 |
|
the same checks as \x{...} characters in non-JavaScript mode. Specifically, |
642 |
|
codepoints that are too big for the mode are faulted, and in a UTF mode, |
643 |
|
disallowed codepoints are also faulted. |
644 |
|
|
645 |
|
39. If PCRE was compiled with UTF support, in three places in the DFA |
646 |
|
matcher there was code that should only have been obeyed in UTF mode, but |
647 |
|
was being obeyed unconditionally. In 8-bit mode this could cause incorrect |
648 |
|
processing when bytes with values greater than 127 were present. In 16-bit |
649 |
|
mode the bug would be provoked by values in the range 0xfc00 to 0xdc00. In |
650 |
|
both cases the values are those that cannot be the first data item in a UTF |
651 |
|
character. The three items that might have provoked this were recursions, |
652 |
|
possessively repeated groups, and atomic groups. |
653 |
|
|
654 |
|
40. Ensure that libpcre is explicitly listed in the link commands for pcretest |
655 |
|
and pcregrep, because some OS require shared objects to be explicitly |
656 |
|
passed to ld, causing the link step to fail if they are not. |
657 |
|
|
658 |
|
41. There were two incorrect #ifdefs in pcre_study.c, meaning that, in 16-bit |
659 |
|
mode, patterns that started with \h* or \R* might be incorrectly matched. |
660 |
|
|
661 |
|
|
662 |
|
Version 8.30 04-February-2012 |
663 |
|
----------------------------- |
664 |
|
|
665 |
|
1. Renamed "isnumber" as "is_a_number" because in some Mac environments this |
666 |
|
name is defined in ctype.h. |
667 |
|
|
668 |
|
2. Fixed a bug in fixed-length calculation for lookbehinds that would show up |
669 |
|
only in quite long subpatterns. |
670 |
|
|
671 |
|
3. Removed the function pcre_info(), which has been obsolete and deprecated |
672 |
|
since it was replaced by pcre_fullinfo() in February 2000. |
673 |
|
|
674 |
|
4. For a non-anchored pattern, if (*SKIP) was given with a name that did not |
675 |
|
match a (*MARK), and the match failed at the start of the subject, a |
676 |
|
reference to memory before the start of the subject could occur. This bug |
677 |
|
was introduced by fix 17 of release 8.21. |
678 |
|
|
679 |
|
5. A reference to an unset group with zero minimum repetition was giving |
680 |
|
totally wrong answers (in non-JavaScript-compatibility mode). For example, |
681 |
|
/(another)?(\1?)test/ matched against "hello world test". This bug was |
682 |
|
introduced in release 8.13. |
683 |
|
|
684 |
|
6. Add support for 16-bit character strings (a large amount of work involving |
685 |
|
many changes and refactorings). |
686 |
|
|
687 |
|
7. RunGrepTest failed on msys because \r\n was replaced by whitespace when the |
688 |
|
command "pattern=`printf 'xxx\r\njkl'`" was run. The pattern is now taken |
689 |
|
from a file. |
690 |
|
|
691 |
|
8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size |
692 |
|
rounding is not applied in this particular case). |
693 |
|
|
694 |
|
9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected |
695 |
|
if they appear, or are escaped, in patterns. |
696 |
|
|
697 |
|
10. Get rid of a number of -Wunused-but-set-variable warnings. |
698 |
|
|
699 |
|
11. The pattern /(?=(*:x))(q|)/ matches an empty string, and returns the mark |
700 |
|
"x". The similar pattern /(?=(*:x))((*:y)q|)/ did not return a mark at all. |
701 |
|
Oddly, Perl behaves the same way. PCRE has been fixed so that this pattern |
702 |
|
also returns the mark "x". This bug applied to capturing parentheses, |
703 |
|
non-capturing parentheses, and atomic parentheses. It also applied to some |
704 |
|
assertions. |
705 |
|
|
706 |
|
12. Stephen Kelly's patch to CMakeLists.txt allows it to parse the version |
707 |
|
information out of configure.ac instead of relying on pcre.h.generic, which |
708 |
|
is not stored in the repository. |
709 |
|
|
710 |
|
13. Applied Dmitry V. Levin's patch for a more portable method for linking with |
711 |
|
-lreadline. |
712 |
|
|
713 |
|
14. ZH added PCRE_CONFIG_JITTARGET; added its output to pcretest -C. |
714 |
|
|
715 |
|
15. Applied Graycode's patch to put the top-level frame on the stack rather |
716 |
|
than the heap when not using the stack for recursion. This gives a |
717 |
|
performance improvement in many cases when recursion is not deep. |
718 |
|
|
719 |
|
16. Experimental code added to "pcretest -C" to output the stack frame size. |
720 |
|
|
721 |
|
|
722 |
|
Version 8.21 12-Dec-2011 |
723 |
|
------------------------ |
724 |
|
|
725 |
|
1. Updating the JIT compiler. |
726 |
|
|
727 |
|
2. JIT compiler now supports OP_NCREF, OP_RREF and OP_NRREF. New test cases |
728 |
|
are added as well. |
729 |
|
|
730 |
|
3. Fix cache-flush issue on PowerPC (It is still an experimental JIT port). |
731 |
|
PCRE_EXTRA_TABLES is not suported by JIT, and should be checked before |
732 |
|
calling _pcre_jit_exec. Some extra comments are added. |
733 |
|
|
734 |
|
4. (*MARK) settings inside atomic groups that do not contain any capturing |
735 |
|
parentheses, for example, (?>a(*:m)), were not being passed out. This bug |
736 |
|
was introduced by change 18 for 8.20. |
737 |
|
|
738 |
|
5. Supporting of \x, \U and \u in JavaScript compatibility mode based on the |
739 |
|
ECMA-262 standard. |
740 |
|
|
741 |
|
6. Lookbehinds such as (?<=a{2}b) that contained a fixed repetition were |
742 |
|
erroneously being rejected as "not fixed length" if PCRE_CASELESS was set. |
743 |
|
This bug was probably introduced by change 9 of 8.13. |
744 |
|
|
745 |
|
7. While fixing 6 above, I noticed that a number of other items were being |
746 |
|
incorrectly rejected as "not fixed length". This arose partly because newer |
747 |
|
opcodes had not been added to the fixed-length checking code. I have (a) |
748 |
|
corrected the bug and added tests for these items, and (b) arranged for an |
749 |
|
error to occur if an unknown opcode is encountered while checking for fixed |
750 |
|
length instead of just assuming "not fixed length". The items that were |
751 |
|
rejected were: (*ACCEPT), (*COMMIT), (*FAIL), (*MARK), (*PRUNE), (*SKIP), |
752 |
|
(*THEN), \h, \H, \v, \V, and single character negative classes with fixed |
753 |
|
repetitions, e.g. [^a]{3}, with and without PCRE_CASELESS. |
754 |
|
|
755 |
|
8. A possessively repeated conditional subpattern such as (?(?=c)c|d)++ was |
756 |
|
being incorrectly compiled and would have given unpredicatble results. |
757 |
|
|
758 |
|
9. A possessively repeated subpattern with minimum repeat count greater than |
759 |
|
one behaved incorrectly. For example, (A){2,}+ behaved as if it was |
760 |
|
(A)(A)++ which meant that, after a subsequent mismatch, backtracking into |
761 |
|
the first (A) could occur when it should not. |
762 |
|
|
763 |
|
10. Add a cast and remove a redundant test from the code. |
764 |
|
|
765 |
|
11. JIT should use pcre_malloc/pcre_free for allocation. |
766 |
|
|
767 |
|
12. Updated pcre-config so that it no longer shows -L/usr/lib, which seems |
768 |
|
best practice nowadays, and helps with cross-compiling. (If the exec_prefix |
769 |
|
is anything other than /usr, -L is still shown). |
770 |
|
|
771 |
|
13. In non-UTF-8 mode, \C is now supported in lookbehinds and DFA matching. |
772 |
|
|
773 |
|
14. Perl does not support \N without a following name in a [] class; PCRE now |
774 |
|
also gives an error. |
775 |
|
|
776 |
|
15. If a forward reference was repeated with an upper limit of around 2000, |
777 |
|
it caused the error "internal error: overran compiling workspace". The |
778 |
|
maximum number of forward references (including repeats) was limited by the |
779 |
|
internal workspace, and dependent on the LINK_SIZE. The code has been |
780 |
|
rewritten so that the workspace expands (via pcre_malloc) if necessary, and |
781 |
|
the default depends on LINK_SIZE. There is a new upper limit (for safety) |
782 |
|
of around 200,000 forward references. While doing this, I also speeded up |
783 |
|
the filling in of repeated forward references. |
784 |
|
|
785 |
|
16. A repeated forward reference in a pattern such as (a)(?2){2}(.) was |
786 |
|
incorrectly expecting the subject to contain another "a" after the start. |
787 |
|
|
788 |
|
17. When (*SKIP:name) is activated without a corresponding (*MARK:name) earlier |
789 |
|
in the match, the SKIP should be ignored. This was not happening; instead |
790 |
|
the SKIP was being treated as NOMATCH. For patterns such as |
791 |
|
/A(*MARK:A)A+(*SKIP:B)Z|AAC/ this meant that the AAC branch was never |
792 |
|
tested. |
793 |
|
|
794 |
|
18. The behaviour of (*MARK), (*PRUNE), and (*THEN) has been reworked and is |
795 |
|
now much more compatible with Perl, in particular in cases where the result |
796 |
|
is a non-match for a non-anchored pattern. For example, if |
797 |
|
/b(*:m)f|a(*:n)w/ is matched against "abc", the non-match returns the name |
798 |
|
"m", where previously it did not return a name. A side effect of this |
799 |
|
change is that for partial matches, the last encountered mark name is |
800 |
|
returned, as for non matches. A number of tests that were previously not |
801 |
|
Perl-compatible have been moved into the Perl-compatible test files. The |
802 |
|
refactoring has had the pleasing side effect of removing one argument from |
803 |
|
the match() function, thus reducing its stack requirements. |
804 |
|
|
805 |
|
19. If the /S+ option was used in pcretest to study a pattern using JIT, |
806 |
|
subsequent uses of /S (without +) incorrectly behaved like /S+. |
807 |
|
|
808 |
|
21. Retrieve executable code size support for the JIT compiler and fixing |
809 |
|
some warnings. |
810 |
|
|
811 |
|
22. A caseless match of a UTF-8 character whose other case uses fewer bytes did |
812 |
|
not work when the shorter character appeared right at the end of the |
813 |
|
subject string. |
814 |
|
|
815 |
|
23. Added some (int) casts to non-JIT modules to reduce warnings on 64-bit |
816 |
|
systems. |
817 |
|
|
818 |
|
24. Added PCRE_INFO_JITSIZE to pass on the value from (21) above, and also |
819 |
|
output it when the /M option is used in pcretest. |
820 |
|
|
821 |
|
25. The CheckMan script was not being included in the distribution. Also, added |
822 |
|
an explicit "perl" to run Perl scripts from the PrepareRelease script |
823 |
|
because this is reportedly needed in Windows. |
824 |
|
|
825 |
|
26. If study data was being save in a file and studying had not found a set of |
826 |
|
"starts with" bytes for the pattern, the data written to the file (though |
827 |
|
never used) was taken from uninitialized memory and so caused valgrind to |
828 |
|
complain. |
829 |
|
|
830 |
|
27. Updated RunTest.bat as provided by Sheri Pierce. |
831 |
|
|
832 |
|
28. Fixed a possible uninitialized memory bug in pcre_jit_compile.c. |
833 |
|
|
834 |
|
29. Computation of memory usage for the table of capturing group names was |
835 |
|
giving an unnecessarily large value. |
836 |
|
|
837 |
|
|
838 |
|
Version 8.20 21-Oct-2011 |
839 |
|
------------------------ |
840 |
|
|
841 |
|
1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had |
842 |
|
a POSIX class. After further experiments with Perl, which convinced me that |
843 |
|
Perl has bugs and confusions, a closing square bracket is no longer allowed |
844 |
|
in a POSIX name. This bug also affected patterns with classes that started |
845 |
|
with full stops. |
846 |
|
|
847 |
|
2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no |
848 |
|
captured substring, but while checking the failing first alternative, |
849 |
|
substring 1 is temporarily captured. If the output vector supplied to |
850 |
|
pcre_exec() was not big enough for this capture, the yield of the function |
851 |
|
was still zero ("insufficient space for captured substrings"). This cannot |
852 |
|
be totally fixed without adding another stack variable, which seems a lot |
853 |
|
of expense for a edge case. However, I have improved the situation in cases |
854 |
|
such as /(a)(b)x|abc/ matched against "abc", where the return code |
855 |
|
indicates that fewer than the maximum number of slots in the ovector have |
856 |
|
been set. |
857 |
|
|
858 |
|
3. Related to (2) above: when there are more back references in a pattern than |
859 |
|
slots in the output vector, pcre_exec() uses temporary memory during |
860 |
|
matching, and copies in the captures as far as possible afterwards. It was |
861 |
|
using the entire output vector, but this conflicts with the specification |
862 |
|
that only 2/3 is used for passing back captured substrings. Now it uses |
863 |
|
only the first 2/3, for compatibility. This is, of course, another edge |
864 |
|
case. |
865 |
|
|
866 |
|
4. Zoltan Herczeg's just-in-time compiler support has been integrated into the |
867 |
|
main code base, and can be used by building with --enable-jit. When this is |
868 |
|
done, pcregrep automatically uses it unless --disable-pcregrep-jit or the |
869 |
|
runtime --no-jit option is given. |
870 |
|
|
871 |
|
5. When the number of matches in a pcre_dfa_exec() run exactly filled the |
872 |
|
ovector, the return from the function was zero, implying that there were |
873 |
|
other matches that did not fit. The correct "exactly full" value is now |
874 |
|
returned. |
875 |
|
|
876 |
|
6. If a subpattern that was called recursively or as a subroutine contained |
877 |
|
(*PRUNE) or any other control that caused it to give a non-standard return, |
878 |
|
invalid errors such as "Error -26 (nested recursion at the same subject |
879 |
|
position)" or even infinite loops could occur. |
880 |
|
|
881 |
|
7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped |
882 |
|
computing the minimum length on reaching *ACCEPT, and so ended up with the |
883 |
|
wrong value of 1 rather than 0. Further investigation indicates that |
884 |
|
computing a minimum subject length in the presence of *ACCEPT is difficult |
885 |
|
(think back references, subroutine calls), and so I have changed the code |
886 |
|
so that no minimum is registered for a pattern that contains *ACCEPT. |
887 |
|
|
888 |
|
8. If (*THEN) was present in the first (true) branch of a conditional group, |
889 |
|
it was not handled as intended. [But see 16 below.] |
890 |
|
|
891 |
|
9. Replaced RunTest.bat and CMakeLists.txt with improved versions provided by |
892 |
|
Sheri Pierce. |
893 |
|
|
894 |
|
10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that |
895 |
|
the first byte in a match must be "a". |
896 |
|
|
897 |
|
11. Change 17 for 8.13 increased the recursion depth for patterns like |
898 |
|
/a(?:.)*?a/ drastically. I've improved things by remembering whether a |
899 |
|
pattern contains any instances of (*THEN). If it does not, the old |
900 |
|
optimizations are restored. It would be nice to do this on a per-group |
901 |
|
basis, but at the moment that is not feasible. |
902 |
|
|
903 |
|
12. In some environments, the output of pcretest -C is CRLF terminated. This |
904 |
|
broke RunTest's code that checks for the link size. A single white space |
905 |
|
character after the value is now allowed for. |
906 |
|
|
907 |
|
13. RunTest now checks for the "fr" locale as well as for "fr_FR" and "french". |
908 |
|
For "fr", it uses the Windows-specific input and output files. |
909 |
|
|
910 |
|
14. If (*THEN) appeared in a group that was called recursively or as a |
911 |
|
subroutine, it did not work as intended. [But see next item.] |
912 |
|
|
913 |
|
15. Consider the pattern /A (B(*THEN)C) | D/ where A, B, C, and D are complex |
914 |
|
pattern fragments (but not containing any | characters). If A and B are |
915 |
|
matched, but there is a failure in C so that it backtracks to (*THEN), PCRE |
916 |
|
was behaving differently to Perl. PCRE backtracked into A, but Perl goes to |
917 |
|
D. In other words, Perl considers parentheses that do not contain any | |
918 |
|
characters to be part of a surrounding alternative, whereas PCRE was |
919 |
|
treading (B(*THEN)C) the same as (B(*THEN)C|(*FAIL)) -- which Perl handles |
920 |
|
differently. PCRE now behaves in the same way as Perl, except in the case |
921 |
|
of subroutine/recursion calls such as (?1) which have in any case always |
922 |
|
been different (but PCRE had them first :-). |
923 |
|
|
924 |
|
16. Related to 15 above: Perl does not treat the | in a conditional group as |
925 |
|
creating alternatives. Such a group is treated in the same way as an |
926 |
|
ordinary group without any | characters when processing (*THEN). PCRE has |
927 |
|
been changed to match Perl's behaviour. |
928 |
|
|
929 |
|
17. If a user had set PCREGREP_COLO(U)R to something other than 1:31, the |
930 |
|
RunGrepTest script failed. |
931 |
|
|
932 |
|
18. Change 22 for version 13 caused atomic groups to use more stack. This is |
933 |
|
inevitable for groups that contain captures, but it can lead to a lot of |
934 |
|
stack use in large patterns. The old behaviour has been restored for atomic |
935 |
|
groups that do not contain any capturing parentheses. |
936 |
|
|
937 |
|
19. If the PCRE_NO_START_OPTIMIZE option was set for pcre_compile(), it did not |
938 |
|
suppress the check for a minimum subject length at run time. (If it was |
939 |
|
given to pcre_exec() or pcre_dfa_exec() it did work.) |
940 |
|
|
941 |
|
20. Fixed an ASCII-dependent infelicity in pcretest that would have made it |
942 |
|
fail to work when decoding hex characters in data strings in EBCDIC |
943 |
|
environments. |
944 |
|
|
945 |
|
21. It appears that in at least one Mac OS environment, the isxdigit() function |
946 |
|
is implemented as a macro that evaluates to its argument more than once, |
947 |
|
contravening the C 90 Standard (I haven't checked a later standard). There |
948 |
|
was an instance in pcretest which caused it to go wrong when processing |
949 |
|
\x{...} escapes in subject strings. The has been rewritten to avoid using |
950 |
|
things like p++ in the argument of isxdigit(). |
951 |
|
|
952 |
|
|
953 |
|
Version 8.13 16-Aug-2011 |
954 |
------------------------ |
------------------------ |
955 |
|
|
956 |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
969 |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
970 |
2-byte code at the end of the subject (it thought there wasn't enough data |
2-byte code at the end of the subject (it thought there wasn't enough data |
971 |
left). |
left). |
972 |
|
|
973 |
5. Comprehensive information about what went wrong is now returned by |
5. Comprehensive information about what went wrong is now returned by |
974 |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
975 |
as the output vector has at least 2 elements. The offset of the start of |
as the output vector has at least 2 elements. The offset of the start of |
976 |
the failing character and a reason code are placed in the vector. |
the failing character and a reason code are placed in the vector. |
977 |
|
|
978 |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
979 |
now returned is for the first byte of the failing character, instead of the |
now returned is for the first byte of the failing character, instead of the |
980 |
last byte inspected. This is an incompatible change, but I hope it is small |
last byte inspected. This is an incompatible change, but I hope it is small |
981 |
enough not to be a problem. It makes the returned offset consistent with |
enough not to be a problem. It makes the returned offset consistent with |
982 |
pcre_exec() and pcre_dfa_exec(). |
pcre_exec() and pcre_dfa_exec(). |
983 |
|
|
984 |
7. pcretest now gives a text phrase as well as the error number when |
7. pcretest now gives a text phrase as well as the error number when |
985 |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
986 |
failure, the offset and reason code are output. |
failure, the offset and reason code are output. |
987 |
|
|
988 |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
989 |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
990 |
back over a single character (\n). This seems wrong (because it treated the |
back over a single character (\n). This seems wrong (because it treated the |
991 |
two characters as a single entity when going forwards), conflicts with the |
two characters as a single entity when going forwards), conflicts with the |
992 |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
993 |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
994 |
has been changed. |
has been changed. |
995 |
|
|
996 |
9. Some internal refactoring has changed the processing so that the handling |
9. Some internal refactoring has changed the processing so that the handling |
997 |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
998 |
time (the PCRE_DOTALL option was changed this way some time ago: version |
time (the PCRE_DOTALL option was changed this way some time ago: version |
999 |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
1000 |
which was always a bit of a fudge. It also means that there is one less |
which was always a bit of a fudge. It also means that there is one less |
1001 |
argument for the match() function, which reduces its stack requirements |
argument for the match() function, which reduces its stack requirements |
1002 |
slightly. This change also fixes an incompatibility with Perl: the pattern |
slightly. This change also fixes an incompatibility with Perl: the pattern |
1003 |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
1004 |
|
|
1005 |
10. More internal refactoring has drastically reduced the number of recursive |
10. More internal refactoring has drastically reduced the number of recursive |
1006 |
calls to match() for possessively repeated groups such as (abc)++ when |
calls to match() for possessively repeated groups such as (abc)++ when |
1007 |
using pcre_exec(). |
using pcre_exec(). |
1008 |
|
|
1009 |
11. While implementing 10, a number of bugs in the handling of groups were |
11. While implementing 10, a number of bugs in the handling of groups were |
1010 |
discovered and fixed: |
discovered and fixed: |
1011 |
|
|
1012 |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
1013 |
(a|)*(?1) gave a compile-time internal error. |
(a|)*(?1) gave a compile-time internal error. |
1014 |
((a|)+)+ did not notice that the outer group could match an empty string. |
((a|)+)+ did not notice that the outer group could match an empty string. |
1015 |
(^a|^)+ was not marked as anchored. |
(^a|^)+ was not marked as anchored. |
1016 |
(.*a|.*)+ was not marked as matching at start or after a newline. |
(.*a|.*)+ was not marked as matching at start or after a newline. |
1017 |
|
|
1018 |
12. Yet more internal refactoring has removed another argument from the match() |
12. Yet more internal refactoring has removed another argument from the match() |
1019 |
function. Special calls to this function are now indicated by setting a |
function. Special calls to this function are now indicated by setting a |
1020 |
value in a variable in the "match data" data block. |
value in a variable in the "match data" data block. |
1021 |
|
|
1022 |
13. Be more explicit in pcre_study() instead of relying on "default" for |
13. Be more explicit in pcre_study() instead of relying on "default" for |
1023 |
opcodes that mean there is no starting character; this means that when new |
opcodes that mean there is no starting character; this means that when new |
1024 |
ones are added and accidentally left out of pcre_study(), testing should |
ones are added and accidentally left out of pcre_study(), testing should |
1025 |
pick them up. |
pick them up. |
1026 |
|
|
1027 |
14. The -s option of pcretest has been documented for ages as being an old |
14. The -s option of pcretest has been documented for ages as being an old |
1028 |
synonym of -m (show memory usage). I have changed it to mean "force study |
synonym of -m (show memory usage). I have changed it to mean "force study |
1029 |
for every regex", that is, assume /S for every regex. This is similar to -i |
for every regex", that is, assume /S for every regex. This is similar to -i |
1030 |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
1031 |
using it. It makes it easier to run collection of tests with study enabled, |
using it. It makes it easier to run collections of tests with and without |
1032 |
and thereby test pcre_study() more easily. |
study enabled, and thereby test pcre_study() more easily. All the standard |
1033 |
|
tests are now run with and without -s (but some patterns can be marked as |
1034 |
|
"never study" - see 20 below). |
1035 |
|
|
1036 |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
1037 |
restoration of the capturing data to the outer values was not happening |
restoration of the capturing data to the outer values was not happening |
1038 |
correctly. |
correctly. |
1039 |
|
|
1040 |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
1041 |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
1042 |
pattern had matched an empty string, and so incorrectly returned a no |
pattern had matched an empty string, and so incorrectly returned a no |
1043 |
match. |
match. |
1044 |
|
|
1045 |
|
17. There was optimizing code for the last branch of non-capturing parentheses, |
1046 |
|
and also for the obeyed branch of a conditional subexpression, which used |
1047 |
|
tail recursion to cut down on stack usage. Unfortunately, now that there is |
1048 |
|
the possibility of (*THEN) occurring in these branches, tail recursion is |
1049 |
|
no longer possible because the return has to be checked for (*THEN). These |
1050 |
|
two optimizations have therefore been removed. [But see 8.20/11 above.] |
1051 |
|
|
1052 |
|
18. If a pattern containing \R was studied, it was assumed that \R always |
1053 |
|
matched two bytes, thus causing the minimum subject length to be |
1054 |
|
incorrectly computed because \R can also match just one byte. |
1055 |
|
|
1056 |
|
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
1057 |
|
was incorrectly computed. |
1058 |
|
|
1059 |
|
20. If /S is present twice on a test pattern in pcretest input, it now |
1060 |
|
*disables* studying, thereby overriding the use of -s on the command line |
1061 |
|
(see 14 above). This is necessary for one or two tests to keep the output |
1062 |
|
identical in both cases. |
1063 |
|
|
1064 |
|
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
1065 |
|
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
1066 |
|
|
1067 |
|
22. When an atomic group that contained a capturing parenthesis was |
1068 |
|
successfully matched, but the branch in which it appeared failed, the |
1069 |
|
capturing was not being forgotten if a higher numbered group was later |
1070 |
|
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
1071 |
|
group 1 to "a", when in fact it should be unset. This applied to multi- |
1072 |
|
branched capturing and non-capturing groups, repeated or not, and also to |
1073 |
|
positive assertions (capturing in negative assertions does not happen |
1074 |
|
in PCRE) and also to nested atomic groups. |
1075 |
|
|
1076 |
|
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
1077 |
|
subject after a captured substring, to make it easier to tell which of a |
1078 |
|
number of identical substrings has been captured. |
1079 |
|
|
1080 |
|
24. The way atomic groups are processed by pcre_exec() has been changed so that |
1081 |
|
if they are repeated, backtracking one repetition now resets captured |
1082 |
|
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
1083 |
|
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
1084 |
|
"aaa". Previously, it would have been "a". As part of this code |
1085 |
|
refactoring, the way recursive calls are handled has also been changed. |
1086 |
|
|
1087 |
|
25. If an assertion condition captured any substrings, they were not passed |
1088 |
|
back unless some other capturing happened later. For example, if |
1089 |
|
(?(?=(a))a) was matched against "a", no capturing was returned. |
1090 |
|
|
1091 |
|
26. When studying a pattern that contained subroutine calls or assertions, |
1092 |
|
the code for finding the minimum length of a possible match was handling |
1093 |
|
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
1094 |
|
group 1 called group 2 while simultaneously a separate group 2 called group |
1095 |
|
1). A stack overflow occurred in this case. I have fixed this by limiting |
1096 |
|
the recursion depth to 10. |
1097 |
|
|
1098 |
|
27. Updated RunTest.bat in the distribution to the version supplied by Tom |
1099 |
|
Fortmann. This supports explicit test numbers on the command line, and has |
1100 |
|
argument validation and error reporting. |
1101 |
|
|
1102 |
|
28. An instance of \X with an unlimited repeat could fail if at any point the |
1103 |
|
first character it looked at was a mark character. |
1104 |
|
|
1105 |
|
29. Some minor code refactoring concerning Unicode properties and scripts |
1106 |
|
should reduce the stack requirement of match() slightly. |
1107 |
|
|
1108 |
|
30. Added the '=' option to pcretest to check the setting of unused capturing |
1109 |
|
slots at the end of the pattern, which are documented as being -1, but are |
1110 |
|
not included in the return count. |
1111 |
|
|
1112 |
|
31. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
1113 |
|
compiled something random. Now it gives a compile-time error (as does |
1114 |
|
Perl). |
1115 |
|
|
1116 |
|
32. A *MARK encountered during the processing of a positive assertion is now |
1117 |
|
recorded and passed back (compatible with Perl). |
1118 |
|
|
1119 |
|
33. If --only-matching or --colour was set on a pcregrep call whose pattern |
1120 |
|
had alternative anchored branches, the search for a second match in a line |
1121 |
|
was done as if at the line start. Thus, for example, /^01|^02/ incorrectly |
1122 |
|
matched the line "0102" twice. The same bug affected patterns that started |
1123 |
|
with a backwards assertion. For example /\b01|\b02/ also matched "0102" |
1124 |
|
twice. |
1125 |
|
|
1126 |
|
34. Previously, PCRE did not allow quantification of assertions. However, Perl |
1127 |
|
does, and because of capturing effects, quantifying parenthesized |
1128 |
|
assertions may at times be useful. Quantifiers are now allowed for |
1129 |
|
parenthesized assertions. |
1130 |
|
|
1131 |
|
35. A minor code tidy in pcre_compile() when checking options for \R usage. |
1132 |
|
|
1133 |
|
36. \g was being checked for fancy things in a character class, when it should |
1134 |
|
just be a literal "g". |
1135 |
|
|
1136 |
|
37. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the |
1137 |
|
appearance of a nested POSIX class supersedes an apparent external class. |
1138 |
|
For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, |
1139 |
|
unescaped square brackets may also appear as part of class names. For |
1140 |
|
example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves |
1141 |
|
more like Perl. (But see 8.20/1 above.) |
1142 |
|
|
1143 |
|
38. PCRE was giving an error for \N with a braced quantifier such as {1,} (this |
1144 |
|
was because it thought it was \N{name}, which is not supported). |
1145 |
|
|
1146 |
|
39. Add minix to OS list not supporting the -S option in pcretest. |
1147 |
|
|
1148 |
|
40. PCRE tries to detect cases of infinite recursion at compile time, but it |
1149 |
|
cannot analyze patterns in sufficient detail to catch mutual recursions |
1150 |
|
such as ((?1))((?2)). There is now a runtime test that gives an error if a |
1151 |
|
subgroup is called recursively as a subpattern for a second time at the |
1152 |
|
same position in the subject string. In previous releases this might have |
1153 |
|
been caught by the recursion limit, or it might have run out of stack. |
1154 |
|
|
1155 |
|
41. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can |
1156 |
|
happen only once. PCRE was, however incorrectly giving a compile time error |
1157 |
|
"recursive call could loop indefinitely" because it cannot analyze the |
1158 |
|
pattern in sufficient detail. The compile time test no longer happens when |
1159 |
|
PCRE is compiling a conditional subpattern, but actual runaway loops are |
1160 |
|
now caught at runtime (see 40 above). |
1161 |
|
|
1162 |
|
42. It seems that Perl allows any characters other than a closing parenthesis |
1163 |
|
to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE |
1164 |
|
has been changed to be the same. |
1165 |
|
|
1166 |
|
43. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so |
1167 |
|
as not to get warnings when autogen.sh is called. Also changed |
1168 |
|
AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). |
1169 |
|
|
1170 |
|
44. To help people who use pcregrep to scan files containing exceedingly long |
1171 |
|
lines, the following changes have been made: |
1172 |
|
|
1173 |
|
(a) The default value of the buffer size parameter has been increased from |
1174 |
|
8K to 20K. (The actual buffer used is three times this size.) |
1175 |
|
|
1176 |
|
(b) The default can be changed by ./configure --with-pcregrep-bufsize when |
1177 |
|
PCRE is built. |
1178 |
|
|
1179 |
|
(c) A --buffer-size=n option has been added to pcregrep, to allow the size |
1180 |
|
to be set at run time. |
1181 |
|
|
1182 |
|
(d) Numerical values in pcregrep options can be followed by K or M, for |
1183 |
|
example --buffer-size=50K. |
1184 |
|
|
1185 |
|
(e) If a line being scanned overflows pcregrep's buffer, an error is now |
1186 |
|
given and the return code is set to 2. |
1187 |
|
|
1188 |
|
45. Add a pointer to the latest mark to the callout data block. |
1189 |
|
|
1190 |
|
46. The pattern /.(*F)/, when applied to "abc" with PCRE_PARTIAL_HARD, gave a |
1191 |
|
partial match of an empty string instead of no match. This was specific to |
1192 |
|
the use of ".". |
1193 |
|
|
1194 |
|
47. The pattern /f.*/8s, when applied to "for" with PCRE_PARTIAL_HARD, gave a |
1195 |
|
complete match instead of a partial match. This bug was dependent on both |
1196 |
|
the PCRE_UTF8 and PCRE_DOTALL options being set. |
1197 |
|
|
1198 |
|
48. For a pattern such as /\babc|\bdef/ pcre_study() was failing to set up the |
1199 |
|
starting byte set, because \b was not being ignored. |
1200 |
|
|
1201 |
|
|
1202 |
Version 8.12 15-Jan-2011 |
Version 8.12 15-Jan-2011 |
1203 |
------------------------ |
------------------------ |
1849 |
7. A pattern that could match an empty string could cause pcregrep to loop; it |
7. A pattern that could match an empty string could cause pcregrep to loop; it |
1850 |
doesn't make sense to accept an empty string match in pcregrep, so I have |
doesn't make sense to accept an empty string match in pcregrep, so I have |
1851 |
locked it out (using PCRE's PCRE_NOTEMPTY option). By experiment, this |
locked it out (using PCRE's PCRE_NOTEMPTY option). By experiment, this |
1852 |
seems to be how GNU grep behaves. |
seems to be how GNU grep behaves. [But see later change 40 for release |
1853 |
|
8.33.] |
1854 |
|
|
1855 |
8. The pattern (?(?=.*b)b|^) was incorrectly compiled as "match must be at |
8. The pattern (?(?=.*b)b|^) was incorrectly compiled as "match must be at |
1856 |
start or after a newline", because the conditional assertion was not being |
start or after a newline", because the conditional assertion was not being |
2093 |
containing () gave an internal compiling error instead of "reference to |
containing () gave an internal compiling error instead of "reference to |
2094 |
non-existent subpattern". Fortunately, when the pattern did exist, the |
non-existent subpattern". Fortunately, when the pattern did exist, the |
2095 |
compiled code was correct. (When scanning forwards to check for the |
compiled code was correct. (When scanning forwards to check for the |
2096 |
existencd of the subpattern, it was treating the data ']' as terminating |
existence of the subpattern, it was treating the data ']' as terminating |
2097 |
the class, so got the count wrong. When actually compiling, the reference |
the class, so got the count wrong. When actually compiling, the reference |
2098 |
was subsequently set up correctly.) |
was subsequently set up correctly.) |
2099 |
|
|