1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.13 30-Apr-2011 |
Version 8.31 |
5 |
|
----------------------------- |
6 |
|
|
7 |
|
1. Fixing a wrong JIT test case and some compiler warnings. |
8 |
|
|
9 |
|
2. Removed a bashism from the RunTest script. |
10 |
|
|
11 |
|
3. Add a cast to pcre_exec.c to fix the warning "unary minus operator applied |
12 |
|
to unsigned type, result still unsigned" that was given by an MS compiler |
13 |
|
on encountering the code "-sizeof(xxx)". |
14 |
|
|
15 |
|
4. Partial matching support is added to the JIT compiler. |
16 |
|
|
17 |
|
5. Fixed several bugs concerned with partial matching of items that consist |
18 |
|
of more than one character: |
19 |
|
|
20 |
|
(a) /^(..)\1/ did not partially match "aba" because checking references was |
21 |
|
done on an "all or nothing" basis. This also applied to repeated |
22 |
|
references. |
23 |
|
|
24 |
|
(b) \R did not give a hard partial match if \r was found at the end of the |
25 |
|
subject. |
26 |
|
|
27 |
|
(c) \X did not give a hard partial match after matching one or more |
28 |
|
characters at the end of the subject. |
29 |
|
|
30 |
|
(d) When newline was set to CRLF, a pattern such as /a$/ did not give a |
31 |
|
partial match for the string "\r". |
32 |
|
|
33 |
|
|
34 |
|
Version 8.30 04-February-2012 |
35 |
|
----------------------------- |
36 |
|
|
37 |
|
1. Renamed "isnumber" as "is_a_number" because in some Mac environments this |
38 |
|
name is defined in ctype.h. |
39 |
|
|
40 |
|
2. Fixed a bug in fixed-length calculation for lookbehinds that would show up |
41 |
|
only in quite long subpatterns. |
42 |
|
|
43 |
|
3. Removed the function pcre_info(), which has been obsolete and deprecated |
44 |
|
since it was replaced by pcre_fullinfo() in February 2000. |
45 |
|
|
46 |
|
4. For a non-anchored pattern, if (*SKIP) was given with a name that did not |
47 |
|
match a (*MARK), and the match failed at the start of the subject, a |
48 |
|
reference to memory before the start of the subject could occur. This bug |
49 |
|
was introduced by fix 17 of release 8.21. |
50 |
|
|
51 |
|
5. A reference to an unset group with zero minimum repetition was giving |
52 |
|
totally wrong answers (in non-JavaScript-compatibility mode). For example, |
53 |
|
/(another)?(\1?)test/ matched against "hello world test". This bug was |
54 |
|
introduced in release 8.13. |
55 |
|
|
56 |
|
6. Add support for 16-bit character strings (a large amount of work involving |
57 |
|
many changes and refactorings). |
58 |
|
|
59 |
|
7. RunGrepTest failed on msys because \r\n was replaced by whitespace when the |
60 |
|
command "pattern=`printf 'xxx\r\njkl'`" was run. The pattern is now taken |
61 |
|
from a file. |
62 |
|
|
63 |
|
8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size |
64 |
|
rounding is not applied in this particular case). |
65 |
|
|
66 |
|
9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected |
67 |
|
if they appear, or are escaped, in patterns. |
68 |
|
|
69 |
|
10. Get rid of a number of -Wunused-but-set-variable warnings. |
70 |
|
|
71 |
|
11. The pattern /(?=(*:x))(q|)/ matches an empty string, and returns the mark |
72 |
|
"x". The similar pattern /(?=(*:x))((*:y)q|)/ did not return a mark at all. |
73 |
|
Oddly, Perl behaves the same way. PCRE has been fixed so that this pattern |
74 |
|
also returns the mark "x". This bug applied to capturing parentheses, |
75 |
|
non-capturing parentheses, and atomic parentheses. It also applied to some |
76 |
|
assertions. |
77 |
|
|
78 |
|
12. Stephen Kelly's patch to CMakeLists.txt allows it to parse the version |
79 |
|
information out of configure.ac instead of relying on pcre.h.generic, which |
80 |
|
is not stored in the repository. |
81 |
|
|
82 |
|
13. Applied Dmitry V. Levin's patch for a more portable method for linking with |
83 |
|
-lreadline. |
84 |
|
|
85 |
|
14. ZH added PCRE_CONFIG_JITTARGET; added its output to pcretest -C. |
86 |
|
|
87 |
|
15. Applied Graycode's patch to put the top-level frame on the stack rather |
88 |
|
than the heap when not using the stack for recursion. This gives a |
89 |
|
performance improvement in many cases when recursion is not deep. |
90 |
|
|
91 |
|
16. Experimental code added to "pcretest -C" to output the stack frame size. |
92 |
|
|
93 |
|
|
94 |
|
Version 8.21 12-Dec-2011 |
95 |
|
------------------------ |
96 |
|
|
97 |
|
1. Updating the JIT compiler. |
98 |
|
|
99 |
|
2. JIT compiler now supports OP_NCREF, OP_RREF and OP_NRREF. New test cases |
100 |
|
are added as well. |
101 |
|
|
102 |
|
3. Fix cache-flush issue on PowerPC (It is still an experimental JIT port). |
103 |
|
PCRE_EXTRA_TABLES is not suported by JIT, and should be checked before |
104 |
|
calling _pcre_jit_exec. Some extra comments are added. |
105 |
|
|
106 |
|
4. (*MARK) settings inside atomic groups that do not contain any capturing |
107 |
|
parentheses, for example, (?>a(*:m)), were not being passed out. This bug |
108 |
|
was introduced by change 18 for 8.20. |
109 |
|
|
110 |
|
5. Supporting of \x, \U and \u in JavaScript compatibility mode based on the |
111 |
|
ECMA-262 standard. |
112 |
|
|
113 |
|
6. Lookbehinds such as (?<=a{2}b) that contained a fixed repetition were |
114 |
|
erroneously being rejected as "not fixed length" if PCRE_CASELESS was set. |
115 |
|
This bug was probably introduced by change 9 of 8.13. |
116 |
|
|
117 |
|
7. While fixing 6 above, I noticed that a number of other items were being |
118 |
|
incorrectly rejected as "not fixed length". This arose partly because newer |
119 |
|
opcodes had not been added to the fixed-length checking code. I have (a) |
120 |
|
corrected the bug and added tests for these items, and (b) arranged for an |
121 |
|
error to occur if an unknown opcode is encountered while checking for fixed |
122 |
|
length instead of just assuming "not fixed length". The items that were |
123 |
|
rejected were: (*ACCEPT), (*COMMIT), (*FAIL), (*MARK), (*PRUNE), (*SKIP), |
124 |
|
(*THEN), \h, \H, \v, \V, and single character negative classes with fixed |
125 |
|
repetitions, e.g. [^a]{3}, with and without PCRE_CASELESS. |
126 |
|
|
127 |
|
8. A possessively repeated conditional subpattern such as (?(?=c)c|d)++ was |
128 |
|
being incorrectly compiled and would have given unpredicatble results. |
129 |
|
|
130 |
|
9. A possessively repeated subpattern with minimum repeat count greater than |
131 |
|
one behaved incorrectly. For example, (A){2,}+ behaved as if it was |
132 |
|
(A)(A)++ which meant that, after a subsequent mismatch, backtracking into |
133 |
|
the first (A) could occur when it should not. |
134 |
|
|
135 |
|
10. Add a cast and remove a redundant test from the code. |
136 |
|
|
137 |
|
11. JIT should use pcre_malloc/pcre_free for allocation. |
138 |
|
|
139 |
|
12. Updated pcre-config so that it no longer shows -L/usr/lib, which seems |
140 |
|
best practice nowadays, and helps with cross-compiling. (If the exec_prefix |
141 |
|
is anything other than /usr, -L is still shown). |
142 |
|
|
143 |
|
13. In non-UTF-8 mode, \C is now supported in lookbehinds and DFA matching. |
144 |
|
|
145 |
|
14. Perl does not support \N without a following name in a [] class; PCRE now |
146 |
|
also gives an error. |
147 |
|
|
148 |
|
15. If a forward reference was repeated with an upper limit of around 2000, |
149 |
|
it caused the error "internal error: overran compiling workspace". The |
150 |
|
maximum number of forward references (including repeats) was limited by the |
151 |
|
internal workspace, and dependent on the LINK_SIZE. The code has been |
152 |
|
rewritten so that the workspace expands (via pcre_malloc) if necessary, and |
153 |
|
the default depends on LINK_SIZE. There is a new upper limit (for safety) |
154 |
|
of around 200,000 forward references. While doing this, I also speeded up |
155 |
|
the filling in of repeated forward references. |
156 |
|
|
157 |
|
16. A repeated forward reference in a pattern such as (a)(?2){2}(.) was |
158 |
|
incorrectly expecting the subject to contain another "a" after the start. |
159 |
|
|
160 |
|
17. When (*SKIP:name) is activated without a corresponding (*MARK:name) earlier |
161 |
|
in the match, the SKIP should be ignored. This was not happening; instead |
162 |
|
the SKIP was being treated as NOMATCH. For patterns such as |
163 |
|
/A(*MARK:A)A+(*SKIP:B)Z|AAC/ this meant that the AAC branch was never |
164 |
|
tested. |
165 |
|
|
166 |
|
18. The behaviour of (*MARK), (*PRUNE), and (*THEN) has been reworked and is |
167 |
|
now much more compatible with Perl, in particular in cases where the result |
168 |
|
is a non-match for a non-anchored pattern. For example, if |
169 |
|
/b(*:m)f|a(*:n)w/ is matched against "abc", the non-match returns the name |
170 |
|
"m", where previously it did not return a name. A side effect of this |
171 |
|
change is that for partial matches, the last encountered mark name is |
172 |
|
returned, as for non matches. A number of tests that were previously not |
173 |
|
Perl-compatible have been moved into the Perl-compatible test files. The |
174 |
|
refactoring has had the pleasing side effect of removing one argument from |
175 |
|
the match() function, thus reducing its stack requirements. |
176 |
|
|
177 |
|
19. If the /S+ option was used in pcretest to study a pattern using JIT, |
178 |
|
subsequent uses of /S (without +) incorrectly behaved like /S+. |
179 |
|
|
180 |
|
21. Retrieve executable code size support for the JIT compiler and fixing |
181 |
|
some warnings. |
182 |
|
|
183 |
|
22. A caseless match of a UTF-8 character whose other case uses fewer bytes did |
184 |
|
not work when the shorter character appeared right at the end of the |
185 |
|
subject string. |
186 |
|
|
187 |
|
23. Added some (int) casts to non-JIT modules to reduce warnings on 64-bit |
188 |
|
systems. |
189 |
|
|
190 |
|
24. Added PCRE_INFO_JITSIZE to pass on the value from (21) above, and also |
191 |
|
output it when the /M option is used in pcretest. |
192 |
|
|
193 |
|
25. The CheckMan script was not being included in the distribution. Also, added |
194 |
|
an explicit "perl" to run Perl scripts from the PrepareRelease script |
195 |
|
because this is reportedly needed in Windows. |
196 |
|
|
197 |
|
26. If study data was being save in a file and studying had not found a set of |
198 |
|
"starts with" bytes for the pattern, the data written to the file (though |
199 |
|
never used) was taken from uninitialized memory and so caused valgrind to |
200 |
|
complain. |
201 |
|
|
202 |
|
27. Updated RunTest.bat as provided by Sheri Pierce. |
203 |
|
|
204 |
|
28. Fixed a possible uninitialized memory bug in pcre_jit_compile.c. |
205 |
|
|
206 |
|
29. Computation of memory usage for the table of capturing group names was |
207 |
|
giving an unnecessarily large value. |
208 |
|
|
209 |
|
|
210 |
|
Version 8.20 21-Oct-2011 |
211 |
|
------------------------ |
212 |
|
|
213 |
|
1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had |
214 |
|
a POSIX class. After further experiments with Perl, which convinced me that |
215 |
|
Perl has bugs and confusions, a closing square bracket is no longer allowed |
216 |
|
in a POSIX name. This bug also affected patterns with classes that started |
217 |
|
with full stops. |
218 |
|
|
219 |
|
2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no |
220 |
|
captured substring, but while checking the failing first alternative, |
221 |
|
substring 1 is temporarily captured. If the output vector supplied to |
222 |
|
pcre_exec() was not big enough for this capture, the yield of the function |
223 |
|
was still zero ("insufficient space for captured substrings"). This cannot |
224 |
|
be totally fixed without adding another stack variable, which seems a lot |
225 |
|
of expense for a edge case. However, I have improved the situation in cases |
226 |
|
such as /(a)(b)x|abc/ matched against "abc", where the return code |
227 |
|
indicates that fewer than the maximum number of slots in the ovector have |
228 |
|
been set. |
229 |
|
|
230 |
|
3. Related to (2) above: when there are more back references in a pattern than |
231 |
|
slots in the output vector, pcre_exec() uses temporary memory during |
232 |
|
matching, and copies in the captures as far as possible afterwards. It was |
233 |
|
using the entire output vector, but this conflicts with the specification |
234 |
|
that only 2/3 is used for passing back captured substrings. Now it uses |
235 |
|
only the first 2/3, for compatibility. This is, of course, another edge |
236 |
|
case. |
237 |
|
|
238 |
|
4. Zoltan Herczeg's just-in-time compiler support has been integrated into the |
239 |
|
main code base, and can be used by building with --enable-jit. When this is |
240 |
|
done, pcregrep automatically uses it unless --disable-pcregrep-jit or the |
241 |
|
runtime --no-jit option is given. |
242 |
|
|
243 |
|
5. When the number of matches in a pcre_dfa_exec() run exactly filled the |
244 |
|
ovector, the return from the function was zero, implying that there were |
245 |
|
other matches that did not fit. The correct "exactly full" value is now |
246 |
|
returned. |
247 |
|
|
248 |
|
6. If a subpattern that was called recursively or as a subroutine contained |
249 |
|
(*PRUNE) or any other control that caused it to give a non-standard return, |
250 |
|
invalid errors such as "Error -26 (nested recursion at the same subject |
251 |
|
position)" or even infinite loops could occur. |
252 |
|
|
253 |
|
7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped |
254 |
|
computing the minimum length on reaching *ACCEPT, and so ended up with the |
255 |
|
wrong value of 1 rather than 0. Further investigation indicates that |
256 |
|
computing a minimum subject length in the presence of *ACCEPT is difficult |
257 |
|
(think back references, subroutine calls), and so I have changed the code |
258 |
|
so that no minimum is registered for a pattern that contains *ACCEPT. |
259 |
|
|
260 |
|
8. If (*THEN) was present in the first (true) branch of a conditional group, |
261 |
|
it was not handled as intended. [But see 16 below.] |
262 |
|
|
263 |
|
9. Replaced RunTest.bat and CMakeLists.txt with improved versions provided by |
264 |
|
Sheri Pierce. |
265 |
|
|
266 |
|
10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that |
267 |
|
the first byte in a match must be "a". |
268 |
|
|
269 |
|
11. Change 17 for 8.13 increased the recursion depth for patterns like |
270 |
|
/a(?:.)*?a/ drastically. I've improved things by remembering whether a |
271 |
|
pattern contains any instances of (*THEN). If it does not, the old |
272 |
|
optimizations are restored. It would be nice to do this on a per-group |
273 |
|
basis, but at the moment that is not feasible. |
274 |
|
|
275 |
|
12. In some environments, the output of pcretest -C is CRLF terminated. This |
276 |
|
broke RunTest's code that checks for the link size. A single white space |
277 |
|
character after the value is now allowed for. |
278 |
|
|
279 |
|
13. RunTest now checks for the "fr" locale as well as for "fr_FR" and "french". |
280 |
|
For "fr", it uses the Windows-specific input and output files. |
281 |
|
|
282 |
|
14. If (*THEN) appeared in a group that was called recursively or as a |
283 |
|
subroutine, it did not work as intended. [But see next item.] |
284 |
|
|
285 |
|
15. Consider the pattern /A (B(*THEN)C) | D/ where A, B, C, and D are complex |
286 |
|
pattern fragments (but not containing any | characters). If A and B are |
287 |
|
matched, but there is a failure in C so that it backtracks to (*THEN), PCRE |
288 |
|
was behaving differently to Perl. PCRE backtracked into A, but Perl goes to |
289 |
|
D. In other words, Perl considers parentheses that do not contain any | |
290 |
|
characters to be part of a surrounding alternative, whereas PCRE was |
291 |
|
treading (B(*THEN)C) the same as (B(*THEN)C|(*FAIL)) -- which Perl handles |
292 |
|
differently. PCRE now behaves in the same way as Perl, except in the case |
293 |
|
of subroutine/recursion calls such as (?1) which have in any case always |
294 |
|
been different (but PCRE had them first :-). |
295 |
|
|
296 |
|
16. Related to 15 above: Perl does not treat the | in a conditional group as |
297 |
|
creating alternatives. Such a group is treated in the same way as an |
298 |
|
ordinary group without any | characters when processing (*THEN). PCRE has |
299 |
|
been changed to match Perl's behaviour. |
300 |
|
|
301 |
|
17. If a user had set PCREGREP_COLO(U)R to something other than 1:31, the |
302 |
|
RunGrepTest script failed. |
303 |
|
|
304 |
|
18. Change 22 for version 13 caused atomic groups to use more stack. This is |
305 |
|
inevitable for groups that contain captures, but it can lead to a lot of |
306 |
|
stack use in large patterns. The old behaviour has been restored for atomic |
307 |
|
groups that do not contain any capturing parentheses. |
308 |
|
|
309 |
|
19. If the PCRE_NO_START_OPTIMIZE option was set for pcre_compile(), it did not |
310 |
|
suppress the check for a minimum subject length at run time. (If it was |
311 |
|
given to pcre_exec() or pcre_dfa_exec() it did work.) |
312 |
|
|
313 |
|
20. Fixed an ASCII-dependent infelicity in pcretest that would have made it |
314 |
|
fail to work when decoding hex characters in data strings in EBCDIC |
315 |
|
environments. |
316 |
|
|
317 |
|
21. It appears that in at least one Mac OS environment, the isxdigit() function |
318 |
|
is implemented as a macro that evaluates to its argument more than once, |
319 |
|
contravening the C 90 Standard (I haven't checked a later standard). There |
320 |
|
was an instance in pcretest which caused it to go wrong when processing |
321 |
|
\x{...} escapes in subject strings. The has been rewritten to avoid using |
322 |
|
things like p++ in the argument of isxdigit(). |
323 |
|
|
324 |
|
|
325 |
|
Version 8.13 16-Aug-2011 |
326 |
------------------------ |
------------------------ |
327 |
|
|
328 |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
341 |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
342 |
2-byte code at the end of the subject (it thought there wasn't enough data |
2-byte code at the end of the subject (it thought there wasn't enough data |
343 |
left). |
left). |
344 |
|
|
345 |
5. Comprehensive information about what went wrong is now returned by |
5. Comprehensive information about what went wrong is now returned by |
346 |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
347 |
as the output vector has at least 2 elements. The offset of the start of |
as the output vector has at least 2 elements. The offset of the start of |
348 |
the failing character and a reason code are placed in the vector. |
the failing character and a reason code are placed in the vector. |
349 |
|
|
350 |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
351 |
now returned is for the first byte of the failing character, instead of the |
now returned is for the first byte of the failing character, instead of the |
352 |
last byte inspected. This is an incompatible change, but I hope it is small |
last byte inspected. This is an incompatible change, but I hope it is small |
353 |
enough not to be a problem. It makes the returned offset consistent with |
enough not to be a problem. It makes the returned offset consistent with |
354 |
pcre_exec() and pcre_dfa_exec(). |
pcre_exec() and pcre_dfa_exec(). |
355 |
|
|
356 |
7. pcretest now gives a text phrase as well as the error number when |
7. pcretest now gives a text phrase as well as the error number when |
357 |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
358 |
failure, the offset and reason code are output. |
failure, the offset and reason code are output. |
359 |
|
|
360 |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
361 |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
362 |
back over a single character (\n). This seems wrong (because it treated the |
back over a single character (\n). This seems wrong (because it treated the |
363 |
two characters as a single entity when going forwards), conflicts with the |
two characters as a single entity when going forwards), conflicts with the |
364 |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
365 |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
366 |
has been changed. |
has been changed. |
367 |
|
|
368 |
9. Some internal refactoring has changed the processing so that the handling |
9. Some internal refactoring has changed the processing so that the handling |
369 |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
370 |
time (the PCRE_DOTALL option was changed this way some time ago: version |
time (the PCRE_DOTALL option was changed this way some time ago: version |
371 |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
372 |
which was always a bit of a fudge. It also means that there is one less |
which was always a bit of a fudge. It also means that there is one less |
373 |
argument for the match() function, which reduces its stack requirements |
argument for the match() function, which reduces its stack requirements |
374 |
slightly. This change also fixes an incompatibility with Perl: the pattern |
slightly. This change also fixes an incompatibility with Perl: the pattern |
375 |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
376 |
|
|
377 |
10. More internal refactoring has drastically reduced the number of recursive |
10. More internal refactoring has drastically reduced the number of recursive |
378 |
calls to match() for possessively repeated groups such as (abc)++ when |
calls to match() for possessively repeated groups such as (abc)++ when |
379 |
using pcre_exec(). |
using pcre_exec(). |
380 |
|
|
381 |
11. While implementing 10, a number of bugs in the handling of groups were |
11. While implementing 10, a number of bugs in the handling of groups were |
382 |
discovered and fixed: |
discovered and fixed: |
383 |
|
|
384 |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
385 |
(a|)*(?1) gave a compile-time internal error. |
(a|)*(?1) gave a compile-time internal error. |
386 |
((a|)+)+ did not notice that the outer group could match an empty string. |
((a|)+)+ did not notice that the outer group could match an empty string. |
387 |
(^a|^)+ was not marked as anchored. |
(^a|^)+ was not marked as anchored. |
388 |
(.*a|.*)+ was not marked as matching at start or after a newline. |
(.*a|.*)+ was not marked as matching at start or after a newline. |
389 |
|
|
390 |
12. Yet more internal refactoring has removed another argument from the match() |
12. Yet more internal refactoring has removed another argument from the match() |
391 |
function. Special calls to this function are now indicated by setting a |
function. Special calls to this function are now indicated by setting a |
392 |
value in a variable in the "match data" data block. |
value in a variable in the "match data" data block. |
393 |
|
|
394 |
13. Be more explicit in pcre_study() instead of relying on "default" for |
13. Be more explicit in pcre_study() instead of relying on "default" for |
395 |
opcodes that mean there is no starting character; this means that when new |
opcodes that mean there is no starting character; this means that when new |
396 |
ones are added and accidentally left out of pcre_study(), testing should |
ones are added and accidentally left out of pcre_study(), testing should |
397 |
pick them up. |
pick them up. |
398 |
|
|
399 |
14. The -s option of pcretest has been documented for ages as being an old |
14. The -s option of pcretest has been documented for ages as being an old |
400 |
synonym of -m (show memory usage). I have changed it to mean "force study |
synonym of -m (show memory usage). I have changed it to mean "force study |
401 |
for every regex", that is, assume /S for every regex. This is similar to -i |
for every regex", that is, assume /S for every regex. This is similar to -i |
402 |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
403 |
using it. It makes it easier to run collections of tests with and without |
using it. It makes it easier to run collections of tests with and without |
404 |
study enabled, and thereby test pcre_study() more easily. All the standard |
study enabled, and thereby test pcre_study() more easily. All the standard |
405 |
tests are now run with and without -s (but some patterns can be marked as |
tests are now run with and without -s (but some patterns can be marked as |
406 |
"never study" - see 20 below). |
"never study" - see 20 below). |
407 |
|
|
408 |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
409 |
restoration of the capturing data to the outer values was not happening |
restoration of the capturing data to the outer values was not happening |
410 |
correctly. |
correctly. |
411 |
|
|
412 |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
413 |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
414 |
pattern had matched an empty string, and so incorrectly returned a no |
pattern had matched an empty string, and so incorrectly returned a no |
415 |
match. |
match. |
416 |
|
|
417 |
17. There was optimizing code for the last branch of non-capturing parentheses, |
17. There was optimizing code for the last branch of non-capturing parentheses, |
418 |
and also for the obeyed branch of a conditional subexpression, which used |
and also for the obeyed branch of a conditional subexpression, which used |
419 |
tail recursion to cut down on stack usage. Unfortunately, not that there is |
tail recursion to cut down on stack usage. Unfortunately, now that there is |
420 |
the possibility of (*THEN) occurring in these branches, tail recursion is |
the possibility of (*THEN) occurring in these branches, tail recursion is |
421 |
no longer possible because the return has to be checked for (*THEN). These |
no longer possible because the return has to be checked for (*THEN). These |
422 |
two optimizations have therefore been removed. |
two optimizations have therefore been removed. [But see 8.20/11 above.] |
423 |
|
|
424 |
18. If a pattern containing \R was studied, it was assumed that \R always |
18. If a pattern containing \R was studied, it was assumed that \R always |
425 |
matched two bytes, thus causing the minimum subject length to be |
matched two bytes, thus causing the minimum subject length to be |
426 |
incorrectly computed because \R can also match just one byte. |
incorrectly computed because \R can also match just one byte. |
427 |
|
|
428 |
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
429 |
was incorrectly computed. |
was incorrectly computed. |
430 |
|
|
431 |
20. If /S is present twice on a test pattern in pcretest input, it *disables* |
20. If /S is present twice on a test pattern in pcretest input, it now |
432 |
studying, thereby overriding the use of -s on the command line. This is |
*disables* studying, thereby overriding the use of -s on the command line |
433 |
necessary for one or two tests to keep the output identical in both cases. |
(see 14 above). This is necessary for one or two tests to keep the output |
434 |
|
identical in both cases. |
435 |
|
|
436 |
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
437 |
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
438 |
|
|
439 |
22. When an atomic group that contained a capturing parenthesis was |
22. When an atomic group that contained a capturing parenthesis was |
440 |
successfully matched, but the branch in which it appeared failed, the |
successfully matched, but the branch in which it appeared failed, the |
441 |
capturing was not being forgotten if a higher numbered group was later |
capturing was not being forgotten if a higher numbered group was later |
442 |
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
443 |
group 1 to "a", when in fact it should be unset. This applied to multi- |
group 1 to "a", when in fact it should be unset. This applied to multi- |
444 |
branched capturing and non-capturing groups, repeated or not, and also to |
branched capturing and non-capturing groups, repeated or not, and also to |
445 |
positive assertions (capturing in negative assertions is not well defined |
positive assertions (capturing in negative assertions does not happen |
446 |
in PCRE) and also to nested atomic groups. |
in PCRE) and also to nested atomic groups. |
447 |
|
|
448 |
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
449 |
subject after a captured substring (to make it easier to tell which of a |
subject after a captured substring, to make it easier to tell which of a |
450 |
number of identical substrings has been captured). |
number of identical substrings has been captured. |
451 |
|
|
452 |
24. The way atomic groups are processed by pcre_exec() has been changed so that |
24. The way atomic groups are processed by pcre_exec() has been changed so that |
453 |
if they are repeated, backtracking one repetition now resets captured |
if they are repeated, backtracking one repetition now resets captured |
454 |
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
455 |
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
456 |
"aaa". Previously, it would have been "a". As part of this code |
"aaa". Previously, it would have been "a". As part of this code |
457 |
refactoring, the way recursive calls are handled has also been changed. |
refactoring, the way recursive calls are handled has also been changed. |
458 |
|
|
459 |
24. If an assertion condition captured any substrings, they were not passed |
25. If an assertion condition captured any substrings, they were not passed |
460 |
back unless some other capturing happened later. For example, if |
back unless some other capturing happened later. For example, if |
461 |
(?(?=(a))a) was matched against "a", no capturing was returned. |
(?(?=(a))a) was matched against "a", no capturing was returned. |
462 |
|
|
463 |
25. When studying a pattern that contained subroutine calls or assertions, |
26. When studying a pattern that contained subroutine calls or assertions, |
464 |
the code for finding the minimum length of a possible match was handling |
the code for finding the minimum length of a possible match was handling |
465 |
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
466 |
group 1 called group 2 while simultaneously a separate group 2 called group |
group 1 called group 2 while simultaneously a separate group 2 called group |
467 |
1). A stack overflow occurred in this case. I have fixed this by limiting |
1). A stack overflow occurred in this case. I have fixed this by limiting |
468 |
the recursion depth to 10. |
the recursion depth to 10. |
469 |
|
|
470 |
26. Updated RunTest.bat in the distribution to the version supplied by Tom |
27. Updated RunTest.bat in the distribution to the version supplied by Tom |
471 |
Fortmann. This supports explicit test numbers on the command line, and has |
Fortmann. This supports explicit test numbers on the command line, and has |
472 |
argument validation and error reporting. |
argument validation and error reporting. |
473 |
|
|
474 |
27. An instance of \X with an unlimited repeat could fail if at any point the |
28. An instance of \X with an unlimited repeat could fail if at any point the |
475 |
first character it looked at was a mark character. |
first character it looked at was a mark character. |
476 |
|
|
477 |
28. Some minor code refactoring concerning Unicode properties and scripts |
29. Some minor code refactoring concerning Unicode properties and scripts |
478 |
should reduce the stack requirement of match() slightly. |
should reduce the stack requirement of match() slightly. |
479 |
|
|
480 |
29. Added the '=' option to pcretest to check the setting of unused capturing |
30. Added the '=' option to pcretest to check the setting of unused capturing |
481 |
slots at the end of the pattern, which are documented as being -1, but are |
slots at the end of the pattern, which are documented as being -1, but are |
482 |
not included in the return count. |
not included in the return count. |
483 |
|
|
484 |
30. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
31. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
485 |
compiled something random. Now it gives a compile-time error (as does |
compiled something random. Now it gives a compile-time error (as does |
486 |
Perl). |
Perl). |
487 |
|
|
488 |
31. A *MARK encountered during the processing of a positive assertion is now |
32. A *MARK encountered during the processing of a positive assertion is now |
489 |
recorded and passed back (compatible with Perl). |
recorded and passed back (compatible with Perl). |
490 |
|
|
491 |
32. If --only-matching or --colour was set on a pcregrep call whose pattern |
33. If --only-matching or --colour was set on a pcregrep call whose pattern |
492 |
had alternative anchored branches, the search for a second match in a line |
had alternative anchored branches, the search for a second match in a line |
493 |
was done as if at the line start. Thus, for example, /^01|^02/ incorrectly |
was done as if at the line start. Thus, for example, /^01|^02/ incorrectly |
494 |
matched the line "0102" twice. The same bug affected patterns that started |
matched the line "0102" twice. The same bug affected patterns that started |
495 |
with a backwards assertion. For example /\b01|\b02/ also matched "0102" |
with a backwards assertion. For example /\b01|\b02/ also matched "0102" |
496 |
twice. |
twice. |
497 |
|
|
498 |
33. Previously, PCRE did not allow quantification of assertions. However, Perl |
34. Previously, PCRE did not allow quantification of assertions. However, Perl |
499 |
does, and because of capturing effects, quantifying parenthesized |
does, and because of capturing effects, quantifying parenthesized |
500 |
assertions may at times be useful. Quantifiers are now allowed for |
assertions may at times be useful. Quantifiers are now allowed for |
501 |
parenthesized assertions. |
parenthesized assertions. |
|
|
|
|
34. A minor code tidy in pcre_compile() when checking options for \R usage. |
|
502 |
|
|
503 |
35. \g was being checked for fancy things in a character class, when it should |
35. A minor code tidy in pcre_compile() when checking options for \R usage. |
504 |
|
|
505 |
|
36. \g was being checked for fancy things in a character class, when it should |
506 |
just be a literal "g". |
just be a literal "g". |
507 |
|
|
508 |
36. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the |
37. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the |
509 |
appearance of a nested POSIX class supersedes an apparent external class. |
appearance of a nested POSIX class supersedes an apparent external class. |
510 |
For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, |
For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, |
511 |
unescaped square brackets may also appear as part of class names. For |
unescaped square brackets may also appear as part of class names. For |
512 |
example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves |
example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves |
513 |
more like Perl. |
more like Perl. (But see 8.20/1 above.) |
514 |
|
|
515 |
37. PCRE was giving an error for \N with a braced quantifier such as {1,} (this |
38. PCRE was giving an error for \N with a braced quantifier such as {1,} (this |
516 |
was because it thought it was \N{name}, which is not supported). |
was because it thought it was \N{name}, which is not supported). |
|
|
|
|
38. Add minix to OS list not supporting the -S option in pcretest. |
|
517 |
|
|
518 |
39. PCRE tries to detect cases of infinite recursion at compile time, but it |
39. Add minix to OS list not supporting the -S option in pcretest. |
519 |
|
|
520 |
|
40. PCRE tries to detect cases of infinite recursion at compile time, but it |
521 |
cannot analyze patterns in sufficient detail to catch mutual recursions |
cannot analyze patterns in sufficient detail to catch mutual recursions |
522 |
such as ((?1))((?2)). There is now a runtime test that gives an error if a |
such as ((?1))((?2)). There is now a runtime test that gives an error if a |
523 |
subgroup is called recursively as a subpattern for a second time at the |
subgroup is called recursively as a subpattern for a second time at the |
524 |
same position in the subject string. In previous releases this might have |
same position in the subject string. In previous releases this might have |
525 |
been caught by the recursion limit, or it might have run out of stack. |
been caught by the recursion limit, or it might have run out of stack. |
526 |
|
|
527 |
40. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can |
41. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can |
528 |
happen only once. PCRE was, however incorrectly giving a compile time error |
happen only once. PCRE was, however incorrectly giving a compile time error |
529 |
"recursive call could loop indefinitely" because it cannot analyze the |
"recursive call could loop indefinitely" because it cannot analyze the |
530 |
pattern in sufficient detail. The compile time test no longer happens when |
pattern in sufficient detail. The compile time test no longer happens when |
531 |
PCRE is compiling a conditional subpattern, but actual runaway loops are |
PCRE is compiling a conditional subpattern, but actual runaway loops are |
532 |
now caught at runtime (see 39 above). |
now caught at runtime (see 40 above). |
533 |
|
|
534 |
41. It seems that Perl allows any characters other than a closing parenthesis |
42. It seems that Perl allows any characters other than a closing parenthesis |
535 |
to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE |
to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE |
536 |
has been changed to be the same. |
has been changed to be the same. |
537 |
|
|
538 |
42. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so |
43. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so |
539 |
as not to get warnings when autogen.sh is called. Also changed |
as not to get warnings when autogen.sh is called. Also changed |
540 |
AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). |
AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). |
541 |
|
|
542 |
43. To help people who use pcregrep to scan files containing exceedingly long |
44. To help people who use pcregrep to scan files containing exceedingly long |
543 |
lines, the following changes have been made: |
lines, the following changes have been made: |
544 |
|
|
545 |
(a) The default value of the buffer size parameter has been increased from |
(a) The default value of the buffer size parameter has been increased from |
546 |
8K to 20K. (A buffer three times this size is actually used.) |
8K to 20K. (The actual buffer used is three times this size.) |
547 |
|
|
548 |
(b) The default can be changed by ./configure --with-pcregrep-bufsiz when |
(b) The default can be changed by ./configure --with-pcregrep-bufsize when |
549 |
PCRE is built. |
PCRE is built. |
550 |
|
|
551 |
(c) A --buffer-size=n option has been added to pcregrep, to allow the size |
(c) A --buffer-size=n option has been added to pcregrep, to allow the size |
552 |
to be set at run time. |
to be set at run time. |
553 |
|
|
554 |
(d) Numerical values in pcregrep options can be followed by K or M, for |
(d) Numerical values in pcregrep options can be followed by K or M, for |
555 |
example --buffer-size=50K. |
example --buffer-size=50K. |
556 |
|
|
557 |
(e) If a line being scanned overflows pcregrep's buffer, an error is now |
(e) If a line being scanned overflows pcregrep's buffer, an error is now |
558 |
given and the return code is set to 2. |
given and the return code is set to 2. |
559 |
|
|
560 |
44. Add a pointer to the latest mark to the callout data block. |
45. Add a pointer to the latest mark to the callout data block. |
561 |
|
|
562 |
|
46. The pattern /.(*F)/, when applied to "abc" with PCRE_PARTIAL_HARD, gave a |
563 |
|
partial match of an empty string instead of no match. This was specific to |
564 |
|
the use of ".". |
565 |
|
|
566 |
|
47. The pattern /f.*/8s, when applied to "for" with PCRE_PARTIAL_HARD, gave a |
567 |
|
complete match instead of a partial match. This bug was dependent on both |
568 |
|
the PCRE_UTF8 and PCRE_DOTALL options being set. |
569 |
|
|
570 |
|
48. For a pattern such as /\babc|\bdef/ pcre_study() was failing to set up the |
571 |
|
starting byte set, because \b was not being ignored. |
572 |
|
|
573 |
|
|
574 |
Version 8.12 15-Jan-2011 |
Version 8.12 15-Jan-2011 |