1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.13 30-Apr-2011 |
Version 8.31 |
5 |
|
----------------------------- |
6 |
|
|
7 |
|
1. Fixing a wrong JIT test case and some compiler warnings. |
8 |
|
|
9 |
|
2. Removed a bashism from the RunTest script. |
10 |
|
|
11 |
|
3. Add a cast to pcre_exec.c to fix the warning "unary minus operator applied |
12 |
|
to unsigned type, result still unsigned" that was given by an MS compiler |
13 |
|
on encountering the code "-sizeof(xxx)". |
14 |
|
|
15 |
|
4. Partial matching support is added to the JIT compiler. |
16 |
|
|
17 |
|
5. Fixed several bugs concerned with partial matching of items that consist |
18 |
|
of more than one character: |
19 |
|
|
20 |
|
(a) /^(..)\1/ did not partially match "aba" because checking references was |
21 |
|
done on an "all or nothing" basis. This also applied to repeated |
22 |
|
references. |
23 |
|
|
24 |
|
(b) \R did not give a hard partial match if \r was found at the end of the |
25 |
|
subject. |
26 |
|
|
27 |
|
(c) \X did not give a hard partial match after matching one or more |
28 |
|
characters at the end of the subject. |
29 |
|
|
30 |
|
(d) When newline was set to CRLF, a pattern such as /a$/ did not recognize |
31 |
|
a partial match for the string "\r". |
32 |
|
|
33 |
|
(e) When newline was set to CRLF, the metacharacter "." did not recognize |
34 |
|
a partial match for a CR character at the end of the subject string. |
35 |
|
|
36 |
|
6. If JIT is requested using /S++ or -s++ (instead of just /S+ or -s+) when |
37 |
|
running pcretest, the text "(JIT)" added to the output whenever JIT is |
38 |
|
actually used to run the match. |
39 |
|
|
40 |
|
7. Individual JIT compile options can be set in pcretest by following -s+[+] |
41 |
|
or /S+[+] with a digit between 1 and 7. |
42 |
|
|
43 |
|
8. OP_NOT now supports any UTF character not just single-byte ones. |
44 |
|
|
45 |
|
|
46 |
|
Version 8.30 04-February-2012 |
47 |
|
----------------------------- |
48 |
|
|
49 |
|
1. Renamed "isnumber" as "is_a_number" because in some Mac environments this |
50 |
|
name is defined in ctype.h. |
51 |
|
|
52 |
|
2. Fixed a bug in fixed-length calculation for lookbehinds that would show up |
53 |
|
only in quite long subpatterns. |
54 |
|
|
55 |
|
3. Removed the function pcre_info(), which has been obsolete and deprecated |
56 |
|
since it was replaced by pcre_fullinfo() in February 2000. |
57 |
|
|
58 |
|
4. For a non-anchored pattern, if (*SKIP) was given with a name that did not |
59 |
|
match a (*MARK), and the match failed at the start of the subject, a |
60 |
|
reference to memory before the start of the subject could occur. This bug |
61 |
|
was introduced by fix 17 of release 8.21. |
62 |
|
|
63 |
|
5. A reference to an unset group with zero minimum repetition was giving |
64 |
|
totally wrong answers (in non-JavaScript-compatibility mode). For example, |
65 |
|
/(another)?(\1?)test/ matched against "hello world test". This bug was |
66 |
|
introduced in release 8.13. |
67 |
|
|
68 |
|
6. Add support for 16-bit character strings (a large amount of work involving |
69 |
|
many changes and refactorings). |
70 |
|
|
71 |
|
7. RunGrepTest failed on msys because \r\n was replaced by whitespace when the |
72 |
|
command "pattern=`printf 'xxx\r\njkl'`" was run. The pattern is now taken |
73 |
|
from a file. |
74 |
|
|
75 |
|
8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size |
76 |
|
rounding is not applied in this particular case). |
77 |
|
|
78 |
|
9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected |
79 |
|
if they appear, or are escaped, in patterns. |
80 |
|
|
81 |
|
10. Get rid of a number of -Wunused-but-set-variable warnings. |
82 |
|
|
83 |
|
11. The pattern /(?=(*:x))(q|)/ matches an empty string, and returns the mark |
84 |
|
"x". The similar pattern /(?=(*:x))((*:y)q|)/ did not return a mark at all. |
85 |
|
Oddly, Perl behaves the same way. PCRE has been fixed so that this pattern |
86 |
|
also returns the mark "x". This bug applied to capturing parentheses, |
87 |
|
non-capturing parentheses, and atomic parentheses. It also applied to some |
88 |
|
assertions. |
89 |
|
|
90 |
|
12. Stephen Kelly's patch to CMakeLists.txt allows it to parse the version |
91 |
|
information out of configure.ac instead of relying on pcre.h.generic, which |
92 |
|
is not stored in the repository. |
93 |
|
|
94 |
|
13. Applied Dmitry V. Levin's patch for a more portable method for linking with |
95 |
|
-lreadline. |
96 |
|
|
97 |
|
14. ZH added PCRE_CONFIG_JITTARGET; added its output to pcretest -C. |
98 |
|
|
99 |
|
15. Applied Graycode's patch to put the top-level frame on the stack rather |
100 |
|
than the heap when not using the stack for recursion. This gives a |
101 |
|
performance improvement in many cases when recursion is not deep. |
102 |
|
|
103 |
|
16. Experimental code added to "pcretest -C" to output the stack frame size. |
104 |
|
|
105 |
|
|
106 |
|
Version 8.21 12-Dec-2011 |
107 |
|
------------------------ |
108 |
|
|
109 |
|
1. Updating the JIT compiler. |
110 |
|
|
111 |
|
2. JIT compiler now supports OP_NCREF, OP_RREF and OP_NRREF. New test cases |
112 |
|
are added as well. |
113 |
|
|
114 |
|
3. Fix cache-flush issue on PowerPC (It is still an experimental JIT port). |
115 |
|
PCRE_EXTRA_TABLES is not suported by JIT, and should be checked before |
116 |
|
calling _pcre_jit_exec. Some extra comments are added. |
117 |
|
|
118 |
|
4. (*MARK) settings inside atomic groups that do not contain any capturing |
119 |
|
parentheses, for example, (?>a(*:m)), were not being passed out. This bug |
120 |
|
was introduced by change 18 for 8.20. |
121 |
|
|
122 |
|
5. Supporting of \x, \U and \u in JavaScript compatibility mode based on the |
123 |
|
ECMA-262 standard. |
124 |
|
|
125 |
|
6. Lookbehinds such as (?<=a{2}b) that contained a fixed repetition were |
126 |
|
erroneously being rejected as "not fixed length" if PCRE_CASELESS was set. |
127 |
|
This bug was probably introduced by change 9 of 8.13. |
128 |
|
|
129 |
|
7. While fixing 6 above, I noticed that a number of other items were being |
130 |
|
incorrectly rejected as "not fixed length". This arose partly because newer |
131 |
|
opcodes had not been added to the fixed-length checking code. I have (a) |
132 |
|
corrected the bug and added tests for these items, and (b) arranged for an |
133 |
|
error to occur if an unknown opcode is encountered while checking for fixed |
134 |
|
length instead of just assuming "not fixed length". The items that were |
135 |
|
rejected were: (*ACCEPT), (*COMMIT), (*FAIL), (*MARK), (*PRUNE), (*SKIP), |
136 |
|
(*THEN), \h, \H, \v, \V, and single character negative classes with fixed |
137 |
|
repetitions, e.g. [^a]{3}, with and without PCRE_CASELESS. |
138 |
|
|
139 |
|
8. A possessively repeated conditional subpattern such as (?(?=c)c|d)++ was |
140 |
|
being incorrectly compiled and would have given unpredicatble results. |
141 |
|
|
142 |
|
9. A possessively repeated subpattern with minimum repeat count greater than |
143 |
|
one behaved incorrectly. For example, (A){2,}+ behaved as if it was |
144 |
|
(A)(A)++ which meant that, after a subsequent mismatch, backtracking into |
145 |
|
the first (A) could occur when it should not. |
146 |
|
|
147 |
|
10. Add a cast and remove a redundant test from the code. |
148 |
|
|
149 |
|
11. JIT should use pcre_malloc/pcre_free for allocation. |
150 |
|
|
151 |
|
12. Updated pcre-config so that it no longer shows -L/usr/lib, which seems |
152 |
|
best practice nowadays, and helps with cross-compiling. (If the exec_prefix |
153 |
|
is anything other than /usr, -L is still shown). |
154 |
|
|
155 |
|
13. In non-UTF-8 mode, \C is now supported in lookbehinds and DFA matching. |
156 |
|
|
157 |
|
14. Perl does not support \N without a following name in a [] class; PCRE now |
158 |
|
also gives an error. |
159 |
|
|
160 |
|
15. If a forward reference was repeated with an upper limit of around 2000, |
161 |
|
it caused the error "internal error: overran compiling workspace". The |
162 |
|
maximum number of forward references (including repeats) was limited by the |
163 |
|
internal workspace, and dependent on the LINK_SIZE. The code has been |
164 |
|
rewritten so that the workspace expands (via pcre_malloc) if necessary, and |
165 |
|
the default depends on LINK_SIZE. There is a new upper limit (for safety) |
166 |
|
of around 200,000 forward references. While doing this, I also speeded up |
167 |
|
the filling in of repeated forward references. |
168 |
|
|
169 |
|
16. A repeated forward reference in a pattern such as (a)(?2){2}(.) was |
170 |
|
incorrectly expecting the subject to contain another "a" after the start. |
171 |
|
|
172 |
|
17. When (*SKIP:name) is activated without a corresponding (*MARK:name) earlier |
173 |
|
in the match, the SKIP should be ignored. This was not happening; instead |
174 |
|
the SKIP was being treated as NOMATCH. For patterns such as |
175 |
|
/A(*MARK:A)A+(*SKIP:B)Z|AAC/ this meant that the AAC branch was never |
176 |
|
tested. |
177 |
|
|
178 |
|
18. The behaviour of (*MARK), (*PRUNE), and (*THEN) has been reworked and is |
179 |
|
now much more compatible with Perl, in particular in cases where the result |
180 |
|
is a non-match for a non-anchored pattern. For example, if |
181 |
|
/b(*:m)f|a(*:n)w/ is matched against "abc", the non-match returns the name |
182 |
|
"m", where previously it did not return a name. A side effect of this |
183 |
|
change is that for partial matches, the last encountered mark name is |
184 |
|
returned, as for non matches. A number of tests that were previously not |
185 |
|
Perl-compatible have been moved into the Perl-compatible test files. The |
186 |
|
refactoring has had the pleasing side effect of removing one argument from |
187 |
|
the match() function, thus reducing its stack requirements. |
188 |
|
|
189 |
|
19. If the /S+ option was used in pcretest to study a pattern using JIT, |
190 |
|
subsequent uses of /S (without +) incorrectly behaved like /S+. |
191 |
|
|
192 |
|
21. Retrieve executable code size support for the JIT compiler and fixing |
193 |
|
some warnings. |
194 |
|
|
195 |
|
22. A caseless match of a UTF-8 character whose other case uses fewer bytes did |
196 |
|
not work when the shorter character appeared right at the end of the |
197 |
|
subject string. |
198 |
|
|
199 |
|
23. Added some (int) casts to non-JIT modules to reduce warnings on 64-bit |
200 |
|
systems. |
201 |
|
|
202 |
|
24. Added PCRE_INFO_JITSIZE to pass on the value from (21) above, and also |
203 |
|
output it when the /M option is used in pcretest. |
204 |
|
|
205 |
|
25. The CheckMan script was not being included in the distribution. Also, added |
206 |
|
an explicit "perl" to run Perl scripts from the PrepareRelease script |
207 |
|
because this is reportedly needed in Windows. |
208 |
|
|
209 |
|
26. If study data was being save in a file and studying had not found a set of |
210 |
|
"starts with" bytes for the pattern, the data written to the file (though |
211 |
|
never used) was taken from uninitialized memory and so caused valgrind to |
212 |
|
complain. |
213 |
|
|
214 |
|
27. Updated RunTest.bat as provided by Sheri Pierce. |
215 |
|
|
216 |
|
28. Fixed a possible uninitialized memory bug in pcre_jit_compile.c. |
217 |
|
|
218 |
|
29. Computation of memory usage for the table of capturing group names was |
219 |
|
giving an unnecessarily large value. |
220 |
|
|
221 |
|
|
222 |
|
Version 8.20 21-Oct-2011 |
223 |
|
------------------------ |
224 |
|
|
225 |
|
1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had |
226 |
|
a POSIX class. After further experiments with Perl, which convinced me that |
227 |
|
Perl has bugs and confusions, a closing square bracket is no longer allowed |
228 |
|
in a POSIX name. This bug also affected patterns with classes that started |
229 |
|
with full stops. |
230 |
|
|
231 |
|
2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no |
232 |
|
captured substring, but while checking the failing first alternative, |
233 |
|
substring 1 is temporarily captured. If the output vector supplied to |
234 |
|
pcre_exec() was not big enough for this capture, the yield of the function |
235 |
|
was still zero ("insufficient space for captured substrings"). This cannot |
236 |
|
be totally fixed without adding another stack variable, which seems a lot |
237 |
|
of expense for a edge case. However, I have improved the situation in cases |
238 |
|
such as /(a)(b)x|abc/ matched against "abc", where the return code |
239 |
|
indicates that fewer than the maximum number of slots in the ovector have |
240 |
|
been set. |
241 |
|
|
242 |
|
3. Related to (2) above: when there are more back references in a pattern than |
243 |
|
slots in the output vector, pcre_exec() uses temporary memory during |
244 |
|
matching, and copies in the captures as far as possible afterwards. It was |
245 |
|
using the entire output vector, but this conflicts with the specification |
246 |
|
that only 2/3 is used for passing back captured substrings. Now it uses |
247 |
|
only the first 2/3, for compatibility. This is, of course, another edge |
248 |
|
case. |
249 |
|
|
250 |
|
4. Zoltan Herczeg's just-in-time compiler support has been integrated into the |
251 |
|
main code base, and can be used by building with --enable-jit. When this is |
252 |
|
done, pcregrep automatically uses it unless --disable-pcregrep-jit or the |
253 |
|
runtime --no-jit option is given. |
254 |
|
|
255 |
|
5. When the number of matches in a pcre_dfa_exec() run exactly filled the |
256 |
|
ovector, the return from the function was zero, implying that there were |
257 |
|
other matches that did not fit. The correct "exactly full" value is now |
258 |
|
returned. |
259 |
|
|
260 |
|
6. If a subpattern that was called recursively or as a subroutine contained |
261 |
|
(*PRUNE) or any other control that caused it to give a non-standard return, |
262 |
|
invalid errors such as "Error -26 (nested recursion at the same subject |
263 |
|
position)" or even infinite loops could occur. |
264 |
|
|
265 |
|
7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped |
266 |
|
computing the minimum length on reaching *ACCEPT, and so ended up with the |
267 |
|
wrong value of 1 rather than 0. Further investigation indicates that |
268 |
|
computing a minimum subject length in the presence of *ACCEPT is difficult |
269 |
|
(think back references, subroutine calls), and so I have changed the code |
270 |
|
so that no minimum is registered for a pattern that contains *ACCEPT. |
271 |
|
|
272 |
|
8. If (*THEN) was present in the first (true) branch of a conditional group, |
273 |
|
it was not handled as intended. [But see 16 below.] |
274 |
|
|
275 |
|
9. Replaced RunTest.bat and CMakeLists.txt with improved versions provided by |
276 |
|
Sheri Pierce. |
277 |
|
|
278 |
|
10. A pathological pattern such as /(*ACCEPT)a/ was miscompiled, thinking that |
279 |
|
the first byte in a match must be "a". |
280 |
|
|
281 |
|
11. Change 17 for 8.13 increased the recursion depth for patterns like |
282 |
|
/a(?:.)*?a/ drastically. I've improved things by remembering whether a |
283 |
|
pattern contains any instances of (*THEN). If it does not, the old |
284 |
|
optimizations are restored. It would be nice to do this on a per-group |
285 |
|
basis, but at the moment that is not feasible. |
286 |
|
|
287 |
|
12. In some environments, the output of pcretest -C is CRLF terminated. This |
288 |
|
broke RunTest's code that checks for the link size. A single white space |
289 |
|
character after the value is now allowed for. |
290 |
|
|
291 |
|
13. RunTest now checks for the "fr" locale as well as for "fr_FR" and "french". |
292 |
|
For "fr", it uses the Windows-specific input and output files. |
293 |
|
|
294 |
|
14. If (*THEN) appeared in a group that was called recursively or as a |
295 |
|
subroutine, it did not work as intended. [But see next item.] |
296 |
|
|
297 |
|
15. Consider the pattern /A (B(*THEN)C) | D/ where A, B, C, and D are complex |
298 |
|
pattern fragments (but not containing any | characters). If A and B are |
299 |
|
matched, but there is a failure in C so that it backtracks to (*THEN), PCRE |
300 |
|
was behaving differently to Perl. PCRE backtracked into A, but Perl goes to |
301 |
|
D. In other words, Perl considers parentheses that do not contain any | |
302 |
|
characters to be part of a surrounding alternative, whereas PCRE was |
303 |
|
treading (B(*THEN)C) the same as (B(*THEN)C|(*FAIL)) -- which Perl handles |
304 |
|
differently. PCRE now behaves in the same way as Perl, except in the case |
305 |
|
of subroutine/recursion calls such as (?1) which have in any case always |
306 |
|
been different (but PCRE had them first :-). |
307 |
|
|
308 |
|
16. Related to 15 above: Perl does not treat the | in a conditional group as |
309 |
|
creating alternatives. Such a group is treated in the same way as an |
310 |
|
ordinary group without any | characters when processing (*THEN). PCRE has |
311 |
|
been changed to match Perl's behaviour. |
312 |
|
|
313 |
|
17. If a user had set PCREGREP_COLO(U)R to something other than 1:31, the |
314 |
|
RunGrepTest script failed. |
315 |
|
|
316 |
|
18. Change 22 for version 13 caused atomic groups to use more stack. This is |
317 |
|
inevitable for groups that contain captures, but it can lead to a lot of |
318 |
|
stack use in large patterns. The old behaviour has been restored for atomic |
319 |
|
groups that do not contain any capturing parentheses. |
320 |
|
|
321 |
|
19. If the PCRE_NO_START_OPTIMIZE option was set for pcre_compile(), it did not |
322 |
|
suppress the check for a minimum subject length at run time. (If it was |
323 |
|
given to pcre_exec() or pcre_dfa_exec() it did work.) |
324 |
|
|
325 |
|
20. Fixed an ASCII-dependent infelicity in pcretest that would have made it |
326 |
|
fail to work when decoding hex characters in data strings in EBCDIC |
327 |
|
environments. |
328 |
|
|
329 |
|
21. It appears that in at least one Mac OS environment, the isxdigit() function |
330 |
|
is implemented as a macro that evaluates to its argument more than once, |
331 |
|
contravening the C 90 Standard (I haven't checked a later standard). There |
332 |
|
was an instance in pcretest which caused it to go wrong when processing |
333 |
|
\x{...} escapes in subject strings. The has been rewritten to avoid using |
334 |
|
things like p++ in the argument of isxdigit(). |
335 |
|
|
336 |
|
|
337 |
|
Version 8.13 16-Aug-2011 |
338 |
------------------------ |
------------------------ |
339 |
|
|
340 |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
1. The Unicode data tables have been updated to Unicode 6.0.0. |
353 |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
354 |
2-byte code at the end of the subject (it thought there wasn't enough data |
2-byte code at the end of the subject (it thought there wasn't enough data |
355 |
left). |
left). |
356 |
|
|
357 |
5. Comprehensive information about what went wrong is now returned by |
5. Comprehensive information about what went wrong is now returned by |
358 |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
359 |
as the output vector has at least 2 elements. The offset of the start of |
as the output vector has at least 2 elements. The offset of the start of |
360 |
the failing character and a reason code are placed in the vector. |
the failing character and a reason code are placed in the vector. |
361 |
|
|
362 |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
363 |
now returned is for the first byte of the failing character, instead of the |
now returned is for the first byte of the failing character, instead of the |
364 |
last byte inspected. This is an incompatible change, but I hope it is small |
last byte inspected. This is an incompatible change, but I hope it is small |
365 |
enough not to be a problem. It makes the returned offset consistent with |
enough not to be a problem. It makes the returned offset consistent with |
366 |
pcre_exec() and pcre_dfa_exec(). |
pcre_exec() and pcre_dfa_exec(). |
367 |
|
|
368 |
7. pcretest now gives a text phrase as well as the error number when |
7. pcretest now gives a text phrase as well as the error number when |
369 |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
370 |
failure, the offset and reason code are output. |
failure, the offset and reason code are output. |
371 |
|
|
372 |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
8. When \R was used with a maximizing quantifier it failed to skip backwards |
373 |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
374 |
back over a single character (\n). This seems wrong (because it treated the |
back over a single character (\n). This seems wrong (because it treated the |
375 |
two characters as a single entity when going forwards), conflicts with the |
two characters as a single entity when going forwards), conflicts with the |
376 |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
377 |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
378 |
has been changed. |
has been changed. |
379 |
|
|
380 |
9. Some internal refactoring has changed the processing so that the handling |
9. Some internal refactoring has changed the processing so that the handling |
381 |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
382 |
time (the PCRE_DOTALL option was changed this way some time ago: version |
time (the PCRE_DOTALL option was changed this way some time ago: version |
383 |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
384 |
which was always a bit of a fudge. It also means that there is one less |
which was always a bit of a fudge. It also means that there is one less |
385 |
argument for the match() function, which reduces its stack requirements |
argument for the match() function, which reduces its stack requirements |
386 |
slightly. This change also fixes an incompatibility with Perl: the pattern |
slightly. This change also fixes an incompatibility with Perl: the pattern |
387 |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
388 |
|
|
389 |
10. More internal refactoring has drastically reduced the number of recursive |
10. More internal refactoring has drastically reduced the number of recursive |
390 |
calls to match() for possessively repeated groups such as (abc)++ when |
calls to match() for possessively repeated groups such as (abc)++ when |
391 |
using pcre_exec(). |
using pcre_exec(). |
392 |
|
|
393 |
11. While implementing 10, a number of bugs in the handling of groups were |
11. While implementing 10, a number of bugs in the handling of groups were |
394 |
discovered and fixed: |
discovered and fixed: |
395 |
|
|
396 |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
397 |
(a|)*(?1) gave a compile-time internal error. |
(a|)*(?1) gave a compile-time internal error. |
398 |
((a|)+)+ did not notice that the outer group could match an empty string. |
((a|)+)+ did not notice that the outer group could match an empty string. |
399 |
(^a|^)+ was not marked as anchored. |
(^a|^)+ was not marked as anchored. |
400 |
(.*a|.*)+ was not marked as matching at start or after a newline. |
(.*a|.*)+ was not marked as matching at start or after a newline. |
401 |
|
|
402 |
12. Yet more internal refactoring has removed another argument from the match() |
12. Yet more internal refactoring has removed another argument from the match() |
403 |
function. Special calls to this function are now indicated by setting a |
function. Special calls to this function are now indicated by setting a |
404 |
value in a variable in the "match data" data block. |
value in a variable in the "match data" data block. |
405 |
|
|
406 |
13. Be more explicit in pcre_study() instead of relying on "default" for |
13. Be more explicit in pcre_study() instead of relying on "default" for |
407 |
opcodes that mean there is no starting character; this means that when new |
opcodes that mean there is no starting character; this means that when new |
408 |
ones are added and accidentally left out of pcre_study(), testing should |
ones are added and accidentally left out of pcre_study(), testing should |
409 |
pick them up. |
pick them up. |
410 |
|
|
411 |
14. The -s option of pcretest has been documented for ages as being an old |
14. The -s option of pcretest has been documented for ages as being an old |
412 |
synonym of -m (show memory usage). I have changed it to mean "force study |
synonym of -m (show memory usage). I have changed it to mean "force study |
413 |
for every regex", that is, assume /S for every regex. This is similar to -i |
for every regex", that is, assume /S for every regex. This is similar to -i |
414 |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
415 |
using it. It makes it easier to run collections of tests with and without |
using it. It makes it easier to run collections of tests with and without |
416 |
study enabled, and thereby test pcre_study() more easily. All the standard |
study enabled, and thereby test pcre_study() more easily. All the standard |
417 |
tests are now run with and without -s (but some patterns can be marked as |
tests are now run with and without -s (but some patterns can be marked as |
418 |
"never study" - see 20 below). |
"never study" - see 20 below). |
419 |
|
|
420 |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
421 |
restoration of the capturing data to the outer values was not happening |
restoration of the capturing data to the outer values was not happening |
422 |
correctly. |
correctly. |
423 |
|
|
424 |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
425 |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
426 |
pattern had matched an empty string, and so incorrectly returned a no |
pattern had matched an empty string, and so incorrectly returned a no |
427 |
match. |
match. |
428 |
|
|
429 |
17. There was optimizing code for the last branch of non-capturing parentheses, |
17. There was optimizing code for the last branch of non-capturing parentheses, |
430 |
and also for the obeyed branch of a conditional subexpression, which used |
and also for the obeyed branch of a conditional subexpression, which used |
431 |
tail recursion to cut down on stack usage. Unfortunately, not that there is |
tail recursion to cut down on stack usage. Unfortunately, now that there is |
432 |
the possibility of (*THEN) occurring in these branches, tail recursion is |
the possibility of (*THEN) occurring in these branches, tail recursion is |
433 |
no longer possible because the return has to be checked for (*THEN). These |
no longer possible because the return has to be checked for (*THEN). These |
434 |
two optimizations have therefore been removed. |
two optimizations have therefore been removed. [But see 8.20/11 above.] |
435 |
|
|
436 |
18. If a pattern containing \R was studied, it was assumed that \R always |
18. If a pattern containing \R was studied, it was assumed that \R always |
437 |
matched two bytes, thus causing the minimum subject length to be |
matched two bytes, thus causing the minimum subject length to be |
438 |
incorrectly computed because \R can also match just one byte. |
incorrectly computed because \R can also match just one byte. |
439 |
|
|
440 |
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
441 |
was incorrectly computed. |
was incorrectly computed. |
442 |
|
|
443 |
20. If /S is present twice on a test pattern in pcretest input, it *disables* |
20. If /S is present twice on a test pattern in pcretest input, it now |
444 |
studying, thereby overriding the use of -s on the command line. This is |
*disables* studying, thereby overriding the use of -s on the command line |
445 |
necessary for one or two tests to keep the output identical in both cases. |
(see 14 above). This is necessary for one or two tests to keep the output |
446 |
|
identical in both cases. |
447 |
|
|
448 |
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
449 |
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
450 |
|
|
451 |
22. When an atomic group that contained a capturing parenthesis was |
22. When an atomic group that contained a capturing parenthesis was |
452 |
successfully matched, but the branch in which it appeared failed, the |
successfully matched, but the branch in which it appeared failed, the |
453 |
capturing was not being forgotten if a higher numbered group was later |
capturing was not being forgotten if a higher numbered group was later |
454 |
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
455 |
group 1 to "a", when in fact it should be unset. This applied to multi- |
group 1 to "a", when in fact it should be unset. This applied to multi- |
456 |
branched capturing and non-capturing groups, repeated or not, and also to |
branched capturing and non-capturing groups, repeated or not, and also to |
457 |
positive assertions (capturing in negative assertions is not well defined |
positive assertions (capturing in negative assertions does not happen |
458 |
in PCRE) and also to nested atomic groups. |
in PCRE) and also to nested atomic groups. |
459 |
|
|
460 |
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
461 |
subject after a captured substring (to make it easier to tell which of a |
subject after a captured substring, to make it easier to tell which of a |
462 |
number of identical substrings has been captured). |
number of identical substrings has been captured. |
463 |
|
|
464 |
24. The way atomic groups are processed by pcre_exec() has been changed so that |
24. The way atomic groups are processed by pcre_exec() has been changed so that |
465 |
if they are repeated, backtracking one repetition now resets captured |
if they are repeated, backtracking one repetition now resets captured |
466 |
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
467 |
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
468 |
"aaa". Previously, it would have been "a". As part of this code |
"aaa". Previously, it would have been "a". As part of this code |
469 |
refactoring, the way recursive calls are handled has also been changed. |
refactoring, the way recursive calls are handled has also been changed. |
470 |
|
|
471 |
24. If an assertion condition captured any substrings, they were not passed |
25. If an assertion condition captured any substrings, they were not passed |
472 |
back unless some other capturing happened later. For example, if |
back unless some other capturing happened later. For example, if |
473 |
(?(?=(a))a) was matched against "a", no capturing was returned. |
(?(?=(a))a) was matched against "a", no capturing was returned. |
474 |
|
|
475 |
25. When studying a pattern that contained subroutine calls or assertions, |
26. When studying a pattern that contained subroutine calls or assertions, |
476 |
the code for finding the minimum length of a possible match was handling |
the code for finding the minimum length of a possible match was handling |
477 |
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
478 |
group 1 called group 2 while simultaneously a separate group 2 called group |
group 1 called group 2 while simultaneously a separate group 2 called group |
479 |
1). A stack overflow occurred in this case. I have fixed this by limiting |
1). A stack overflow occurred in this case. I have fixed this by limiting |
480 |
the recursion depth to 10. |
the recursion depth to 10. |
481 |
|
|
482 |
26. Updated RunTest.bat in the distribution to the version supplied by Tom |
27. Updated RunTest.bat in the distribution to the version supplied by Tom |
483 |
Fortmann. This supports explicit test numbers on the command line, and has |
Fortmann. This supports explicit test numbers on the command line, and has |
484 |
argument validation and error reporting. |
argument validation and error reporting. |
485 |
|
|
486 |
27. An instance of \X with an unlimited repeat could fail if at any point the |
28. An instance of \X with an unlimited repeat could fail if at any point the |
487 |
first character it looked at was a mark character. |
first character it looked at was a mark character. |
488 |
|
|
489 |
28. Some minor code refactoring concerning Unicode properties and scripts |
29. Some minor code refactoring concerning Unicode properties and scripts |
490 |
should reduce the stack requirement of match() slightly. |
should reduce the stack requirement of match() slightly. |
491 |
|
|
492 |
29. Added the '=' option to pcretest to check the setting of unused capturing |
30. Added the '=' option to pcretest to check the setting of unused capturing |
493 |
slots at the end of the pattern, which are documented as being -1, but are |
slots at the end of the pattern, which are documented as being -1, but are |
494 |
not included in the return count. |
not included in the return count. |
495 |
|
|
496 |
|
31. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
497 |
|
compiled something random. Now it gives a compile-time error (as does |
498 |
|
Perl). |
499 |
|
|
500 |
|
32. A *MARK encountered during the processing of a positive assertion is now |
501 |
|
recorded and passed back (compatible with Perl). |
502 |
|
|
503 |
|
33. If --only-matching or --colour was set on a pcregrep call whose pattern |
504 |
|
had alternative anchored branches, the search for a second match in a line |
505 |
|
was done as if at the line start. Thus, for example, /^01|^02/ incorrectly |
506 |
|
matched the line "0102" twice. The same bug affected patterns that started |
507 |
|
with a backwards assertion. For example /\b01|\b02/ also matched "0102" |
508 |
|
twice. |
509 |
|
|
510 |
|
34. Previously, PCRE did not allow quantification of assertions. However, Perl |
511 |
|
does, and because of capturing effects, quantifying parenthesized |
512 |
|
assertions may at times be useful. Quantifiers are now allowed for |
513 |
|
parenthesized assertions. |
514 |
|
|
515 |
|
35. A minor code tidy in pcre_compile() when checking options for \R usage. |
516 |
|
|
517 |
|
36. \g was being checked for fancy things in a character class, when it should |
518 |
|
just be a literal "g". |
519 |
|
|
520 |
|
37. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the |
521 |
|
appearance of a nested POSIX class supersedes an apparent external class. |
522 |
|
For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, |
523 |
|
unescaped square brackets may also appear as part of class names. For |
524 |
|
example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves |
525 |
|
more like Perl. (But see 8.20/1 above.) |
526 |
|
|
527 |
|
38. PCRE was giving an error for \N with a braced quantifier such as {1,} (this |
528 |
|
was because it thought it was \N{name}, which is not supported). |
529 |
|
|
530 |
|
39. Add minix to OS list not supporting the -S option in pcretest. |
531 |
|
|
532 |
|
40. PCRE tries to detect cases of infinite recursion at compile time, but it |
533 |
|
cannot analyze patterns in sufficient detail to catch mutual recursions |
534 |
|
such as ((?1))((?2)). There is now a runtime test that gives an error if a |
535 |
|
subgroup is called recursively as a subpattern for a second time at the |
536 |
|
same position in the subject string. In previous releases this might have |
537 |
|
been caught by the recursion limit, or it might have run out of stack. |
538 |
|
|
539 |
|
41. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can |
540 |
|
happen only once. PCRE was, however incorrectly giving a compile time error |
541 |
|
"recursive call could loop indefinitely" because it cannot analyze the |
542 |
|
pattern in sufficient detail. The compile time test no longer happens when |
543 |
|
PCRE is compiling a conditional subpattern, but actual runaway loops are |
544 |
|
now caught at runtime (see 40 above). |
545 |
|
|
546 |
|
42. It seems that Perl allows any characters other than a closing parenthesis |
547 |
|
to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE |
548 |
|
has been changed to be the same. |
549 |
|
|
550 |
|
43. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so |
551 |
|
as not to get warnings when autogen.sh is called. Also changed |
552 |
|
AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). |
553 |
|
|
554 |
|
44. To help people who use pcregrep to scan files containing exceedingly long |
555 |
|
lines, the following changes have been made: |
556 |
|
|
557 |
|
(a) The default value of the buffer size parameter has been increased from |
558 |
|
8K to 20K. (The actual buffer used is three times this size.) |
559 |
|
|
560 |
|
(b) The default can be changed by ./configure --with-pcregrep-bufsize when |
561 |
|
PCRE is built. |
562 |
|
|
563 |
|
(c) A --buffer-size=n option has been added to pcregrep, to allow the size |
564 |
|
to be set at run time. |
565 |
|
|
566 |
|
(d) Numerical values in pcregrep options can be followed by K or M, for |
567 |
|
example --buffer-size=50K. |
568 |
|
|
569 |
|
(e) If a line being scanned overflows pcregrep's buffer, an error is now |
570 |
|
given and the return code is set to 2. |
571 |
|
|
572 |
|
45. Add a pointer to the latest mark to the callout data block. |
573 |
|
|
574 |
|
46. The pattern /.(*F)/, when applied to "abc" with PCRE_PARTIAL_HARD, gave a |
575 |
|
partial match of an empty string instead of no match. This was specific to |
576 |
|
the use of ".". |
577 |
|
|
578 |
|
47. The pattern /f.*/8s, when applied to "for" with PCRE_PARTIAL_HARD, gave a |
579 |
|
complete match instead of a partial match. This bug was dependent on both |
580 |
|
the PCRE_UTF8 and PCRE_DOTALL options being set. |
581 |
|
|
582 |
|
48. For a pattern such as /\babc|\bdef/ pcre_study() was failing to set up the |
583 |
|
starting byte set, because \b was not being ignored. |
584 |
|
|
585 |
|
|
586 |
Version 8.12 15-Jan-2011 |
Version 8.12 15-Jan-2011 |