1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.10 03 May-2010 |
Version 8.20 12-Sep-2011 |
5 |
|
------------------------ |
6 |
|
|
7 |
|
1. Change 37 of 8.13 broke patterns like [:a]...[b:] because it thought it had |
8 |
|
a POSIX class. After further experiments with Perl, which convinced me that |
9 |
|
Perl has bugs and confusions, a closing square bracket is no longer allowed |
10 |
|
in a POSIX name. This bug also affected patterns with classes that started |
11 |
|
with full stops. |
12 |
|
|
13 |
|
2. If a pattern such as /(a)b|ac/ is matched against "ac", there is no captured |
14 |
|
substring, but while checking the failing first alternative, substring 1 is |
15 |
|
temporarily captured. If the output vector supplied to pcre_exec() was not |
16 |
|
big enough for this capture, the yield of the function was still zero |
17 |
|
("insufficient space for captured substrings"). This cannot be totally fixed |
18 |
|
without adding another stack variable, which seems a lot of expense for a |
19 |
|
edge case. However, I have improved the situation in cases such as |
20 |
|
/(a)(b)x|abc/ matched against "abc", where the return code indicates that |
21 |
|
fewer than the maximum number of slots in the ovector have been set. |
22 |
|
|
23 |
|
3. Related to (2) above: when there are more back references in a pattern than |
24 |
|
slots in the output vector, pcre_exec() uses temporary memory during |
25 |
|
matching, and copies in the captures as far as possible afterwards. It was |
26 |
|
using the entire output vector, but this conflicts with the specification |
27 |
|
that only 2/3 is used for passing back captured substrings. Now it uses only |
28 |
|
the first 2/3, for compatibility. This is, of course, another edge case. |
29 |
|
|
30 |
|
4. Zoltan Herczeg's just-in-time compiler support has been integrated into the |
31 |
|
main code base, and can be used by building with --enable-jit. When this is |
32 |
|
done, pcregrep automatically uses it unless --disable-pcregrep-jit or the |
33 |
|
runtime --no-jit option is given. |
34 |
|
|
35 |
|
5. When the number of matches in a pcre_dfa_exec() run exactly filled the |
36 |
|
ovector, the return from the function was zero, implying that there were |
37 |
|
other matches that did not fit. The correct "exactly full" value is now |
38 |
|
returned. |
39 |
|
|
40 |
|
6. If a subpattern that was called recursively or as a subroutine contained |
41 |
|
(*PRUNE) or any other control that caused it to give a non-standard return, |
42 |
|
invalid errors such as "Error -26 (nested recursion at the same subject |
43 |
|
position)" or even infinite loops could occur. |
44 |
|
|
45 |
|
7. If a pattern such as /a(*SKIP)c|b(*ACCEPT)|/ was studied, it stopped |
46 |
|
computing the minimum length on reaching *ACCEPT, and so ended up with the |
47 |
|
wrong value of 1 rather than 0. Further investigation indicates that |
48 |
|
computing a minimum subject length in the presence of *ACCEPT is difficult |
49 |
|
(think back references, subroutine calls), and so I have changed the code so |
50 |
|
that no minimum is registered for a pattern that contains *ACCEPT. |
51 |
|
|
52 |
|
8. If (*THEN) was present in the first (true) branch of a conditional group, |
53 |
|
it was not handled as intended. |
54 |
|
|
55 |
|
|
56 |
|
Version 8.13 16-Aug-2011 |
57 |
|
------------------------ |
58 |
|
|
59 |
|
1. The Unicode data tables have been updated to Unicode 6.0.0. |
60 |
|
|
61 |
|
2. Two minor typos in pcre_internal.h have been fixed. |
62 |
|
|
63 |
|
3. Added #include <string.h> to pcre_scanner_unittest.cc, pcrecpp.cc, and |
64 |
|
pcrecpp_unittest.cc. They are needed for strcmp(), memset(), and strchr() |
65 |
|
in some environments (e.g. Solaris 10/SPARC using Sun Studio 12U2). |
66 |
|
|
67 |
|
4. There were a number of related bugs in the code for matching backrefences |
68 |
|
caselessly in UTF-8 mode when codes for the characters concerned were |
69 |
|
different numbers of bytes. For example, U+023A and U+2C65 are an upper |
70 |
|
and lower case pair, using 2 and 3 bytes, respectively. The main bugs were: |
71 |
|
(a) A reference to 3 copies of a 2-byte code matched only 2 of a 3-byte |
72 |
|
code. (b) A reference to 2 copies of a 3-byte code would not match 2 of a |
73 |
|
2-byte code at the end of the subject (it thought there wasn't enough data |
74 |
|
left). |
75 |
|
|
76 |
|
5. Comprehensive information about what went wrong is now returned by |
77 |
|
pcre_exec() and pcre_dfa_exec() when the UTF-8 string check fails, as long |
78 |
|
as the output vector has at least 2 elements. The offset of the start of |
79 |
|
the failing character and a reason code are placed in the vector. |
80 |
|
|
81 |
|
6. When the UTF-8 string check fails for pcre_compile(), the offset that is |
82 |
|
now returned is for the first byte of the failing character, instead of the |
83 |
|
last byte inspected. This is an incompatible change, but I hope it is small |
84 |
|
enough not to be a problem. It makes the returned offset consistent with |
85 |
|
pcre_exec() and pcre_dfa_exec(). |
86 |
|
|
87 |
|
7. pcretest now gives a text phrase as well as the error number when |
88 |
|
pcre_exec() or pcre_dfa_exec() fails; if the error is a UTF-8 check |
89 |
|
failure, the offset and reason code are output. |
90 |
|
|
91 |
|
8. When \R was used with a maximizing quantifier it failed to skip backwards |
92 |
|
over a \r\n pair if the subsequent match failed. Instead, it just skipped |
93 |
|
back over a single character (\n). This seems wrong (because it treated the |
94 |
|
two characters as a single entity when going forwards), conflicts with the |
95 |
|
documentation that \R is equivalent to (?>\r\n|\n|...etc), and makes the |
96 |
|
behaviour of \R* different to (\R)*, which also seems wrong. The behaviour |
97 |
|
has been changed. |
98 |
|
|
99 |
|
9. Some internal refactoring has changed the processing so that the handling |
100 |
|
of the PCRE_CASELESS and PCRE_MULTILINE options is done entirely at compile |
101 |
|
time (the PCRE_DOTALL option was changed this way some time ago: version |
102 |
|
7.7 change 16). This has made it possible to abolish the OP_OPT op code, |
103 |
|
which was always a bit of a fudge. It also means that there is one less |
104 |
|
argument for the match() function, which reduces its stack requirements |
105 |
|
slightly. This change also fixes an incompatibility with Perl: the pattern |
106 |
|
(?i:([^b]))(?1) should not match "ab", but previously PCRE gave a match. |
107 |
|
|
108 |
|
10. More internal refactoring has drastically reduced the number of recursive |
109 |
|
calls to match() for possessively repeated groups such as (abc)++ when |
110 |
|
using pcre_exec(). |
111 |
|
|
112 |
|
11. While implementing 10, a number of bugs in the handling of groups were |
113 |
|
discovered and fixed: |
114 |
|
|
115 |
|
(?<=(a)+) was not diagnosed as invalid (non-fixed-length lookbehind). |
116 |
|
(a|)*(?1) gave a compile-time internal error. |
117 |
|
((a|)+)+ did not notice that the outer group could match an empty string. |
118 |
|
(^a|^)+ was not marked as anchored. |
119 |
|
(.*a|.*)+ was not marked as matching at start or after a newline. |
120 |
|
|
121 |
|
12. Yet more internal refactoring has removed another argument from the match() |
122 |
|
function. Special calls to this function are now indicated by setting a |
123 |
|
value in a variable in the "match data" data block. |
124 |
|
|
125 |
|
13. Be more explicit in pcre_study() instead of relying on "default" for |
126 |
|
opcodes that mean there is no starting character; this means that when new |
127 |
|
ones are added and accidentally left out of pcre_study(), testing should |
128 |
|
pick them up. |
129 |
|
|
130 |
|
14. The -s option of pcretest has been documented for ages as being an old |
131 |
|
synonym of -m (show memory usage). I have changed it to mean "force study |
132 |
|
for every regex", that is, assume /S for every regex. This is similar to -i |
133 |
|
and -d etc. It's slightly incompatible, but I'm hoping nobody is still |
134 |
|
using it. It makes it easier to run collections of tests with and without |
135 |
|
study enabled, and thereby test pcre_study() more easily. All the standard |
136 |
|
tests are now run with and without -s (but some patterns can be marked as |
137 |
|
"never study" - see 20 below). |
138 |
|
|
139 |
|
15. When (*ACCEPT) was used in a subpattern that was called recursively, the |
140 |
|
restoration of the capturing data to the outer values was not happening |
141 |
|
correctly. |
142 |
|
|
143 |
|
16. If a recursively called subpattern ended with (*ACCEPT) and matched an |
144 |
|
empty string, and PCRE_NOTEMPTY was set, pcre_exec() thought the whole |
145 |
|
pattern had matched an empty string, and so incorrectly returned a no |
146 |
|
match. |
147 |
|
|
148 |
|
17. There was optimizing code for the last branch of non-capturing parentheses, |
149 |
|
and also for the obeyed branch of a conditional subexpression, which used |
150 |
|
tail recursion to cut down on stack usage. Unfortunately, now that there is |
151 |
|
the possibility of (*THEN) occurring in these branches, tail recursion is |
152 |
|
no longer possible because the return has to be checked for (*THEN). These |
153 |
|
two optimizations have therefore been removed. |
154 |
|
|
155 |
|
18. If a pattern containing \R was studied, it was assumed that \R always |
156 |
|
matched two bytes, thus causing the minimum subject length to be |
157 |
|
incorrectly computed because \R can also match just one byte. |
158 |
|
|
159 |
|
19. If a pattern containing (*ACCEPT) was studied, the minimum subject length |
160 |
|
was incorrectly computed. |
161 |
|
|
162 |
|
20. If /S is present twice on a test pattern in pcretest input, it now |
163 |
|
*disables* studying, thereby overriding the use of -s on the command line |
164 |
|
(see 14 above). This is necessary for one or two tests to keep the output |
165 |
|
identical in both cases. |
166 |
|
|
167 |
|
21. When (*ACCEPT) was used in an assertion that matched an empty string and |
168 |
|
PCRE_NOTEMPTY was set, PCRE applied the non-empty test to the assertion. |
169 |
|
|
170 |
|
22. When an atomic group that contained a capturing parenthesis was |
171 |
|
successfully matched, but the branch in which it appeared failed, the |
172 |
|
capturing was not being forgotten if a higher numbered group was later |
173 |
|
captured. For example, /(?>(a))b|(a)c/ when matching "ac" set capturing |
174 |
|
group 1 to "a", when in fact it should be unset. This applied to multi- |
175 |
|
branched capturing and non-capturing groups, repeated or not, and also to |
176 |
|
positive assertions (capturing in negative assertions does not happen |
177 |
|
in PCRE) and also to nested atomic groups. |
178 |
|
|
179 |
|
23. Add the ++ qualifier feature to pcretest, to show the remainder of the |
180 |
|
subject after a captured substring, to make it easier to tell which of a |
181 |
|
number of identical substrings has been captured. |
182 |
|
|
183 |
|
24. The way atomic groups are processed by pcre_exec() has been changed so that |
184 |
|
if they are repeated, backtracking one repetition now resets captured |
185 |
|
values correctly. For example, if ((?>(a+)b)+aabab) is matched against |
186 |
|
"aaaabaaabaabab" the value of captured group 2 is now correctly recorded as |
187 |
|
"aaa". Previously, it would have been "a". As part of this code |
188 |
|
refactoring, the way recursive calls are handled has also been changed. |
189 |
|
|
190 |
|
25. If an assertion condition captured any substrings, they were not passed |
191 |
|
back unless some other capturing happened later. For example, if |
192 |
|
(?(?=(a))a) was matched against "a", no capturing was returned. |
193 |
|
|
194 |
|
26. When studying a pattern that contained subroutine calls or assertions, |
195 |
|
the code for finding the minimum length of a possible match was handling |
196 |
|
direct recursions such as (xxx(?1)|yyy) but not mutual recursions (where |
197 |
|
group 1 called group 2 while simultaneously a separate group 2 called group |
198 |
|
1). A stack overflow occurred in this case. I have fixed this by limiting |
199 |
|
the recursion depth to 10. |
200 |
|
|
201 |
|
27. Updated RunTest.bat in the distribution to the version supplied by Tom |
202 |
|
Fortmann. This supports explicit test numbers on the command line, and has |
203 |
|
argument validation and error reporting. |
204 |
|
|
205 |
|
28. An instance of \X with an unlimited repeat could fail if at any point the |
206 |
|
first character it looked at was a mark character. |
207 |
|
|
208 |
|
29. Some minor code refactoring concerning Unicode properties and scripts |
209 |
|
should reduce the stack requirement of match() slightly. |
210 |
|
|
211 |
|
30. Added the '=' option to pcretest to check the setting of unused capturing |
212 |
|
slots at the end of the pattern, which are documented as being -1, but are |
213 |
|
not included in the return count. |
214 |
|
|
215 |
|
31. If \k was not followed by a braced, angle-bracketed, or quoted name, PCRE |
216 |
|
compiled something random. Now it gives a compile-time error (as does |
217 |
|
Perl). |
218 |
|
|
219 |
|
32. A *MARK encountered during the processing of a positive assertion is now |
220 |
|
recorded and passed back (compatible with Perl). |
221 |
|
|
222 |
|
33. If --only-matching or --colour was set on a pcregrep call whose pattern |
223 |
|
had alternative anchored branches, the search for a second match in a line |
224 |
|
was done as if at the line start. Thus, for example, /^01|^02/ incorrectly |
225 |
|
matched the line "0102" twice. The same bug affected patterns that started |
226 |
|
with a backwards assertion. For example /\b01|\b02/ also matched "0102" |
227 |
|
twice. |
228 |
|
|
229 |
|
34. Previously, PCRE did not allow quantification of assertions. However, Perl |
230 |
|
does, and because of capturing effects, quantifying parenthesized |
231 |
|
assertions may at times be useful. Quantifiers are now allowed for |
232 |
|
parenthesized assertions. |
233 |
|
|
234 |
|
35. A minor code tidy in pcre_compile() when checking options for \R usage. |
235 |
|
|
236 |
|
36. \g was being checked for fancy things in a character class, when it should |
237 |
|
just be a literal "g". |
238 |
|
|
239 |
|
37. PCRE was rejecting [:a[:digit:]] whereas Perl was not. It seems that the |
240 |
|
appearance of a nested POSIX class supersedes an apparent external class. |
241 |
|
For example, [:a[:digit:]b:] matches "a", "b", ":", or a digit. Also, |
242 |
|
unescaped square brackets may also appear as part of class names. For |
243 |
|
example, [:a[:abc]b:] gives unknown class "[:abc]b:]". PCRE now behaves |
244 |
|
more like Perl. (But see 8.20/1 above.) |
245 |
|
|
246 |
|
38. PCRE was giving an error for \N with a braced quantifier such as {1,} (this |
247 |
|
was because it thought it was \N{name}, which is not supported). |
248 |
|
|
249 |
|
39. Add minix to OS list not supporting the -S option in pcretest. |
250 |
|
|
251 |
|
40. PCRE tries to detect cases of infinite recursion at compile time, but it |
252 |
|
cannot analyze patterns in sufficient detail to catch mutual recursions |
253 |
|
such as ((?1))((?2)). There is now a runtime test that gives an error if a |
254 |
|
subgroup is called recursively as a subpattern for a second time at the |
255 |
|
same position in the subject string. In previous releases this might have |
256 |
|
been caught by the recursion limit, or it might have run out of stack. |
257 |
|
|
258 |
|
41. A pattern such as /(?(R)a+|(?R)b)/ is quite safe, as the recursion can |
259 |
|
happen only once. PCRE was, however incorrectly giving a compile time error |
260 |
|
"recursive call could loop indefinitely" because it cannot analyze the |
261 |
|
pattern in sufficient detail. The compile time test no longer happens when |
262 |
|
PCRE is compiling a conditional subpattern, but actual runaway loops are |
263 |
|
now caught at runtime (see 40 above). |
264 |
|
|
265 |
|
42. It seems that Perl allows any characters other than a closing parenthesis |
266 |
|
to be part of the NAME in (*MARK:NAME) and other backtracking verbs. PCRE |
267 |
|
has been changed to be the same. |
268 |
|
|
269 |
|
43. Updated configure.ac to put in more quoting round AC_LANG_PROGRAM etc. so |
270 |
|
as not to get warnings when autogen.sh is called. Also changed |
271 |
|
AC_PROG_LIBTOOL (deprecated) to LT_INIT (the current macro). |
272 |
|
|
273 |
|
44. To help people who use pcregrep to scan files containing exceedingly long |
274 |
|
lines, the following changes have been made: |
275 |
|
|
276 |
|
(a) The default value of the buffer size parameter has been increased from |
277 |
|
8K to 20K. (The actual buffer used is three times this size.) |
278 |
|
|
279 |
|
(b) The default can be changed by ./configure --with-pcregrep-bufsize when |
280 |
|
PCRE is built. |
281 |
|
|
282 |
|
(c) A --buffer-size=n option has been added to pcregrep, to allow the size |
283 |
|
to be set at run time. |
284 |
|
|
285 |
|
(d) Numerical values in pcregrep options can be followed by K or M, for |
286 |
|
example --buffer-size=50K. |
287 |
|
|
288 |
|
(e) If a line being scanned overflows pcregrep's buffer, an error is now |
289 |
|
given and the return code is set to 2. |
290 |
|
|
291 |
|
45. Add a pointer to the latest mark to the callout data block. |
292 |
|
|
293 |
|
46. The pattern /.(*F)/, when applied to "abc" with PCRE_PARTIAL_HARD, gave a |
294 |
|
partial match of an empty string instead of no match. This was specific to |
295 |
|
the use of ".". |
296 |
|
|
297 |
|
47. The pattern /f.*/8s, when applied to "for" with PCRE_PARTIAL_HARD, gave a |
298 |
|
complete match instead of a partial match. This bug was dependent on both |
299 |
|
the PCRE_UTF8 and PCRE_DOTALL options being set. |
300 |
|
|
301 |
|
48. For a pattern such as /\babc|\bdef/ pcre_study() was failing to set up the |
302 |
|
starting byte set, because \b was not being ignored. |
303 |
|
|
304 |
|
|
305 |
|
Version 8.12 15-Jan-2011 |
306 |
|
------------------------ |
307 |
|
|
308 |
|
1. Fixed some typos in the markup of the man pages, and wrote a script that |
309 |
|
checks for such things as part of the documentation building process. |
310 |
|
|
311 |
|
2. On a big-endian 64-bit system, pcregrep did not correctly process the |
312 |
|
--match-limit and --recursion-limit options (added for 8.11). In |
313 |
|
particular, this made one of the standard tests fail. (The integer value |
314 |
|
went into the wrong half of a long int.) |
315 |
|
|
316 |
|
3. If the --colour option was given to pcregrep with -v (invert match), it |
317 |
|
did strange things, either producing crazy output, or crashing. It should, |
318 |
|
of course, ignore a request for colour when reporting lines that do not |
319 |
|
match. |
320 |
|
|
321 |
|
4. Another pcregrep bug caused similar problems if --colour was specified with |
322 |
|
-M (multiline) and the pattern match finished with a line ending. |
323 |
|
|
324 |
|
5. In pcregrep, when a pattern that ended with a literal newline sequence was |
325 |
|
matched in multiline mode, the following line was shown as part of the |
326 |
|
match. This seems wrong, so I have changed it. |
327 |
|
|
328 |
|
6. Another pcregrep bug in multiline mode, when --colour was specified, caused |
329 |
|
the check for further matches in the same line (so they could be coloured) |
330 |
|
to overrun the end of the current line. If another match was found, it was |
331 |
|
incorrectly shown (and then shown again when found in the next line). |
332 |
|
|
333 |
|
7. If pcregrep was compiled under Windows, there was a reference to the |
334 |
|
function pcregrep_exit() before it was defined. I am assuming this was |
335 |
|
the cause of the "error C2371: 'pcregrep_exit' : redefinition;" that was |
336 |
|
reported by a user. I've moved the definition above the reference. |
337 |
|
|
338 |
|
|
339 |
|
Version 8.11 10-Dec-2010 |
340 |
|
------------------------ |
341 |
|
|
342 |
|
1. (*THEN) was not working properly if there were untried alternatives prior |
343 |
|
to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it |
344 |
|
backtracked to try for "b" instead of moving to the next alternative branch |
345 |
|
at the same level (in this case, to look for "c"). The Perl documentation |
346 |
|
is clear that when (*THEN) is backtracked onto, it goes to the "next |
347 |
|
alternative in the innermost enclosing group". |
348 |
|
|
349 |
|
2. (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern |
350 |
|
such as (A(*COMMIT)B(*THEN)C|D) any failure after matching A should |
351 |
|
result in overall failure. Similarly, (*COMMIT) now overrides (*PRUNE) and |
352 |
|
(*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides |
353 |
|
(*THEN). |
354 |
|
|
355 |
|
3. If \s appeared in a character class, it removed the VT character from |
356 |
|
the class, even if it had been included by some previous item, for example |
357 |
|
in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part |
358 |
|
of \s, but is part of the POSIX "space" class.) |
359 |
|
|
360 |
|
4. A partial match never returns an empty string (because you can always |
361 |
|
match an empty string at the end of the subject); however the checking for |
362 |
|
an empty string was starting at the "start of match" point. This has been |
363 |
|
changed to the "earliest inspected character" point, because the returned |
364 |
|
data for a partial match starts at this character. This means that, for |
365 |
|
example, /(?<=abc)def/ gives a partial match for the subject "abc" |
366 |
|
(previously it gave "no match"). |
367 |
|
|
368 |
|
5. Changes have been made to the way PCRE_PARTIAL_HARD affects the matching |
369 |
|
of $, \z, \Z, \b, and \B. If the match point is at the end of the string, |
370 |
|
previously a full match would be given. However, setting PCRE_PARTIAL_HARD |
371 |
|
has an implication that the given string is incomplete (because a partial |
372 |
|
match is preferred over a full match). For this reason, these items now |
373 |
|
give a partial match in this situation. [Aside: previously, the one case |
374 |
|
/t\b/ matched against "cat" with PCRE_PARTIAL_HARD set did return a partial |
375 |
|
match rather than a full match, which was wrong by the old rules, but is |
376 |
|
now correct.] |
377 |
|
|
378 |
|
6. There was a bug in the handling of #-introduced comments, recognized when |
379 |
|
PCRE_EXTENDED is set, when PCRE_NEWLINE_ANY and PCRE_UTF8 were also set. |
380 |
|
If a UTF-8 multi-byte character included the byte 0x85 (e.g. +U0445, whose |
381 |
|
UTF-8 encoding is 0xd1,0x85), this was misinterpreted as a newline when |
382 |
|
scanning for the end of the comment. (*Character* 0x85 is an "any" newline, |
383 |
|
but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several |
384 |
|
places in pcre_compile(). |
385 |
|
|
386 |
|
7. Related to (6) above, when pcre_compile() was skipping #-introduced |
387 |
|
comments when looking ahead for named forward references to subpatterns, |
388 |
|
the only newline sequence it recognized was NL. It now handles newlines |
389 |
|
according to the set newline convention. |
390 |
|
|
391 |
|
8. SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the |
392 |
|
former, but used strtoul(), whereas pcretest avoided strtoul() but did not |
393 |
|
cater for a lack of strerror(). These oversights have been fixed. |
394 |
|
|
395 |
|
9. Added --match-limit and --recursion-limit to pcregrep. |
396 |
|
|
397 |
|
10. Added two casts needed to build with Visual Studio when NO_RECURSE is set. |
398 |
|
|
399 |
|
11. When the -o option was used, pcregrep was setting a return code of 1, even |
400 |
|
when matches were found, and --line-buffered was not being honoured. |
401 |
|
|
402 |
|
12. Added an optional parentheses number to the -o and --only-matching options |
403 |
|
of pcregrep. |
404 |
|
|
405 |
|
13. Imitating Perl's /g action for multiple matches is tricky when the pattern |
406 |
|
can match an empty string. The code to do it in pcretest and pcredemo |
407 |
|
needed fixing: |
408 |
|
|
409 |
|
(a) When the newline convention was "crlf", pcretest got it wrong, skipping |
410 |
|
only one byte after an empty string match just before CRLF (this case |
411 |
|
just got forgotten; "any" and "anycrlf" were OK). |
412 |
|
|
413 |
|
(b) The pcretest code also had a bug, causing it to loop forever in UTF-8 |
414 |
|
mode when an empty string match preceded an ASCII character followed by |
415 |
|
a non-ASCII character. (The code for advancing by one character rather |
416 |
|
than one byte was nonsense.) |
417 |
|
|
418 |
|
(c) The pcredemo.c sample program did not have any code at all to handle |
419 |
|
the cases when CRLF is a valid newline sequence. |
420 |
|
|
421 |
|
14. Neither pcre_exec() nor pcre_dfa_exec() was checking that the value given |
422 |
|
as a starting offset was within the subject string. There is now a new |
423 |
|
error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is |
424 |
|
negative or greater than the length of the string. In order to test this, |
425 |
|
pcretest is extended to allow the setting of negative starting offsets. |
426 |
|
|
427 |
|
15. In both pcre_exec() and pcre_dfa_exec() the code for checking that the |
428 |
|
starting offset points to the beginning of a UTF-8 character was |
429 |
|
unnecessarily clumsy. I tidied it up. |
430 |
|
|
431 |
|
16. Added PCRE_ERROR_SHORTUTF8 to make it possible to distinguish between a |
432 |
|
bad UTF-8 sequence and one that is incomplete when using PCRE_PARTIAL_HARD. |
433 |
|
|
434 |
|
17. Nobody had reported that the --include_dir option, which was added in |
435 |
|
release 7.7 should have been called --include-dir (hyphen, not underscore) |
436 |
|
for compatibility with GNU grep. I have changed it to --include-dir, but |
437 |
|
left --include_dir as an undocumented synonym, and the same for |
438 |
|
--exclude-dir, though that is not available in GNU grep, at least as of |
439 |
|
release 2.5.4. |
440 |
|
|
441 |
|
18. At a user's suggestion, the macros GETCHAR and friends (which pick up UTF-8 |
442 |
|
characters from a string of bytes) have been redefined so as not to use |
443 |
|
loops, in order to improve performance in some environments. At the same |
444 |
|
time, I abstracted some of the common code into auxiliary macros to save |
445 |
|
repetition (this should not affect the compiled code). |
446 |
|
|
447 |
|
19. If \c was followed by a multibyte UTF-8 character, bad things happened. A |
448 |
|
compile-time error is now given if \c is not followed by an ASCII |
449 |
|
character, that is, a byte less than 128. (In EBCDIC mode, the code is |
450 |
|
different, and any byte value is allowed.) |
451 |
|
|
452 |
|
20. Recognize (*NO_START_OPT) at the start of a pattern to set the PCRE_NO_ |
453 |
|
START_OPTIMIZE option, which is now allowed at compile time - but just |
454 |
|
passed through to pcre_exec() or pcre_dfa_exec(). This makes it available |
455 |
|
to pcregrep and other applications that have no direct access to PCRE |
456 |
|
options. The new /Y option in pcretest sets this option when calling |
457 |
|
pcre_compile(). |
458 |
|
|
459 |
|
21. Change 18 of release 8.01 broke the use of named subpatterns for recursive |
460 |
|
back references. Groups containing recursive back references were forced to |
461 |
|
be atomic by that change, but in the case of named groups, the amount of |
462 |
|
memory required was incorrectly computed, leading to "Failed: internal |
463 |
|
error: code overflow". This has been fixed. |
464 |
|
|
465 |
|
22. Some patches to pcre_stringpiece.h, pcre_stringpiece_unittest.cc, and |
466 |
|
pcretest.c, to avoid build problems in some Borland environments. |
467 |
|
|
468 |
|
|
469 |
|
Version 8.10 25-Jun-2010 |
470 |
------------------------ |
------------------------ |
471 |
|
|
472 |
1. Added support for (*MARK:ARG) and for ARG additions to PRUNE, SKIP, and |
1. Added support for (*MARK:ARG) and for ARG additions to PRUNE, SKIP, and |
474 |
|
|
475 |
2. (*ACCEPT) was not working when inside an atomic group. |
2. (*ACCEPT) was not working when inside an atomic group. |
476 |
|
|
477 |
3. Inside a character class, \B is treated as a literal by default, but |
3. Inside a character class, \B is treated as a literal by default, but |
478 |
faulted if PCRE_EXTRA is set. This mimics Perl's behaviour (the -w option |
faulted if PCRE_EXTRA is set. This mimics Perl's behaviour (the -w option |
479 |
causes the error). The code is unchanged, but I tidied the documentation. |
causes the error). The code is unchanged, but I tidied the documentation. |
480 |
|
|
481 |
4. Inside a character class, PCRE always treated \R and \X as literals, |
4. Inside a character class, PCRE always treated \R and \X as literals, |
482 |
whereas Perl faults them if its -w option is set. I have changed PCRE so |
whereas Perl faults them if its -w option is set. I have changed PCRE so |
483 |
that it faults them when PCRE_EXTRA is set. |
that it faults them when PCRE_EXTRA is set. |
484 |
|
|
485 |
5. Added support for \N, which always matches any character other than |
5. Added support for \N, which always matches any character other than |
486 |
newline. (It is the same as "." when PCRE_DOTALL is not set.) |
newline. (It is the same as "." when PCRE_DOTALL is not set.) |
487 |
|
|
488 |
6. When compiling pcregrep with newer versions of gcc which may have |
6. When compiling pcregrep with newer versions of gcc which may have |
489 |
FORTIFY_SOURCE set, several warnings "ignoring return value of 'fwrite', |
FORTIFY_SOURCE set, several warnings "ignoring return value of 'fwrite', |
490 |
declared with attribute warn_unused_result" were given. Just casting the |
declared with attribute warn_unused_result" were given. Just casting the |
491 |
result to (void) does not stop the warnings; a more elaborate fudge is |
result to (void) does not stop the warnings; a more elaborate fudge is |
492 |
needed. I've used a macro to implement this. |
needed. I've used a macro to implement this. |
493 |
|
|
494 |
7. Minor change to pcretest.c to avoid a compiler warning. |
7. Minor change to pcretest.c to avoid a compiler warning. |
495 |
|
|
496 |
8. Added four artifical Unicode properties to help with an option to make |
8. Added four artifical Unicode properties to help with an option to make |
497 |
\s etc use properties (see next item). The new properties are: Xan |
\s etc use properties (see next item). The new properties are: Xan |
498 |
(alphanumeric), Xsp (Perl space), Xps (POSIX space), and Xwd (word). |
(alphanumeric), Xsp (Perl space), Xps (POSIX space), and Xwd (word). |
499 |
|
|
500 |
9. Added PCRE_UCP to make \b, \d, \s, \w, and certain POSIX character classes |
9. Added PCRE_UCP to make \b, \d, \s, \w, and certain POSIX character classes |
501 |
use Unicode properties. (*UCP) at the start of a pattern can be used to set |
use Unicode properties. (*UCP) at the start of a pattern can be used to set |
502 |
this option. Modified pcretest to add /W to test this facility. Added |
this option. Modified pcretest to add /W to test this facility. Added |
503 |
REG_UCP to make it available via the POSIX interface. |
REG_UCP to make it available via the POSIX interface. |
|
|
|
|
10. Added --line-buffered to pcregrep. |
|
504 |
|
|
505 |
11. In UTF-8 mode, if a pattern that was compiled with PCRE_CASELESS was |
10. Added --line-buffered to pcregrep. |
506 |
studied, and the match started with a letter with a code point greater than |
|
507 |
|
11. In UTF-8 mode, if a pattern that was compiled with PCRE_CASELESS was |
508 |
|
studied, and the match started with a letter with a code point greater than |
509 |
127 whose first byte was different to the first byte of the other case of |
127 whose first byte was different to the first byte of the other case of |
510 |
the letter, the other case of this starting letter was not recognized. |
the letter, the other case of this starting letter was not recognized |
511 |
|
(#976). |
512 |
12. pcreposix.c included pcre.h before including pcre_internal.h. This caused a |
|
513 |
conflict in the definition of PCRE_EXP_DECL. I have removed the include of |
12. If a pattern that was studied started with a repeated Unicode property |
514 |
pcre.h as pcre_internal.h includes pcre.h itself. (This may be a bit of |
test, for example, \p{Nd}+, there was the theoretical possibility of |
515 |
historical tidying that never got done.) |
setting up an incorrect bitmap of starting bytes, but fortunately it could |
516 |
|
not have actually happened in practice until change 8 above was made (it |
517 |
|
added property types that matched character-matching opcodes). |
518 |
|
|
519 |
|
13. pcre_study() now recognizes \h, \v, and \R when constructing a bit map of |
520 |
|
possible starting bytes for non-anchored patterns. |
521 |
|
|
522 |
|
14. Extended the "auto-possessify" feature of pcre_compile(). It now recognizes |
523 |
|
\R, and also a number of cases that involve Unicode properties, both |
524 |
|
explicit and implicit when PCRE_UCP is set. |
525 |
|
|
526 |
|
15. If a repeated Unicode property match (e.g. \p{Lu}*) was used with non-UTF-8 |
527 |
|
input, it could crash or give wrong results if characters with values |
528 |
|
greater than 0xc0 were present in the subject string. (Detail: it assumed |
529 |
|
UTF-8 input when processing these items.) |
530 |
|
|
531 |
|
16. Added a lot of (int) casts to avoid compiler warnings in systems where |
532 |
|
size_t is 64-bit (#991). |
533 |
|
|
534 |
|
17. Added a check for running out of memory when PCRE is compiled with |
535 |
|
--disable-stack-for-recursion (#990). |
536 |
|
|
537 |
|
18. If the last data line in a file for pcretest does not have a newline on |
538 |
|
the end, a newline was missing in the output. |
539 |
|
|
540 |
|
19. The default pcre_chartables.c file recognizes only ASCII characters (values |
541 |
|
less than 128) in its various bitmaps. However, there is a facility for |
542 |
|
generating tables according to the current locale when PCRE is compiled. It |
543 |
|
turns out that in some environments, 0x85 and 0xa0, which are Unicode space |
544 |
|
characters, are recognized by isspace() and therefore were getting set in |
545 |
|
these tables, and indeed these tables seem to approximate to ISO 8859. This |
546 |
|
caused a problem in UTF-8 mode when pcre_study() was used to create a list |
547 |
|
of bytes that can start a match. For \s, it was including 0x85 and 0xa0, |
548 |
|
which of course cannot start UTF-8 characters. I have changed the code so |
549 |
|
that only real ASCII characters (less than 128) and the correct starting |
550 |
|
bytes for UTF-8 encodings are set for characters greater than 127 when in |
551 |
|
UTF-8 mode. (When PCRE_UCP is set - see 9 above - the code is different |
552 |
|
altogether.) |
553 |
|
|
554 |
|
20. Added the /T option to pcretest so as to be able to run tests with non- |
555 |
|
standard character tables, thus making it possible to include the tests |
556 |
|
used for 19 above in the standard set of tests. |
557 |
|
|
558 |
|
21. A pattern such as (?&t)(?#()(?(DEFINE)(?<t>a)) which has a forward |
559 |
|
reference to a subpattern the other side of a comment that contains an |
560 |
|
opening parenthesis caused either an internal compiling error, or a |
561 |
|
reference to the wrong subpattern. |
562 |
|
|
563 |
|
|
564 |
Version 8.02 19-Mar-2010 |
Version 8.02 19-Mar-2010 |