1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 8.30 |
Version 8.32 |
5 |
------------ |
------------ |
6 |
|
|
7 |
|
1. Improved JIT compiler optimizations for first character search and single |
8 |
|
character iterators. |
9 |
|
|
10 |
|
2. Supporting IBM XL C compilers for PPC architectures in the JIT compiler. |
11 |
|
Patch by Daniel Richard G. |
12 |
|
|
13 |
|
3. Single character iterator optimizations in the JIT compiler. |
14 |
|
|
15 |
|
4. Improved JIT compiler optimizations for character ranges. |
16 |
|
|
17 |
|
5. Rename the "leave" variable names to "quit" to improve WinCE compatibility. |
18 |
|
Reported by Giuseppe D'Angelo. |
19 |
|
|
20 |
|
6. The PCRE_STARTLINE bit, indicating that a match can occur only at the start |
21 |
|
of a line, was being set incorrectly in cases where .* appeared inside |
22 |
|
atomic brackets at the start of a pattern, or where there was a subsequent |
23 |
|
*PRUNE or *SKIP. |
24 |
|
|
25 |
|
7. Improved instruction cache flush for POWER/PowerPC. |
26 |
|
Patch by Daniel Richard G. |
27 |
|
|
28 |
|
8. Fixed a number of issues in pcregrep, making it more compatible with GNU |
29 |
|
grep: |
30 |
|
|
31 |
|
(a) There is now no limit to the number of patterns to be matched. |
32 |
|
|
33 |
|
(b) An error is given if a pattern is too long. |
34 |
|
|
35 |
|
(c) Multiple uses of --exclude, --exclude-dir, --include, and --include-dir |
36 |
|
are now supported. |
37 |
|
|
38 |
|
(d) --exclude-from and --include-from (multiple use) have been added. |
39 |
|
|
40 |
|
(e) Exclusions and inclusions now apply to all files and directories, not |
41 |
|
just to those obtained from scanning a directory recursively. |
42 |
|
|
43 |
|
(f) Multiple uses of -f and --file-list are now supported. |
44 |
|
|
45 |
|
(g) In a Windows environment, the default for -d has been changed from |
46 |
|
"read" (the GNU grep default) to "skip", because otherwise the presence |
47 |
|
of a directory in the file list provokes an error. |
48 |
|
|
49 |
|
(h) The documentation has been revised and clarified in places. |
50 |
|
|
51 |
|
9. Improve the matching speed of capturing brackets. |
52 |
|
|
53 |
|
10. Changed the meaning of \X so that it now matches a Unicode extended |
54 |
|
grapheme cluster. |
55 |
|
|
56 |
|
11. Patch by Daniel Richard G to the autoconf files to add a macro for sorting |
57 |
|
out POSIX threads when JIT support is configured. |
58 |
|
|
59 |
|
12. Added support for PCRE_STUDY_EXTRA_NEEDED. |
60 |
|
|
61 |
|
13. In the POSIX wrapper regcomp() function, setting re_nsub field in the preg |
62 |
|
structure could go wrong in environments where size_t is not the same size |
63 |
|
as int. |
64 |
|
|
65 |
|
14. Applied user-supplied patch to pcrecpp.cc to allow PCRE_NO_UTF8_CHECK to be |
66 |
|
set. |
67 |
|
|
68 |
|
15. The EBCDIC support had decayed; later updates to the code had included |
69 |
|
explicit references to (e.g.) \x0a instead of CHAR_LF. There has been a |
70 |
|
general tidy up of EBCDIC-related issues, and the documentation was also |
71 |
|
not quite right. There is now a test that can be run on ASCII systems to |
72 |
|
check some of the EBCDIC-related things (but is it not a full test). |
73 |
|
|
74 |
|
16. The new PCRE_STUDY_EXTRA_NEEDED option is now used by pcregrep, resulting |
75 |
|
in a small tidy to the code. |
76 |
|
|
77 |
|
17. Fix JIT tests when UTF is disabled and both 8 and 16 bit mode are enabled. |
78 |
|
|
79 |
|
18. If the --only-matching (-o) option in pcregrep is specified multiple |
80 |
|
times, each one causes appropriate output. For example, -o1 -o2 outputs the |
81 |
|
substrings matched by the 1st and 2nd capturing parentheses. A separating |
82 |
|
string can be specified by --om-separator (default empty). |
83 |
|
|
84 |
|
19. Improving the first n character searches. |
85 |
|
|
86 |
|
20. Turn case lists for horizontal and vertical white space into macros so that |
87 |
|
they are defined only once. |
88 |
|
|
89 |
|
21. This set of changes together give more compatible Unicode case-folding |
90 |
|
behaviour for characters that have more than one other case when UCP |
91 |
|
support is available. |
92 |
|
|
93 |
|
(a) The Unicode property table now has offsets into a new table of sets of |
94 |
|
three or more characters that are case-equivalent. The MultiStage2.py |
95 |
|
script that generates these tables (the pcre_ucd.c file) now scans |
96 |
|
CaseFolding.txt instead of UnicodeData.txt for character case |
97 |
|
information. |
98 |
|
|
99 |
|
(b) The code for adding characters or ranges of characters to a character |
100 |
|
class has been abstracted into a generalized function that also handles |
101 |
|
case-independence. In UTF-mode with UCP support, this uses the new data |
102 |
|
to handle characters with more than one other case. |
103 |
|
|
104 |
|
(c) A bug that is fixed as a result of (b) is that codepoints less than 256 |
105 |
|
whose other case is greater than 256 are now correctly matched |
106 |
|
caselessly. Previously, the high codepoint matched the low one, but not |
107 |
|
vice versa. |
108 |
|
|
109 |
|
(d) The processing of \h, \H, \v, and \ in character classes now makes use |
110 |
|
of the new class addition function, using character lists defined as |
111 |
|
macros alongside the case definitions of 20 above. |
112 |
|
|
113 |
|
(e) Caseless back references now work with characters that have more than |
114 |
|
one other case. |
115 |
|
|
116 |
|
(f) General caseless matching of characters with more than one other case |
117 |
|
is supported. |
118 |
|
|
119 |
|
22. Unicode character properties were updated from Unicode 6.2.0 |
120 |
|
|
121 |
|
Version 8.31 06-July-2012 |
122 |
|
------------------------- |
123 |
|
|
124 |
|
1. Fixing a wrong JIT test case and some compiler warnings. |
125 |
|
|
126 |
|
2. Removed a bashism from the RunTest script. |
127 |
|
|
128 |
|
3. Add a cast to pcre_exec.c to fix the warning "unary minus operator applied |
129 |
|
to unsigned type, result still unsigned" that was given by an MS compiler |
130 |
|
on encountering the code "-sizeof(xxx)". |
131 |
|
|
132 |
|
4. Partial matching support is added to the JIT compiler. |
133 |
|
|
134 |
|
5. Fixed several bugs concerned with partial matching of items that consist |
135 |
|
of more than one character: |
136 |
|
|
137 |
|
(a) /^(..)\1/ did not partially match "aba" because checking references was |
138 |
|
done on an "all or nothing" basis. This also applied to repeated |
139 |
|
references. |
140 |
|
|
141 |
|
(b) \R did not give a hard partial match if \r was found at the end of the |
142 |
|
subject. |
143 |
|
|
144 |
|
(c) \X did not give a hard partial match after matching one or more |
145 |
|
characters at the end of the subject. |
146 |
|
|
147 |
|
(d) When newline was set to CRLF, a pattern such as /a$/ did not recognize |
148 |
|
a partial match for the string "\r". |
149 |
|
|
150 |
|
(e) When newline was set to CRLF, the metacharacter "." did not recognize |
151 |
|
a partial match for a CR character at the end of the subject string. |
152 |
|
|
153 |
|
6. If JIT is requested using /S++ or -s++ (instead of just /S+ or -s+) when |
154 |
|
running pcretest, the text "(JIT)" added to the output whenever JIT is |
155 |
|
actually used to run the match. |
156 |
|
|
157 |
|
7. Individual JIT compile options can be set in pcretest by following -s+[+] |
158 |
|
or /S+[+] with a digit between 1 and 7. |
159 |
|
|
160 |
|
8. OP_NOT now supports any UTF character not just single-byte ones. |
161 |
|
|
162 |
|
9. (*MARK) control verb is now supported by the JIT compiler. |
163 |
|
|
164 |
|
10. The command "./RunTest list" lists the available tests without actually |
165 |
|
running any of them. (Because I keep forgetting what they all are.) |
166 |
|
|
167 |
|
11. Add PCRE_INFO_MAXLOOKBEHIND. |
168 |
|
|
169 |
|
12. Applied a (slightly modified) user-supplied patch that improves performance |
170 |
|
when the heap is used for recursion (compiled with --disable-stack-for- |
171 |
|
recursion). Instead of malloc and free for each heap frame each time a |
172 |
|
logical recursion happens, frames are retained on a chain and re-used where |
173 |
|
possible. This sometimes gives as much as 30% improvement. |
174 |
|
|
175 |
|
13. As documented, (*COMMIT) is now confined to within a recursive subpattern |
176 |
|
call. |
177 |
|
|
178 |
|
14. As documented, (*COMMIT) is now confined to within a positive assertion. |
179 |
|
|
180 |
|
15. It is now possible to link pcretest with libedit as an alternative to |
181 |
|
libreadline. |
182 |
|
|
183 |
|
16. (*COMMIT) control verb is now supported by the JIT compiler. |
184 |
|
|
185 |
|
17. The Unicode data tables have been updated to Unicode 6.1.0. |
186 |
|
|
187 |
|
18. Added --file-list option to pcregrep. |
188 |
|
|
189 |
|
19. Added binary file support to pcregrep, including the -a, --binary-files, |
190 |
|
-I, and --text options. |
191 |
|
|
192 |
|
20. The madvise function is renamed for posix_madvise for QNX compatibility |
193 |
|
reasons. Fixed by Giuseppe D'Angelo. |
194 |
|
|
195 |
|
21. Fixed a bug for backward assertions with REVERSE 0 in the JIT compiler. |
196 |
|
|
197 |
|
22. Changed the option for creating symbolic links for 16-bit man pages from |
198 |
|
-s to -sf so that re-installing does not cause issues. |
199 |
|
|
200 |
|
23. Support PCRE_NO_START_OPTIMIZE in JIT as (*MARK) support requires it. |
201 |
|
|
202 |
|
24. Fixed a very old bug in pcretest that caused errors with restarted DFA |
203 |
|
matches in certain environments (the workspace was not being correctly |
204 |
|
retained). Also added to pcre_dfa_exec() a simple plausibility check on |
205 |
|
some of the workspace data at the beginning of a restart. |
206 |
|
|
207 |
|
25. \s*\R was auto-possessifying the \s* when it should not, whereas \S*\R |
208 |
|
was not doing so when it should - probably a typo introduced by SVN 528 |
209 |
|
(change 8.10/14). |
210 |
|
|
211 |
|
26. When PCRE_UCP was not set, \w+\x{c4} was incorrectly auto-possessifying the |
212 |
|
\w+ when the character tables indicated that \x{c4} was a word character. |
213 |
|
There were several related cases, all because the tests for doing a table |
214 |
|
lookup were testing for characters less than 127 instead of 255. |
215 |
|
|
216 |
|
27. If a pattern contains capturing parentheses that are not used in a match, |
217 |
|
their slots in the ovector are set to -1. For those that are higher than |
218 |
|
any matched groups, this happens at the end of processing. In the case when |
219 |
|
there were back references that the ovector was too small to contain |
220 |
|
(causing temporary malloc'd memory to be used during matching), and the |
221 |
|
highest capturing number was not used, memory off the end of the ovector |
222 |
|
was incorrectly being set to -1. (It was using the size of the temporary |
223 |
|
memory instead of the true size.) |
224 |
|
|
225 |
|
28. To catch bugs like 27 using valgrind, when pcretest is asked to specify an |
226 |
|
ovector size, it uses memory at the end of the block that it has got. |
227 |
|
|
228 |
|
29. Check for an overlong MARK name and give an error at compile time. The |
229 |
|
limit is 255 for the 8-bit library and 65535 for the 16-bit library. |
230 |
|
|
231 |
|
30. JIT compiler update. |
232 |
|
|
233 |
|
31. JIT is now supported on jailbroken iOS devices. Thanks for Ruiger |
234 |
|
Rill for the patch. |
235 |
|
|
236 |
|
32. Put spaces around SLJIT_PRINT_D in the JIT compiler. Required by CXX11. |
237 |
|
|
238 |
|
33. Variable renamings in the PCRE-JIT compiler. No functionality change. |
239 |
|
|
240 |
|
34. Fixed typos in pcregrep: in two places there was SUPPORT_LIBZ2 instead of |
241 |
|
SUPPORT_LIBBZ2. This caused a build problem when bzip2 but not gzip (zlib) |
242 |
|
was enabled. |
243 |
|
|
244 |
|
35. Improve JIT code generation for greedy plus quantifier. |
245 |
|
|
246 |
|
36. When /((?:a?)*)*c/ or /((?>a?)*)*c/ was matched against "aac", it set group |
247 |
|
1 to "aa" instead of to an empty string. The bug affected repeated groups |
248 |
|
that could potentially match an empty string. |
249 |
|
|
250 |
|
37. Optimizing single character iterators in JIT. |
251 |
|
|
252 |
|
38. Wide characters specified with \uxxxx in JavaScript mode are now subject to |
253 |
|
the same checks as \x{...} characters in non-JavaScript mode. Specifically, |
254 |
|
codepoints that are too big for the mode are faulted, and in a UTF mode, |
255 |
|
disallowed codepoints are also faulted. |
256 |
|
|
257 |
|
39. If PCRE was compiled with UTF support, in three places in the DFA |
258 |
|
matcher there was code that should only have been obeyed in UTF mode, but |
259 |
|
was being obeyed unconditionally. In 8-bit mode this could cause incorrect |
260 |
|
processing when bytes with values greater than 127 were present. In 16-bit |
261 |
|
mode the bug would be provoked by values in the range 0xfc00 to 0xdc00. In |
262 |
|
both cases the values are those that cannot be the first data item in a UTF |
263 |
|
character. The three items that might have provoked this were recursions, |
264 |
|
possessively repeated groups, and atomic groups. |
265 |
|
|
266 |
|
40. Ensure that libpcre is explicitly listed in the link commands for pcretest |
267 |
|
and pcregrep, because some OS require shared objects to be explicitly |
268 |
|
passed to ld, causing the link step to fail if they are not. |
269 |
|
|
270 |
|
41. There were two incorrect #ifdefs in pcre_study.c, meaning that, in 16-bit |
271 |
|
mode, patterns that started with \h* or \R* might be incorrectly matched. |
272 |
|
|
273 |
|
|
274 |
|
Version 8.30 04-February-2012 |
275 |
|
----------------------------- |
276 |
|
|
277 |
1. Renamed "isnumber" as "is_a_number" because in some Mac environments this |
1. Renamed "isnumber" as "is_a_number" because in some Mac environments this |
278 |
name is defined in ctype.h. |
name is defined in ctype.h. |
279 |
|
|
302 |
|
|
303 |
8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size |
8. Ovector size of 2 is also supported by JIT based pcre_exec (the ovector size |
304 |
rounding is not applied in this particular case). |
rounding is not applied in this particular case). |
305 |
|
|
306 |
9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected |
9. The invalid Unicode surrogate codepoints U+D800 to U+DFFF are now rejected |
307 |
if they appear, or are escaped, in patterns. |
if they appear, or are escaped, in patterns. |
308 |
|
|
309 |
10. Get rid of a number of -Wunused-but-set-variable warnings. |
10. Get rid of a number of -Wunused-but-set-variable warnings. |
310 |
|
|
311 |
|
11. The pattern /(?=(*:x))(q|)/ matches an empty string, and returns the mark |
312 |
|
"x". The similar pattern /(?=(*:x))((*:y)q|)/ did not return a mark at all. |
313 |
|
Oddly, Perl behaves the same way. PCRE has been fixed so that this pattern |
314 |
|
also returns the mark "x". This bug applied to capturing parentheses, |
315 |
|
non-capturing parentheses, and atomic parentheses. It also applied to some |
316 |
|
assertions. |
317 |
|
|
318 |
|
12. Stephen Kelly's patch to CMakeLists.txt allows it to parse the version |
319 |
|
information out of configure.ac instead of relying on pcre.h.generic, which |
320 |
|
is not stored in the repository. |
321 |
|
|
322 |
|
13. Applied Dmitry V. Levin's patch for a more portable method for linking with |
323 |
|
-lreadline. |
324 |
|
|
325 |
|
14. ZH added PCRE_CONFIG_JITTARGET; added its output to pcretest -C. |
326 |
|
|
327 |
|
15. Applied Graycode's patch to put the top-level frame on the stack rather |
328 |
|
than the heap when not using the stack for recursion. This gives a |
329 |
|
performance improvement in many cases when recursion is not deep. |
330 |
|
|
331 |
|
16. Experimental code added to "pcretest -C" to output the stack frame size. |
332 |
|
|
333 |
|
|
334 |
Version 8.21 12-Dec-2011 |
Version 8.21 12-Dec-2011 |