1 |
ChangeLog for PCRE |
ChangeLog for PCRE |
2 |
------------------ |
------------------ |
3 |
|
|
4 |
Version 7.7 05-Mar-08 |
Version 7.9 xx-xxx-09 |
5 |
|
--------------------- |
6 |
|
|
7 |
|
1. When building with support for bzlib/zlib (pcregrep) and/or readline |
8 |
|
(pcretest), all targets were linked against these libraries. This included |
9 |
|
libpcre, libpcreposix, and libpcrecpp, even though they do not use these |
10 |
|
libraries. This caused unwanted dependencies to be created. This problem |
11 |
|
has been fixed, and now only pcregrep is linked with bzlib/zlib and only |
12 |
|
pcretest is linked with readline. |
13 |
|
|
14 |
|
2. The "typedef int BOOL" in pcre_internal.h that was included inside the |
15 |
|
"#ifndef FALSE" condition by an earlier change (probably 7.8/18) has been |
16 |
|
moved outside it again, because FALSE and TRUE are already defined in AIX, |
17 |
|
but BOOL is not. |
18 |
|
|
19 |
|
3. The pcre_config() function was treating the PCRE_MATCH_LIMIT and |
20 |
|
PCRE_MATCH_LIMIT_RETURSION values as ints, when they should be long ints. |
21 |
|
|
22 |
|
4. The pcregrep documentation said spaces were inserted as well as colons (or |
23 |
|
hyphens) following file names and line numbers when outputting matching |
24 |
|
lines. This is not true; no spaces are inserted. I have also clarified the |
25 |
|
wording for the --colour (or --color) option. |
26 |
|
|
27 |
|
5. In pcregrep, when --colour was used with -o, the list of matching strings |
28 |
|
was not coloured; this is different to GNU grep, so I have changed it to be |
29 |
|
the same. |
30 |
|
|
31 |
|
6. When --colo(u)r was used in pcregrep, only the first matching substring in |
32 |
|
each matching line was coloured. Now it goes on to look for further matches |
33 |
|
of any of the test patterns, which is the same behaviour as GNU grep. |
34 |
|
|
35 |
|
7. A pattern that could match an empty string could cause pcregrep to loop; it |
36 |
|
doesn't make sense to accept an empty string match in pcregrep, so I have |
37 |
|
locked it out (using PCRE's PCRE_NOTEMPTY option). By experiment, this |
38 |
|
seems to be how GNU grep behaves. |
39 |
|
|
40 |
|
8. The pattern (?(?=.*b)b|^) was incorrectly compiled as "match must be at |
41 |
|
start or after a newline", because the conditional assertion was not being |
42 |
|
correctly handled. The rule now is that both the assertion and what follows |
43 |
|
in the first alternative must satisfy the test. |
44 |
|
|
45 |
|
9. If auto-callout was enabled in a pattern with a conditional group, PCRE |
46 |
|
could crash during matching. |
47 |
|
|
48 |
|
10. The PCRE_DOLLAR_ENDONLY option was not working when pcre_dfa_exec() was |
49 |
|
used for matching. |
50 |
|
|
51 |
|
11. Unicode property support in character classes was not working for |
52 |
|
characters (bytes) greater than 127 when not in UTF-8 mode. |
53 |
|
|
54 |
|
12. Added the -M command line option to pcretest. |
55 |
|
|
56 |
|
|
57 |
|
Version 7.8 05-Sep-08 |
58 |
|
--------------------- |
59 |
|
|
60 |
|
1. Replaced UCP searching code with optimized version as implemented for Ad |
61 |
|
Muncher (http://www.admuncher.com/) by Peter Kankowski. This uses a two- |
62 |
|
stage table and inline lookup instead of a function, giving speed ups of 2 |
63 |
|
to 5 times on some simple patterns that I tested. Permission was given to |
64 |
|
distribute the MultiStage2.py script that generates the tables (it's not in |
65 |
|
the tarball, but is in the Subversion repository). |
66 |
|
|
67 |
|
2. Updated the Unicode datatables to Unicode 5.1.0. This adds yet more |
68 |
|
scripts. |
69 |
|
|
70 |
|
3. Change 12 for 7.7 introduced a bug in pcre_study() when a pattern contained |
71 |
|
a group with a zero qualifier. The result of the study could be incorrect, |
72 |
|
or the function might crash, depending on the pattern. |
73 |
|
|
74 |
|
4. Caseless matching was not working for non-ASCII characters in back |
75 |
|
references. For example, /(\x{de})\1/8i was not matching \x{de}\x{fe}. |
76 |
|
It now works when Unicode Property Support is available. |
77 |
|
|
78 |
|
5. In pcretest, an escape such as \x{de} in the data was always generating |
79 |
|
a UTF-8 string, even in non-UTF-8 mode. Now it generates a single byte in |
80 |
|
non-UTF-8 mode. If the value is greater than 255, it gives a warning about |
81 |
|
truncation. |
82 |
|
|
83 |
|
6. Minor bugfix in pcrecpp.cc (change "" == ... to NULL == ...). |
84 |
|
|
85 |
|
7. Added two (int) casts to pcregrep when printing the difference of two |
86 |
|
pointers, in case they are 64-bit values. |
87 |
|
|
88 |
|
8. Added comments about Mac OS X stack usage to the pcrestack man page and to |
89 |
|
test 2 if it fails. |
90 |
|
|
91 |
|
9. Added PCRE_CALL_CONVENTION just before the names of all exported functions, |
92 |
|
and a #define of that name to empty if it is not externally set. This is to |
93 |
|
allow users of MSVC to set it if necessary. |
94 |
|
|
95 |
|
10. The PCRE_EXP_DEFN macro which precedes exported functions was missing from |
96 |
|
the convenience functions in the pcre_get.c source file. |
97 |
|
|
98 |
|
11. An option change at the start of a pattern that had top-level alternatives |
99 |
|
could cause overwriting and/or a crash. This command provoked a crash in |
100 |
|
some environments: |
101 |
|
|
102 |
|
printf "/(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8\n" | pcretest |
103 |
|
|
104 |
|
This potential security problem was recorded as CVE-2008-2371. |
105 |
|
|
106 |
|
12. For a pattern where the match had to start at the beginning or immediately |
107 |
|
after a newline (e.g /.*anything/ without the DOTALL flag), pcre_exec() and |
108 |
|
pcre_dfa_exec() could read past the end of the passed subject if there was |
109 |
|
no match. To help with detecting such bugs (e.g. with valgrind), I modified |
110 |
|
pcretest so that it places the subject at the end of its malloc-ed buffer. |
111 |
|
|
112 |
|
13. The change to pcretest in 12 above threw up a couple more cases when pcre_ |
113 |
|
exec() might read past the end of the data buffer in UTF-8 mode. |
114 |
|
|
115 |
|
14. A similar bug to 7.3/2 existed when the PCRE_FIRSTLINE option was set and |
116 |
|
the data contained the byte 0x85 as part of a UTF-8 character within its |
117 |
|
first line. This applied both to normal and DFA matching. |
118 |
|
|
119 |
|
15. Lazy qualifiers were not working in some cases in UTF-8 mode. For example, |
120 |
|
/^[^d]*?$/8 failed to match "abc". |
121 |
|
|
122 |
|
16. Added a missing copyright notice to pcrecpp_internal.h. |
123 |
|
|
124 |
|
17. Make it more clear in the documentation that values returned from |
125 |
|
pcre_exec() in ovector are byte offsets, not character counts. |
126 |
|
|
127 |
|
18. Tidied a few places to stop certain compilers from issuing warnings. |
128 |
|
|
129 |
|
19. Updated the Virtual Pascal + BCC files to compile the latest v7.7, as |
130 |
|
supplied by Stefan Weber. I made a further small update for 7.8 because |
131 |
|
there is a change of source arrangements: the pcre_searchfuncs.c module is |
132 |
|
replaced by pcre_ucd.c. |
133 |
|
|
134 |
|
|
135 |
|
Version 7.7 07-May-08 |
136 |
--------------------- |
--------------------- |
137 |
|
|
138 |
1. Applied Craig's patch to sort out a long long problem: "If we can't convert |
1. Applied Craig's patch to sort out a long long problem: "If we can't convert |
139 |
a string to a long long, pretend we don't even have a long long." This is |
a string to a long long, pretend we don't even have a long long." This is |
140 |
done by checking for the strtoq, strtoll, and _strtoi64 functions. |
done by checking for the strtoq, strtoll, and _strtoi64 functions. |
141 |
|
|
142 |
2. Applied Craig's patch to pcrecpp.cc to restore ABI compatibility with |
2. Applied Craig's patch to pcrecpp.cc to restore ABI compatibility with |
143 |
pre-7.6 versions, which defined a global no_arg variable instead of putting |
pre-7.6 versions, which defined a global no_arg variable instead of putting |
144 |
it in the RE class. (See also #8 below.) |
it in the RE class. (See also #8 below.) |
145 |
|
|
146 |
3. Remove a line of dead code, identified by coverity and reported by Nuno |
3. Remove a line of dead code, identified by coverity and reported by Nuno |
147 |
Lopes. |
Lopes. |
148 |
|
|
149 |
4. Fixed two related pcregrep bugs involving -r with --include or --exclude: |
4. Fixed two related pcregrep bugs involving -r with --include or --exclude: |
150 |
|
|
151 |
(1) The include/exclude patterns were being applied to the whole pathnames |
(1) The include/exclude patterns were being applied to the whole pathnames |
152 |
of files, instead of just to the final components. |
of files, instead of just to the final components. |
153 |
|
|
154 |
(2) If there was more than one level of directory, the subdirectories were |
(2) If there was more than one level of directory, the subdirectories were |
155 |
skipped unless they satisfied the include/exclude conditions. This is |
skipped unless they satisfied the include/exclude conditions. This is |
156 |
inconsistent with GNU grep (and could even be seen as contrary to the |
inconsistent with GNU grep (and could even be seen as contrary to the |
157 |
pcregrep specification - which I improved to make it absolutely clear). |
pcregrep specification - which I improved to make it absolutely clear). |
158 |
The action now is always to scan all levels of directory, and just |
The action now is always to scan all levels of directory, and just |
159 |
apply the include/exclude patterns to regular files. |
apply the include/exclude patterns to regular files. |
160 |
|
|
161 |
5. Added the --include_dir and --exclude_dir patterns to pcregrep, and used |
5. Added the --include_dir and --exclude_dir patterns to pcregrep, and used |
162 |
--exclude_dir in the tests to avoid scanning .svn directories. |
--exclude_dir in the tests to avoid scanning .svn directories. |
163 |
|
|
164 |
6. Applied Craig's patch to the QuoteMeta function so that it escapes the |
6. Applied Craig's patch to the QuoteMeta function so that it escapes the |
165 |
NUL character as backslash + 0 rather than backslash + NUL, because PCRE |
NUL character as backslash + 0 rather than backslash + NUL, because PCRE |
166 |
doesn't support NULs in patterns. |
doesn't support NULs in patterns. |
167 |
|
|
168 |
7. Added some missing "const"s to declarations of static tables in |
7. Added some missing "const"s to declarations of static tables in |
169 |
pcre_compile.c and pcre_dfa_exec.c. |
pcre_compile.c and pcre_dfa_exec.c. |
170 |
|
|
171 |
8. Applied Craig's patch to pcrecpp.cc to fix a problem in OS X that was |
8. Applied Craig's patch to pcrecpp.cc to fix a problem in OS X that was |
172 |
caused by fix #2 above. (Subsequently also a second patch to fix the |
caused by fix #2 above. (Subsequently also a second patch to fix the |
173 |
first patch. And a third patch - this was a messy problem.) |
first patch. And a third patch - this was a messy problem.) |
174 |
|
|
175 |
9. Applied Craig's patch to remove the use of push_back(). |
9. Applied Craig's patch to remove the use of push_back(). |
176 |
|
|
177 |
10. Applied Alan Lehotsky's patch to add REG_STARTEND support to the POSIX |
10. Applied Alan Lehotsky's patch to add REG_STARTEND support to the POSIX |
178 |
matching function regexec(). |
matching function regexec(). |
179 |
|
|
180 |
11. Added support for the Oniguruma syntax \g<name>, \g<n>, \g'name', \g'n', |
11. Added support for the Oniguruma syntax \g<name>, \g<n>, \g'name', \g'n', |
181 |
which, however, unlike Perl's \g{...}, are subroutine calls, not back |
which, however, unlike Perl's \g{...}, are subroutine calls, not back |
182 |
references. PCRE supports relative numbers with this syntax (I don't think |
references. PCRE supports relative numbers with this syntax (I don't think |
183 |
Oniguruma does). |
Oniguruma does). |
184 |
|
|
185 |
12. Previously, a group with a zero repeat such as (...){0} was completely |
12. Previously, a group with a zero repeat such as (...){0} was completely |
186 |
omitted from the compiled regex. However, this means that if the group |
omitted from the compiled regex. However, this means that if the group |
187 |
was called as a subroutine from elsewhere in the pattern, things went wrong |
was called as a subroutine from elsewhere in the pattern, things went wrong |
188 |
(an internal error was given). Such groups are now left in the compiled |
(an internal error was given). Such groups are now left in the compiled |
189 |
pattern, with a new opcode that causes them to be skipped at execution |
pattern, with a new opcode that causes them to be skipped at execution |
190 |
time. |
time. |
191 |
|
|
192 |
13. Added the PCRE_JAVASCRIPT_COMPAT option. This currently does two things: |
13. Added the PCRE_JAVASCRIPT_COMPAT option. This makes the following changes |
193 |
|
to the way PCRE behaves: |
194 |
|
|
195 |
(a) A lone ] character is dis-allowed (Perl treats it as data). |
(a) A lone ] character is dis-allowed (Perl treats it as data). |
196 |
(b) A back reference to an unmatched subpattern matches an empty string |
|
197 |
|
(b) A back reference to an unmatched subpattern matches an empty string |
198 |
(Perl fails the current match path). |
(Perl fails the current match path). |
199 |
|
|
200 |
|
(c) A data ] in a character class must be notated as \] because if the |
201 |
|
first data character in a class is ], it defines an empty class. (In |
202 |
|
Perl it is not possible to have an empty class.) The empty class [] |
203 |
|
never matches; it forces failure and is equivalent to (*FAIL) or (?!). |
204 |
|
The negative empty class [^] matches any one character, independently |
205 |
|
of the DOTALL setting. |
206 |
|
|
207 |
|
14. A pattern such as /(?2)[]a()b](abc)/ which had a forward reference to a |
208 |
|
non-existent subpattern following a character class starting with ']' and |
209 |
|
containing () gave an internal compiling error instead of "reference to |
210 |
|
non-existent subpattern". Fortunately, when the pattern did exist, the |
211 |
|
compiled code was correct. (When scanning forwards to check for the |
212 |
|
existencd of the subpattern, it was treating the data ']' as terminating |
213 |
|
the class, so got the count wrong. When actually compiling, the reference |
214 |
|
was subsequently set up correctly.) |
215 |
|
|
216 |
|
15. The "always fail" assertion (?!) is optimzed to (*FAIL) by pcre_compile; |
217 |
|
it was being rejected as not supported by pcre_dfa_exec(), even though |
218 |
|
other assertions are supported. I have made pcre_dfa_exec() support |
219 |
|
(*FAIL). |
220 |
|
|
221 |
|
16. The implementation of 13c above involved the invention of a new opcode, |
222 |
|
OP_ALLANY, which is like OP_ANY but doesn't check the /s flag. Since /s |
223 |
|
cannot be changed at match time, I realized I could make a small |
224 |
|
improvement to matching performance by compiling OP_ALLANY instead of |
225 |
|
OP_ANY for "." when DOTALL was set, and then removing the runtime tests |
226 |
|
on the OP_ANY path. |
227 |
|
|
228 |
|
17. Compiling pcretest on Windows with readline support failed without the |
229 |
|
following two fixes: (1) Make the unistd.h include conditional on |
230 |
|
HAVE_UNISTD_H; (2) #define isatty and fileno as _isatty and _fileno. |
231 |
|
|
232 |
|
18. Changed CMakeLists.txt and cmake/FindReadline.cmake to arrange for the |
233 |
|
ncurses library to be included for pcretest when ReadLine support is |
234 |
|
requested, but also to allow for it to be overridden. This patch came from |
235 |
|
Daniel Bergström. |
236 |
|
|
237 |
|
19. There was a typo in the file ucpinternal.h where f0_rangeflag was defined |
238 |
|
as 0x00f00000 instead of 0x00800000. Luckily, this would not have caused |
239 |
|
any errors with the current Unicode tables. Thanks to Peter Kankowski for |
240 |
|
spotting this. |
241 |
|
|
242 |
|
|
243 |
Version 7.6 28-Jan-08 |
Version 7.6 28-Jan-08 |
244 |
--------------------- |
--------------------- |