/[pcre]/code/trunk/ChangeLog
ViewVC logotype

Diff of /code/trunk/ChangeLog

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 73 by nigel, Sat Feb 24 21:40:30 2007 UTC revision 81 by nigel, Sat Feb 24 21:40:59 2007 UTC
# Line 1  Line 1 
1  ChangeLog for PCRE  ChangeLog for PCRE
2  ------------------  ------------------
3    
4    Version 6.2 01-Aug-05
5    ---------------------
6    
7     1. There was no test for integer overflow of quantifier values. A construction
8        such as {1111111111111111} would give undefined results. What is worse, if
9        a minimum quantifier for a parenthesized subpattern overflowed and became
10        negative, the calculation of the memory size went wrong. This could have
11        led to memory overwriting.
12    
13     2. Building PCRE using VPATH was broken. Hopefully it is now fixed.
14    
15     3. Added "b" to the 2nd argument of fopen() in dftables.c, for non-Unix-like
16        operating environments where this matters.
17    
18     4. Applied Giuseppe Maxia's patch to add additional features for controlling
19        PCRE options from within the C++ wrapper.
20    
21     5. Named capturing subpatterns were not being correctly counted when a pattern
22        was compiled. This caused two problems: (a) If there were more than 100
23        such subpatterns, the calculation of the memory needed for the whole
24        compiled pattern went wrong, leading to an overflow error. (b) Numerical
25        back references of the form \12, where the number was greater than 9, were
26        not recognized as back references, even though there were sufficient
27        previous subpatterns.
28    
29     6. Two minor patches to pcrecpp.cc in order to allow it to compile on older
30        versions of gcc, e.g. 2.95.4.
31    
32    
33    Version 6.1 21-Jun-05
34    ---------------------
35    
36     1. There was one reference to the variable "posix" in pcretest.c that was not
37        surrounded by "#if !defined NOPOSIX".
38    
39     2. Make it possible to compile pcretest without DFA support, UTF8 support, or
40        the cross-check on the old pcre_info() function, for the benefit of the
41        cut-down version of PCRE that is currently imported into Exim.
42    
43     3. A (silly) pattern starting with (?i)(?-i) caused an internal space
44        allocation error. I've done the easy fix, which wastes 2 bytes for sensible
45        patterns that start (?i) but I don't think that matters. The use of (?i) is
46        just an example; this all applies to the other options as well.
47    
48     4. Since libtool seems to echo the compile commands it is issuing, the output
49        from "make" can be reduced a bit by putting "@" in front of each libtool
50        compile command.
51    
52     5. Patch from the folks at Google for configure.in to be a bit more thorough
53        in checking for a suitable C++ installation before trying to compile the
54        C++ stuff. This should fix a reported problem when a compiler was present,
55        but no suitable headers.
56    
57     6. The man pages all had just "PCRE" as their title. I have changed them to
58        be the relevant file name. I have also arranged that these names are
59        retained in the file doc/pcre.txt, which is a concatenation in text format
60        of all the man pages except the little individual ones for each function.
61    
62     7. The NON-UNIX-USE file had not been updated for the different set of source
63        files that come with release 6. I also added a few comments about the C++
64        wrapper.
65    
66    
67    Version 6.0 07-Jun-05
68    ---------------------
69    
70     1. Some minor internal re-organization to help with my DFA experiments.
71    
72     2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that
73        didn't matter for the library itself when fully configured, but did matter
74        when compiling without UCP support, or within Exim, where the ucp files are
75        not imported.
76    
77     3. Refactoring of the library code to split up the various functions into
78        different source modules. The addition of the new DFA matching code (see
79        below) to a single monolithic source would have made it really too
80        unwieldy, quite apart from causing all the code to be include in a
81        statically linked application, when only some functions are used. This is
82        relevant even without the DFA addition now that patterns can be compiled in
83        one application and matched in another.
84    
85        The downside of splitting up is that there have to be some external
86        functions and data tables that are used internally in different modules of
87        the library but which are not part of the API. These have all had their
88        names changed to start with "_pcre_" so that they are unlikely to clash
89        with other external names.
90    
91     4. Added an alternate matching function, pcre_dfa_exec(), which matches using
92        a different (DFA) algorithm. Although it is slower than the original
93        function, it does have some advantages for certain types of matching
94        problem.
95    
96     5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(),
97        including restarting after a partial match.
98    
99     6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not
100        defined when compiling for Windows was sent to me. I have put it into the
101        code, though I have no means of testing or verifying it.
102    
103     7. Added the pcre_refcount() auxiliary function.
104    
105     8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to
106        match before or at the first newline in the subject string. In pcretest,
107        the /f option on a pattern can be used to set this.
108    
109     9. A repeated \w when used in UTF-8 mode with characters greater than 256
110        would behave wrongly. This has been present in PCRE since release 4.0.
111    
112    10. A number of changes to the pcregrep command:
113    
114        (a) Refactored how -x works; insert ^(...)$ instead of setting
115            PCRE_ANCHORED and checking the length, in preparation for adding
116            something similar for -w.
117    
118        (b) Added the -w (match as a word) option.
119    
120        (c) Refactored the way lines are read and buffered so as to have more
121            than one at a time available.
122    
123        (d) Implemented a pcregrep test script.
124    
125        (e) Added the -M (multiline match) option. This allows patterns to match
126            over several lines of the subject. The buffering ensures that at least
127            8K, or the rest of the document (whichever is the shorter) is available
128            for matching (and similarly the previous 8K for lookbehind assertions).
129    
130        (f) Changed the --help output so that it now says
131    
132              -w, --word-regex(p)
133    
134            instead of two lines, one with "regex" and the other with "regexp"
135            because that confused at least one person since the short forms are the
136            same. (This required a bit of code, as the output is generated
137            automatically from a table. It wasn't just a text change.)
138    
139        (g) -- can be used to terminate pcregrep options if the next thing isn't an
140            option but starts with a hyphen. Could be a pattern or a path name
141            starting with a hyphen, for instance.
142    
143        (h) "-" can be given as a file name to represent stdin.
144    
145        (i) When file names are being printed, "(standard input)" is used for
146            the standard input, for compatibility with GNU grep. Previously
147            "<stdin>" was used.
148    
149        (j) The option --label=xxx can be used to supply a name to be used for
150            stdin when file names are being printed. There is no short form.
151    
152        (k) Re-factored the options decoding logic because we are going to add
153            two more options that take data. Such options can now be given in four
154            different ways, e.g. "-fname", "-f name", "--file=name", "--file name".
155    
156        (l) Added the -A, -B, and -C options for requesting that lines of context
157            around matches be printed.
158    
159        (m) Added the -L option to print the names of files that do not contain
160            any matching lines, that is, the complement of -l.
161    
162        (n) The return code is 2 if any file cannot be opened, but pcregrep does
163            continue to scan other files.
164    
165        (o) The -s option was incorrectly implemented. For compatibility with other
166            greps, it now suppresses the error message for a non-existent or non-
167            accessible file (but not the return code). There is a new option called
168            -q that suppresses the output of matching lines, which was what -s was
169            previously doing.
170    
171        (p) Added --include and --exclude options to specify files for inclusion
172            and exclusion when recursing.
173    
174    11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly.
175        Hopefully, it now does.
176    
177    12. Missing cast in pcre_study().
178    
179    13. Added an "uninstall" target to the makefile.
180    
181    14. Replaced "extern" in the function prototypes in Makefile.in with
182        "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix
183        world, but is set differently for Windows.
184    
185    15. Added a second compiling function called pcre_compile2(). The only
186        difference is that it has an extra argument, which is a pointer to an
187        integer error code. When there is a compile-time failure, this is set
188        non-zero, in addition to the error test pointer being set to point to an
189        error message. The new argument may be NULL if no error number is required
190        (but then you may as well call pcre_compile(), which is now just a
191        wrapper). This facility is provided because some applications need a
192        numeric error indication, but it has also enabled me to tidy up the way
193        compile-time errors are handled in the POSIX wrapper.
194    
195    16. Added VPATH=.libs to the makefile; this should help when building with one
196        prefix path and installing with another. (Or so I'm told by someone who
197        knows more about this stuff than I do.)
198    
199    17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This
200        passes PCRE_DOTALL to the pcre_compile() function, making the "." character
201        match everything, including newlines. This is not POSIX-compatible, but
202        somebody wanted the feature. From pcretest it can be activated by using
203        both the P and the s flags.
204    
205    18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one.
206    
207    19. libpcre.pc was being incorrectly installed as executable.
208    
209    20. A couple of places in pcretest check for end-of-line by looking for '\n';
210        it now also looks for '\r' so that it will work unmodified on Windows.
211    
212    21. Added Google's contributed C++ wrapper to the distribution.
213    
214    22. Added some untidy missing memory free() calls in pcretest, to keep
215        Electric Fence happy when testing.
216    
217    
218    
219    Version 5.0 13-Sep-04
220    ---------------------
221    
222     1. Internal change: literal characters are no longer packed up into items
223        containing multiple characters in a single byte-string. Each character
224        is now matched using a separate opcode. However, there may be more than one
225        byte in the character in UTF-8 mode.
226    
227     2. The pcre_callout_block structure has two new fields: pattern_position and
228        next_item_length. These contain the offset in the pattern to the next match
229        item, and its length, respectively.
230    
231     3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic
232        insertion of callouts before each pattern item. Added the /C option to
233        pcretest to make use of this.
234    
235     4. On the advice of a Windows user, the lines
236    
237          #if defined(_WIN32) || defined(WIN32)
238          _setmode( _fileno( stdout ), 0x8000 );
239          #endif  /* defined(_WIN32) || defined(WIN32) */
240    
241        have been added to the source of pcretest. This apparently does useful
242        magic in relation to line terminators.
243    
244     5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb"
245        for the benefit of those environments where the "b" makes a difference.
246    
247     6. The icc compiler has the same options as gcc, but "configure" doesn't seem
248        to know about it. I have put a hack into configure.in that adds in code
249        to set GCC=yes if CC=icc. This seems to end up at a point in the
250        generated configure script that is early enough to affect the setting of
251        compiler options, which is what is needed, but I have no means of testing
252        whether it really works. (The user who reported this had patched the
253        generated configure script, which of course I cannot do.)
254    
255        LATER: After change 22 below (new libtool files), the configure script
256        seems to know about icc (and also ecc). Therefore, I have commented out
257        this hack in configure.in.
258    
259     7. Added support for pkg-config (2 patches were sent in).
260    
261     8. Negated POSIX character classes that used a combination of internal tables
262        were completely broken. These were [[:^alpha:]], [[:^alnum:]], and
263        [[:^ascii]]. Typically, they would match almost any characters. The other
264        POSIX classes were not broken in this way.
265    
266     9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed
267        to find the match, as PCRE was deluded into thinking that the match had to
268        start at the start point or following a newline. The same bug applied to
269        patterns with negative forward assertions or any backward assertions
270        preceding ".*" at the start, unless the pattern required a fixed first
271        character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed.
272    
273    10. In UTF-8 mode, when moving forwards in the subject after a failed match
274        starting at the last subject character, bytes beyond the end of the subject
275        string were read.
276    
277    11. Renamed the variable "class" as "classbits" to make life easier for C++
278        users. (Previously there was a macro definition, but it apparently wasn't
279        enough.)
280    
281    12. Added the new field "tables" to the extra data so that tables can be passed
282        in at exec time, or the internal tables can be re-selected. This allows
283        a compiled regex to be saved and re-used at a later time by a different
284        program that might have everything at different addresses.
285    
286    13. Modified the pcre-config script so that, when run on Solaris, it shows a
287        -R library as well as a -L library.
288    
289    14. The debugging options of pcretest (-d on the command line or D on a
290        pattern) showed incorrect output for anything following an extended class
291        that contained multibyte characters and which was followed by a quantifier.
292    
293    15. Added optional support for general category Unicode character properties
294        via the \p, \P, and \X escapes. Unicode property support implies UTF-8
295        support. It adds about 90K to the size of the library. The meanings of the
296        inbuilt class escapes such as \d and \s have NOT been changed.
297    
298    16. Updated pcredemo.c to include calls to free() to release the memory for the
299        compiled pattern.
300    
301    17. The generated file chartables.c was being created in the source directory
302        instead of in the building directory. This caused the build to fail if the
303        source directory was different from the building directory, and was
304        read-only.
305    
306    18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE
307        file. No doubt somebody will tell me if they don't make sense... Also added
308        Dan Mooney's comments about building on OpenVMS.
309    
310    19. Added support for partial matching via the PCRE_PARTIAL option for
311        pcre_exec() and the \P data escape in pcretest.
312    
313    20. Extended pcretest with 3 new pattern features:
314    
315        (i)   A pattern option of the form ">rest-of-line" causes pcretest to
316              write the compiled pattern to the file whose name is "rest-of-line".
317              This is a straight binary dump of the data, with the saved pointer to
318              the character tables forced to be NULL. The study data, if any, is
319              written too. After writing, pcretest reads a new pattern.
320    
321        (ii)  If, instead of a pattern, "<rest-of-line" is given, pcretest reads a
322              compiled pattern from the given file. There must not be any
323              occurrences of "<" in the file name (pretty unlikely); if there are,
324              pcretest will instead treat the initial "<" as a pattern delimiter.
325              After reading in the pattern, pcretest goes on to read data lines as
326              usual.
327    
328        (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit
329              and 16-bit fields in a compiled pattern, to simulate a pattern that
330              was compiled on a host of opposite endianness.
331    
332    21. The pcre-exec() function can now cope with patterns that were compiled on
333        hosts of opposite endianness, with this restriction:
334    
335          As for any compiled expression that is saved and used later, the tables
336          pointer field cannot be preserved; the extra_data field in the arguments
337          to pcre_exec() should be used to pass in a tables address if a value
338          other than the default internal tables were used at compile time.
339    
340    22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is
341        now diagnosed as an error. Previously, most of the time, a negative number
342        would have been treated as zero, but if in addition "ovector" was passed as
343        NULL, a crash could occur.
344    
345    23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with
346        new versions from the libtool 1.5 distribution (the last one is a copy of
347        a file called libtool.m4). This seems to have fixed the need to patch
348        "configure" to support Darwin 1.3 (which I used to do). However, I still
349        had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my
350        workstation).
351    
352    24. Changed the PCRE licence to be the more standard "BSD" licence.
353    
354    
355  Version 4.5 01-Dec-03  Version 4.5 01-Dec-03
356  ---------------------  ---------------------
357    

Legend:
Removed from v.73  
changed lines
  Added in v.81

  ViewVC Help
Powered by ViewVC 1.1.5