/[pcre]/code/trunk/ChangeLog
ViewVC logotype

Diff of /code/trunk/ChangeLog

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 71 by nigel, Sat Feb 24 21:40:24 2007 UTC revision 79 by nigel, Sat Feb 24 21:40:52 2007 UTC
# Line 1  Line 1 
1  ChangeLog for PCRE  ChangeLog for PCRE
2  ------------------  ------------------
3    
4    Version 6.1 21-Jun-05
5    ---------------------
6    
7     1. There was one reference to the variable "posix" in pcretest.c that was not
8        surrounded by "#if !defined NOPOSIX".
9    
10     2. Make it possible to compile pcretest without DFA support, UTF8 support, or
11        the cross-check on the old pcre_info() function, for the benefit of the
12        cut-down version of PCRE that is currently imported into Exim.
13    
14     3. A (silly) pattern starting with (?i)(?-i) caused an internal space
15        allocation error. I've done the easy fix, which wastes 2 bytes for sensible
16        patterns that start (?i) but I don't think that matters. The use of (?i) is
17        just an example; this all applies to the other options as well.
18    
19     4. Since libtool seems to echo the compile commands it is issuing, the output
20        from "make" can be reduced a bit by putting "@" in front of each libtool
21        compile command.
22    
23     5. Patch from the folks at Google for configure.in to be a bit more thorough
24        in checking for a suitable C++ installation before trying to compile the
25        C++ stuff. This should fix a reported problem when a compiler was present,
26        but no suitable headers.
27    
28     6. The man pages all had just "PCRE" as their title. I have changed them to
29        be the relevant file name. I have also arranged that these names are
30        retained in the file doc/pcre.txt, which is a concatenation in text format
31        of all the man pages except the little individual ones for each function.
32    
33     7. The NON-UNIX-USE file had not been updated for the different set of source
34        files that come with release 6. I also added a few comments about the C++
35        wrapper.
36    
37    
38    Version 6.0 07-Jun-05
39    ---------------------
40    
41     1. Some minor internal re-organization to help with my DFA experiments.
42    
43     2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that
44        didn't matter for the library itself when fully configured, but did matter
45        when compiling without UCP support, or within Exim, where the ucp files are
46        not imported.
47    
48     3. Refactoring of the library code to split up the various functions into
49        different source modules. The addition of the new DFA matching code (see
50        below) to a single monolithic source would have made it really too
51        unwieldy, quite apart from causing all the code to be include in a
52        statically linked application, when only some functions are used. This is
53        relevant even without the DFA addition now that patterns can be compiled in
54        one application and matched in another.
55    
56        The downside of splitting up is that there have to be some external
57        functions and data tables that are used internally in different modules of
58        the library but which are not part of the API. These have all had their
59        names changed to start with "_pcre_" so that they are unlikely to clash
60        with other external names.
61    
62     4. Added an alternate matching function, pcre_dfa_exec(), which matches using
63        a different (DFA) algorithm. Although it is slower than the original
64        function, it does have some advantages for certain types of matching
65        problem.
66    
67     5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(),
68        including restarting after a partial match.
69    
70     6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not
71        defined when compiling for Windows was sent to me. I have put it into the
72        code, though I have no means of testing or verifying it.
73    
74     7. Added the pcre_refcount() auxiliary function.
75    
76     8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to
77        match before or at the first newline in the subject string. In pcretest,
78        the /f option on a pattern can be used to set this.
79    
80     9. A repeated \w when used in UTF-8 mode with characters greater than 256
81        would behave wrongly. This has been present in PCRE since release 4.0.
82    
83    10. A number of changes to the pcregrep command:
84    
85        (a) Refactored how -x works; insert ^(...)$ instead of setting
86            PCRE_ANCHORED and checking the length, in preparation for adding
87            something similar for -w.
88    
89        (b) Added the -w (match as a word) option.
90    
91        (c) Refactored the way lines are read and buffered so as to have more
92            than one at a time available.
93    
94        (d) Implemented a pcregrep test script.
95    
96        (e) Added the -M (multiline match) option. This allows patterns to match
97            over several lines of the subject. The buffering ensures that at least
98            8K, or the rest of the document (whichever is the shorter) is available
99            for matching (and similarly the previous 8K for lookbehind assertions).
100    
101        (f) Changed the --help output so that it now says
102    
103              -w, --word-regex(p)
104    
105            instead of two lines, one with "regex" and the other with "regexp"
106            because that confused at least one person since the short forms are the
107            same. (This required a bit of code, as the output is generated
108            automatically from a table. It wasn't just a text change.)
109    
110        (g) -- can be used to terminate pcregrep options if the next thing isn't an
111            option but starts with a hyphen. Could be a pattern or a path name
112            starting with a hyphen, for instance.
113    
114        (h) "-" can be given as a file name to represent stdin.
115    
116        (i) When file names are being printed, "(standard input)" is used for
117            the standard input, for compatibility with GNU grep. Previously
118            "<stdin>" was used.
119    
120        (j) The option --label=xxx can be used to supply a name to be used for
121            stdin when file names are being printed. There is no short form.
122    
123        (k) Re-factored the options decoding logic because we are going to add
124            two more options that take data. Such options can now be given in four
125            different ways, e.g. "-fname", "-f name", "--file=name", "--file name".
126    
127        (l) Added the -A, -B, and -C options for requesting that lines of context
128            around matches be printed.
129    
130        (m) Added the -L option to print the names of files that do not contain
131            any matching lines, that is, the complement of -l.
132    
133        (n) The return code is 2 if any file cannot be opened, but pcregrep does
134            continue to scan other files.
135    
136        (o) The -s option was incorrectly implemented. For compatibility with other
137            greps, it now suppresses the error message for a non-existent or non-
138            accessible file (but not the return code). There is a new option called
139            -q that suppresses the output of matching lines, which was what -s was
140            previously doing.
141    
142        (p) Added --include and --exclude options to specify files for inclusion
143            and exclusion when recursing.
144    
145    11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly.
146        Hopefully, it now does.
147    
148    12. Missing cast in pcre_study().
149    
150    13. Added an "uninstall" target to the makefile.
151    
152    14. Replaced "extern" in the function prototypes in Makefile.in with
153        "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix
154        world, but is set differently for Windows.
155    
156    15. Added a second compiling function called pcre_compile2(). The only
157        difference is that it has an extra argument, which is a pointer to an
158        integer error code. When there is a compile-time failure, this is set
159        non-zero, in addition to the error test pointer being set to point to an
160        error message. The new argument may be NULL if no error number is required
161        (but then you may as well call pcre_compile(), which is now just a
162        wrapper). This facility is provided because some applications need a
163        numeric error indication, but it has also enabled me to tidy up the way
164        compile-time errors are handled in the POSIX wrapper.
165    
166    16. Added VPATH=.libs to the makefile; this should help when building with one
167        prefix path and installing with another. (Or so I'm told by someone who
168        knows more about this stuff than I do.)
169    
170    17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This
171        passes PCRE_DOTALL to the pcre_compile() function, making the "." character
172        match everything, including newlines. This is not POSIX-compatible, but
173        somebody wanted the feature. From pcretest it can be activated by using
174        both the P and the s flags.
175    
176    18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one.
177    
178    19. libpcre.pc was being incorrectly installed as executable.
179    
180    20. A couple of places in pcretest check for end-of-line by looking for '\n';
181        it now also looks for '\r' so that it will work unmodified on Windows.
182    
183    21. Added Google's contributed C++ wrapper to the distribution.
184    
185    22. Added some untidy missing memory free() calls in pcretest, to keep
186        Electric Fence happy when testing.
187    
188    
189    
190    Version 5.0 13-Sep-04
191    ---------------------
192    
193     1. Internal change: literal characters are no longer packed up into items
194        containing multiple characters in a single byte-string. Each character
195        is now matched using a separate opcode. However, there may be more than one
196        byte in the character in UTF-8 mode.
197    
198     2. The pcre_callout_block structure has two new fields: pattern_position and
199        next_item_length. These contain the offset in the pattern to the next match
200        item, and its length, respectively.
201    
202     3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic
203        insertion of callouts before each pattern item. Added the /C option to
204        pcretest to make use of this.
205    
206     4. On the advice of a Windows user, the lines
207    
208          #if defined(_WIN32) || defined(WIN32)
209          _setmode( _fileno( stdout ), 0x8000 );
210          #endif  /* defined(_WIN32) || defined(WIN32) */
211    
212        have been added to the source of pcretest. This apparently does useful
213        magic in relation to line terminators.
214    
215     5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb"
216        for the benefit of those environments where the "b" makes a difference.
217    
218     6. The icc compiler has the same options as gcc, but "configure" doesn't seem
219        to know about it. I have put a hack into configure.in that adds in code
220        to set GCC=yes if CC=icc. This seems to end up at a point in the
221        generated configure script that is early enough to affect the setting of
222        compiler options, which is what is needed, but I have no means of testing
223        whether it really works. (The user who reported this had patched the
224        generated configure script, which of course I cannot do.)
225    
226        LATER: After change 22 below (new libtool files), the configure script
227        seems to know about icc (and also ecc). Therefore, I have commented out
228        this hack in configure.in.
229    
230     7. Added support for pkg-config (2 patches were sent in).
231    
232     8. Negated POSIX character classes that used a combination of internal tables
233        were completely broken. These were [[:^alpha:]], [[:^alnum:]], and
234        [[:^ascii]]. Typically, they would match almost any characters. The other
235        POSIX classes were not broken in this way.
236    
237     9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed
238        to find the match, as PCRE was deluded into thinking that the match had to
239        start at the start point or following a newline. The same bug applied to
240        patterns with negative forward assertions or any backward assertions
241        preceding ".*" at the start, unless the pattern required a fixed first
242        character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed.
243    
244    10. In UTF-8 mode, when moving forwards in the subject after a failed match
245        starting at the last subject character, bytes beyond the end of the subject
246        string were read.
247    
248    11. Renamed the variable "class" as "classbits" to make life easier for C++
249        users. (Previously there was a macro definition, but it apparently wasn't
250        enough.)
251    
252    12. Added the new field "tables" to the extra data so that tables can be passed
253        in at exec time, or the internal tables can be re-selected. This allows
254        a compiled regex to be saved and re-used at a later time by a different
255        program that might have everything at different addresses.
256    
257    13. Modified the pcre-config script so that, when run on Solaris, it shows a
258        -R library as well as a -L library.
259    
260    14. The debugging options of pcretest (-d on the command line or D on a
261        pattern) showed incorrect output for anything following an extended class
262        that contained multibyte characters and which was followed by a quantifier.
263    
264    15. Added optional support for general category Unicode character properties
265        via the \p, \P, and \X escapes. Unicode property support implies UTF-8
266        support. It adds about 90K to the size of the library. The meanings of the
267        inbuilt class escapes such as \d and \s have NOT been changed.
268    
269    16. Updated pcredemo.c to include calls to free() to release the memory for the
270        compiled pattern.
271    
272    17. The generated file chartables.c was being created in the source directory
273        instead of in the building directory. This caused the build to fail if the
274        source directory was different from the building directory, and was
275        read-only.
276    
277    18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE
278        file. No doubt somebody will tell me if they don't make sense... Also added
279        Dan Mooney's comments about building on OpenVMS.
280    
281    19. Added support for partial matching via the PCRE_PARTIAL option for
282        pcre_exec() and the \P data escape in pcretest.
283    
284    20. Extended pcretest with 3 new pattern features:
285    
286        (i)   A pattern option of the form ">rest-of-line" causes pcretest to
287              write the compiled pattern to the file whose name is "rest-of-line".
288              This is a straight binary dump of the data, with the saved pointer to
289              the character tables forced to be NULL. The study data, if any, is
290              written too. After writing, pcretest reads a new pattern.
291    
292        (ii)  If, instead of a pattern, "<rest-of-line" is given, pcretest reads a
293              compiled pattern from the given file. There must not be any
294              occurrences of "<" in the file name (pretty unlikely); if there are,
295              pcretest will instead treat the initial "<" as a pattern delimiter.
296              After reading in the pattern, pcretest goes on to read data lines as
297              usual.
298    
299        (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit
300              and 16-bit fields in a compiled pattern, to simulate a pattern that
301              was compiled on a host of opposite endianness.
302    
303    21. The pcre-exec() function can now cope with patterns that were compiled on
304        hosts of opposite endianness, with this restriction:
305    
306          As for any compiled expression that is saved and used later, the tables
307          pointer field cannot be preserved; the extra_data field in the arguments
308          to pcre_exec() should be used to pass in a tables address if a value
309          other than the default internal tables were used at compile time.
310    
311    22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is
312        now diagnosed as an error. Previously, most of the time, a negative number
313        would have been treated as zero, but if in addition "ovector" was passed as
314        NULL, a crash could occur.
315    
316    23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with
317        new versions from the libtool 1.5 distribution (the last one is a copy of
318        a file called libtool.m4). This seems to have fixed the need to patch
319        "configure" to support Darwin 1.3 (which I used to do). However, I still
320        had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my
321        workstation).
322    
323    24. Changed the PCRE licence to be the more standard "BSD" licence.
324    
325    
326    Version 4.5 01-Dec-03
327    ---------------------
328    
329     1. There has been some re-arrangement of the code for the match() function so
330        that it can be compiled in a version that does not call itself recursively.
331        Instead, it keeps those local variables that need separate instances for
332        each "recursion" in a frame on the heap, and gets/frees frames whenever it
333        needs to "recurse". Keeping track of where control must go is done by means
334        of setjmp/longjmp. The whole thing is implemented by a set of macros that
335        hide most of the details from the main code, and operates only if
336        NO_RECURSE is defined while compiling pcre.c. If PCRE is built using the
337        "configure" mechanism, "--disable-stack-for-recursion" turns on this way of
338        operating.
339    
340        To make it easier for callers to provide specially tailored get/free
341        functions for this usage, two new functions, pcre_stack_malloc, and
342        pcre_stack_free, are used. They are always called in strict stacking order,
343        and the size of block requested is always the same.
344    
345        The PCRE_CONFIG_STACKRECURSE info parameter can be used to find out whether
346        PCRE has been compiled to use the stack or the heap for recursion. The
347        -C option of pcretest uses this to show which version is compiled.
348    
349        A new data escape \S, is added to pcretest; it causes the amounts of store
350        obtained and freed by both kinds of malloc/free at match time to be added
351        to the output.
352    
353     2. Changed the locale test to use "fr_FR" instead of "fr" because that's
354        what's available on my current Linux desktop machine.
355    
356     3. When matching a UTF-8 string, the test for a valid string at the start has
357        been extended. If start_offset is not zero, PCRE now checks that it points
358        to a byte that is the start of a UTF-8 character. If not, it returns
359        PCRE_ERROR_BADUTF8_OFFSET (-11). Note: the whole string is still checked;
360        this is necessary because there may be backward assertions in the pattern.
361        When matching the same subject several times, it may save resources to use
362        PCRE_NO_UTF8_CHECK on all but the first call if the string is long.
363    
364     4. The code for checking the validity of UTF-8 strings has been tightened so
365        that it rejects (a) strings containing 0xfe or 0xff bytes and (b) strings
366        containing "overlong sequences".
367    
368     5. Fixed a bug (appearing twice) that I could not find any way of exploiting!
369        I had written "if ((digitab[*p++] && chtab_digit) == 0)" where the "&&"
370        should have been "&", but it just so happened that all the cases this let
371        through by mistake were picked up later in the function.
372    
373     6. I had used a variable called "isblank" - this is a C99 function, causing
374        some compilers to warn. To avoid this, I renamed it (as "blankclass").
375    
376     7. Cosmetic: (a) only output another newline at the end of pcretest if it is
377        prompting; (b) run "./pcretest /dev/null" at the start of the test script
378        so the version is shown; (c) stop "make test" echoing "./RunTest".
379    
380     8. Added patches from David Burgess to enable PCRE to run on EBCDIC systems.
381    
382     9. The prototype for memmove() for systems that don't have it was using
383        size_t, but the inclusion of the header that defines size_t was later. I've
384        moved the #includes for the C headers earlier to avoid this.
385    
386    10. Added some adjustments to the code to make it easier to compiler on certain
387        special systems:
388    
389          (a) Some "const" qualifiers were missing.
390          (b) Added the macro EXPORT before all exported functions; by default this
391              is defined to be empty.
392          (c) Changed the dftables auxiliary program (that builds chartables.c) so
393              that it reads its output file name as an argument instead of writing
394              to the standard output and assuming this can be redirected.
395    
396    11. In UTF-8 mode, if a recursive reference (e.g. (?1)) followed a character
397        class containing characters with values greater than 255, PCRE compilation
398        went into a loop.
399    
400    12. A recursive reference to a subpattern that was within another subpattern
401        that had a minimum quantifier of zero caused PCRE to crash. For example,
402        (x(y(?2))z)? provoked this bug with a subject that got as far as the
403        recursion. If the recursively-called subpattern itself had a zero repeat,
404        that was OK.
405    
406    13. In pcretest, the buffer for reading a data line was set at 30K, but the
407        buffer into which it was copied (for escape processing) was still set at
408        1024, so long lines caused crashes.
409    
410    14. A pattern such as /[ab]{1,3}+/ failed to compile, giving the error
411        "internal error: code overflow...". This applied to any character class
412        that was followed by a possessive quantifier.
413    
414    15. Modified the Makefile to add libpcre.la as a prerequisite for
415        libpcreposix.la because I was told this is needed for a parallel build to
416        work.
417    
418    16. If a pattern that contained .* following optional items at the start was
419        studied, the wrong optimizing data was generated, leading to matching
420        errors. For example, studying /[ab]*.*c/ concluded, erroneously, that any
421        matching string must start with a or b or c. The correct conclusion for
422        this pattern is that a match can start with any character.
423    
424    
425  Version 4.4 13-Aug-03  Version 4.4 13-Aug-03
426  ---------------------  ---------------------
427    

Legend:
Removed from v.71  
changed lines
  Added in v.79

  ViewVC Help
Powered by ViewVC 1.1.5