ViewVC logotype

Diff of /code/trunk/ChangeLog

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 69 by nigel, Sat Feb 24 21:40:18 2007 UTC revision 79 by nigel, Sat Feb 24 21:40:52 2007 UTC
# Line 1  Line 1 
1  ChangeLog for PCRE  ChangeLog for PCRE
2  ------------------  ------------------
4    Version 6.1 21-Jun-05
5    ---------------------
7     1. There was one reference to the variable "posix" in pcretest.c that was not
8        surrounded by "#if !defined NOPOSIX".
10     2. Make it possible to compile pcretest without DFA support, UTF8 support, or
11        the cross-check on the old pcre_info() function, for the benefit of the
12        cut-down version of PCRE that is currently imported into Exim.
14     3. A (silly) pattern starting with (?i)(?-i) caused an internal space
15        allocation error. I've done the easy fix, which wastes 2 bytes for sensible
16        patterns that start (?i) but I don't think that matters. The use of (?i) is
17        just an example; this all applies to the other options as well.
19     4. Since libtool seems to echo the compile commands it is issuing, the output
20        from "make" can be reduced a bit by putting "@" in front of each libtool
21        compile command.
23     5. Patch from the folks at Google for configure.in to be a bit more thorough
24        in checking for a suitable C++ installation before trying to compile the
25        C++ stuff. This should fix a reported problem when a compiler was present,
26        but no suitable headers.
28     6. The man pages all had just "PCRE" as their title. I have changed them to
29        be the relevant file name. I have also arranged that these names are
30        retained in the file doc/pcre.txt, which is a concatenation in text format
31        of all the man pages except the little individual ones for each function.
33     7. The NON-UNIX-USE file had not been updated for the different set of source
34        files that come with release 6. I also added a few comments about the C++
35        wrapper.
38    Version 6.0 07-Jun-05
39    ---------------------
41     1. Some minor internal re-organization to help with my DFA experiments.
43     2. Some missing #ifdef SUPPORT_UCP conditionals in pcretest and printint that
44        didn't matter for the library itself when fully configured, but did matter
45        when compiling without UCP support, or within Exim, where the ucp files are
46        not imported.
48     3. Refactoring of the library code to split up the various functions into
49        different source modules. The addition of the new DFA matching code (see
50        below) to a single monolithic source would have made it really too
51        unwieldy, quite apart from causing all the code to be include in a
52        statically linked application, when only some functions are used. This is
53        relevant even without the DFA addition now that patterns can be compiled in
54        one application and matched in another.
56        The downside of splitting up is that there have to be some external
57        functions and data tables that are used internally in different modules of
58        the library but which are not part of the API. These have all had their
59        names changed to start with "_pcre_" so that they are unlikely to clash
60        with other external names.
62     4. Added an alternate matching function, pcre_dfa_exec(), which matches using
63        a different (DFA) algorithm. Although it is slower than the original
64        function, it does have some advantages for certain types of matching
65        problem.
67     5. Upgrades to pcretest in order to test the features of pcre_dfa_exec(),
68        including restarting after a partial match.
70     6. A patch for pcregrep that defines INVALID_FILE_ATTRIBUTES if it is not
71        defined when compiling for Windows was sent to me. I have put it into the
72        code, though I have no means of testing or verifying it.
74     7. Added the pcre_refcount() auxiliary function.
76     8. Added the PCRE_FIRSTLINE option. This constrains an unanchored pattern to
77        match before or at the first newline in the subject string. In pcretest,
78        the /f option on a pattern can be used to set this.
80     9. A repeated \w when used in UTF-8 mode with characters greater than 256
81        would behave wrongly. This has been present in PCRE since release 4.0.
83    10. A number of changes to the pcregrep command:
85        (a) Refactored how -x works; insert ^(...)$ instead of setting
86            PCRE_ANCHORED and checking the length, in preparation for adding
87            something similar for -w.
89        (b) Added the -w (match as a word) option.
91        (c) Refactored the way lines are read and buffered so as to have more
92            than one at a time available.
94        (d) Implemented a pcregrep test script.
96        (e) Added the -M (multiline match) option. This allows patterns to match
97            over several lines of the subject. The buffering ensures that at least
98            8K, or the rest of the document (whichever is the shorter) is available
99            for matching (and similarly the previous 8K for lookbehind assertions).
101        (f) Changed the --help output so that it now says
103              -w, --word-regex(p)
105            instead of two lines, one with "regex" and the other with "regexp"
106            because that confused at least one person since the short forms are the
107            same. (This required a bit of code, as the output is generated
108            automatically from a table. It wasn't just a text change.)
110        (g) -- can be used to terminate pcregrep options if the next thing isn't an
111            option but starts with a hyphen. Could be a pattern or a path name
112            starting with a hyphen, for instance.
114        (h) "-" can be given as a file name to represent stdin.
116        (i) When file names are being printed, "(standard input)" is used for
117            the standard input, for compatibility with GNU grep. Previously
118            "<stdin>" was used.
120        (j) The option --label=xxx can be used to supply a name to be used for
121            stdin when file names are being printed. There is no short form.
123        (k) Re-factored the options decoding logic because we are going to add
124            two more options that take data. Such options can now be given in four
125            different ways, e.g. "-fname", "-f name", "--file=name", "--file name".
127        (l) Added the -A, -B, and -C options for requesting that lines of context
128            around matches be printed.
130        (m) Added the -L option to print the names of files that do not contain
131            any matching lines, that is, the complement of -l.
133        (n) The return code is 2 if any file cannot be opened, but pcregrep does
134            continue to scan other files.
136        (o) The -s option was incorrectly implemented. For compatibility with other
137            greps, it now suppresses the error message for a non-existent or non-
138            accessible file (but not the return code). There is a new option called
139            -q that suppresses the output of matching lines, which was what -s was
140            previously doing.
142        (p) Added --include and --exclude options to specify files for inclusion
143            and exclusion when recursing.
145    11. The Makefile was not using the Autoconf-supported LDFLAGS macro properly.
146        Hopefully, it now does.
148    12. Missing cast in pcre_study().
150    13. Added an "uninstall" target to the makefile.
152    14. Replaced "extern" in the function prototypes in Makefile.in with
153        "PCRE_DATA_SCOPE", which defaults to 'extern' or 'extern "C"' in the Unix
154        world, but is set differently for Windows.
156    15. Added a second compiling function called pcre_compile2(). The only
157        difference is that it has an extra argument, which is a pointer to an
158        integer error code. When there is a compile-time failure, this is set
159        non-zero, in addition to the error test pointer being set to point to an
160        error message. The new argument may be NULL if no error number is required
161        (but then you may as well call pcre_compile(), which is now just a
162        wrapper). This facility is provided because some applications need a
163        numeric error indication, but it has also enabled me to tidy up the way
164        compile-time errors are handled in the POSIX wrapper.
166    16. Added VPATH=.libs to the makefile; this should help when building with one
167        prefix path and installing with another. (Or so I'm told by someone who
168        knows more about this stuff than I do.)
170    17. Added a new option, REG_DOTALL, to the POSIX function regcomp(). This
171        passes PCRE_DOTALL to the pcre_compile() function, making the "." character
172        match everything, including newlines. This is not POSIX-compatible, but
173        somebody wanted the feature. From pcretest it can be activated by using
174        both the P and the s flags.
176    18. AC_PROG_LIBTOOL appeared twice in Makefile.in. Removed one.
178    19. libpcre.pc was being incorrectly installed as executable.
180    20. A couple of places in pcretest check for end-of-line by looking for '\n';
181        it now also looks for '\r' so that it will work unmodified on Windows.
183    21. Added Google's contributed C++ wrapper to the distribution.
185    22. Added some untidy missing memory free() calls in pcretest, to keep
186        Electric Fence happy when testing.
190    Version 5.0 13-Sep-04
191    ---------------------
193     1. Internal change: literal characters are no longer packed up into items
194        containing multiple characters in a single byte-string. Each character
195        is now matched using a separate opcode. However, there may be more than one
196        byte in the character in UTF-8 mode.
198     2. The pcre_callout_block structure has two new fields: pattern_position and
199        next_item_length. These contain the offset in the pattern to the next match
200        item, and its length, respectively.
202     3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic
203        insertion of callouts before each pattern item. Added the /C option to
204        pcretest to make use of this.
206     4. On the advice of a Windows user, the lines
208          #if defined(_WIN32) || defined(WIN32)
209          _setmode( _fileno( stdout ), 0x8000 );
210          #endif  /* defined(_WIN32) || defined(WIN32) */
212        have been added to the source of pcretest. This apparently does useful
213        magic in relation to line terminators.
215     5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb"
216        for the benefit of those environments where the "b" makes a difference.
218     6. The icc compiler has the same options as gcc, but "configure" doesn't seem
219        to know about it. I have put a hack into configure.in that adds in code
220        to set GCC=yes if CC=icc. This seems to end up at a point in the
221        generated configure script that is early enough to affect the setting of
222        compiler options, which is what is needed, but I have no means of testing
223        whether it really works. (The user who reported this had patched the
224        generated configure script, which of course I cannot do.)
226        LATER: After change 22 below (new libtool files), the configure script
227        seems to know about icc (and also ecc). Therefore, I have commented out
228        this hack in configure.in.
230     7. Added support for pkg-config (2 patches were sent in).
232     8. Negated POSIX character classes that used a combination of internal tables
233        were completely broken. These were [[:^alpha:]], [[:^alnum:]], and
234        [[:^ascii]]. Typically, they would match almost any characters. The other
235        POSIX classes were not broken in this way.
237     9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed
238        to find the match, as PCRE was deluded into thinking that the match had to
239        start at the start point or following a newline. The same bug applied to
240        patterns with negative forward assertions or any backward assertions
241        preceding ".*" at the start, unless the pattern required a fixed first
242        character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed.
244    10. In UTF-8 mode, when moving forwards in the subject after a failed match
245        starting at the last subject character, bytes beyond the end of the subject
246        string were read.
248    11. Renamed the variable "class" as "classbits" to make life easier for C++
249        users. (Previously there was a macro definition, but it apparently wasn't
250        enough.)
252    12. Added the new field "tables" to the extra data so that tables can be passed
253        in at exec time, or the internal tables can be re-selected. This allows
254        a compiled regex to be saved and re-used at a later time by a different
255        program that might have everything at different addresses.
257    13. Modified the pcre-config script so that, when run on Solaris, it shows a
258        -R library as well as a -L library.
260    14. The debugging options of pcretest (-d on the command line or D on a
261        pattern) showed incorrect output for anything following an extended class
262        that contained multibyte characters and which was followed by a quantifier.
264    15. Added optional support for general category Unicode character properties
265        via the \p, \P, and \X escapes. Unicode property support implies UTF-8
266        support. It adds about 90K to the size of the library. The meanings of the
267        inbuilt class escapes such as \d and \s have NOT been changed.
269    16. Updated pcredemo.c to include calls to free() to release the memory for the
270        compiled pattern.
272    17. The generated file chartables.c was being created in the source directory
273        instead of in the building directory. This caused the build to fail if the
274        source directory was different from the building directory, and was
275        read-only.
277    18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE
278        file. No doubt somebody will tell me if they don't make sense... Also added
279        Dan Mooney's comments about building on OpenVMS.
281    19. Added support for partial matching via the PCRE_PARTIAL option for
282        pcre_exec() and the \P data escape in pcretest.
284    20. Extended pcretest with 3 new pattern features:
286        (i)   A pattern option of the form ">rest-of-line" causes pcretest to
287              write the compiled pattern to the file whose name is "rest-of-line".
288              This is a straight binary dump of the data, with the saved pointer to
289              the character tables forced to be NULL. The study data, if any, is
290              written too. After writing, pcretest reads a new pattern.
292        (ii)  If, instead of a pattern, "<rest-of-line" is given, pcretest reads a
293              compiled pattern from the given file. There must not be any
294              occurrences of "<" in the file name (pretty unlikely); if there are,
295              pcretest will instead treat the initial "<" as a pattern delimiter.
296              After reading in the pattern, pcretest goes on to read data lines as
297              usual.
299        (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit
300              and 16-bit fields in a compiled pattern, to simulate a pattern that
301              was compiled on a host of opposite endianness.
303    21. The pcre-exec() function can now cope with patterns that were compiled on
304        hosts of opposite endianness, with this restriction:
306          As for any compiled expression that is saved and used later, the tables
307          pointer field cannot be preserved; the extra_data field in the arguments
308          to pcre_exec() should be used to pass in a tables address if a value
309          other than the default internal tables were used at compile time.
311    22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is
312        now diagnosed as an error. Previously, most of the time, a negative number
313        would have been treated as zero, but if in addition "ovector" was passed as
314        NULL, a crash could occur.
316    23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with
317        new versions from the libtool 1.5 distribution (the last one is a copy of
318        a file called libtool.m4). This seems to have fixed the need to patch
319        "configure" to support Darwin 1.3 (which I used to do). However, I still
320        had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my
321        workstation).
323    24. Changed the PCRE licence to be the more standard "BSD" licence.
326    Version 4.5 01-Dec-03
327    ---------------------
329     1. There has been some re-arrangement of the code for the match() function so
330        that it can be compiled in a version that does not call itself recursively.
331        Instead, it keeps those local variables that need separate instances for
332        each "recursion" in a frame on the heap, and gets/frees frames whenever it
333        needs to "recurse". Keeping track of where control must go is done by means
334        of setjmp/longjmp. The whole thing is implemented by a set of macros that
335        hide most of the details from the main code, and operates only if
336        NO_RECURSE is defined while compiling pcre.c. If PCRE is built using the
337        "configure" mechanism, "--disable-stack-for-recursion" turns on this way of
338        operating.
340        To make it easier for callers to provide specially tailored get/free
341        functions for this usage, two new functions, pcre_stack_malloc, and
342        pcre_stack_free, are used. They are always called in strict stacking order,
343        and the size of block requested is always the same.
345        The PCRE_CONFIG_STACKRECURSE info parameter can be used to find out whether
346        PCRE has been compiled to use the stack or the heap for recursion. The
347        -C option of pcretest uses this to show which version is compiled.
349        A new data escape \S, is added to pcretest; it causes the amounts of store
350        obtained and freed by both kinds of malloc/free at match time to be added
351        to the output.
353     2. Changed the locale test to use "fr_FR" instead of "fr" because that's
354        what's available on my current Linux desktop machine.
356     3. When matching a UTF-8 string, the test for a valid string at the start has
357        been extended. If start_offset is not zero, PCRE now checks that it points
358        to a byte that is the start of a UTF-8 character. If not, it returns
359        PCRE_ERROR_BADUTF8_OFFSET (-11). Note: the whole string is still checked;
360        this is necessary because there may be backward assertions in the pattern.
361        When matching the same subject several times, it may save resources to use
362        PCRE_NO_UTF8_CHECK on all but the first call if the string is long.
364     4. The code for checking the validity of UTF-8 strings has been tightened so
365        that it rejects (a) strings containing 0xfe or 0xff bytes and (b) strings
366        containing "overlong sequences".
368     5. Fixed a bug (appearing twice) that I could not find any way of exploiting!
369        I had written "if ((digitab[*p++] && chtab_digit) == 0)" where the "&&"
370        should have been "&", but it just so happened that all the cases this let
371        through by mistake were picked up later in the function.
373     6. I had used a variable called "isblank" - this is a C99 function, causing
374        some compilers to warn. To avoid this, I renamed it (as "blankclass").
376     7. Cosmetic: (a) only output another newline at the end of pcretest if it is
377        prompting; (b) run "./pcretest /dev/null" at the start of the test script
378        so the version is shown; (c) stop "make test" echoing "./RunTest".
380     8. Added patches from David Burgess to enable PCRE to run on EBCDIC systems.
382     9. The prototype for memmove() for systems that don't have it was using
383        size_t, but the inclusion of the header that defines size_t was later. I've
384        moved the #includes for the C headers earlier to avoid this.
386    10. Added some adjustments to the code to make it easier to compiler on certain
387        special systems:
389          (a) Some "const" qualifiers were missing.
390          (b) Added the macro EXPORT before all exported functions; by default this
391              is defined to be empty.
392          (c) Changed the dftables auxiliary program (that builds chartables.c) so
393              that it reads its output file name as an argument instead of writing
394              to the standard output and assuming this can be redirected.
396    11. In UTF-8 mode, if a recursive reference (e.g. (?1)) followed a character
397        class containing characters with values greater than 255, PCRE compilation
398        went into a loop.
400    12. A recursive reference to a subpattern that was within another subpattern
401        that had a minimum quantifier of zero caused PCRE to crash. For example,
402        (x(y(?2))z)? provoked this bug with a subject that got as far as the
403        recursion. If the recursively-called subpattern itself had a zero repeat,
404        that was OK.
406    13. In pcretest, the buffer for reading a data line was set at 30K, but the
407        buffer into which it was copied (for escape processing) was still set at
408        1024, so long lines caused crashes.
410    14. A pattern such as /[ab]{1,3}+/ failed to compile, giving the error
411        "internal error: code overflow...". This applied to any character class
412        that was followed by a possessive quantifier.
414    15. Modified the Makefile to add libpcre.la as a prerequisite for
415        libpcreposix.la because I was told this is needed for a parallel build to
416        work.
418    16. If a pattern that contained .* following optional items at the start was
419        studied, the wrong optimizing data was generated, leading to matching
420        errors. For example, studying /[ab]*.*c/ concluded, erroneously, that any
421        matching string must start with a or b or c. The correct conclusion for
422        this pattern is that a match can start with any character.
425    Version 4.4 13-Aug-03
426    ---------------------
428     1. In UTF-8 mode, a character class containing characters with values between
429        127 and 255 was not handled correctly if the compiled pattern was studied.
430        In fixing this, I have also improved the studying algorithm for such
431        classes (slightly).
433     2. Three internal functions had redundant arguments passed to them. Removal
434        might give a very teeny performance improvement.
436     3. Documentation bug: the value of the capture_top field in a callout is *one
437        more than* the number of the hightest numbered captured substring.
439     4. The Makefile linked pcretest and pcregrep with -lpcre, which could result
440        in incorrectly linking with a previously installed version. They now link
441        explicitly with libpcre.la.
443     5. configure.in no longer needs to recognize Cygwin specially.
445     6. A problem in pcre.in for Windows platforms is fixed.
447     7. If a pattern was successfully studied, and the -d (or /D) flag was given to
448        pcretest, it used to include the size of the study block as part of its
449        output. Unfortunately, the structure contains a field that has a different
450        size on different hardware architectures. This meant that the tests that
451        showed this size failed. As the block is currently always of a fixed size,
452        this information isn't actually particularly useful in pcretest output, so
453        I have just removed it.
455     8. Three pre-processor statements accidentally did not start in column 1.
456        Sadly, there are *still* compilers around that complain, even though
457        standard C has not required this for well over a decade. Sigh.
459     9. In pcretest, the code for checking callouts passed small integers in the
460        callout_data field, which is a void * field. However, some picky compilers
461        complained about the casts involved for this on 64-bit systems. Now
462        pcretest passes the address of the small integer instead, which should get
463        rid of the warnings.
465    10. By default, when in UTF-8 mode, PCRE now checks for valid UTF-8 strings at
466        both compile and run time, and gives an error if an invalid UTF-8 sequence
467        is found. There is a option for disabling this check in cases where the
468        string is known to be correct and/or the maximum performance is wanted.
470    11. In response to a bug report, I changed one line in Makefile.in from
472            -Wl,--out-implib,.libs/lib@WIN_PREFIX@pcreposix.dll.a \
473        to
474            -Wl,--out-implib,.libs/@WIN_PREFIX@libpcreposix.dll.a \
476        to look similar to other lines, but I have no way of telling whether this
477        is the right thing to do, as I do not use Windows. No doubt I'll get told
478        if it's wrong...
481  Version 4.3 21-May-03  Version 4.3 21-May-03
482  ---------------------  ---------------------

Removed from v.69  
changed lines
  Added in v.79

  ViewVC Help
Powered by ViewVC 1.1.5