ViewVC logotype

Diff of /code/trunk/ChangeLog

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 51 by nigel, Sat Feb 24 21:39:37 2007 UTC revision 75 by nigel, Sat Feb 24 21:40:37 2007 UTC
# Line 1  Line 1 
1  ChangeLog for PCRE  ChangeLog for PCRE
2  ------------------  ------------------
4    Version 5.0 13-Sep-04
5    ---------------------
7     1. Internal change: literal characters are no longer packed up into items
8        containing multiple characters in a single byte-string. Each character
9        is now matched using a separate opcode. However, there may be more than one
10        byte in the character in UTF-8 mode.
12     2. The pcre_callout_block structure has two new fields: pattern_position and
13        next_item_length. These contain the offset in the pattern to the next match
14        item, and its length, respectively.
16     3. The PCRE_AUTO_CALLOUT option for pcre_compile() requests the automatic
17        insertion of callouts before each pattern item. Added the /C option to
18        pcretest to make use of this.
20     4. On the advice of a Windows user, the lines
22          #if defined(_WIN32) || defined(WIN32)
23          _setmode( _fileno( stdout ), 0x8000 );
24          #endif  /* defined(_WIN32) || defined(WIN32) */
26        have been added to the source of pcretest. This apparently does useful
27        magic in relation to line terminators.
29     5. Changed "r" and "w" in the calls to fopen() in pcretest to "rb" and "wb"
30        for the benefit of those environments where the "b" makes a difference.
32     6. The icc compiler has the same options as gcc, but "configure" doesn't seem
33        to know about it. I have put a hack into configure.in that adds in code
34        to set GCC=yes if CC=icc. This seems to end up at a point in the
35        generated configure script that is early enough to affect the setting of
36        compiler options, which is what is needed, but I have no means of testing
37        whether it really works. (The user who reported this had patched the
38        generated configure script, which of course I cannot do.)
40        LATER: After change 22 below (new libtool files), the configure script
41        seems to know about icc (and also ecc). Therefore, I have commented out
42        this hack in configure.in.
44     7. Added support for pkg-config (2 patches were sent in).
46     8. Negated POSIX character classes that used a combination of internal tables
47        were completely broken. These were [[:^alpha:]], [[:^alnum:]], and
48        [[:^ascii]]. Typically, they would match almost any characters. The other
49        POSIX classes were not broken in this way.
51     9. Matching the pattern "\b.*?" against "ab cd", starting at offset 1, failed
52        to find the match, as PCRE was deluded into thinking that the match had to
53        start at the start point or following a newline. The same bug applied to
54        patterns with negative forward assertions or any backward assertions
55        preceding ".*" at the start, unless the pattern required a fixed first
56        character. This was a failing pattern: "(?!.bcd).*". The bug is now fixed.
58    10. In UTF-8 mode, when moving forwards in the subject after a failed match
59        starting at the last subject character, bytes beyond the end of the subject
60        string were read.
62    11. Renamed the variable "class" as "classbits" to make life easier for C++
63        users. (Previously there was a macro definition, but it apparently wasn't
64        enough.)
66    12. Added the new field "tables" to the extra data so that tables can be passed
67        in at exec time, or the internal tables can be re-selected. This allows
68        a compiled regex to be saved and re-used at a later time by a different
69        program that might have everything at different addresses.
71    13. Modified the pcre-config script so that, when run on Solaris, it shows a
72        -R library as well as a -L library.
74    14. The debugging options of pcretest (-d on the command line or D on a
75        pattern) showed incorrect output for anything following an extended class
76        that contained multibyte characters and which was followed by a quantifier.
78    15. Added optional support for general category Unicode character properties
79        via the \p, \P, and \X escapes. Unicode property support implies UTF-8
80        support. It adds about 90K to the size of the library. The meanings of the
81        inbuilt class escapes such as \d and \s have NOT been changed.
83    16. Updated pcredemo.c to include calls to free() to release the memory for the
84        compiled pattern.
86    17. The generated file chartables.c was being created in the source directory
87        instead of in the building directory. This caused the build to fail if the
88        source directory was different from the building directory, and was
89        read-only.
91    18. Added some sample Win commands from Mark Tetrode into the NON-UNIX-USE
92        file. No doubt somebody will tell me if they don't make sense... Also added
93        Dan Mooney's comments about building on OpenVMS.
95    19. Added support for partial matching via the PCRE_PARTIAL option for
96        pcre_exec() and the \P data escape in pcretest.
98    20. Extended pcretest with 3 new pattern features:
100        (i)   A pattern option of the form ">rest-of-line" causes pcretest to
101              write the compiled pattern to the file whose name is "rest-of-line".
102              This is a straight binary dump of the data, with the saved pointer to
103              the character tables forced to be NULL. The study data, if any, is
104              written too. After writing, pcretest reads a new pattern.
106        (ii)  If, instead of a pattern, "<rest-of-line" is given, pcretest reads a
107              compiled pattern from the given file. There must not be any
108              occurrences of "<" in the file name (pretty unlikely); if there are,
109              pcretest will instead treat the initial "<" as a pattern delimiter.
110              After reading in the pattern, pcretest goes on to read data lines as
111              usual.
113        (iii) The F pattern option causes pcretest to flip the bytes in the 32-bit
114              and 16-bit fields in a compiled pattern, to simulate a pattern that
115              was compiled on a host of opposite endianness.
117    21. The pcre-exec() function can now cope with patterns that were compiled on
118        hosts of opposite endianness, with this restriction:
120          As for any compiled expression that is saved and used later, the tables
121          pointer field cannot be preserved; the extra_data field in the arguments
122          to pcre_exec() should be used to pass in a tables address if a value
123          other than the default internal tables were used at compile time.
125    22. Calling pcre_exec() with a negative value of the "ovecsize" parameter is
126        now diagnosed as an error. Previously, most of the time, a negative number
127        would have been treated as zero, but if in addition "ovector" was passed as
128        NULL, a crash could occur.
130    23. Updated the files ltmain.sh, config.sub, config.guess, and aclocal.m4 with
131        new versions from the libtool 1.5 distribution (the last one is a copy of
132        a file called libtool.m4). This seems to have fixed the need to patch
133        "configure" to support Darwin 1.3 (which I used to do). However, I still
134        had to patch ltmain.sh to ensure that ${SED} is set (it isn't on my
135        workstation).
137    24. Changed the PCRE licence to be the more standard "BSD" licence.
140    Version 4.5 01-Dec-03
141    ---------------------
143     1. There has been some re-arrangement of the code for the match() function so
144        that it can be compiled in a version that does not call itself recursively.
145        Instead, it keeps those local variables that need separate instances for
146        each "recursion" in a frame on the heap, and gets/frees frames whenever it
147        needs to "recurse". Keeping track of where control must go is done by means
148        of setjmp/longjmp. The whole thing is implemented by a set of macros that
149        hide most of the details from the main code, and operates only if
150        NO_RECURSE is defined while compiling pcre.c. If PCRE is built using the
151        "configure" mechanism, "--disable-stack-for-recursion" turns on this way of
152        operating.
154        To make it easier for callers to provide specially tailored get/free
155        functions for this usage, two new functions, pcre_stack_malloc, and
156        pcre_stack_free, are used. They are always called in strict stacking order,
157        and the size of block requested is always the same.
159        The PCRE_CONFIG_STACKRECURSE info parameter can be used to find out whether
160        PCRE has been compiled to use the stack or the heap for recursion. The
161        -C option of pcretest uses this to show which version is compiled.
163        A new data escape \S, is added to pcretest; it causes the amounts of store
164        obtained and freed by both kinds of malloc/free at match time to be added
165        to the output.
167     2. Changed the locale test to use "fr_FR" instead of "fr" because that's
168        what's available on my current Linux desktop machine.
170     3. When matching a UTF-8 string, the test for a valid string at the start has
171        been extended. If start_offset is not zero, PCRE now checks that it points
172        to a byte that is the start of a UTF-8 character. If not, it returns
173        PCRE_ERROR_BADUTF8_OFFSET (-11). Note: the whole string is still checked;
174        this is necessary because there may be backward assertions in the pattern.
175        When matching the same subject several times, it may save resources to use
176        PCRE_NO_UTF8_CHECK on all but the first call if the string is long.
178     4. The code for checking the validity of UTF-8 strings has been tightened so
179        that it rejects (a) strings containing 0xfe or 0xff bytes and (b) strings
180        containing "overlong sequences".
182     5. Fixed a bug (appearing twice) that I could not find any way of exploiting!
183        I had written "if ((digitab[*p++] && chtab_digit) == 0)" where the "&&"
184        should have been "&", but it just so happened that all the cases this let
185        through by mistake were picked up later in the function.
187     6. I had used a variable called "isblank" - this is a C99 function, causing
188        some compilers to warn. To avoid this, I renamed it (as "blankclass").
190     7. Cosmetic: (a) only output another newline at the end of pcretest if it is
191        prompting; (b) run "./pcretest /dev/null" at the start of the test script
192        so the version is shown; (c) stop "make test" echoing "./RunTest".
194     8. Added patches from David Burgess to enable PCRE to run on EBCDIC systems.
196     9. The prototype for memmove() for systems that don't have it was using
197        size_t, but the inclusion of the header that defines size_t was later. I've
198        moved the #includes for the C headers earlier to avoid this.
200    10. Added some adjustments to the code to make it easier to compiler on certain
201        special systems:
203          (a) Some "const" qualifiers were missing.
204          (b) Added the macro EXPORT before all exported functions; by default this
205              is defined to be empty.
206          (c) Changed the dftables auxiliary program (that builds chartables.c) so
207              that it reads its output file name as an argument instead of writing
208              to the standard output and assuming this can be redirected.
210    11. In UTF-8 mode, if a recursive reference (e.g. (?1)) followed a character
211        class containing characters with values greater than 255, PCRE compilation
212        went into a loop.
214    12. A recursive reference to a subpattern that was within another subpattern
215        that had a minimum quantifier of zero caused PCRE to crash. For example,
216        (x(y(?2))z)? provoked this bug with a subject that got as far as the
217        recursion. If the recursively-called subpattern itself had a zero repeat,
218        that was OK.
220    13. In pcretest, the buffer for reading a data line was set at 30K, but the
221        buffer into which it was copied (for escape processing) was still set at
222        1024, so long lines caused crashes.
224    14. A pattern such as /[ab]{1,3}+/ failed to compile, giving the error
225        "internal error: code overflow...". This applied to any character class
226        that was followed by a possessive quantifier.
228    15. Modified the Makefile to add libpcre.la as a prerequisite for
229        libpcreposix.la because I was told this is needed for a parallel build to
230        work.
232    16. If a pattern that contained .* following optional items at the start was
233        studied, the wrong optimizing data was generated, leading to matching
234        errors. For example, studying /[ab]*.*c/ concluded, erroneously, that any
235        matching string must start with a or b or c. The correct conclusion for
236        this pattern is that a match can start with any character.
239    Version 4.4 13-Aug-03
240    ---------------------
242     1. In UTF-8 mode, a character class containing characters with values between
243        127 and 255 was not handled correctly if the compiled pattern was studied.
244        In fixing this, I have also improved the studying algorithm for such
245        classes (slightly).
247     2. Three internal functions had redundant arguments passed to them. Removal
248        might give a very teeny performance improvement.
250     3. Documentation bug: the value of the capture_top field in a callout is *one
251        more than* the number of the hightest numbered captured substring.
253     4. The Makefile linked pcretest and pcregrep with -lpcre, which could result
254        in incorrectly linking with a previously installed version. They now link
255        explicitly with libpcre.la.
257     5. configure.in no longer needs to recognize Cygwin specially.
259     6. A problem in pcre.in for Windows platforms is fixed.
261     7. If a pattern was successfully studied, and the -d (or /D) flag was given to
262        pcretest, it used to include the size of the study block as part of its
263        output. Unfortunately, the structure contains a field that has a different
264        size on different hardware architectures. This meant that the tests that
265        showed this size failed. As the block is currently always of a fixed size,
266        this information isn't actually particularly useful in pcretest output, so
267        I have just removed it.
269     8. Three pre-processor statements accidentally did not start in column 1.
270        Sadly, there are *still* compilers around that complain, even though
271        standard C has not required this for well over a decade. Sigh.
273     9. In pcretest, the code for checking callouts passed small integers in the
274        callout_data field, which is a void * field. However, some picky compilers
275        complained about the casts involved for this on 64-bit systems. Now
276        pcretest passes the address of the small integer instead, which should get
277        rid of the warnings.
279    10. By default, when in UTF-8 mode, PCRE now checks for valid UTF-8 strings at
280        both compile and run time, and gives an error if an invalid UTF-8 sequence
281        is found. There is a option for disabling this check in cases where the
282        string is known to be correct and/or the maximum performance is wanted.
284    11. In response to a bug report, I changed one line in Makefile.in from
286            -Wl,--out-implib,.libs/lib@WIN_PREFIX@pcreposix.dll.a \
287        to
288            -Wl,--out-implib,.libs/@WIN_PREFIX@libpcreposix.dll.a \
290        to look similar to other lines, but I have no way of telling whether this
291        is the right thing to do, as I do not use Windows. No doubt I'll get told
292        if it's wrong...
295    Version 4.3 21-May-03
296    ---------------------
298    1. Two instances of @WIN_PREFIX@ omitted from the Windows targets in the
299       Makefile.
301    2. Some refactoring to improve the quality of the code:
303       (i)   The utf8_table... variables are now declared "const".
305       (ii)  The code for \cx, which used the "case flipping" table to upper case
306             lower case letters, now just substracts 32. This is ASCII-specific,
307             but the whole concept of \cx is ASCII-specific, so it seems
308             reasonable.
310       (iii) PCRE was using its character types table to recognize decimal and
311             hexadecimal digits in the pattern. This is silly, because it handles
312             only 0-9, a-f, and A-F, but the character types table is locale-
313             specific, which means strange things might happen. A private
314             table is now used for this - though it costs 256 bytes, a table is
315             much faster than multiple explicit tests. Of course, the standard
316             character types table is still used for matching digits in subject
317             strings against \d.
319       (iv)  Strictly, the identifier ESC_t is reserved by POSIX (all identifiers
320             ending in _t are). So I've renamed it as ESC_tee.
322    3. The first argument for regexec() in the POSIX wrapper should have been
323       defined as "const".
325    4. Changed pcretest to use malloc() for its buffers so that they can be
326       Electric Fenced for debugging.
328    5. There were several places in the code where, in UTF-8 mode, PCRE would try
329       to read one or more bytes before the start of the subject string. Often this
330       had no effect on PCRE's behaviour, but in some circumstances it could
331       provoke a segmentation fault.
333    6. A lookbehind at the start of a pattern in UTF-8 mode could also cause PCRE
334       to try to read one or more bytes before the start of the subject string.
336    7. A lookbehind in a pattern matched in non-UTF-8 mode on a PCRE compiled with
337       UTF-8 support could misbehave in various ways if the subject string
338       contained bytes with the 0x80 bit set and the 0x40 bit unset in a lookbehind
339       area. (PCRE was not checking for the UTF-8 mode flag, and trying to move
340       back over UTF-8 characters.)
343    Version 4.2 14-Apr-03
344    ---------------------
346    1. Typo "#if SUPPORT_UTF8" instead of "#ifdef SUPPORT_UTF8" fixed.
348    2. Changes to the building process, supplied by Ronald Landheer-Cieslak
349         [ON_WINDOWS]: new variable, "#" on non-Windows platforms
350         [NOT_ON_WINDOWS]: new variable, "#" on Windows platforms
351         [WIN_PREFIX]: new variable, "cyg" for Cygwin
352         * Makefile.in: use autoconf substitution for OBJEXT, EXEEXT, BUILD_OBJEXT
353           and BUILD_EXEEXT
354         Note: automatic setting of the BUILD variables is not yet working
355         set CPPFLAGS and BUILD_CPPFLAGS (but don't use yet) - should be used at
356           compile-time but not at link-time
357         [LINK]: use for linking executables only
358         make different versions for Windows and non-Windows
359         [LINKLIB]: new variable, copy of UNIX-style LINK, used for linking
360           libraries
361         [LINK_FOR_BUILD]: new variable
362         [OBJEXT]: use throughout
363         [EXEEXT]: use throughout
364         <winshared>: new target
365         <wininstall>: new target
366         <dftables.o>: use native compiler
367         <dftables>: use native linker
368         <install>: handle Windows platform correctly
369         <clean>: ditto
370         <check>: ditto
371         copy DLL to top builddir before testing
373       As part of these changes, -no-undefined was removed again. This was reported
374       to give trouble on HP-UX 11.0, so getting rid of it seems like a good idea
375       in any case.
377    3. Some tidies to get rid of compiler warnings:
379       . In the match_data structure, match_limit was an unsigned long int, whereas
380         match_call_count was an int. I've made them both unsigned long ints.
382       . In pcretest the fact that a const uschar * doesn't automatically cast to
383         a void * provoked a warning.
385       . Turning on some more compiler warnings threw up some "shadow" variables
386         and a few more missing casts.
388    4. If PCRE was complied with UTF-8 support, but called without the PCRE_UTF8
389       option, a class that contained a single character with a value between 128
390       and 255 (e.g. /[\xFF]/) caused PCRE to crash.
392    5. If PCRE was compiled with UTF-8 support, but called without the PCRE_UTF8
393       option, a class that contained several characters, but with at least one
394       whose value was between 128 and 255 caused PCRE to crash.
397    Version 4.1 12-Mar-03
398    ---------------------
400    1. Compiling with gcc -pedantic found a couple of places where casts were
401    needed, and a string in dftables.c that was longer than standard compilers are
402    required to support.
404    2. Compiling with Sun's compiler found a few more places where the code could
405    be tidied up in order to avoid warnings.
407    3. The variables for cross-compiling were called HOST_CC and HOST_CFLAGS; the
408    first of these names is deprecated in the latest Autoconf in favour of the name
409    CC_FOR_BUILD, because "host" is typically used to mean the system on which the
410    compiled code will be run. I can't find a reference for HOST_CFLAGS, but by
411    analogy I have changed it to CFLAGS_FOR_BUILD.
413    4. Added -no-undefined to the linking command in the Makefile, because this is
414    apparently helpful for Windows. To make it work, also added "-L. -lpcre" to the
415    linking step for the pcreposix library.
417    5. PCRE was failing to diagnose the case of two named groups with the same
418    name.
420    6. A problem with one of PCRE's optimizations was discovered. PCRE remembers a
421    literal character that is needed in the subject for a match, and scans along to
422    ensure that it is present before embarking on the full matching process. This
423    saves time in cases of nested unlimited repeats that are never going to match.
424    Problem: the scan can take a lot of time if the subject is very long (e.g.
425    megabytes), thus penalizing straightforward matches. It is now done only if the
426    amount of subject to be scanned is less than 1000 bytes.
428    7. A lesser problem with the same optimization is that it was recording the
429    first character of an anchored pattern as "needed", thus provoking a search
430    right along the subject, even when the first match of the pattern was going to
431    fail. The "needed" character is now not set for anchored patterns, unless it
432    follows something in the pattern that is of non-fixed length. Thus, it still
433    fulfils its original purpose of finding quick non-matches in cases of nested
434    unlimited repeats, but isn't used for simple anchored patterns such as /^abc/.
437    Version 4.0 17-Feb-03
438    ---------------------
440    1. If a comment in an extended regex that started immediately after a meta-item
441    extended to the end of string, PCRE compiled incorrect data. This could lead to
442    all kinds of weird effects. Example: /#/ was bad; /()#/ was bad; /a#/ was not.
444    2. Moved to autoconf 2.53 and libtool 1.4.2.
446    3. Perl 5.8 no longer needs "use utf8" for doing UTF-8 things. Consequently,
447    the special perltest8 script is no longer needed - all the tests can be run
448    from a single perltest script.
450    4. From 5.004, Perl has not included the VT character (0x0b) in the set defined
451    by \s. It has now been removed in PCRE. This means it isn't recognized as
452    whitespace in /x regexes too, which is the same as Perl. Note that the POSIX
453    class [:space:] *does* include VT, thereby creating a mess.
455    5. Added the class [:blank:] (a GNU extension from Perl 5.8) to match only
456    space and tab.
458    6. Perl 5.005 was a long time ago. It's time to amalgamate the tests that use
459    its new features into the main test script, reducing the number of scripts.
461    7. Perl 5.8 has changed the meaning of patterns like /a(?i)b/. Earlier versions
462    were backward compatible, and made the (?i) apply to the whole pattern, as if
463    /i were given. Now it behaves more logically, and applies the option setting
464    only to what follows. PCRE has been changed to follow suit. However, if it
465    finds options settings right at the start of the pattern, it extracts them into
466    the global options, as before. Thus, they show up in the info data.
468    8. Added support for the \Q...\E escape sequence. Characters in between are
469    treated as literals. This is slightly different from Perl in that $ and @ are
470    also handled as literals inside the quotes. In Perl, they will cause variable
471    interpolation. Note the following examples:
473        Pattern            PCRE matches      Perl matches
475        \Qabc$xyz\E        abc$xyz           abc followed by the contents of $xyz
476        \Qabc\$xyz\E       abc\$xyz          abc\$xyz
477        \Qabc\E\$\Qxyz\E   abc$xyz           abc$xyz
479    For compatibility with Perl, \Q...\E sequences are recognized inside character
480    classes as well as outside them.
482    9. Re-organized 3 code statements in pcretest to avoid "overflow in
483    floating-point constant arithmetic" warnings from a Microsoft compiler. Added a
484    (size_t) cast to one statement in pcretest and one in pcreposix to avoid
485    signed/unsigned warnings.
487    10. SunOS4 doesn't have strtoul(). This was used only for unpicking the -o
488    option for pcretest, so I've replaced it by a simple function that does just
489    that job.
491    11. pcregrep was ending with code 0 instead of 2 for the commands "pcregrep" or
492    "pcregrep -".
494    12. Added "possessive quantifiers" ?+, *+, ++, and {,}+ which come from Sun's
495    Java package. This provides some syntactic sugar for simple cases of what my
496    documentation calls "once-only subpatterns". A pattern such as x*+ is the same
497    as (?>x*). In other words, if what is inside (?>...) is just a single repeated
498    item, you can use this simplified notation. Note that only makes sense with
499    greedy quantifiers. Consequently, the use of the possessive quantifier forces
500    greediness, whatever the setting of the PCRE_UNGREEDY option.
502    13. A change of greediness default within a pattern was not taking effect at
503    the current level for patterns like /(b+(?U)a+)/. It did apply to parenthesized
504    subpatterns that followed. Patterns like /b+(?U)a+/ worked because the option
505    was abstracted outside.
507    14. PCRE now supports the \G assertion. It is true when the current matching
508    position is at the start point of the match. This differs from \A when the
509    starting offset is non-zero. Used with the /g option of pcretest (or similar
510    code), it works in the same way as it does for Perl's /g option. If all
511    alternatives of a regex begin with \G, the expression is anchored to the start
512    match position, and the "anchored" flag is set in the compiled expression.
514    15. Some bugs concerning the handling of certain option changes within patterns
515    have been fixed. These applied to options other than (?ims). For example,
516    "a(?x: b c )d" did not match "XabcdY" but did match "Xa b c dY". It should have
517    been the other way round. Some of this was related to change 7 above.
519    16. PCRE now gives errors for /[.x.]/ and /[=x=]/ as unsupported POSIX
520    features, as Perl does. Previously, PCRE gave the warnings only for /[[.x.]]/
521    and /[[=x=]]/. PCRE now also gives an error for /[:name:]/ because it supports
522    POSIX classes only within a class (e.g. /[[:alpha:]]/).
524    17. Added support for Perl's \C escape. This matches one byte, even in UTF8
525    mode. Unlike ".", it always matches newline, whatever the setting of
526    PCRE_DOTALL. However, PCRE does not permit \C to appear in lookbehind
527    assertions. Perl allows it, but it doesn't (in general) work because it can't
528    calculate the length of the lookbehind. At least, that's the case for Perl
529    5.8.0 - I've been told they are going to document that it doesn't work in
530    future.
532    18. Added an error diagnosis for escapes that PCRE does not support: these are
533    \L, \l, \N, \P, \p, \U, \u, and \X.
535    19. Although correctly diagnosing a missing ']' in a character class, PCRE was
536    reading past the end of the pattern in cases such as /[abcd/.
538    20. PCRE was getting more memory than necessary for patterns with classes that
539    contained both POSIX named classes and other characters, e.g. /[[:space:]abc/.
541    21. Added some code, conditional on #ifdef VPCOMPAT, to make life easier for
542    compiling PCRE for use with Virtual Pascal.
544    22. Small fix to the Makefile to make it work properly if the build is done
545    outside the source tree.
547    23. Added a new extension: a condition to go with recursion. If a conditional
548    subpattern starts with (?(R) the "true" branch is used if recursion has
549    happened, whereas the "false" branch is used only at the top level.
551    24. When there was a very long string of literal characters (over 255 bytes
552    without UTF support, over 250 bytes with UTF support), the computation of how
553    much memory was required could be incorrect, leading to segfaults or other
554    strange effects.
556    25. PCRE was incorrectly assuming anchoring (either to start of subject or to
557    start of line for a non-DOTALL pattern) when a pattern started with (.*) and
558    there was a subsequent back reference to those brackets. This meant that, for
559    example, /(.*)\d+\1/ failed to match "abc123bc". Unfortunately, it isn't
560    possible to check for precisely this case. All we can do is abandon the
561    optimization if .* occurs inside capturing brackets when there are any back
562    references whatsoever. (See below for a better fix that came later.)
564    26. The handling of the optimization for finding the first character of a
565    non-anchored pattern, and for finding a character that is required later in the
566    match were failing in some cases. This didn't break the matching; it just
567    failed to optimize when it could. The way this is done has been re-implemented.
569    27. Fixed typo in error message for invalid (?R item (it said "(?p").
571    28. Added a new feature that provides some of the functionality that Perl
572    provides with (?{...}). The facility is termed a "callout". The way it is done
573    in PCRE is for the caller to provide an optional function, by setting
574    pcre_callout to its entry point. Like pcre_malloc and pcre_free, this is a
575    global variable. By default it is unset, which disables all calling out. To get
576    the function called, the regex must include (?C) at appropriate points. This
577    is, in fact, equivalent to (?C0), and any number <= 255 may be given with (?C).
578    This provides a means of identifying different callout points. When PCRE
579    reaches such a point in the regex, if pcre_callout has been set, the external
580    function is called. It is provided with data in a structure called
581    pcre_callout_block, which is defined in pcre.h. If the function returns 0,
582    matching continues; if it returns a non-zero value, the match at the current
583    point fails. However, backtracking will occur if possible. [This was changed
584    later and other features added - see item 49 below.]
586    29. pcretest is upgraded to test the callout functionality. It provides a
587    callout function that displays information. By default, it shows the start of
588    the match and the current position in the text. There are some new data escapes
589    to vary what happens:
591        \C+         in addition, show current contents of captured substrings
592        \C-         do not supply a callout function
593        \C!n        return 1 when callout number n is reached
594        \C!n!m      return 1 when callout number n is reached for the mth time
596    30. If pcregrep was called with the -l option and just a single file name, it
597    output "<stdin>" if a match was found, instead of the file name.
599    31. Improve the efficiency of the POSIX API to PCRE. If the number of capturing
600    slots is less than POSIX_MALLOC_THRESHOLD, use a block on the stack to pass to
601    pcre_exec(). This saves a malloc/free per call. The default value of
602    POSIX_MALLOC_THRESHOLD is 10; it can be changed by --with-posix-malloc-threshold
603    when configuring.
605    32. The default maximum size of a compiled pattern is 64K. There have been a
606    few cases of people hitting this limit. The code now uses macros to handle the
607    storing of links as offsets within the compiled pattern. It defaults to 2-byte
608    links, but this can be changed to 3 or 4 bytes by --with-link-size when
609    configuring. Tests 2 and 5 work only with 2-byte links because they output
610    debugging information about compiled patterns.
612    33. Internal code re-arrangements:
614    (a) Moved the debugging function for printing out a compiled regex into
615        its own source file (printint.c) and used #include to pull it into
616        pcretest.c and, when DEBUG is defined, into pcre.c, instead of having two
617        separate copies.
619    (b) Defined the list of op-code names for debugging as a macro in
620        internal.h so that it is next to the definition of the opcodes.
622    (c) Defined a table of op-code lengths for simpler skipping along compiled
623        code. This is again a macro in internal.h so that it is next to the
624        definition of the opcodes.
626    34. Added support for recursive calls to individual subpatterns, along the
627    lines of Robin Houston's patch (but implemented somewhat differently).
629    35. Further mods to the Makefile to help Win32. Also, added code to pcregrep to
630    allow it to read and process whole directories in Win32. This code was
631    contributed by Lionel Fourquaux; it has not been tested by me.
633    36. Added support for named subpatterns. The Python syntax (?P<name>...) is
634    used to name a group. Names consist of alphanumerics and underscores, and must
635    be unique. Back references use the syntax (?P=name) and recursive calls use
636    (?P>name) which is a PCRE extension to the Python extension. Groups still have
637    numbers. The function pcre_fullinfo() can be used after compilation to extract
638    a name/number map. There are three relevant calls:
640      PCRE_INFO_NAMEENTRYSIZE        yields the size of each entry in the map
641      PCRE_INFO_NAMECOUNT            yields the number of entries
642      PCRE_INFO_NAMETABLE            yields a pointer to the map.
644    The map is a vector of fixed-size entries. The size of each entry depends on
645    the length of the longest name used. The first two bytes of each entry are the
646    group number, most significant byte first. There follows the corresponding
647    name, zero terminated. The names are in alphabetical order.
649    37. Make the maximum literal string in the compiled code 250 for the non-UTF-8
650    case instead of 255. Making it the same both with and without UTF-8 support
651    means that the same test output works with both.
653    38. There was a case of malloc(0) in the POSIX testing code in pcretest. Avoid
654    calling malloc() with a zero argument.
656    39. Change 25 above had to resort to a heavy-handed test for the .* anchoring
657    optimization. I've improved things by keeping a bitmap of backreferences with
658    numbers 1-31 so that if .* occurs inside capturing brackets that are not in
659    fact referenced, the optimization can be applied. It is unlikely that a
660    relevant occurrence of .* (i.e. one which might indicate anchoring or forcing
661    the match to follow \n) will appear inside brackets with a number greater than
662    31, but if it does, any back reference > 31 suppresses the optimization.
664    40. Added a new compile-time option PCRE_NO_AUTO_CAPTURE. This has the effect
665    of disabling numbered capturing parentheses. Any opening parenthesis that is
666    not followed by ? behaves as if it were followed by ?: but named parentheses
667    can still be used for capturing (and they will acquire numbers in the usual
668    way).
670    41. Redesigned the return codes from the match() function into yes/no/error so
671    that errors can be passed back from deep inside the nested calls. A malloc
672    failure while inside a recursive subpattern call now causes the
673    PCRE_ERROR_NOMEMORY return instead of quietly going wrong.
675    42. It is now possible to set a limit on the number of times the match()
676    function is called in a call to pcre_exec(). This facility makes it possible to
677    limit the amount of recursion and backtracking, though not in a directly
678    obvious way, because the match() function is used in a number of different
679    circumstances. The count starts from zero for each position in the subject
680    string (for non-anchored patterns). The default limit is, for compatibility, a
681    large number, namely 10 000 000. You can change this in two ways:
683    (a) When configuring PCRE before making, you can use --with-match-limit=n
684        to set a default value for the compiled library.
686    (b) For each call to pcre_exec(), you can pass a pcre_extra block in which
687        a different value is set. See 45 below.
689    If the limit is exceeded, pcre_exec() returns PCRE_ERROR_MATCHLIMIT.
691    43. Added a new function pcre_config(int, void *) to enable run-time extraction
692    of things that can be changed at compile time. The first argument specifies
693    what is wanted and the second points to where the information is to be placed.
694    The current list of available information is:
698    The output is an integer that is set to one if UTF-8 support is available;
699    otherwise it is set to zero.
703    The output is an integer that it set to the value of the code that is used for
704    newline. It is either LF (10) or CR (13).
708    The output is an integer that contains the number of bytes used for internal
709    linkage in compiled expressions. The value is 2, 3, or 4. See item 32 above.
713    The output is an integer that contains the threshold above which the POSIX
714    interface uses malloc() for output vectors. See item 31 above.
718    The output is an unsigned integer that contains the default limit of the number
719    of match() calls in a pcre_exec() execution. See 42 above.
721    44. pcretest has been upgraded by the addition of the -C option. This causes it
722    to extract all the available output from the new pcre_config() function, and to
723    output it. The program then exits immediately.
725    45. A need has arisen to pass over additional data with calls to pcre_exec() in
726    order to support additional features. One way would have been to define
727    pcre_exec2() (for example) with extra arguments, but this would not have been
728    extensible, and would also have required all calls to the original function to
729    be mapped to the new one. Instead, I have chosen to extend the mechanism that
730    is used for passing in "extra" data from pcre_study().
732    The pcre_extra structure is now exposed and defined in pcre.h. It currently
733    contains the following fields:
735      flags         a bitmap indicating which of the following fields are set
736      study_data    opaque data from pcre_study()
737      match_limit   a way of specifying a limit on match() calls for a specific
738                      call to pcre_exec()
739      callout_data  data for callouts (see 49 below)
741    The flag bits are also defined in pcre.h, and are
747    The pcre_study() function now returns one of these new pcre_extra blocks, with
748    the actual study data pointed to by the study_data field, and the
749    PCRE_EXTRA_STUDY_DATA flag set. This can be passed directly to pcre_exec() as
750    before. That is, this change is entirely upwards-compatible and requires no
751    change to existing code.
753    If you want to pass in additional data to pcre_exec(), you can either place it
754    in a pcre_extra block provided by pcre_study(), or create your own pcre_extra
755    block.
757    46. pcretest has been extended to test the PCRE_EXTRA_MATCH_LIMIT feature. If a
758    data string contains the escape sequence \M, pcretest calls pcre_exec() several
759    times with different match limits, until it finds the minimum value needed for
760    pcre_exec() to complete. The value is then output. This can be instructive; for
761    most simple matches the number is quite small, but for pathological cases it
762    gets very large very quickly.
764    47. There's a new option for pcre_fullinfo() called PCRE_INFO_STUDYSIZE. It
765    returns the size of the data block pointed to by the study_data field in a
766    pcre_extra block, that is, the value that was passed as the argument to
767    pcre_malloc() when PCRE was getting memory in which to place the information
768    created by pcre_study(). The fourth argument should point to a size_t variable.
769    pcretest has been extended so that this information is shown after a successful
770    pcre_study() call when information about the compiled regex is being displayed.
772    48. Cosmetic change to Makefile: there's no need to have / after $(DESTDIR)
773    because what follows is always an absolute path. (Later: it turns out that this
774    is more than cosmetic for MinGW, because it doesn't like empty path
775    components.)
777    49. Some changes have been made to the callout feature (see 28 above):
779    (i)  A callout function now has three choices for what it returns:
781           0  =>  success, carry on matching
782         > 0  =>  failure at this point, but backtrack if possible
783         < 0  =>  serious error, return this value from pcre_exec()
785         Negative values should normally be chosen from the set of PCRE_ERROR_xxx
786         values. In particular, returning PCRE_ERROR_NOMATCH forces a standard
787         "match failed" error. The error number PCRE_ERROR_CALLOUT is reserved for
788         use by callout functions. It will never be used by PCRE itself.
790    (ii) The pcre_extra structure (see 45 above) has a void * field called
791         callout_data, with corresponding flag bit PCRE_EXTRA_CALLOUT_DATA. The
792         pcre_callout_block structure has a field of the same name. The contents of
793         the field passed in the pcre_extra structure are passed to the callout
794         function in the corresponding field in the callout block. This makes it
795         easier to use the same callout-containing regex from multiple threads. For
796         testing, the pcretest program has a new data escape
798           \C*n        pass the number n (may be negative) as callout_data
800         If the callout function in pcretest receives a non-zero value as
801         callout_data, it returns that value.
803    50. Makefile wasn't handling CFLAGS properly when compiling dftables. Also,
804    there were some redundant $(CFLAGS) in commands that are now specified as
805    $(LINK), which already includes $(CFLAGS).
807    51. Extensions to UTF-8 support are listed below. These all apply when (a) PCRE
808    has been compiled with UTF-8 support *and* pcre_compile() has been compiled
809    with the PCRE_UTF8 flag. Patterns that are compiled without that flag assume
810    one-byte characters throughout. Note that case-insensitive matching applies
811    only to characters whose values are less than 256. PCRE doesn't support the
812    notion of cases for higher-valued characters.
814    (i)   A character class whose characters are all within 0-255 is handled as
815          a bit map, and the map is inverted for negative classes. Previously, a
816          character > 255 always failed to match such a class; however it should
817          match if the class was a negative one (e.g. [^ab]). This has been fixed.
819    (ii)  A negated character class with a single character < 255 is coded as
820          "not this character" (OP_NOT). This wasn't working properly when the test
821          character was multibyte, either singly or repeated.
823    (iii) Repeats of multibyte characters are now handled correctly in UTF-8
824          mode, for example: \x{100}{2,3}.
826    (iv)  The character escapes \b, \B, \d, \D, \s, \S, \w, and \W (either
827          singly or repeated) now correctly test multibyte characters. However,
828          PCRE doesn't recognize any characters with values greater than 255 as
829          digits, spaces, or word characters. Such characters always match \D, \S,
830          and \W, and never match \d, \s, or \w.
832    (v)   Classes may now contain characters and character ranges with values
833          greater than 255. For example: [ab\x{100}-\x{400}].
835    (vi)  pcregrep now has a --utf-8 option (synonym -u) which makes it call
836          PCRE in UTF-8 mode.
838    52. The info request value PCRE_INFO_FIRSTCHAR has been renamed
839    PCRE_INFO_FIRSTBYTE because it is a byte value. However, the old name is
840    retained for backwards compatibility. (Note that LASTLITERAL is also a byte
841    value.)
843    53. The single man page has become too large. I have therefore split it up into
844    a number of separate man pages. These also give rise to individual HTML pages;
845    these are now put in a separate directory, and there is an index.html page that
846    lists them all. Some hyperlinking between the pages has been installed.
848    54. Added convenience functions for handling named capturing parentheses.
850    55. Unknown escapes inside character classes (e.g. [\M]) and escapes that
851    aren't interpreted therein (e.g. [\C]) are literals in Perl. This is now also
852    true in PCRE, except when the PCRE_EXTENDED option is set, in which case they
853    are faulted.
855    56. Introduced HOST_CC and HOST_CFLAGS which can be set in the environment when
856    calling configure. These values are used when compiling the dftables.c program
857    which is run to generate the source of the default character tables. They
858    default to the values of CC and CFLAGS. If you are cross-compiling PCRE,
859    you will need to set these values.
861    57. Updated the building process for Windows DLL, as provided by Fred Cox.
864    Version 3.9 02-Jan-02
865    ---------------------
867    1. A bit of extraneous text had somehow crept into the pcregrep documentation.
869    2. If --disable-static was given, the building process failed when trying to
870    build pcretest and pcregrep. (For some reason it was using libtool to compile
871    them, which is not right, as they aren't part of the library.)
874    Version 3.8 18-Dec-01
875    ---------------------
877    1. The experimental UTF-8 code was completely screwed up. It was packing the
878    bytes in the wrong order. How dumb can you get?
881    Version 3.7 29-Oct-01
882    ---------------------
884    1. In updating pcretest to check change 1 of version 3.6, I screwed up.
885    This caused pcretest, when used on the test data, to segfault. Unfortunately,
886    this didn't happen under Solaris 8, where I normally test things.
888    2. The Makefile had to be changed to make it work on BSD systems, where 'make'
889    doesn't seem to recognize that ./xxx and xxx are the same file. (This entry
890    isn't in ChangeLog distributed with 3.7 because I forgot when I hastily made
891    this fix an hour or so after the initial 3.7 release.)
894    Version 3.6 23-Oct-01
895    ---------------------
897    1. Crashed with /(sens|respons)e and \1ibility/ and "sense and sensibility" if
898    offsets passed as NULL with zero offset count.
900    2. The config.guess and config.sub files had not been updated when I moved to
901    the latest autoconf.
904    Version 3.5 15-Aug-01
905    ---------------------
907    1. Added some missing #if !defined NOPOSIX conditionals in pcretest.c that
908    had been forgotten.
910    2. By using declared but undefined structures, we can avoid using "void"
911    definitions in pcre.h while keeping the internal definitions of the structures
912    private.
914    3. The distribution is now built using autoconf 2.50 and libtool 1.4. From a
915    user point of view, this means that both static and shared libraries are built
916    by default, but this can be individually controlled. More of the work of
917    handling this static/shared cases is now inside libtool instead of PCRE's make
918    file.
920    4. The pcretest utility is now installed along with pcregrep because it is
921    useful for users (to test regexs) and by doing this, it automatically gets
922    relinked by libtool. The documentation has been turned into a man page, so
923    there are now .1, .txt, and .html versions in /doc.
925    5. Upgrades to pcregrep:
926       (i)   Added long-form option names like gnu grep.
927       (ii)  Added --help to list all options with an explanatory phrase.
928       (iii) Added -r, --recursive to recurse into sub-directories.
929       (iv)  Added -f, --file to read patterns from a file.
931    6. pcre_exec() was referring to its "code" argument before testing that
932    argument for NULL (and giving an error if it was NULL).
934    7. Upgraded Makefile.in to allow for compiling in a different directory from
935    the source directory.
937    8. Tiny buglet in pcretest: when pcre_fullinfo() was called to retrieve the
938    options bits, the pointer it was passed was to an int instead of to an unsigned
939    long int. This mattered only on 64-bit systems.
941    9. Fixed typo (3.4/1) in pcre.h again. Sigh. I had changed pcre.h (which is
942    generated) instead of pcre.in, which it its source. Also made the same change
943    in several of the .c files.
945    10. A new release of gcc defines printf() as a macro, which broke pcretest
946    because it had an ifdef in the middle of a string argument for printf(). Fixed
947    by using separate calls to printf().
949    11. Added --enable-newline-is-cr and --enable-newline-is-lf to the configure
950    script, to force use of CR or LF instead of \n in the source. On non-Unix
951    systems, the value can be set in config.h.
953    12. The limit of 200 on non-capturing parentheses is a _nesting_ limit, not an
954    absolute limit. Changed the text of the error message to make this clear, and
955    likewise updated the man page.
957    13. The limit of 99 on the number of capturing subpatterns has been removed.
958    The new limit is 65535, which I hope will not be a "real" limit.
961  Version 3.4 22-Aug-00  Version 3.4 22-Aug-00
962  ---------------------  ---------------------

Removed from v.51  
changed lines
  Added in v.75

  ViewVC Help
Powered by ViewVC 1.1.5