/[pcre]/code/trunk/ChangeLog

Diff of /code/trunk/ChangeLog

Parent Directory | Revision Log | View Patch Patch

-revision 484 by ph10,
Mon Jan  4 17:11:54 2010 UTC
+revision 567 by ph10,
Sat Nov  6 17:10:00 2010 UTC
 Line 1
  ChangeLog for PCRE
  ------------------
- Version 8.01 11-Dec-09
+ Version 8.11 10-Oct-2010
- ----------------------
+ ------------------------
+.  (*THEN) was not working properly if there were untried alternatives prior
+     to it in the current branch. For example, in ((a|b)(*THEN)(*F)|c..) it
+     backtracked to try for "b" instead of moving to the next alternative branch
+     at the same level (in this case, to look for "c"). The Perl documentation
+     is clear that when (*THEN) is backtracked onto, it goes to the "next
+     alternative in the innermost enclosing group".
+.  (*COMMIT) was not overriding (*THEN), as it does in Perl. In a pattern
+     such as   (A(*COMMIT)B(*THEN)C|D)  any failure after matching A should
+     result in overall failure. Similarly, (*COMMIT) now overrides (*PRUNE) and
+     (*SKIP), (*SKIP) overrides (*PRUNE) and (*THEN), and (*PRUNE) overrides
+     (*THEN).
+.  If \s appeared in a character class, it removed the VT character from
+     the class, even if it had been included by some previous item, for example
+     in [\x00-\xff\s]. (This was a bug related to the fact that VT is not part
+     of \s, but is part of the POSIX "space" class.)
+.  A partial match never returns an empty string (because you can always
+     match an empty string at the end of the subject); however the checking for
+     an empty string was starting at the "start of match" point. This has been
+     changed to the "earliest inspected character" point, because the returned
+     data for a partial match starts at this character. This means that, for
+     example, /(?<=abc)def/ gives a partial match for the subject "abc"
+     (previously it gave "no match").
+.  Changes have been made to the way PCRE_PARTIAL_HARD affects the matching
+     of $, \z, \Z, \b, and \B. If the match point is at the end of the string,
+     previously a full match would be given. However, setting PCRE_PARTIAL_HARD
+     has an implication that the given string is incomplete (because a partial
+     match is preferred over a full match). For this reason, these items now
+     give a partial match in this situation. [Aside: previously, the one case
+     /t\b/ matched against "cat" with PCRE_PARTIAL_HARD set did return a partial
+     match rather than a full match, which was wrong by the old rules, but is
+     now correct.]
+.  There was a bug in the handling of #-introduced comments, recognized when
+     PCRE_EXTENDED is set, when PCRE_NEWLINE_ANY and PCRE_UTF8 were also set.
+     If a UTF-8 multi-byte character included the byte 0x85 (e.g. +U0445, whose
+     UTF-8 encoding is 0xd1,0x85), this was misinterpreted as a newline when
+     scanning for the end of the comment. (*Character* 0x85 is an "any" newline,
+     but *byte* 0x85 is not, in UTF-8 mode). This bug was present in several
+     places in pcre_compile().
+.  Related to (6) above, when pcre_compile() was skipping #-introduced
+     comments when looking ahead for named forward references to subpatterns,
+     the only newline sequence it recognized was NL. It now handles newlines
+     according to the set newline convention.
+.  SunOS4 doesn't have strerror() or strtoul(); pcregrep dealt with the
+     former, but used strtoul(), whereas pcretest avoided strtoul() but did not
+     cater for a lack of strerror(). These oversights have been fixed.
+.  Added --match-limit and --recursion-limit to pcregrep.
+. Added two casts needed to build with Visual Studio when NO_RECURSE is set.
+. When the -o option was used, pcregrep was setting a return code of 1, even
+     when matches were found, and --line-buffered was not being honoured.
+. Added an optional parentheses number to the -o and --only-matching options
+     of pcregrep.
+. Imitating Perl's /g action for multiple matches is tricky when the pattern
+     can match an empty string. The code to do it in pcretest and pcredemo
+     needed fixing:
+     (a) When the newline convention was "crlf", pcretest got it wrong, skipping
+         only one byte after an empty string match just before CRLF (this case
+         just got forgotten; "any" and "anycrlf" were OK).
+     (b) The pcretest code also had a bug, causing it to loop forever in UTF-8
+         mode when an empty string match preceded an ASCII character followed by
+         a non-ASCII character. (The code for advancing by one character rather
+         than one byte was nonsense.)
+     (c) The pcredemo.c sample program did not have any code at all to handle
+         the cases when CRLF is a valid newline sequence.
+. Neither pcre_exec() nor pcre_dfa_exec() was checking that the value given
+     as a starting offset was within the subject string. There is now a new
+     error, PCRE_ERROR_BADOFFSET, which is returned if the starting offset is
+     negative or greater than the length of the string. In order to test this,
+     pcretest is extended to allow the setting of negative starting offsets.
+. In both pcre_exec() and pcre_dfa_exec() the code for checking that the
+     starting offset points to the beginning of a UTF-8 character was
+     unnecessarily clumsy. I tidied it up.
+ Version 8.10 25-Jun-2010
+ ------------------------
+.  Added support for (*MARK:ARG) and for ARG additions to PRUNE, SKIP, and
+     THEN.
+.  (*ACCEPT) was not working when inside an atomic group.
+.  Inside a character class, \B is treated as a literal by default, but
+     faulted if PCRE_EXTRA is set. This mimics Perl's behaviour (the -w option
+     causes the error). The code is unchanged, but I tidied the documentation.
+.  Inside a character class, PCRE always treated \R and \X as literals,
+     whereas Perl faults them if its -w option is set. I have changed PCRE so
+     that it faults them when PCRE_EXTRA is set.
+.  Added support for \N, which always matches any character other than
+     newline. (It is the same as "." when PCRE_DOTALL is not set.)
+.  When compiling pcregrep with newer versions of gcc which may have
+     FORTIFY_SOURCE set, several warnings "ignoring return value of 'fwrite',
+     declared with attribute warn_unused_result" were given. Just casting the
+     result to (void) does not stop the warnings; a more elaborate fudge is
+     needed. I've used a macro to implement this.
+.  Minor change to pcretest.c to avoid a compiler warning.
+.  Added four artifical Unicode properties to help with an option to make
+     \s etc use properties (see next item). The new properties are: Xan
+     (alphanumeric), Xsp (Perl space), Xps (POSIX space), and Xwd (word).
+.  Added PCRE_UCP to make \b, \d, \s, \w, and certain POSIX character classes
+     use Unicode properties. (*UCP) at the start of a pattern can be used to set
+     this option. Modified pcretest to add /W to test this facility. Added
+     REG_UCP to make it available via the POSIX interface.
+. Added --line-buffered to pcregrep.
+. In UTF-8 mode, if a pattern that was compiled with PCRE_CASELESS was
+     studied, and the match started with a letter with a code point greater than
+whose first byte was different to the first byte of the other case of
+     the letter, the other case of this starting letter was not recognized
+     (#976).
+. If a pattern that was studied started with a repeated Unicode property
+     test, for example, \p{Nd}+, there was the theoretical possibility of
+     setting up an incorrect bitmap of starting bytes, but fortunately it could
+     not have actually happened in practice until change 8 above was made (it
+     added property types that matched character-matching opcodes).
+. pcre_study() now recognizes \h, \v, and \R when constructing a bit map of
+     possible starting bytes for non-anchored patterns.
+. Extended the "auto-possessify" feature of pcre_compile(). It now recognizes
+     \R, and also a number of cases that involve Unicode properties, both
+     explicit and implicit when PCRE_UCP is set.
+. If a repeated Unicode property match (e.g. \p{Lu}*) was used with non-UTF-8
+     input, it could crash or give wrong results if characters with values
+     greater than 0xc0 were present in the subject string. (Detail: it assumed
+     UTF-8 input when processing these items.)
+. Added a lot of (int) casts to avoid compiler warnings in systems where
+     size_t is 64-bit (#991).
+. Added a check for running out of memory when PCRE is compiled with
+     --disable-stack-for-recursion (#990).
+. If the last data line in a file for pcretest does not have a newline on
+     the end, a newline was missing in the output.
+. The default pcre_chartables.c file recognizes only ASCII characters (values
+     less than 128) in its various bitmaps. However, there is a facility for
+     generating tables according to the current locale when PCRE is compiled. It
+     turns out that in some environments, 0x85 and 0xa0, which are Unicode space
+     characters, are recognized by isspace() and therefore were getting set in
+     these tables, and indeed these tables seem to approximate to ISO 8859. This
+     caused a problem in UTF-8 mode when pcre_study() was used to create a list
+     of bytes that can start a match. For \s, it was including 0x85 and 0xa0,
+     which of course cannot start UTF-8 characters. I have changed the code so
+     that only real ASCII characters (less than 128) and the correct starting
+     bytes for UTF-8 encodings are set for characters greater than 127 when in
+     UTF-8 mode. (When PCRE_UCP is set - see 9 above - the code is different
+     altogether.)
+. Added the /T option to pcretest so as to be able to run tests with non-
+     standard character tables, thus making it possible to include the tests
+     used for 19 above in the standard set of tests.
+. A pattern such as (?&t)(?#()(?(DEFINE)(?<t>a)) which has a forward
+     reference to a subpattern the other side of a comment that contains an
+     opening parenthesis caused either an internal compiling error, or a
+     reference to the wrong subpattern.
+ Version 8.02 19-Mar-2010
+ ------------------------
+.  The Unicode data tables have been updated to Unicode 5.2.0.
+.  Added the option --libs-cpp to pcre-config, but only when C++ support is
+     configured.
+.  Updated the licensing terms in the pcregexp.pas file, as agreed with the
+     original author of that file, following a query about its status.
+.  On systems that do not have stdint.h (e.g. Solaris), check for and include
+     inttypes.h instead. This fixes a bug that was introduced by change 8.01/8.
+.  A pattern such as (?&t)*+(?(DEFINE)(?<t>.)) which has a possessive
+     quantifier applied to a forward-referencing subroutine call, could compile
+     incorrect code or give the error "internal error: previously-checked
+     referenced subpattern not found".
+.  Both MS Visual Studio and Symbian OS have problems with initializing
+     variables to point to external functions. For these systems, therefore,
+     pcre_malloc etc. are now initialized to local functions that call the
+     relevant global functions.
+.  There were two entries missing in the vectors called coptable and poptable
+     in pcre_dfa_exec.c. This could lead to memory accesses outsize the vectors.
+     I've fixed the data, and added a kludgy way of testing at compile time that
+     the lengths are correct (equal to the number of opcodes).
+.  Following on from 7, I added a similar kludge to check the length of the
+     eint vector in pcreposix.c.
+.  Error texts for pcre_compile() are held as one long string to avoid too
+     much relocation at load time. To find a text, the string is searched,
+     counting zeros. There was no check for running off the end of the string,
+     which could happen if a new error number was added without updating the
+     string.
+. \K gave a compile-time error if it appeared in a lookbehind assersion.
+. \K was not working if it appeared in an atomic group or in a group that
+     was called as a "subroutine", or in an assertion. Perl 5.11 documents that
+     \K is "not well defined" if used in an assertion. PCRE now accepts it if
+     the assertion is positive, but not if it is negative.
+. Change 11 fortuitously reduced the size of the stack frame used in the
+     "match()" function of pcre_exec.c by one pointer. Forthcoming
+     implementation of support for (*MARK) will need an extra pointer on the
+     stack; I have reserved it now, so that the stack frame size does not
+     decrease.
+. A pattern such as (?P<L1>(?P<L2>0)|(?P>L2)(?P>L1)) in which the only other
+     item in branch that calls a recursion is a subroutine call - as in the
+     second branch in the above example - was incorrectly given the compile-
+     time error "recursive call could loop indefinitely" because pcre_compile()
+     was not correctly checking the subroutine for matching a non-empty string.
+. The checks for overrunning compiling workspace could trigger after an
+     overrun had occurred. This is a "should never occur" error, but it can be
+     triggered by pathological patterns such as hundreds of nested parentheses.
+     The checks now trigger 100 bytes before the end of the workspace.
+. Fix typo in configure.ac: "srtoq" should be "strtoq".
+ Version 8.01 19-Jan-2010
+ ------------------------
 .  If a pattern contained a conditional subpattern with only one branch (in
-     particular, this includes all (DEFINE) patterns), a call to pcre_study()
+     particular, this includes all (*DEFINE) patterns), a call to pcre_study()
      computed the wrong minimum data length (which is of course zero for such
-     subpatterns).
+     subpatterns). This could cause incorrect "no match" results.
 .  For patterns such as (?i)a(?-i)b|c where an option setting at the start of
      the pattern is reset in the first branch, pcre_compile() failed with
-Line 61 
 Version 8.01 11-Dec-09
+Line 314 
 Version 8.01 11-Dec-09
      custom one, because of the following reported problem in Windows:
        - libbz2 uses the Pascal calling convention (WINAPI) for the functions
-         under Win32.
+           under Win32.
        - The standard autoconf AC_CHECK_LIB fails to include "bzlib.h",
-         therefore missing the function definition.
+           therefore missing the function definition.
        - The compiler thus generates a "C" signature for the test function.
        - The linker fails to find the "C" function.
        - PCRE fails to configure if asked to do so against libbz2.
 . When running libtoolize from libtool-2.2.6b as part of autogen.sh, these
      messages were output:
        Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.ac and
        rerunning libtoolize, to keep the correct libtool macros in-tree.
        Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
      I have done both of these things.
 . Although pcre_dfa_exec() does not use nearly as much stack as pcre_exec()
      most of the time, it *can* run out if it is given a pattern that contains a
      runaway infinite recursion. I updated the discussion in the pcrestack man
      page.
 . Now that we have gone to the x.xx style of version numbers, the minor
      version may start with zero. Using 08 or 09 is a bad idea because users
      might check the value of PCRE_MINOR in their code, and 08 or 09 may be
      interpreted as invalid octal numbers. I've updated the previous comment in
      configure.ac, and also added a check that gives an error if 08 or 09 are
      used.
 . Change 8.00/11 was not quite complete: code had been accidentally omitted,
      causing partial matching to fail when the end of the subject matched \W
      in a UTF-8 pattern where \W was quantified with a minimum of 3.
 . There were some discrepancies between the declarations in pcre_internal.h
      of _pcre_is_newline(), _pcre_was_newline(), and _pcre_valid_utf8() and
      their definitions. The declarations used "const uschar *" and the
      definitions used USPTR. Even though USPTR is normally defined as "const
      unsigned char *" (and uschar is typedeffed as "unsigned char"), it was
      reported that: "This difference in casting confuses some C++ compilers, for
      example, SunCC recognizes above declarations as different functions and
      generates broken code for hbpcre." I have changed the declarations to use
      USPTR.
 . GNU libtool is named differently on some systems. The autogen.sh script now
      tries several variants such as glibtoolize (MacOSX) and libtoolize1x
      (FreeBSD).
+. Applied Craig's patch that fixes an HP aCC compile error in pcre 8.00
+     (strtoXX undefined when compiling pcrecpp.cc). The patch contains this
+     comment: "Figure out how to create a longlong from a string: strtoll and
+     equivalent. It's not enough to call AC_CHECK_FUNCS: hpux has a strtoll, for
+     instance, but it only takes 2 args instead of 3!"
+. A subtle bug concerned with back references has been fixed by a change of
+     specification, with a corresponding code fix. A pattern such as
+     ^(xa|=?\1a)+$ which contains a back reference inside the group to which it
+     refers, was giving matches when it shouldn't. For example, xa=xaaa would
+     match that pattern. Interestingly, Perl (at least up to 5.11.3) has the
+     same bug. Such groups have to be quantified to be useful, or contained
+     inside another quantified group. (If there's no repetition, the reference
+     can never match.) The problem arises because, having left the group and
+     moved on to the rest of the pattern, a later failure that backtracks into
+     the group uses the captured value from the final iteration of the group
+     rather than the correct earlier one. I have fixed this in PCRE by forcing
+     any group that contains a reference to itself to be an atomic group; that
+     is, there cannot be any backtracking into it once it has completed. This is
+     similar to recursive and subroutine calls.
  Version 8.00 19-Oct-09

 Legend:



Removed from v.484
 


changed lines


 
Added in v.567
 Legend:



Removed from v.484
 


changed lines


 
Added in v.567
-Removed from v.484
+Added in v.567

	ViewVC Help
Powered by ViewVC 1.1.5