/[pcre]/code/trunk/ChangeLog

Diff of /code/trunk/ChangeLog

Parent Directory | Revision Log | View Patch Patch

-revision 143 by ph10,
Mon Apr  2 10:08:14 2007 UTC
+revision 248 by ph10,
Mon Sep 17 10:09:22 2007 UTC
 Line 1
  ChangeLog for PCRE
  ------------------
- Version 7.1 12-Mar-07
+ Version 7.4 10-Sep-07
+ ---------------------
+.  Change 7.3/28 was implemented for classes by looking at the bitmap. This
+     means that a class such as [\s] counted as "explicit reference to CR or
+     LF". That isn't really right - the whole point of the change was to try to
+     help when there was an actual mention of one of the two characters. So now
+     the change happens only if \r or \n (or a literal CR or LF) character is
+     encountered.
+.  The 32-bit options word was also used for 6 internal flags, but the numbers
+     of both had grown to the point where there were only 3 bits left.
+     Fortunately, there was spare space in the data structure, and so I have
+     moved the internal flags into a new 16-bit field to free up more option
+     bits.
+.  The appearance of (?J) at the start of a pattern set the DUPNAMES option,
+     but did not set the internal JCHANGED flag - either of these is enough to
+     control the way the "get" function works - but the PCRE_INFO_JCHANGED
+     facility is supposed to tell if (?J) was ever used, so now (?J) at the
+     start sets both bits.
+.  Added options (at build time, compile time, exec time) to change \R from
+     matching any Unicode line ending sequence to just matching CR, LF, or CRLF.
+.  doc/pcresyntax.html was missing from the distribution.
+.  Put back the definition of PCRE_ERROR_NULLWSLIMIT, for backward
+     compatibility, even though it is no longer used.
+.  Added macro for snprintf to pcrecpp_unittest.cc and also for strtoll and
+     strtoull to pcrecpp.cc to select the available functions in WIN32 when the
+     windows.h file is present (where different names are used).
+.  Changed all #include <config.h> to #include "config.h". There were also
+     some further <pcre.h> cases that I changed to "pcre.h".
+.  When pcregrep was used with the --colour option, it missed the line ending
+     sequence off the lines that it output.
+. It was pointed out to me that arrays of string pointers cause lots of
+     relocations when a shared library is dynamically loaded. A technique of
+     using a single long string with a table of offsets can drastically reduce
+     these. I have refactored PCRE in four places to do this. The result is
+     dramatic:
+       Originally:                          290
+       After changing UCP table:            187
+       After changing error message table:   43
+       After changing table of "verbs"       36
+       After changing table of Posix names   22
+     Thanks to the folks working on Gregex for glib for this insight.
+. --disable-stack-for-recursion caused compiling to fail unless -enable-
+     unicode-properties was also set.
+. Updated the tests so that they work when \R is defaulted to ANYCRLF.
+ Version 7.3 28-Aug-07
+ ---------------------
+. In the rejigging of the build system that eventually resulted in 7.1, the
+     line "#include <pcre.h>" was included in pcre_internal.h. The use of angle
+     brackets there is not right, since it causes compilers to look for an
+     installed pcre.h, not the version that is in the source that is being
+     compiled (which of course may be different). I have changed it back to:
+       #include "pcre.h"
+     I have a vague recollection that the change was concerned with compiling in
+     different directories, but in the new build system, that is taken care of
+     by the VPATH setting the Makefile.
+. The pattern .*$ when run in not-DOTALL UTF-8 mode with newline=any failed
+     when the subject happened to end in the byte 0x85 (e.g. if the last
+     character was \x{1ec5}). *Character* 0x85 is one of the "any" newline
+     characters but of course it shouldn't be taken as a newline when it is part
+     of another character. The bug was that, for an unlimited repeat of . in
+     not-DOTALL UTF-8 mode, PCRE was advancing by bytes rather than by
+     characters when looking for a newline.
+. A small performance improvement in the DOTALL UTF-8 mode .* case.
+. Debugging: adjusted the names of opcodes for different kinds of parentheses
+     in debug output.
+. Arrange to use "%I64d" instead of "%lld" and "%I64u" instead of "%llu" for
+     long printing in the pcrecpp unittest when running under MinGW.
+. ESC_K was left out of the EBCDIC table.
+. Change 7.0/38 introduced a new limit on the number of nested non-capturing
+     parentheses; I made it 1000, which seemed large enough. Unfortunately, the
+     limit also applies to "virtual nesting" when a pattern is recursive, and in
+     this case 1000 isn't so big. I have been able to remove this limit at the
+     expense of backing off one optimization in certain circumstances. Normally,
+     when pcre_exec() would call its internal match() function recursively and
+     immediately return the result unconditionally, it uses a "tail recursion"
+     feature to save stack. However, when a subpattern that can match an empty
+     string has an unlimited repetition quantifier, it no longer makes this
+     optimization. That gives it a stack frame in which to save the data for
+     checking that an empty string has been matched. Previously this was taken
+     from the 1000-entry workspace that had been reserved. So now there is no
+     explicit limit, but more stack is used.
+. Applied Daniel's patches to solve problems with the import/export magic
+     syntax that is required for Windows, and which was going wrong for the
+     pcreposix and pcrecpp parts of the library. These were overlooked when this
+     problem was solved for the main library.
+. There were some crude static tests to avoid integer overflow when computing
+     the size of patterns that contain repeated groups with explicit upper
+     limits. As the maximum quantifier is 65535, the maximum group length was
+     set at 30,000 so that the product of these two numbers did not overflow a
+-bit integer. However, it turns out that people want to use groups that
+     are longer than 30,000 bytes (though not repeat them that many times).
+     Change 7.0/17 (the refactoring of the way the pattern size is computed) has
+     made it possible to implement the integer overflow checks in a much more
+     dynamic way, which I have now done. The artificial limitation on group
+     length has been removed - we now have only the limit on the total length of
+     the compiled pattern, which depends on the LINK_SIZE setting.
+. Fixed a bug in the documentation for get/copy named substring when
+     duplicate names are permitted. If none of the named substrings are set, the
+     functions return PCRE_ERROR_NOSUBSTRING (7); the doc said they returned an
+     empty string.
+. Because Perl interprets \Q...\E at a high level, and ignores orphan \E
+     instances, patterns such as [\Q\E] or [\E] or even [^\E] cause an error,
+     because the ] is interpreted as the first data character and the
+     terminating ] is not found. PCRE has been made compatible with Perl in this
+     regard. Previously, it interpreted [\Q\E] as an empty class, and [\E] could
+     cause memory overwriting.
+. Like Perl, PCRE automatically breaks an unlimited repeat after an empty
+     string has been matched (to stop an infinite loop). It was not recognizing
+     a conditional subpattern that could match an empty string if that
+     subpattern was within another subpattern. For example, it looped when
+     trying to match  (((?(1)X|))*)  but it was OK with  ((?(1)X|)*)  where the
+     condition was not nested. This bug has been fixed.
+. A pattern like \X?\d or \P{L}?\d in non-UTF-8 mode could cause a backtrack
+     past the start of the subject in the presence of bytes with the top bit
+     set, for example "\x8aBCD".
+. Added Perl 5.10 experimental backtracking controls (*FAIL), (*F), (*PRUNE),
+     (*SKIP), (*THEN), (*COMMIT), and (*ACCEPT).
+. Optimized (?!) to (*FAIL).
+. Updated the test for a valid UTF-8 string to conform to the later RFC 3629.
+     This restricts code points to be within the range 0 to 0x10FFFF, excluding
+     the "low surrogate" sequence 0xD800 to 0xDFFF. Previously, PCRE allowed the
+     full range 0 to 0x7FFFFFFF, as defined by RFC 2279. Internally, it still
+     does: it's just the validity check that is more restrictive.
+. Inserted checks for integer overflows during escape sequence (backslash)
+     processing, and also fixed erroneous offset values for syntax errors during
+     backslash processing.
+. Fixed another case of looking too far back in non-UTF-8 mode (cf 12 above)
+     for patterns like [\PPP\x8a]{1,}\x80 with the subject "A\x80".
+. An unterminated class in a pattern like (?1)\c[ with a "forward reference"
+     caused an overrun.
+. A pattern like (?:[\PPa*]*){8,} which had an "extended class" (one with
+     something other than just ASCII characters) inside a group that had an
+     unlimited repeat caused a loop at compile time (while checking to see
+     whether the group could match an empty string).
+. Debugging a pattern containing \p or \P could cause a crash. For example,
+     [\P{Any}] did so. (Error in the code for printing property names.)
+. An orphan \E inside a character class could cause a crash.
+. A repeated capturing bracket such as (A)? could cause a wild memory
+     reference during compilation.
+. There are several functions in pcre_compile() that scan along a compiled
+     expression for various reasons (e.g. to see if it's fixed length for look
+     behind). There were bugs in these functions when a repeated \p or \P was
+     present in the pattern. These operators have additional parameters compared
+     with \d, etc, and these were not being taken into account when moving along
+     the compiled data. Specifically:
+     (a) A item such as \p{Yi}{3} in a lookbehind was not treated as fixed
+         length.
+     (b) An item such as \pL+ within a repeated group could cause crashes or
+         loops.
+     (c) A pattern such as \p{Yi}+(\P{Yi}+)(?1) could give an incorrect
+         "reference to non-existent subpattern" error.
+     (d) A pattern like (\P{Yi}{2}\277)? could loop at compile time.
+. A repeated \S or \W in UTF-8 mode could give wrong answers when multibyte
+     characters were involved (for example /\S{2}/8g with "A\x{a3}BC").
+. Using pcregrep in multiline, inverted mode (-Mv) caused it to loop.
+. Patterns such as [\P{Yi}A] which include \p or \P and just one other
+     character were causing crashes (broken optimization).
+. Patterns such as (\P{Yi}*\277)* (group with possible zero repeat containing
+     \p or \P) caused a compile-time loop.
+. More problems have arisen in unanchored patterns when CRLF is a valid line
+     break. For example, the unstudied pattern [\r\n]A does not match the string
+     "\r\nA" because change 7.0/46 below moves the current point on by two
+     characters after failing to match at the start. However, the pattern \nA
+     *does* match, because it doesn't start till \n, and if [\r\n]A is studied,
+     the same is true. There doesn't seem any very clean way out of this, but
+     what I have chosen to do makes the common cases work: PCRE now takes note
+     of whether there can be an explicit match for \r or \n anywhere in the
+     pattern, and if so, 7.0/46 no longer applies. As part of this change,
+     there's a new PCRE_INFO_HASCRORLF option for finding out whether a compiled
+     pattern has explicit CR or LF references.
+. Added (*CR) etc for changing newline setting at start of pattern.
+ Version 7.2 19-Jun-07
+ ---------------------
+. If the fr_FR locale cannot be found for test 3, try the "french" locale,
+     which is apparently normally available under Windows.
+. Re-jig the pcregrep tests with different newline settings in an attempt
+     to make them independent of the local environment's newline setting.
+. Add code to configure.ac to remove -g from the CFLAGS default settings.
+. Some of the "internals" tests were previously cut out when the link size
+     was not 2, because the output contained actual offsets. The recent new
+     "Z" feature of pcretest means that these can be cut out, making the tests
+     usable with all link sizes.
+. Implemented Stan Switzer's goto replacement for longjmp() when not using
+     stack recursion. This gives a massive performance boost under BSD, but just
+     a small improvement under Linux. However, it saves one field in the frame
+     in all cases.
+. Added more features from the forthcoming Perl 5.10:
+     (a) (?-n) (where n is a string of digits) is a relative subroutine or
+         recursion call. It refers to the nth most recently opened parentheses.
+     (b) (?+n) is also a relative subroutine call; it refers to the nth next
+         to be opened parentheses.
+     (c) Conditions that refer to capturing parentheses can be specified
+         relatively, for example, (?(-2)... or (?(+3)...
+     (d) \K resets the start of the current match so that everything before
+         is not part of it.
+     (e) \k{name} is synonymous with \k<name> and \k'name' (.NET compatible).
+     (f) \g{name} is another synonym - part of Perl 5.10's unification of
+         reference syntax.
+     (g) (?| introduces a group in which the numbering of parentheses in each
+         alternative starts with the same number.
+     (h) \h, \H, \v, and \V match horizontal and vertical whitespace.
+. Added two new calls to pcre_fullinfo(): PCRE_INFO_OKPARTIAL and
+     PCRE_INFO_JCHANGED.
+. A pattern such as  (.*(.)?)*  caused pcre_exec() to fail by either not
+     terminating or by crashing. Diagnosed by Viktor Griph; it was in the code
+     for detecting groups that can match an empty string.
+. A pattern with a very large number of alternatives (more than several
+     hundred) was running out of internal workspace during the pre-compile
+     phase, where pcre_compile() figures out how much memory will be needed. A
+     bit of new cunning has reduced the workspace needed for groups with
+     alternatives. The 1000-alternative test pattern now uses 12 bytes of
+     workspace instead of running out of the 4096 that are available.
+. Inserted some missing (unsigned int) casts to get rid of compiler warnings.
+. Applied patch from Google to remove an optimization that didn't quite work.
+     The report of the bug said:
+       pcrecpp::RE("a*").FullMatch("aaa") matches, while
+       pcrecpp::RE("a*?").FullMatch("aaa") does not, and
+       pcrecpp::RE("a*?\\z").FullMatch("aaa") does again.
+. If \p or \P was used in non-UTF-8 mode on a character greater than 127
+     it matched the wrong number of bytes.
+ Version 7.1 24-Apr-07
  ---------------------
 . Applied Bob Rossi and Daniel G's patches to convert the build system to one
-Line 136 
 Version 7.1 12-Mar-07
+Line 433 
 Version 7.1 12-Mar-07
      string at the end of a line did not allow for this case. They now check for
      it specially.
+. pcretest is supposed to handle patterns and data of any length, by
+     extending its buffers when necessary. It was getting this wrong when the
+     buffer for a data line had to be extended.
+. Added PCRE_NEWLINE_ANYCRLF which is like ANY, but matches only CR, LF, or
+     CRLF as a newline sequence.
+. Code for handling Unicode properties in pcre_dfa_exec() wasn't being cut
+     out by #ifdef SUPPORT_UCP. This did no harm, as it could never be used, but
+     I have nevertheless tidied it up.
+. Added some casts to kill warnings from HP-UX ia64 compiler.
+. Added a man page for pcre-config.
  Version 7.0 19-Dec-06
  ---------------------

 Legend:



Removed from v.143
 


changed lines


 
Added in v.248
 Legend:



Removed from v.143
 


changed lines


 
Added in v.248
-Removed from v.143
+Added in v.248

	ViewVC Help
Powered by ViewVC 1.1.5