/[pcre]/code/trunk/ChangeLog

Diff of /code/trunk/ChangeLog

Parent Directory | Revision Log | View Patch Patch

-revision 296 by ph10,
Tue Jan  1 20:09:30 2008 UTC
+revision 365 by ph10,
Fri Jul 11 17:06:55 2008 UTC
 Line 1
  ChangeLog for PCRE
  ------------------
- Version 7.5 31-Dec-07
+ Version 8.0 02 Jul-08
+ ---------------------
+.  Replaced UCP searching code with optimized version as implemented for Ad
+     Muncher (http://www.admuncher.com/) by Peter Kankowski. This uses a two-
+     stage table and inline lookup instead of a function, giving speed ups of 2
+     to 5 times on some simple patterns that I tested. Permission was given to
+     distribute the MultiStage2.py script that generates the tables (it's not in
+     the tarball, but is in the Subversion repository).
+.  Updated the Unicode datatables to Unicode 5.1.0. This adds yet more
+     scripts.
+.  Change 12 for 7.7 introduced a bug in pcre_study() when a pattern contained
+     a group with a zero qualifier. The result of the study could be incorrect,
+     or the function might crash, depending on the pattern.
+.  Caseless matching was not working for non-ASCII characters in back
+     references. For example, /(\x{de})\1/8i was not matching \x{de}\x{fe}.
+     It now works when Unicode Property Support is available.
+.  In pcretest, an escape such as \x{de} in the data was always generating
+     a UTF-8 string, even in non-UTF-8 mode. Now it generates a single byte in
+     non-UTF-8 mode. If the value is greater than 255, it gives a warning about
+     truncation.
+.  Minor bugfix in pcrecpp.cc (change "" == ... to NULL == ...).
+.  Added two (int) casts to pcregrep when printing the difference of two
+     pointers, in case they are 64-bit values.
+.  Added comments about Mac OS X stack usage to the pcrestack man page and to
+     test 2 if it fails.
+.  Added PCRE_CALL_CONVENTION just before the names of all exported functions,
+     and a #define of that name to empty if it is not externally set. This is to
+     allow users of MSVC to set it if necessary.
+. The PCRE_EXP_DEFN macro which precedes exported functions was missing from
+     the convenience functions in the pcre_get.c source file.
+. An option change at the start of a pattern that had top-level alternatives
+     could cause overwriting and/or a crash. This command provoked a crash in
+     some environments:
+       printf "/(?i)[\xc3\xa9\xc3\xbd]|[\xc3\xa9\xc3\xbdA]/8\n" | pcretest
+     This potential security problem was recorded as CVE-2008-2371.
+. For a pattern where the match had to start at the beginning or immediately
+     after a newline (e.g /.*anything/ without the DOTALL flag), pcre_exec() and
+     pcre_dfa_exec() could read past the end of the passed subject if there was
+     no match. To help with detecting such bugs (e.g. with valgrind), I modified
+     pcretest so that it places the subject at the end of its malloc-ed buffer.
+. The change to pcretest in 12 above threw up a couple more cases when pcre_
+     exec() might read past the end of the data buffer in UTF-8 mode.
+. A similar bug to 7.3/2 existed when the PCRE_FIRSTLINE option was set and
+     the data contained the byte 0x85 as part of a UTF-8 character within its
+     first line. This applied both to normal and DFA matching.
+ Version 7.7 07-May-08
+ ---------------------
+.  Applied Craig's patch to sort out a long long problem: "If we can't convert
+     a string to a long long, pretend we don't even have a long long." This is
+     done by checking for the strtoq, strtoll, and _strtoi64 functions.
+.  Applied Craig's patch to pcrecpp.cc to restore ABI compatibility with
+     pre-7.6 versions, which defined a global no_arg variable instead of putting
+     it in the RE class. (See also #8 below.)
+.  Remove a line of dead code, identified by coverity and reported by Nuno
+     Lopes.
+.  Fixed two related pcregrep bugs involving -r with --include or --exclude:
+     (1) The include/exclude patterns were being applied to the whole pathnames
+         of files, instead of just to the final components.
+     (2) If there was more than one level of directory, the subdirectories were
+         skipped unless they satisfied the include/exclude conditions. This is
+         inconsistent with GNU grep (and could even be seen as contrary to the
+         pcregrep specification - which I improved to make it absolutely clear).
+         The action now is always to scan all levels of directory, and just
+         apply the include/exclude patterns to regular files.
+.  Added the --include_dir and --exclude_dir patterns to pcregrep, and used
+     --exclude_dir in the tests to avoid scanning .svn directories.
+.  Applied Craig's patch to the QuoteMeta function so that it escapes the
+     NUL character as backslash + 0 rather than backslash + NUL, because PCRE
+     doesn't support NULs in patterns.
+.  Added some missing "const"s to declarations of static tables in
+     pcre_compile.c and pcre_dfa_exec.c.
+.  Applied Craig's patch to pcrecpp.cc to fix a problem in OS X that was
+     caused by fix #2  above. (Subsequently also a second patch to fix the
+     first patch. And a third patch - this was a messy problem.)
+.  Applied Craig's patch to remove the use of push_back().
+. Applied Alan Lehotsky's patch to add REG_STARTEND support to the POSIX
+     matching function regexec().
+. Added support for the Oniguruma syntax \g<name>, \g<n>, \g'name', \g'n',
+     which, however, unlike Perl's \g{...}, are subroutine calls, not back
+     references. PCRE supports relative numbers with this syntax (I don't think
+     Oniguruma does).
+. Previously, a group with a zero repeat such as (...){0} was completely
+     omitted from the compiled regex. However, this means that if the group
+     was called as a subroutine from elsewhere in the pattern, things went wrong
+     (an internal error was given). Such groups are now left in the compiled
+     pattern, with a new opcode that causes them to be skipped at execution
+     time.
+. Added the PCRE_JAVASCRIPT_COMPAT option. This makes the following changes
+     to the way PCRE behaves:
+     (a) A lone ] character is dis-allowed (Perl treats it as data).
+     (b) A back reference to an unmatched subpattern matches an empty string
+         (Perl fails the current match path).
+     (c) A data ] in a character class must be notated as \] because if the
+         first data character in a class is ], it defines an empty class. (In
+         Perl it is not possible to have an empty class.) The empty class []
+         never matches; it forces failure and is equivalent to (*FAIL) or (?!).
+         The negative empty class [^] matches any one character, independently
+         of the DOTALL setting.
+. A pattern such as /(?2)[]a()b](abc)/ which had a forward reference to a
+     non-existent subpattern following a character class starting with ']' and
+     containing () gave an internal compiling error instead of "reference to
+     non-existent subpattern". Fortunately, when the pattern did exist, the
+     compiled code was correct. (When scanning forwards to check for the
+     existencd of the subpattern, it was treating the data ']' as terminating
+     the class, so got the count wrong. When actually compiling, the reference
+     was subsequently set up correctly.)
+. The "always fail" assertion (?!) is optimzed to (*FAIL) by pcre_compile;
+     it was being rejected as not supported by pcre_dfa_exec(), even though
+     other assertions are supported. I have made pcre_dfa_exec() support
+     (*FAIL).
+. The implementation of 13c above involved the invention of a new opcode,
+     OP_ALLANY, which is like OP_ANY but doesn't check the /s flag. Since /s
+     cannot be changed at match time, I realized I could make a small
+     improvement to matching performance by compiling OP_ALLANY instead of
+     OP_ANY for "." when DOTALL was set, and then removing the runtime tests
+     on the OP_ANY path.
+. Compiling pcretest on Windows with readline support failed without the
+     following two fixes: (1) Make the unistd.h include conditional on
+     HAVE_UNISTD_H; (2) #define isatty and fileno as _isatty and _fileno.
+. Changed CMakeLists.txt and cmake/FindReadline.cmake to arrange for the
+     ncurses library to be included for pcretest when ReadLine support is
+     requested, but also to allow for it to be overridden. This patch came from
+     Daniel Bergstr�m.
+. There was a typo in the file ucpinternal.h where f0_rangeflag was defined
+     as 0x00f00000 instead of 0x00800000. Luckily, this would not have caused
+     any errors with the current Unicode tables. Thanks to Peter Kankowski for
+     spotting this.
+ Version 7.6 28-Jan-08
+ ---------------------
+.  A character class containing a very large number of characters with
+     codepoints greater than 255 (in UTF-8 mode, of course) caused a buffer
+     overflow.
+.  Patch to cut out the "long long" test in pcrecpp_unittest when
+     HAVE_LONG_LONG is not defined.
+.  Applied Christian Ehrlicher's patch to update the CMake build files to
+     bring them up to date and include new features. This patch includes:
+     - Fixed PH's badly added libz and libbz2 support.
+     - Fixed a problem with static linking.
+     - Added pcredemo. [But later removed - see 7 below.]
+     - Fixed dftables problem and added an option.
+     - Added a number of HAVE_XXX tests, including HAVE_WINDOWS_H and
+         HAVE_LONG_LONG.
+     - Added readline support for pcretest.
+     - Added an listing of the option settings after cmake has run.
+.  A user submitted a patch to Makefile that makes it easy to create
+     "pcre.dll" under mingw when using Configure/Make. I added stuff to
+     Makefile.am that cause it to include this special target, without
+     affecting anything else. Note that the same mingw target plus all
+     the other distribution libraries and programs are now supported
+     when configuring with CMake (see 6 below) instead of with
+     Configure/Make.
+.  Applied Craig's patch that moves no_arg into the RE class in the C++ code.
+     This is an attempt to solve the reported problem "pcrecpp::no_arg is not
+     exported in the Windows port". It has not yet been confirmed that the patch
+     solves the problem, but it does no harm.
+.  Applied Sheri's patch to CMakeLists.txt to add NON_STANDARD_LIB_PREFIX and
+     NON_STANDARD_LIB_SUFFIX for dll names built with mingw when configured
+     with CMake, and also correct the comment about stack recursion.
+.  Remove the automatic building of pcredemo from the ./configure system and
+     from CMakeLists.txt. The whole idea of pcredemo.c is that it is an example
+     of a program that users should build themselves after PCRE is installed, so
+     building it automatically is not really right. What is more, it gave
+     trouble in some build environments.
+.  Further tidies to CMakeLists.txt from Sheri and Christian.
+ Version 7.5 10-Jan-08
  ---------------------
 .  Applied a patch from Craig: "This patch makes it possible to 'ignore'
-Line 99 
 Version 7.5 31-Dec-07
+Line 318 
 Version 7.5 31-Dec-07
 . In pcrecpp.cc, the variable 'count' was incremented twice in
      RE::GlobalReplace(). As a result, the number of replacements returned was
-     double what it should be. I have removed one of the increments.
+     double what it should be. I removed one of the increments, but Craig sent a
+     later patch that removed the other one (the right fix) and added unit tests
+     that check the return values (which was not done before).
 . Several CMake things:
-Line 110 
 Version 7.5 31-Dec-07
+Line 331 
 Version 7.5 31-Dec-07
          linked with the newly-built libraries, not previously installed ones.
      (3) Added PCRE_SUPPORT_LIBREADLINE, PCRE_SUPPORT_LIBZ, PCRE_SUPPORT_LIBBZ2.
 . In UTF-8 mode, with newline set to "any", a pattern such as .*a.*=.b.*
      crashed when matching a string such as a\x{2029}b (note that \x{2029} is a
      UTF-8 newline character). The key issue is that the pattern starts .*;
      this means that the match must be either at the beginning, or after a
      newline. The bug was in the code for advancing after a failed match and
      checking that the new position followed a newline. It was not taking
      account of UTF-8 characters correctly.
 . PCRE was behaving differently from Perl in the way it recognized POSIX
      character classes. PCRE was not treating the sequence [:...:] as a
      character class unless the ... were all letters. Perl, however, seems to
      allow any characters between [: and :], though of course it rejects as
      unknown any "names" that contain non-letters, because all the known class
      names consist only of letters. Thus, Perl gives an error for [[:1234:]],
      for example, whereas PCRE did not - it did not recognize a POSIX character
      class. This seemed a bit dangerous, so the code has been changed to be
      closer to Perl. The behaviour is not identical to Perl, because PCRE will
      diagnose an unknown class for, for example, [[:l\ower:]] where Perl will
      treat it as [[:lower:]]. However, PCRE does now give "unknown" errors where
      Perl does, and where it didn't before.
 . Rewrite so as to remove the single use of %n from pcregrep because in some
      Windows environments %n is disabled by default.

 Legend:



Removed from v.296
 


changed lines


 
Added in v.365
 Legend:



Removed from v.296
 


changed lines


 
Added in v.365
-Removed from v.296
+Added in v.365

	ViewVC Help
Powered by ViewVC 1.1.5