/[pcre]/code/trunk/README

Diff of /code/trunk/README

Parent Directory | Revision Log | View Patch Patch

-revision 7 by nigel,
Sat Feb 24 21:38:09 2007 UTC
+revision 27 by nigel,
Sat Feb 24 21:38:49 2007 UTC
 Line 1
  README file for PCRE (Perl-compatible regular expressions)
  ----------------------------------------------------------
+ *******************************************************************************
+ *           IMPORTANT FOR THOSE UPGRADING FROM VERSIONS BEFORE 2.00           *
+ *                                                                             *
+ * Please note that there has been a change in the API such that a larger      *
+ * ovector is required at matching time, to provide some additional workspace. *
+ * The new man page has details. This change was necessary in order to support *
+ * some of the new functionality in Perl 5.005.                                *
+ *                                                                             *
+ *           IMPORTANT FOR THOSE UPGRADING FROM VERSION 2.00                   *
+ *                                                                             *
+ * Another (I hope this is the last!) change has been made to the API for the  *
+ * pcre_compile() function. An additional argument has been added to make it   *
+ * possible to pass over a pointer to character tables built in the current    *
+ * locale by pcre_maketables(). To use the default tables, this new arguement  *
+ * should be passed as NULL.                                                   *
+ *******************************************************************************
  The distribution should contain the following files:
    ChangeLog         log of changes to the code
    Makefile          for building PCRE
-   Performance       notes on performance
    README            this file
+   RunTest           a shell script for running tests
    Tech.Notes        notes on the encoding
    pcre.3            man page for the functions
    pcreposix.3       man page for the POSIX wrapper API
-   maketables.c      auxiliary program for building chartables.c
+   dftables.c        auxiliary program for building chartables.c
+   maketables.c      )
    study.c           ) source of
    pcre.c            )   the functions
    pcreposix.c       )
-Line 21 
 The distribution should contain the foll
+Line 39 
 The distribution should contain the foll
    pgrep.1           man page for pgrep
    pgrep.c           source of a grep utility that uses PCRE
    perltest          Perl test program
-   testinput         test data, compatible with Perl
+   testinput         test data, compatible with Perl 5.004 and 5.005
    testinput2        test data for error messages and non-Perl things
+   testinput3        test data, compatible with Perl 5.005
+   testinput4        test data for locale-specific tests
    testoutput        test results corresponding to testinput
    testoutput2       test results corresponding to testinput2
+   testoutput3       test results corresponding to testinput3
+   testoutput4       test results corresponding to testinput4
- To build PCRE, edit Makefile for your system (it is a fairly simple make file)
+ To build PCRE, edit Makefile for your system (it is a fairly simple make file,
- and then run it. It builds a two libraries called libpcre.a and libpcreposix.a,
+ and there are some comments at the top) and then run it. It builds two
- a test program called pcretest, and the pgrep command.
+ libraries called libpcre.a and libpcreposix.a, a test program called pcretest,
+ and the pgrep command.
- To test PCRE, run pcretest on the file testinput, and compare the output with
- the contents of testoutput. There should be no differences. For example:
+ To test PCRE, run the RunTest script in the pcre directory. This runs pcretest
+ on each of the testinput files in turn, and compares the output with the
-   pcretest testinput some.file
+ contents of the corresponding testoutput file. A file called testtry is used to
-   diff some.file testoutput
+ hold the output from pcretest (which is documented below).
- Do the same with testinput2, comparing the output with testoutput2, but this
+ To run pcretest on just one of the test files, give its number as an argument
- time using the -i flag for pcretest, i.e.
+ to RunTest, for example:
-   pcretest -i testinput2 some.file
+   RunTest 3
-   diff some.file testoutput2
+ The first and third test files can also be fed directly into the perltest
+ program to check that Perl gives the same results. The third file requires the
+ additional features of release 5.005, which is why it is kept separate from the
+ main test input, which needs only Perl 5.004. In the long run, when 5.005 is
+ widespread, these two test files may get amalgamated.
+ The second set of tests check pcre_info(), pcre_study(), error detection and
+ run-time flags that are specific to PCRE, as well as the POSIX wrapper API.
+ The fourth set of tests checks pcre_maketables(), the facility for building a
+ set of character tables for a specific locale and using them instead of the
+ default tables. The tests make use of the "fr" (French) locale. Before running
+ the test, the script checks for the presence of this locale by running the
+ "locale" command. If that command fails, or if it doesn't include "fr" in the
+ list of available locales, the fourth test cannot be run, and a comment is
+ output to say why. If running this test produces instances of the error
- The make target "runtest" runs both these tests, using the file "testtry" to
+   ** Failed to set locale "fr"
- store the intermediate output, deleting it at the end if all goes well.
- There are two sets of tests because the first set can also be fed directly into
+ in the comparison output, it means that locale is not available on your system,
- the perltest program to check that Perl gives the same results. The second set
+ despite being listed by "locale". This does not mean that PCRE is broken.
- of tests check pcre_info(), pcre_study(), error detection and run-time flags
- that are specific to PCRE, as well as the POSIX wrapper API.
  To install PCRE, copy libpcre.a to any suitable library directory (e.g.
  /usr/local/lib), pcre.h to any suitable include directory (e.g.
-Line 66 
 themselves still follow Perl syntax and
+Line 101 
 themselves still follow Perl syntax and
  for the POSIX-style functions is called pcreposix.h. The official POSIX name is
  regex.h, but I didn't want to risk possible problems with existing files of
  that name by distributing it that way. To use it with an existing program that
- uses the POSIX API it will have to be renamed or pointed at by a link.
+ uses the POSIX API, it will have to be renamed or pointed at by a link.
  Character tables
  ----------------
- PCRE uses four tables for manipulating and identifying characters. These are
+ PCRE uses four tables for manipulating and identifying characters. The final
- compiled from a source file called chartables.c. This is not supplied in
+ argument of the pcre_compile() function is a pointer to a block of memory
- the distribution, but is built by the program maketables (compiled from
+ containing the concatenated tables. A call to pcre_maketables() is used to
- maketables.c), which uses the ANSI C character handling functions such as
+ generate a set of tables in the current locale. However, if the final argument
- isalnum(), isalpha(), isupper(), islower(), etc. to build the table sources.
+ is passed as NULL, a set of default tables that is built into the binary is
- This means that the default C locale set in your system may affect the contents
+ used.
- of the tables. You can change the tables by editing chartables.c and then
- re-building PCRE. If you do this, you should probably also edit Makefile to
+ The source file called chartables.c contains the default set of tables. This is
- ensure that the file doesn't ever get re-generated.
+ not supplied in the distribution, but is built by the program dftables
+ (compiled from dftables.c), which uses the ANSI C character handling functions
- The first two tables pcre_lcc[] and pcre_fcc[] provide lower casing and a
+ such as isalnum(), isalpha(), isupper(), islower(), etc. to build the table
- case flipping functions, respectively. The pcre_cbits[] table consists of four
+ sources. This means that the default C locale set your system will control the
--byte bit maps which identify digits, letters, "word" characters, and white
+ contents of the tables. You can change the default tables by editing
- space, respectively. These are used when building 32-byte bit maps that
+ chartables.c and then re-building PCRE. If you do this, you should probably
- represent character classes.
+ also edit Makefile to ensure that the file doesn't ever get re-generated.
+ The first two 256-byte tables provide lower casing and case flipping functions,
+ respectively. The next table consists of three 32-byte bit maps which identify
+ digits, "word" characters, and white space, respectively. These are used when
+ building 32-byte bit maps that represent character classes.
- The pcre_ctypes[] table has bits indicating various character types, as
+ The final 256-byte table has bits indicating various character types, as
  follows:
   white space character
-Line 127 
 same effect as they do in Perl.
+Line 167 
 same effect as they do in Perl.
  There are also some upper case options that do not match Perl options: /A, /E,
  and /X set PCRE_ANCHORED, PCRE_DOLLAR_ENDONLY, and PCRE_EXTRA respectively.
- The /D option is a PCRE debugging feature. It causes the internal form of
- compiled regular expressions to be output after compilation. The /S option
+ The /L option must be followed directly by the name of a locale, for example,
- causes pcre_study() to be called after the expression has been compiled, and
- the results used when the expression is matched. If /I is present as well as
+   /pattern/Lfr
- /S, then pcre_study() is called with the PCRE_CASELESS option.
+ For this reason, it must be the last option letter. The given locale is set,
+ pcre_maketables() is called to build a set of character tables for the locale,
+ and this is then passed to pcre_compile() when compiling the regular
+ expression. Without an /L option, NULL is passed as the tables pointer; that
+ is, /L applies only to the expression on which it appears.
+ The /I option requests that pcretest output information about the compiled
+ expression (whether it is anchored, has a fixed first character, and so on). It
+ does this by calling pcre_info() after compiling an expression, and outputting
+ the information it gets back. If the pattern is studied, the results of that
+ are also output.
+ The /D option is a PCRE debugging feature, which also assumes /I. It causes the
+ internal form of compiled regular expressions to be output after compilation.
+ The /S option causes pcre_study() to be called after the expression has been
+ compiled, and the results used when the expression is matched.
  Finally, the /P option causes pcretest to call PCRE via the POSIX wrapper API
  rather than its native API. When this is done, all other options except /i and
-Line 140 
 is present. The wrapper functions force
+Line 197 
 is present. The wrapper functions force
  PCRE_DOTALL unless REG_NEWLINE is set.
  A regular expression can extend over several lines of input; the newlines are
- included in it. See the testinput file for many examples.
+ included in it. See the testinput files for many examples.
  Before each data line is passed to pcre_exec(), leading and trailing whitespace
  is removed, and it is then scanned for \ escapes. The following are recognized:
-Line 158 
 is removed, and it is then scanned for \
+Line 215 
 is removed, and it is then scanned for \
    \A     pass the PCRE_ANCHORED option to pcre_exec()
    \B     pass the PCRE_NOTBOL option to pcre_exec()
-   \E     pass the PCRE_DOLLAR_ENDONLY option to pcre_exec()
-   \I     pass the PCRE_CASELESS option to pcre_exec()
-   \M     pass the PCRE_MULTILINE option to pcre_exec()
-   \S     pass the PCRE_DOTALL option to pcre_exec()
    \Odd   set the size of the output vector passed to pcre_exec() to dd
             (any number of decimal digits)
    \Z     pass the PCRE_NOTEOL option to pcre_exec()
-Line 182 
 whole pattern. Here is an example of an
+Line 235 
 whole pattern. Here is an example of an
    Testing Perl-Compatible Regular Expressions
    PCRE version 0.90 08-Sep-1997
-       re> /^abc(\d+)/
+     re> /^abc(\d+)/
-     data> abc123
+   data> abc123
 : abc123
 : 123
-     data> xyz
+   data> xyz
    No match
  Note that while patterns can be continued over several lines (a plain ">"
-Line 200 
 following flags has any effect in this c
+Line 253 
 following flags has any effect in this c
  If the option -d is given to pcretest, it is equivalent to adding /D to each
  regular expression: the internal form is output after compilation.
- If the option -i (for "information") is given to pcretest, it calls pcre_info()
+ If the option -i is given to pcretest, it is equivalent to adding /I to each
- after compiling an expression, and outputs the information it gets back. If the
+ regular expression: information about the compiled pattern is given after
- pattern is studied, the results of that are also output.
+ compilation.
  If the option -s is given to pcretest, it outputs the size of each compiled
  pattern after it has been compiled.
- If the -t option is given, each compile, study, and match is run 2000 times
+ If the -t option is given, each compile, study, and match is run 10000 times
  while being timed, and the resulting time per compile or match is output in
  milliseconds. Do not set -t with -s, because you will then get the size output
-times and the timing will be distorted.
+times and the timing will be distorted. If you want to change the number
+ of repetitions used for timing, edit the definition of LOOPREPEAT at the top of
+ pcretest.c
-Line 219 
 The perltest program
+Line 274 
 The perltest program
  The perltest program tests Perl's regular expressions; it has the same
  specification as pcretest, and so can be given identical input, except that
- input patterns can be followed only by Perl's lower case options.
+ input patterns can be followed only by Perl's lower case options. The contents
+ of testinput and testinput3 meet this condition.
  The data lines are processed as Perl strings, so if they contain $ or @
  characters, these have to be escaped. For this reason, all such characters in
-Line 228 
 for pcretest, and the special upper case
+Line 284 
 for pcretest, and the special upper case
  recognizes are not used in this file. The output should be identical, apart
  from the initial identifying banner.
- The testinput2 file is not suitable for feeding to Perltest, since it does
+ The testinput2 and testinput4 files are not suitable for feeding to Perltest,
- make use of the special upper case options and escapes that pcretest uses to
+ since they do make use of the special upper case options and escapes that
- test additional features of PCRE.
+ pcretest uses to test some features of PCRE. The first of these files also
+ contains malformed regular expressions, in order to check that PCRE diagnoses
+ them correctly.
  Philip Hazel <ph10@cam.ac.uk>
- October 1997
+ January 1999

 Legend:



Removed from v.7
 


changed lines


 
Added in v.27
 Legend:



Removed from v.7
 


changed lines


 
Added in v.27
-Removed from v.7
+Added in v.27

	ViewVC Help
Powered by ViewVC 1.1.5