1 |
README file for PCRE (Perl-compatible regular expressions) |
README file for PCRE (Perl-compatible regular expression library) |
2 |
---------------------------------------------------------- |
----------------------------------------------------------------- |
3 |
|
|
4 |
******************************************************************************* |
The latest release of PCRE is always available from |
|
* IMPORTANT FOR THOSE UPGRADING FROM VERSIONS BEFORE 2.00 * |
|
|
* * |
|
|
* Please note that there has been a change in the API such that a larger * |
|
|
* ovector is required at matching time, to provide some additional workspace. * |
|
|
* The new man page has details. This change was necessary in order to support * |
|
|
* some of the new functionality in Perl 5.005. * |
|
|
* * |
|
|
* IMPORTANT FOR THOSE UPGRADING FROM VERSION 2.00 * |
|
|
* * |
|
|
* Another (I hope this is the last!) change has been made to the API for the * |
|
|
* pcre_compile() function. An additional argument has been added to make it * |
|
|
* possible to pass over a pointer to character tables built in the current * |
|
|
* locale by pcre_maketables(). To use the default tables, this new arguement * |
|
|
* should be passed as NULL. * |
|
|
* * |
|
|
* IMPORTANT FOR THOSE UPGRADING FROM VERSION 2.05 * |
|
|
* * |
|
|
* Yet another (and again I hope this really is the last) change has been made * |
|
|
* to the API for the pcre_exec() function. An additional argument has been * |
|
|
* added to make it possible to start the match other than at the start of the * |
|
|
* subject string. This is important if there are lookbehinds. The new man * |
|
|
* page has the details, but you just want to convert existing programs, all * |
|
|
* you need to do is to stick in a new fifth argument to pcre_exec(), with a * |
|
|
* value of zero. For example, change * |
|
|
* * |
|
|
* pcre_exec(pattern, extra, subject, length, options, ovec, ovecsize) * |
|
|
* to * |
|
|
* pcre_exec(pattern, extra, subject, length, 0, options, ovec, ovecsize) * |
|
|
******************************************************************************* |
|
5 |
|
|
6 |
|
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz |
7 |
|
|
8 |
The distribution should contain the following files: |
Please read the NEWS file if you are upgrading from a previous release. |
9 |
|
|
10 |
|
PCRE has its own native API, but a set of "wrapper" functions that are based on |
11 |
|
the POSIX API are also supplied in the library libpcreposix. Note that this |
12 |
|
just provides a POSIX calling interface to PCRE: the regular expressions |
13 |
|
themselves still follow Perl syntax and semantics. The header file |
14 |
|
for the POSIX-style functions is called pcreposix.h. The official POSIX name is |
15 |
|
regex.h, but I didn't want to risk possible problems with existing files of |
16 |
|
that name by distributing it that way. To use it with an existing program that |
17 |
|
uses the POSIX API, it will have to be renamed or pointed at by a link. |
18 |
|
|
19 |
|
|
20 |
|
Contributions by users of PCRE |
21 |
|
------------------------------ |
22 |
|
|
23 |
|
You can find contributions from PCRE users in the directory |
24 |
|
|
25 |
|
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib |
26 |
|
|
27 |
|
where there is also a README file giving brief descriptions of what they are. |
28 |
|
Several of them provide support for compiling PCRE on various flavours of |
29 |
|
Windows systems (I myself do not use Windows). Some are complete in themselves; |
30 |
|
others are pointers to URLs containing relevant files. |
31 |
|
|
32 |
|
|
33 |
|
Building PCRE on a Unix-like system |
34 |
|
----------------------------------- |
35 |
|
|
36 |
|
To build PCRE on a Unix-like system, first run the "configure" command from the |
37 |
|
PCRE distribution directory, with your current directory set to the directory |
38 |
|
where you want the files to be created. This command is a standard GNU |
39 |
|
"autoconf" configuration script, for which generic instructions are supplied in |
40 |
|
INSTALL. |
41 |
|
|
42 |
|
Most commonly, people build PCRE within its own distribution directory, and in |
43 |
|
this case, on many systems, just running "./configure" is sufficient, but the |
44 |
|
usual methods of changing standard defaults are available. For example, |
45 |
|
|
46 |
|
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local |
47 |
|
|
48 |
|
specifies that the C compiler should be run with the flags '-O2 -Wall' instead |
49 |
|
of the default, and that "make install" should install PCRE under /opt/local |
50 |
|
instead of the default /usr/local. |
51 |
|
|
52 |
|
If you want to build in a different directory, just run "configure" with that |
53 |
|
directory as current. For example, suppose you have unpacked the PCRE source |
54 |
|
into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx: |
55 |
|
|
56 |
|
cd /build/pcre/pcre-xxx |
57 |
|
/source/pcre/pcre-xxx/configure |
58 |
|
|
59 |
|
There are some optional features that can be included or omitted from the PCRE |
60 |
|
library. You can read more about them in the pcrebuild man page. |
61 |
|
|
62 |
|
. If you want to make use of the support for UTF-8 character strings in PCRE, |
63 |
|
you must add --enable-utf8 to the "configure" command. Without it, the code |
64 |
|
for handling UTF-8 is not included in the library. (Even when included, it |
65 |
|
still has to be enabled by an option at run time.) |
66 |
|
|
67 |
|
. You can build PCRE to recognized CR or NL as the newline character, instead |
68 |
|
of whatever your compiler uses for "\n", by adding --newline-is-cr or |
69 |
|
--newline-is-nl to the "configure" command, respectively. Only do this if you |
70 |
|
really understand what you are doing. On traditional Unix-like systems, the |
71 |
|
newline character is NL. |
72 |
|
|
73 |
|
. When called via the POSIX interface, PCRE uses malloc() to get additional |
74 |
|
storage for processing capturing parentheses if there are more than 10 of |
75 |
|
them. You can increase this threshold by setting, for example, |
76 |
|
|
77 |
|
--with-posix-malloc-threshold=20 |
78 |
|
|
79 |
|
on the "configure" command. |
80 |
|
|
81 |
|
. PCRE has a counter which can be set to limit the amount of resources it uses. |
82 |
|
If the limit is exceeded during a match, the match fails. The default is ten |
83 |
|
million. You can change the default by setting, for example, |
84 |
|
|
85 |
|
--with-match-limit=500000 |
86 |
|
|
87 |
|
on the "configure" command. This is just the default; individual calls to |
88 |
|
pcre_exec() can supply their own value. There is discussion on the pcreapi |
89 |
|
man page. |
90 |
|
|
91 |
|
. The default maximum compiled pattern size is around 64K. You can increase |
92 |
|
this by adding --with-link-size=3 to the "configure" command. You can |
93 |
|
increase it even more by setting --with-link-size=4, but this is unlikely |
94 |
|
ever to be necessary. If you build PCRE with an increased link size, test 2 |
95 |
|
(and 5 if you are using UTF-8) will fail. Part of the output of these tests |
96 |
|
is a representation of the compiled pattern, and this changes with the link |
97 |
|
size. |
98 |
|
|
99 |
|
The "configure" script builds five files: |
100 |
|
|
101 |
|
. libtool is a script that builds shared and/or static libraries |
102 |
|
. Makefile is built by copying Makefile.in and making substitutions. |
103 |
|
. config.h is built by copying config.in and making substitutions. |
104 |
|
. pcre-config is built by copying pcre-config.in and making substitutions. |
105 |
|
. RunTest is a script for running tests |
106 |
|
|
107 |
|
Once "configure" has run, you can run "make". It builds two libraries called |
108 |
|
libpcre and libpcreposix, a test program called pcretest, and the pcregrep |
109 |
|
command. You can use "make install" to copy these, the public header files |
110 |
|
pcre.h and pcreposix.h, and the man pages to appropriate live directories on |
111 |
|
your system, in the normal way. |
112 |
|
|
113 |
|
Running "make install" also installs the command pcre-config, which can be used |
114 |
|
to recall information about the PCRE configuration and installation. For |
115 |
|
example, |
116 |
|
|
117 |
|
pcre-config --version |
118 |
|
|
119 |
ChangeLog log of changes to the code |
prints the version number, and |
|
LICENCE conditions for the use of PCRE |
|
|
Makefile for building PCRE |
|
|
README this file |
|
|
RunTest a shell script for running tests |
|
|
Tech.Notes notes on the encoding |
|
|
pcre.3 man page for the functions |
|
|
pcreposix.3 man page for the POSIX wrapper API |
|
|
dftables.c auxiliary program for building chartables.c |
|
|
get.c ) |
|
|
maketables.c ) |
|
|
study.c ) source of |
|
|
pcre.c ) the functions |
|
|
pcreposix.c ) |
|
|
pcre.h header for the external API |
|
|
pcreposix.h header for the external POSIX wrapper API |
|
|
internal.h header for internal use |
|
|
pcretest.c test program |
|
|
pgrep.1 man page for pgrep |
|
|
pgrep.c source of a grep utility that uses PCRE |
|
|
perltest Perl test program |
|
|
testinput1 test data, compatible with Perl 5.004 and 5.005 |
|
|
testinput2 test data for error messages and non-Perl things |
|
|
testinput3 test data, compatible with Perl 5.005 |
|
|
testinput4 test data for locale-specific tests |
|
|
testoutput1 test results corresponding to testinput1 |
|
|
testoutput2 test results corresponding to testinput2 |
|
|
testoutput3 test results corresponding to testinput3 |
|
|
testoutput4 test results corresponding to testinput4 |
|
|
|
|
|
To build PCRE, edit Makefile for your system (it is a fairly simple make file, |
|
|
and there are some comments at the top) and then run it. It builds two |
|
|
libraries called libpcre.a and libpcreposix.a, a test program called pcretest, |
|
|
and the pgrep command. |
|
|
|
|
|
To test PCRE, run the RunTest script in the pcre directory. This runs pcretest |
|
|
on each of the testinput files in turn, and compares the output with the |
|
|
contents of the corresponding testoutput file. A file called testtry is used to |
|
|
hold the output from pcretest (which is documented below). |
|
|
|
|
|
To run pcretest on just one of the test files, give its number as an argument |
|
|
to RunTest, for example: |
|
|
|
|
|
RunTest 3 |
|
|
|
|
|
The first and third test files can also be fed directly into the perltest |
|
|
program to check that Perl gives the same results. The third file requires the |
|
|
additional features of release 5.005, which is why it is kept separate from the |
|
|
main test input, which needs only Perl 5.004. In the long run, when 5.005 is |
|
|
widespread, these two test files may get amalgamated. |
|
|
|
|
|
The second set of tests check pcre_info(), pcre_study(), pcre_copy_substring(), |
|
|
pcre_get_substring(), pcre_get_substring_list(), error detection and run-time |
|
|
flags that are specific to PCRE, as well as the POSIX wrapper API. |
|
120 |
|
|
121 |
The fourth set of tests checks pcre_maketables(), the facility for building a |
pcre-config --libs |
122 |
|
|
123 |
|
outputs information about where the library is installed. This command can be |
124 |
|
included in makefiles for programs that use PCRE, saving the programmer from |
125 |
|
having to remember too many details. |
126 |
|
|
127 |
|
|
128 |
|
Cross-compiling PCRE on a Unix-like system |
129 |
|
------------------------------------------ |
130 |
|
|
131 |
|
PCRE needs to compile and run an auxiliary program as part of the building |
132 |
|
process. Obviously, if the real compilation is for some other system, it can't |
133 |
|
use the same CC and CFLAGS values when it is doing this. For cross compilation, |
134 |
|
therefore, you must set CC_FOR_BUILD to the local host's compiler, and you can |
135 |
|
set flags in CFLAGS_FOR_BUILD if you need to. |
136 |
|
|
137 |
|
|
138 |
|
Shared libraries on Unix-like systems |
139 |
|
------------------------------------- |
140 |
|
|
141 |
|
The default distribution builds PCRE as two shared libraries and two static |
142 |
|
libraries, as long as the operating system supports shared libraries. Shared |
143 |
|
library support relies on the "libtool" script which is built as part of the |
144 |
|
"configure" process. |
145 |
|
|
146 |
|
The libtool script is used to compile and link both shared and static |
147 |
|
libraries. They are placed in a subdirectory called .libs when they are newly |
148 |
|
built. The programs pcretest and pcregrep are built to use these uninstalled |
149 |
|
libraries (by means of wrapper scripts in the case of shared libraries). When |
150 |
|
you use "make install" to install shared libraries, pcregrep and pcretest are |
151 |
|
automatically re-built to use the newly installed shared libraries before being |
152 |
|
installed themselves. However, the versions left in the source directory still |
153 |
|
use the uninstalled libraries. |
154 |
|
|
155 |
|
To build PCRE using static libraries only you must use --disable-shared when |
156 |
|
configuring it. For example |
157 |
|
|
158 |
|
./configure --prefix=/usr/gnu --disable-shared |
159 |
|
|
160 |
|
Then run "make" in the usual way. Similarly, you can use --disable-static to |
161 |
|
build only shared libraries. |
162 |
|
|
163 |
|
|
164 |
|
Cross-compiling on a Unix-like system |
165 |
|
------------------------------------- |
166 |
|
|
167 |
|
You can specify CC and CFLAGS in the normal way to the "configure" command, in |
168 |
|
order to cross-compile PCRE for some other host. However, during the building |
169 |
|
process, the dftables.c source file is compiled *and run* on the local host, in |
170 |
|
order to generate the default character tables (the chartables.c file). It |
171 |
|
therefore needs to be compiled with the local compiler, not the cross compiler. |
172 |
|
You can do this by specifying HOST_CC (and if necessary HOST_CFLAGS) when |
173 |
|
calling the "configure" command. If they are not specified, they default to the |
174 |
|
values of CC and CFLAGS. |
175 |
|
|
176 |
|
|
177 |
|
Building on non-Unix systems |
178 |
|
---------------------------- |
179 |
|
|
180 |
|
For a non-Unix system, read the comments in the file NON-UNIX-USE. PCRE has |
181 |
|
been compiled on Windows systems and on Macintoshes, but I don't know the |
182 |
|
details because I don't use those systems. It should be straightforward to |
183 |
|
build PCRE on any system that has a Standard C compiler, because it uses only |
184 |
|
Standard C functions. |
185 |
|
|
186 |
|
|
187 |
|
Testing PCRE |
188 |
|
------------ |
189 |
|
|
190 |
|
To test PCRE on a Unix system, run the RunTest script that is created by the |
191 |
|
configuring process. (This can also be run by "make runtest", "make check", or |
192 |
|
"make test".) For other systems, see the instruction in NON-UNIX-USE. |
193 |
|
|
194 |
|
The script runs the pcretest test program (which is documented in its own man |
195 |
|
page) on each of the testinput files (in the testdata directory) in turn, |
196 |
|
and compares the output with the contents of the corresponding testoutput file. |
197 |
|
A file called testtry is used to hold the output from pcretest. To run pcretest |
198 |
|
on just one of the test files, give its number as an argument to RunTest, for |
199 |
|
example: |
200 |
|
|
201 |
|
RunTest 2 |
202 |
|
|
203 |
|
The first file can also be fed directly into the perltest script to check that |
204 |
|
Perl gives the same results. The only difference you should see is in the first |
205 |
|
few lines, where the Perl version is given instead of the PCRE version. |
206 |
|
|
207 |
|
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(), |
208 |
|
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error |
209 |
|
detection, and run-time flags that are specific to PCRE, as well as the POSIX |
210 |
|
wrapper API. It also uses the debugging flag to check some of the internals of |
211 |
|
pcre_compile(). |
212 |
|
|
213 |
|
If you build PCRE with a locale setting that is not the standard C locale, the |
214 |
|
character tables may be different (see next paragraph). In some cases, this may |
215 |
|
cause failures in the second set of tests. For example, in a locale where the |
216 |
|
isprint() function yields TRUE for characters in the range 128-255, the use of |
217 |
|
[:isascii:] inside a character class defines a different set of characters, and |
218 |
|
this shows up in this test as a difference in the compiled code, which is being |
219 |
|
listed for checking. Where the comparison test output contains [\x00-\x7f] the |
220 |
|
test will contain [\x00-\xff], and similarly in some other cases. This is not a |
221 |
|
bug in PCRE. |
222 |
|
|
223 |
|
The third set of tests checks pcre_maketables(), the facility for building a |
224 |
set of character tables for a specific locale and using them instead of the |
set of character tables for a specific locale and using them instead of the |
225 |
default tables. The tests make use of the "fr" (French) locale. Before running |
default tables. The tests make use of the "fr" (French) locale. Before running |
226 |
the test, the script checks for the presence of this locale by running the |
the test, the script checks for the presence of this locale by running the |
227 |
"locale" command. If that command fails, or if it doesn't include "fr" in the |
"locale" command. If that command fails, or if it doesn't include "fr" in the |
228 |
list of available locales, the fourth test cannot be run, and a comment is |
list of available locales, the third test cannot be run, and a comment is |
229 |
output to say why. If running this test produces instances of the error |
output to say why. If running this test produces instances of the error |
230 |
|
|
231 |
** Failed to set locale "fr" |
** Failed to set locale "fr" |
233 |
in the comparison output, it means that locale is not available on your system, |
in the comparison output, it means that locale is not available on your system, |
234 |
despite being listed by "locale". This does not mean that PCRE is broken. |
despite being listed by "locale". This does not mean that PCRE is broken. |
235 |
|
|
236 |
To install PCRE, copy libpcre.a to any suitable library directory (e.g. |
The fourth test checks the UTF-8 support. It is not run automatically unless |
237 |
/usr/local/lib), pcre.h to any suitable include directory (e.g. |
PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when |
238 |
/usr/local/include), and pcre.3 to any suitable man directory (e.g. |
running "configure". This file can be also fed directly to the perltest script, |
239 |
/usr/local/man/man3). |
provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch, |
240 |
|
commented in the script, can be be used.) |
|
To install the pgrep command, copy it to any suitable binary directory, (e.g. |
|
|
/usr/local/bin) and pgrep.1 to any suitable man directory (e.g. |
|
|
/usr/local/man/man1). |
|
241 |
|
|
242 |
PCRE has its own native API, but a set of "wrapper" functions that are based on |
The fifth and final file tests error handling with UTF-8 encoding, and internal |
243 |
the POSIX API are also supplied in the library libpcreposix.a. Note that this |
UTF-8 features of PCRE that are not relevant to Perl. |
|
just provides a POSIX calling interface to PCRE: the regular expressions |
|
|
themselves still follow Perl syntax and semantics. The header file |
|
|
for the POSIX-style functions is called pcreposix.h. The official POSIX name is |
|
|
regex.h, but I didn't want to risk possible problems with existing files of |
|
|
that name by distributing it that way. To use it with an existing program that |
|
|
uses the POSIX API, it will have to be renamed or pointed at by a link. |
|
244 |
|
|
245 |
|
|
246 |
Character tables |
Character tables |
282 |
will cause PCRE to malfunction. |
will cause PCRE to malfunction. |
283 |
|
|
284 |
|
|
285 |
The pcretest program |
Manifest |
286 |
-------------------- |
-------- |
|
|
|
|
This program is intended for testing PCRE, but it can also be used for |
|
|
experimenting with regular expressions. |
|
287 |
|
|
288 |
If it is given two filename arguments, it reads from the first and writes to |
The distribution should contain the following files: |
|
the second. If it is given only one filename argument, it reads from that file |
|
|
and writes to stdout. Otherwise, it reads from stdin and writes to stdout, and |
|
|
prompts for each line of input. |
|
|
|
|
|
The program handles any number of sets of input on a single input file. Each |
|
|
set starts with a regular expression, and continues with any number of data |
|
|
lines to be matched against the pattern. An empty line signals the end of the |
|
|
set. The regular expressions are given enclosed in any non-alphameric |
|
|
delimiters other than backslash, for example |
|
|
|
|
|
/(a|bc)x+yz/ |
|
|
|
|
|
White space before the initial delimiter is ignored. A regular expression may |
|
|
be continued over several input lines, in which case the newline characters are |
|
|
included within it. See the testinput files for many examples. It is possible |
|
|
to include the delimiter within the pattern by escaping it, for example |
|
|
|
|
|
/abc\/def/ |
|
|
|
|
|
If you do so, the escape and the delimiter form part of the pattern, but since |
|
|
delimiters are always non-alphameric, this does not affect its interpretation. |
|
|
If the terminating delimiter is immediately followed by a backslash, for |
|
|
example, |
|
|
|
|
|
/abc/\ |
|
289 |
|
|
290 |
then a backslash is added to the end of the pattern. This is done to provide a |
(A) The actual source files of the PCRE library functions and their |
291 |
way of testing the error condition that arises if a pattern finishes with a |
headers: |
|
backslash, because |
|
292 |
|
|
293 |
/abc\/ |
dftables.c auxiliary program for building chartables.c |
294 |
|
get.c ) |
295 |
|
maketables.c ) |
296 |
|
study.c ) source of |
297 |
|
pcre.c ) the functions |
298 |
|
pcreposix.c ) |
299 |
|
printint.c ) |
300 |
|
pcre.in "source" for the header for the external API; pcre.h |
301 |
|
is built from this by "configure" |
302 |
|
pcreposix.h header for the external POSIX wrapper API |
303 |
|
internal.h header for internal use |
304 |
|
config.in template for config.h, which is built by configure |
305 |
|
|
306 |
|
(B) Auxiliary files: |
307 |
|
|
308 |
|
AUTHORS information about the author of PCRE |
309 |
|
ChangeLog log of changes to the code |
310 |
|
INSTALL generic installation instructions |
311 |
|
LICENCE conditions for the use of PCRE |
312 |
|
COPYING the same, using GNU's standard name |
313 |
|
Makefile.in template for Unix Makefile, which is built by configure |
314 |
|
NEWS important changes in this release |
315 |
|
NON-UNIX-USE notes on building PCRE on non-Unix systems |
316 |
|
README this file |
317 |
|
RunTest.in template for a Unix shell script for running tests |
318 |
|
config.guess ) files used by libtool, |
319 |
|
config.sub ) used only when building a shared library |
320 |
|
configure a configuring shell script (built by autoconf) |
321 |
|
configure.in the autoconf input used to build configure |
322 |
|
doc/Tech.Notes notes on the encoding |
323 |
|
doc/*.3 man page sources for the PCRE functions |
324 |
|
doc/*.1 man page sources for pcregrep and pcretest |
325 |
|
doc/html/* HTML documentation |
326 |
|
doc/pcre.txt plain text version of the man pages |
327 |
|
doc/pcretest.txt plain text documentation of test program |
328 |
|
doc/perltest.txt plain text documentation of Perl test program |
329 |
|
install-sh a shell script for installing files |
330 |
|
ltmain.sh file used to build a libtool script |
331 |
|
pcretest.c comprehensive test program |
332 |
|
pcredemo.c simple demonstration of coding calls to PCRE |
333 |
|
perltest Perl test program |
334 |
|
pcregrep.c source of a grep utility that uses PCRE |
335 |
|
pcre-config.in source of script which retains PCRE information |
336 |
|
testdata/testinput1 test data, compatible with Perl |
337 |
|
testdata/testinput2 test data for error messages and non-Perl things |
338 |
|
testdata/testinput3 test data for locale-specific tests |
339 |
|
testdata/testinput4 test data for UTF-8 tests compatible with Perl |
340 |
|
testdata/testinput5 test data for other UTF-8 tests |
341 |
|
testdata/testoutput1 test results corresponding to testinput1 |
342 |
|
testdata/testoutput2 test results corresponding to testinput2 |
343 |
|
testdata/testoutput3 test results corresponding to testinput3 |
344 |
|
testdata/testoutput4 test results corresponding to testinput4 |
345 |
|
testdata/testoutput5 test results corresponding to testinput5 |
346 |
|
|
347 |
is interpreted as the first line of a pattern that starts with "abc/", causing |
(C) Auxiliary files for Win32 DLL |
|
pcretest to read the next line as a continuation of the regular expression. |
|
348 |
|
|
349 |
The pattern may be followed by i, m, s, or x to set the PCRE_CASELESS, |
dll.mk |
350 |
PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively. For |
pcre.def |
|
example: |
|
351 |
|
|
352 |
/caseless/i |
(D) Auxiliary file for VPASCAL |
353 |
|
|
354 |
These modifier letters have the same effect as they do in Perl. There are |
makevp.bat |
|
others which set PCRE options that do not correspond to anything in Perl: /A, |
|
|
/E, and /X set PCRE_ANCHORED, PCRE_DOLLAR_ENDONLY, and PCRE_EXTRA respectively. |
|
|
|
|
|
Searching for all possible matches within each subject string can be requested |
|
|
by the /g or /G modifier. The /g modifier behaves similarly to the way it does |
|
|
in Perl. After finding a match, PCRE is called again to search the remainder of |
|
|
the subject string. The difference between /g and /G is that the former uses |
|
|
the start_offset argument to pcre_exec() to start searching at a new point |
|
|
within the entire string, whereas the latter passes over a shortened substring. |
|
|
This makes a difference to the matching process if the pattern begins with a |
|
|
lookbehind assertion (including \b or \B). |
|
|
|
|
|
There are a number of other modifiers for controlling the way pcretest |
|
|
operates. |
|
|
|
|
|
The /+ modifier requests that as well as outputting the substring that matched |
|
|
the entire pattern, pcretest should in addition output the remainder of the |
|
|
subject string. This is useful for tests where the subject contains multiple |
|
|
copies of the same substring. |
|
|
|
|
|
The /L modifier must be followed directly by the name of a locale, for example, |
|
|
|
|
|
/pattern/Lfr |
|
|
|
|
|
For this reason, it must be the last modifier letter. The given locale is set, |
|
|
pcre_maketables() is called to build a set of character tables for the locale, |
|
|
and this is then passed to pcre_compile() when compiling the regular |
|
|
expression. Without an /L modifier, NULL is passed as the tables pointer; that |
|
|
is, /L applies only to the expression on which it appears. |
|
|
|
|
|
The /I modifier requests that pcretest output information about the compiled |
|
|
expression (whether it is anchored, has a fixed first character, and so on). It |
|
|
does this by calling pcre_info() after compiling an expression, and outputting |
|
|
the information it gets back. If the pattern is studied, the results of that |
|
|
are also output. |
|
|
|
|
|
The /D modifier is a PCRE debugging feature, which also assumes /I. It causes |
|
|
the internal form of compiled regular expressions to be output after |
|
|
compilation. |
|
|
|
|
|
The /S modifier causes pcre_study() to be called after the expression has been |
|
|
compiled, and the results used when the expression is matched. |
|
|
|
|
|
The /M modifier causes information about the size of memory block used to hold |
|
|
the compile pattern to be output. |
|
|
|
|
|
Finally, the /P modifier causes pcretest to call PCRE via the POSIX wrapper API |
|
|
rather than its native API. When this is done, all other modifiers except /i, |
|
|
/m, and /+ are ignored. REG_ICASE is set if /i is present, and REG_NEWLINE is |
|
|
set if /m is present. The wrapper functions force PCRE_DOLLAR_ENDONLY always, |
|
|
and PCRE_DOTALL unless REG_NEWLINE is set. |
|
|
|
|
|
Before each data line is passed to pcre_exec(), leading and trailing whitespace |
|
|
is removed, and it is then scanned for \ escapes. The following are recognized: |
|
|
|
|
|
\a alarm (= BEL) |
|
|
\b backspace |
|
|
\e escape |
|
|
\f formfeed |
|
|
\n newline |
|
|
\r carriage return |
|
|
\t tab |
|
|
\v vertical tab |
|
|
\nnn octal character (up to 3 octal digits) |
|
|
\xhh hexadecimal character (up to 2 hex digits) |
|
|
|
|
|
\A pass the PCRE_ANCHORED option to pcre_exec() |
|
|
\B pass the PCRE_NOTBOL option to pcre_exec() |
|
|
\Cdd call pcre_copy_substring() for substring dd after a successful match |
|
|
(any decimal number less than 32) |
|
|
\Gdd call pcre_get_substring() for substring dd after a successful match |
|
|
(any decimal number less than 32) |
|
|
\L call pcre_get_substringlist() after a successful match |
|
|
\Odd set the size of the output vector passed to pcre_exec() to dd |
|
|
(any number of decimal digits) |
|
|
\Z pass the PCRE_NOTEOL option to pcre_exec() |
|
|
|
|
|
A backslash followed by anything else just escapes the anything else. If the |
|
|
very last character is a backslash, it is ignored. This gives a way of passing |
|
|
an empty line as data, since a real empty line terminates the data input. |
|
|
|
|
|
If /P was present on the regex, causing the POSIX wrapper API to be used, only |
|
|
\B, and \Z have any effect, causing REG_NOTBOL and REG_NOTEOL to be passed to |
|
|
regexec() respectively. |
|
|
|
|
|
When a match succeeds, pcretest outputs the list of captured substrings that |
|
|
pcre_exec() returns, starting with number 0 for the string that matched the |
|
|
whole pattern. Here is an example of an interactive pcretest run. |
|
|
|
|
|
$ pcretest |
|
|
PCRE version 2.06 08-Jun-1999 |
|
|
|
|
|
re> /^abc(\d+)/ |
|
|
data> abc123 |
|
|
0: abc123 |
|
|
1: 123 |
|
|
data> xyz |
|
|
No match |
|
|
|
|
|
If the strings contain any non-printing characters, they are output as \0x |
|
|
escapes. If the pattern has the /+ modifier, then the output for substring 0 is |
|
|
followed by the the rest of the subject string, identified by "0+" like this: |
|
|
|
|
|
re> /cat/+ |
|
|
data> cataract |
|
|
0: cat |
|
|
0+ aract |
|
|
|
|
|
If the pattern has the /g or /G modifier, the results of successive matching |
|
|
attempts are output in sequence, like this: |
|
|
|
|
|
re> /\Bi(\w\w)/g |
|
|
data> Mississippi |
|
|
0: iss |
|
|
1: ss |
|
|
0: iss |
|
|
1: ss |
|
|
0: ipp |
|
|
1: pp |
|
|
|
|
|
"No match" is output only if the first match attempt fails. |
|
|
|
|
|
If any of \C, \G, or \L are present in a data line that is successfully |
|
|
matched, the substrings extracted by the convenience functions are output with |
|
|
C, G, or L after the string number instead of a colon. This is in addition to |
|
|
the normal full list. The string length (that is, the return from the |
|
|
extraction function) is given in parentheses after each string for \C and \G. |
|
|
|
|
|
Note that while patterns can be continued over several lines (a plain ">" |
|
|
prompt is used for continuations), data lines may not. However newlines can be |
|
|
included in data by means of the \n escape. |
|
|
|
|
|
If the -p option is given to pcretest, it is equivalent to adding /P to each |
|
|
regular expression: the POSIX wrapper API is used to call PCRE. None of the |
|
|
following flags has any effect in this case. |
|
|
|
|
|
If the option -d is given to pcretest, it is equivalent to adding /D to each |
|
|
regular expression: the internal form is output after compilation. |
|
|
|
|
|
If the option -i is given to pcretest, it is equivalent to adding /I to each |
|
|
regular expression: information about the compiled pattern is given after |
|
|
compilation. |
|
|
|
|
|
If the option -m is given to pcretest, it outputs the size of each compiled |
|
|
pattern after it has been compiled. It is equivalent to adding /M to each |
|
|
regular expression. For compatibility with earlier versions of pcretest, -s is |
|
|
a synonym for -m. |
|
|
|
|
|
If the -t option is given, each compile, study, and match is run 20000 times |
|
|
while being timed, and the resulting time per compile or match is output in |
|
|
milliseconds. Do not set -t with -s, because you will then get the size output |
|
|
20000 times and the timing will be distorted. If you want to change the number |
|
|
of repetitions used for timing, edit the definition of LOOPREPEAT at the top of |
|
|
pcretest.c |
|
|
|
|
|
|
|
|
|
|
|
The perltest program |
|
|
-------------------- |
|
|
|
|
|
The perltest program tests Perl's regular expressions; it has the same |
|
|
specification as pcretest, and so can be given identical input, except that |
|
|
input patterns can be followed only by Perl's lower case modifiers. The |
|
|
contents of testinput1 and testinput3 meet this condition. |
|
|
|
|
|
The data lines are processed as Perl strings, so if they contain $ or @ |
|
|
characters, these have to be escaped. For this reason, all such characters in |
|
|
testinput1 and testinput3 are escaped so that they can be used for perltest as |
|
|
well as for pcretest, and the special upper case modifiers such as /A that |
|
|
pcretest recognizes are not used in these files. The output should be |
|
|
identical, apart from the initial identifying banner. |
|
|
|
|
|
The testinput2 and testinput4 files are not suitable for feeding to perltest, |
|
|
since they do make use of the special upper case modifiers and escapes that |
|
|
pcretest uses to test some features of PCRE. The first of these files also |
|
|
contains malformed regular expressions, in order to check that PCRE diagnoses |
|
|
them correctly. |
|
355 |
|
|
356 |
Philip Hazel <ph10@cam.ac.uk> |
Philip Hazel <ph10@cam.ac.uk> |
357 |
June 1999 |
February 2003 |