1 |
README file for PCRE (Perl-compatible regular expression library)
|
2 |
-----------------------------------------------------------------
|
3 |
|
4 |
The latest release of PCRE is always available from
|
5 |
|
6 |
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-xxx.tar.gz
|
7 |
|
8 |
There is a mailing list for discussion about the development of PCRE at
|
9 |
|
10 |
pcre-dev@exim.org
|
11 |
|
12 |
Please read the NEWS file if you are upgrading from a previous release.
|
13 |
The contents of this README file are:
|
14 |
|
15 |
The PCRE APIs
|
16 |
Documentation for PCRE
|
17 |
Contributions by users of PCRE
|
18 |
Building PCRE on non-Unix systems
|
19 |
Building PCRE on Unix-like systems
|
20 |
Retrieving configuration information on Unix-like systems
|
21 |
Shared libraries on Unix-like systems
|
22 |
Cross-compiling on Unix-like systems
|
23 |
Using HP's ANSI C++ compiler (aCC)
|
24 |
Making new tarballs
|
25 |
Testing PCRE
|
26 |
Character tables
|
27 |
File manifest
|
28 |
|
29 |
|
30 |
The PCRE APIs
|
31 |
-------------
|
32 |
|
33 |
PCRE is written in C, and it has its own API. The distribution also includes a
|
34 |
set of C++ wrapper functions (see the pcrecpp man page for details), courtesy
|
35 |
of Google Inc.
|
36 |
|
37 |
In addition, there is a set of C wrapper functions that are based on the POSIX
|
38 |
regular expression API (see the pcreposix man page). These end up in the
|
39 |
library called libpcreposix. Note that this just provides a POSIX calling
|
40 |
interface to PCRE; the regular expressions themselves still follow Perl syntax
|
41 |
and semantics. The POSIX API is restricted, and does not give full access to
|
42 |
all of PCRE's facilities.
|
43 |
|
44 |
The header file for the POSIX-style functions is called pcreposix.h. The
|
45 |
official POSIX name is regex.h, but I did not want to risk possible problems
|
46 |
with existing files of that name by distributing it that way. To use PCRE with
|
47 |
an existing program that uses the POSIX API, pcreposix.h will have to be
|
48 |
renamed or pointed at by a link.
|
49 |
|
50 |
If you are using the POSIX interface to PCRE and there is already a POSIX regex
|
51 |
library installed on your system, as well as worrying about the regex.h header
|
52 |
file (as mentioned above), you must also take care when linking programs to
|
53 |
ensure that they link with PCRE's libpcreposix library. Otherwise they may pick
|
54 |
up the POSIX functions of the same name from the other library.
|
55 |
|
56 |
One way of avoiding this confusion is to compile PCRE with the addition of
|
57 |
-Dregcomp=PCREregcomp (and similarly for the other POSIX functions) to the
|
58 |
compiler flags (CFLAGS if you are using "configure" -- see below). This has the
|
59 |
effect of renaming the functions so that the names no longer clash. Of course,
|
60 |
you have to do the same thing for your applications, or write them using the
|
61 |
new names.
|
62 |
|
63 |
|
64 |
Documentation for PCRE
|
65 |
----------------------
|
66 |
|
67 |
If you install PCRE in the normal way on a Unix-like system, you will end up
|
68 |
with a set of man pages whose names all start with "pcre". The one that is just
|
69 |
called "pcre" lists all the others. In addition to these man pages, the PCRE
|
70 |
documentation is supplied in two other forms:
|
71 |
|
72 |
1. There are files called doc/pcre.txt, doc/pcregrep.txt, and
|
73 |
doc/pcretest.txt in the source distribution. The first of these is a
|
74 |
concatenation of the text forms of all the section 3 man pages except
|
75 |
those that summarize individual functions. The other two are the text
|
76 |
forms of the section 1 man pages for the pcregrep and pcretest commands.
|
77 |
These text forms are provided for ease of scanning with text editors or
|
78 |
similar tools. They are installed in <prefix>/share/doc/pcre, where
|
79 |
<prefix> is the installation prefix (defaulting to /usr/local).
|
80 |
|
81 |
2. A set of files containing all the documentation in HTML form, hyperlinked
|
82 |
in various ways, and rooted in a file called index.html, is distributed in
|
83 |
doc/html and installed in <prefix>/share/doc/pcre/html.
|
84 |
|
85 |
|
86 |
Contributions by users of PCRE
|
87 |
------------------------------
|
88 |
|
89 |
You can find contributions from PCRE users in the directory
|
90 |
|
91 |
ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/Contrib
|
92 |
|
93 |
There is a README file giving brief descriptions of what they are. Some are
|
94 |
complete in themselves; others are pointers to URLs containing relevant files.
|
95 |
Some of this material is likely to be well out-of-date. Several of the earlier
|
96 |
contributions provided support for compiling PCRE on various flavours of
|
97 |
Windows (I myself do not use Windows). Nowadays there is more Windows support
|
98 |
in the standard distribution, so these contibutions have been archived.
|
99 |
|
100 |
|
101 |
Building PCRE on non-Unix systems
|
102 |
---------------------------------
|
103 |
|
104 |
For a non-Unix system, please read the comments in the file NON-UNIX-USE,
|
105 |
though if your system supports the use of "configure" and "make" you may be
|
106 |
able to build PCRE in the same way as for Unix-like systems.
|
107 |
|
108 |
PCRE has been compiled on many different operating systems. It should be
|
109 |
straightforward to build PCRE on any system that has a Standard C compiler and
|
110 |
library, because it uses only Standard C functions.
|
111 |
|
112 |
|
113 |
Building PCRE on Unix-like systems
|
114 |
----------------------------------
|
115 |
|
116 |
If you are using HP's ANSI C++ compiler (aCC), please see the special note
|
117 |
in the section entitled "Using HP's ANSI C++ compiler (aCC)" below.
|
118 |
|
119 |
The following instructions assume the use of the widely used "configure, make,
|
120 |
make install" process. There is also some experimental support for "cmake" in
|
121 |
the PCRE distribution, but it is incomplete and not documented. However, if you
|
122 |
are a "cmake" user, you might want to try it.
|
123 |
|
124 |
To build PCRE on a Unix-like system, first run the "configure" command from the
|
125 |
PCRE distribution directory, with your current directory set to the directory
|
126 |
where you want the files to be created. This command is a standard GNU
|
127 |
"autoconf" configuration script, for which generic instructions are supplied in
|
128 |
the file INSTALL.
|
129 |
|
130 |
Most commonly, people build PCRE within its own distribution directory, and in
|
131 |
this case, on many systems, just running "./configure" is sufficient. However,
|
132 |
the usual methods of changing standard defaults are available. For example:
|
133 |
|
134 |
CFLAGS='-O2 -Wall' ./configure --prefix=/opt/local
|
135 |
|
136 |
specifies that the C compiler should be run with the flags '-O2 -Wall' instead
|
137 |
of the default, and that "make install" should install PCRE under /opt/local
|
138 |
instead of the default /usr/local.
|
139 |
|
140 |
If you want to build in a different directory, just run "configure" with that
|
141 |
directory as current. For example, suppose you have unpacked the PCRE source
|
142 |
into /source/pcre/pcre-xxx, but you want to build it in /build/pcre/pcre-xxx:
|
143 |
|
144 |
cd /build/pcre/pcre-xxx
|
145 |
/source/pcre/pcre-xxx/configure
|
146 |
|
147 |
PCRE is written in C and is normally compiled as a C library. However, it is
|
148 |
possible to build it as a C++ library, though the provided building apparatus
|
149 |
does not have any features to support this.
|
150 |
|
151 |
There are some optional features that can be included or omitted from the PCRE
|
152 |
library. You can read more about them in the pcrebuild man page.
|
153 |
|
154 |
. If you want to suppress the building of the C++ wrapper library, you can add
|
155 |
--disable-cpp to the "configure" command. Otherwise, when "configure" is run,
|
156 |
it will try to find a C++ compiler and C++ header files, and if it succeeds,
|
157 |
it will try to build the C++ wrapper.
|
158 |
|
159 |
. If you want to make use of the support for UTF-8 character strings in PCRE,
|
160 |
you must add --enable-utf8 to the "configure" command. Without it, the code
|
161 |
for handling UTF-8 is not included in the library. (Even when included, it
|
162 |
still has to be enabled by an option at run time.)
|
163 |
|
164 |
. If, in addition to support for UTF-8 character strings, you want to include
|
165 |
support for the \P, \p, and \X sequences that recognize Unicode character
|
166 |
properties, you must add --enable-unicode-properties to the "configure"
|
167 |
command. This adds about 30K to the size of the library (in the form of a
|
168 |
property table); only the basic two-letter properties such as Lu are
|
169 |
supported.
|
170 |
|
171 |
. You can build PCRE to recognize either CR or LF or the sequence CRLF or any
|
172 |
of the preceding, or any of the Unicode newline sequences as indicating the
|
173 |
end of a line. Whatever you specify at build time is the default; the caller
|
174 |
of PCRE can change the selection at run time. The default newline indicator
|
175 |
is a single LF character (the Unix standard). You can specify the default
|
176 |
newline indicator by adding --enable-newline-is-cr or --enable-newline-is-lf
|
177 |
or --enable-newline-is-crlf or --enable-newline-is-anycrlf or
|
178 |
--enable-newline-is-any to the "configure" command, respectively.
|
179 |
|
180 |
If you specify --enable-newline-is-cr or --enable-newline-is-crlf, some of
|
181 |
the standard tests will fail, because the lines in the test files end with
|
182 |
LF. Even if the files are edited to change the line endings, there are likely
|
183 |
to be some failures. With --enable-newline-is-anycrlf or
|
184 |
--enable-newline-is-any, many tests should succeed, but there may be some
|
185 |
failures.
|
186 |
|
187 |
. By default, the sequence \R in a pattern matches any Unicode line ending
|
188 |
sequence. This is independent of the option specifying what PCRE considers to
|
189 |
be the end of a line (see above). However, the caller of PCRE can restrict \R
|
190 |
to match only CR, LF, or CRLF. You can make this the default by adding
|
191 |
--enable-bsr-anycrlf to the "configure" command (bsr = "backslash R").
|
192 |
|
193 |
. When called via the POSIX interface, PCRE uses malloc() to get additional
|
194 |
storage for processing capturing parentheses if there are more than 10 of
|
195 |
them in a pattern. You can increase this threshold by setting, for example,
|
196 |
|
197 |
--with-posix-malloc-threshold=20
|
198 |
|
199 |
on the "configure" command.
|
200 |
|
201 |
. PCRE has a counter that can be set to limit the amount of resources it uses.
|
202 |
If the limit is exceeded during a match, the match fails. The default is ten
|
203 |
million. You can change the default by setting, for example,
|
204 |
|
205 |
--with-match-limit=500000
|
206 |
|
207 |
on the "configure" command. This is just the default; individual calls to
|
208 |
pcre_exec() can supply their own value. There is more discussion on the
|
209 |
pcreapi man page.
|
210 |
|
211 |
. There is a separate counter that limits the depth of recursive function calls
|
212 |
during a matching process. This also has a default of ten million, which is
|
213 |
essentially "unlimited". You can change the default by setting, for example,
|
214 |
|
215 |
--with-match-limit-recursion=500000
|
216 |
|
217 |
Recursive function calls use up the runtime stack; running out of stack can
|
218 |
cause programs to crash in strange ways. There is a discussion about stack
|
219 |
sizes in the pcrestack man page.
|
220 |
|
221 |
. The default maximum compiled pattern size is around 64K. You can increase
|
222 |
this by adding --with-link-size=3 to the "configure" command. You can
|
223 |
increase it even more by setting --with-link-size=4, but this is unlikely
|
224 |
ever to be necessary. Increasing the internal link size will reduce
|
225 |
performance.
|
226 |
|
227 |
. You can build PCRE so that its internal match() function that is called from
|
228 |
pcre_exec() does not call itself recursively. Instead, it uses memory blocks
|
229 |
obtained from the heap via the special functions pcre_stack_malloc() and
|
230 |
pcre_stack_free() to save data that would otherwise be saved on the stack. To
|
231 |
build PCRE like this, use
|
232 |
|
233 |
--disable-stack-for-recursion
|
234 |
|
235 |
on the "configure" command. PCRE runs more slowly in this mode, but it may be
|
236 |
necessary in environments with limited stack sizes. This applies only to the
|
237 |
pcre_exec() function; it does not apply to pcre_dfa_exec(), which does not
|
238 |
use deeply nested recursion. There is a discussion about stack sizes in the
|
239 |
pcrestack man page.
|
240 |
|
241 |
. For speed, PCRE uses four tables for manipulating and identifying characters
|
242 |
whose code point values are less than 256. By default, it uses a set of
|
243 |
tables for ASCII encoding that is part of the distribution. If you specify
|
244 |
|
245 |
--enable-rebuild-chartables
|
246 |
|
247 |
a program called dftables is compiled and run in the default C locale when
|
248 |
you obey "make". It builds a source file called pcre_chartables.c. If you do
|
249 |
not specify this option, pcre_chartables.c is created as a copy of
|
250 |
pcre_chartables.c.dist. See "Character tables" below for further information.
|
251 |
|
252 |
. It is possible to compile PCRE for use on systems that use EBCDIC as their
|
253 |
default character code (as opposed to ASCII) by specifying
|
254 |
|
255 |
--enable-ebcdic
|
256 |
|
257 |
This automatically implies --enable-rebuild-chartables (see above).
|
258 |
|
259 |
The "configure" script builds the following files for the basic C library:
|
260 |
|
261 |
. Makefile is the makefile that builds the library
|
262 |
. config.h contains build-time configuration options for the library
|
263 |
. pcre.h is the public PCRE header file
|
264 |
. pcre-config is a script that shows the settings of "configure" options
|
265 |
. libpcre.pc is data for the pkg-config command
|
266 |
. libtool is a script that builds shared and/or static libraries
|
267 |
. RunTest is a script for running tests on the basic C library
|
268 |
. RunGrepTest is a script for running tests on the pcregrep command
|
269 |
|
270 |
Versions of config.h and pcre.h are distributed in the PCRE tarballs under
|
271 |
the names config.h.generic and pcre.h.generic. These are provided for the
|
272 |
benefit of those who have to built PCRE without the benefit of "configure". If
|
273 |
you use "configure", the .generic versions are not used.
|
274 |
|
275 |
If a C++ compiler is found, the following files are also built:
|
276 |
|
277 |
. libpcrecpp.pc is data for the pkg-config command
|
278 |
. pcrecpparg.h is a header file for programs that call PCRE via the C++ wrapper
|
279 |
. pcre_stringpiece.h is the header for the C++ "stringpiece" functions
|
280 |
|
281 |
The "configure" script also creates config.status, which is an executable
|
282 |
script that can be run to recreate the configuration, and config.log, which
|
283 |
contains compiler output from tests that "configure" runs.
|
284 |
|
285 |
Once "configure" has run, you can run "make". It builds two libraries, called
|
286 |
libpcre and libpcreposix, a test program called pcretest, a demonstration
|
287 |
program called pcredemo, and the pcregrep command. If a C++ compiler was found
|
288 |
on your system, "make" also builds the C++ wrapper library, which is called
|
289 |
libpcrecpp, and some test programs called pcrecpp_unittest,
|
290 |
pcre_scanner_unittest, and pcre_stringpiece_unittest. Building the C++ wrapper
|
291 |
can be disabled by adding --disable-cpp to the "configure" command.
|
292 |
|
293 |
The command "make check" runs all the appropriate tests. Details of the PCRE
|
294 |
tests are given below in a separate section of this document.
|
295 |
|
296 |
You can use "make install" to install PCRE into live directories on your
|
297 |
system. The following are installed (file names are all relative to the
|
298 |
<prefix> that is set when "configure" is run):
|
299 |
|
300 |
Commands (bin):
|
301 |
pcretest
|
302 |
pcregrep
|
303 |
pcre-config
|
304 |
|
305 |
Libraries (lib):
|
306 |
libpcre
|
307 |
libpcreposix
|
308 |
libpcrecpp (if C++ support is enabled)
|
309 |
|
310 |
Configuration information (lib/pkgconfig):
|
311 |
libpcre.pc
|
312 |
libpcrecpp.pc (if C++ support is enabled)
|
313 |
|
314 |
Header files (include):
|
315 |
pcre.h
|
316 |
pcreposix.h
|
317 |
pcre_scanner.h )
|
318 |
pcre_stringpiece.h ) if C++ support is enabled
|
319 |
pcrecpp.h )
|
320 |
pcrecpparg.h )
|
321 |
|
322 |
Man pages (share/man/man{1,3}):
|
323 |
pcregrep.1
|
324 |
pcretest.1
|
325 |
pcre.3
|
326 |
pcre*.3 (lots more pages, all starting "pcre")
|
327 |
|
328 |
HTML documentation (share/doc/pcre/html):
|
329 |
index.html
|
330 |
*.html (lots more pages, hyperlinked from index.html)
|
331 |
|
332 |
Text file documentation (share/doc/pcre):
|
333 |
AUTHORS
|
334 |
COPYING
|
335 |
ChangeLog
|
336 |
LICENCE
|
337 |
NEWS
|
338 |
README
|
339 |
pcre.txt (a concatenation of the man(3) pages)
|
340 |
pcretest.txt the pcretest man page
|
341 |
pcregrep.txt the pcregrep man page
|
342 |
|
343 |
Note that the pcredemo program that is built by "configure" is *not* installed
|
344 |
anywhere. It is a demonstration for programmers wanting to use PCRE.
|
345 |
|
346 |
If you want to remove PCRE from your system, you can run "make uninstall".
|
347 |
This removes all the files that "make install" installed. However, it does not
|
348 |
remove any directories, because these are often shared with other programs.
|
349 |
|
350 |
|
351 |
Retrieving configuration information on Unix-like systems
|
352 |
---------------------------------------------------------
|
353 |
|
354 |
Running "make install" installs the command pcre-config, which can be used to
|
355 |
recall information about the PCRE configuration and installation. For example:
|
356 |
|
357 |
pcre-config --version
|
358 |
|
359 |
prints the version number, and
|
360 |
|
361 |
pcre-config --libs
|
362 |
|
363 |
outputs information about where the library is installed. This command can be
|
364 |
included in makefiles for programs that use PCRE, saving the programmer from
|
365 |
having to remember too many details.
|
366 |
|
367 |
The pkg-config command is another system for saving and retrieving information
|
368 |
about installed libraries. Instead of separate commands for each library, a
|
369 |
single command is used. For example:
|
370 |
|
371 |
pkg-config --cflags pcre
|
372 |
|
373 |
The data is held in *.pc files that are installed in a directory called
|
374 |
<prefix>/lib/pkgconfig.
|
375 |
|
376 |
|
377 |
Shared libraries on Unix-like systems
|
378 |
-------------------------------------
|
379 |
|
380 |
The default distribution builds PCRE as shared libraries and static libraries,
|
381 |
as long as the operating system supports shared libraries. Shared library
|
382 |
support relies on the "libtool" script which is built as part of the
|
383 |
"configure" process.
|
384 |
|
385 |
The libtool script is used to compile and link both shared and static
|
386 |
libraries. They are placed in a subdirectory called .libs when they are newly
|
387 |
built. The programs pcretest and pcregrep are built to use these uninstalled
|
388 |
libraries (by means of wrapper scripts in the case of shared libraries). When
|
389 |
you use "make install" to install shared libraries, pcregrep and pcretest are
|
390 |
automatically re-built to use the newly installed shared libraries before being
|
391 |
installed themselves. However, the versions left in the build directory still
|
392 |
use the uninstalled libraries.
|
393 |
|
394 |
To build PCRE using static libraries only you must use --disable-shared when
|
395 |
configuring it. For example:
|
396 |
|
397 |
./configure --prefix=/usr/gnu --disable-shared
|
398 |
|
399 |
Then run "make" in the usual way. Similarly, you can use --disable-static to
|
400 |
build only shared libraries.
|
401 |
|
402 |
|
403 |
Cross-compiling on Unix-like systems
|
404 |
------------------------------------
|
405 |
|
406 |
You can specify CC and CFLAGS in the normal way to the "configure" command, in
|
407 |
order to cross-compile PCRE for some other host. However, you should NOT
|
408 |
specify --enable-rebuild-chartables, because if you do, the dftables.c source
|
409 |
file is compiled and run on the local host, in order to generate the inbuilt
|
410 |
character tables (the pcre_chartables.c file). This will probably not work,
|
411 |
because dftables.c needs to be compiled with the local compiler, not the cross
|
412 |
compiler.
|
413 |
|
414 |
When --enable-rebuild-chartables is not specified, pcre_chartables.c is created
|
415 |
by making a copy of pcre_chartables.c.dist, which is a default set of tables
|
416 |
that assumes ASCII code. Cross-compiling with the default tables should not be
|
417 |
a problem.
|
418 |
|
419 |
If you need to modify the character tables when cross-compiling, you should
|
420 |
move pcre_chartables.c.dist out of the way, then compile dftables.c by hand and
|
421 |
run it on the local host to make a new version of pcre_chartables.c.dist.
|
422 |
Then when you cross-compile PCRE this new version of the tables will be used.
|
423 |
|
424 |
|
425 |
Using HP's ANSI C++ compiler (aCC)
|
426 |
----------------------------------
|
427 |
|
428 |
Unless C++ support is disabled by specifying the "--disable-cpp" option of the
|
429 |
"configure" script, you must include the "-AA" option in the CXXFLAGS
|
430 |
environment variable in order for the C++ components to compile correctly.
|
431 |
|
432 |
Also, note that the aCC compiler on PA-RISC platforms may have a defect whereby
|
433 |
needed libraries fail to get included when specifying the "-AA" compiler
|
434 |
option. If you experience unresolved symbols when linking the C++ programs,
|
435 |
use the workaround of specifying the following environment variable prior to
|
436 |
running the "configure" script:
|
437 |
|
438 |
CXXLDFLAGS="-lstd_v2 -lCsup_v2"
|
439 |
|
440 |
|
441 |
Making new tarballs
|
442 |
-------------------
|
443 |
|
444 |
The command "make dist" creates three PCRE tarballs, in tar.gz, tar.bz2, and
|
445 |
zip formats. The command "make distcheck" does the same, but then does a trial
|
446 |
build of the new distribution to ensure that it works.
|
447 |
|
448 |
If you have modified any of the man page sources in the doc directory, you
|
449 |
should first run the PrepareRelease script before making a distribution. This
|
450 |
script creates the .txt and HTML forms of the documentation from the man pages.
|
451 |
|
452 |
|
453 |
Testing PCRE
|
454 |
------------
|
455 |
|
456 |
To test the basic PCRE library on a Unix system, run the RunTest script that is
|
457 |
created by the configuring process. There is also a script called RunGrepTest
|
458 |
that tests the options of the pcregrep command. If the C++ wrapper library is
|
459 |
built, three test programs called pcrecpp_unittest, pcre_scanner_unittest, and
|
460 |
pcre_stringpiece_unittest are also built.
|
461 |
|
462 |
Both the scripts and all the program tests are run if you obey "make check" or
|
463 |
"make test". For other systems, see the instructions in NON-UNIX-USE.
|
464 |
|
465 |
The RunTest script runs the pcretest test program (which is documented in its
|
466 |
own man page) on each of the testinput files in the testdata directory in
|
467 |
turn, and compares the output with the contents of the corresponding testoutput
|
468 |
files. A file called testtry is used to hold the main output from pcretest
|
469 |
(testsavedregex is also used as a working file). To run pcretest on just one of
|
470 |
the test files, give its number as an argument to RunTest, for example:
|
471 |
|
472 |
RunTest 2
|
473 |
|
474 |
The first test file can also be fed directly into the perltest.pl script to
|
475 |
check that Perl gives the same results. The only difference you should see is
|
476 |
in the first few lines, where the Perl version is given instead of the PCRE
|
477 |
version.
|
478 |
|
479 |
The second set of tests check pcre_fullinfo(), pcre_info(), pcre_study(),
|
480 |
pcre_copy_substring(), pcre_get_substring(), pcre_get_substring_list(), error
|
481 |
detection, and run-time flags that are specific to PCRE, as well as the POSIX
|
482 |
wrapper API. It also uses the debugging flags to check some of the internals of
|
483 |
pcre_compile().
|
484 |
|
485 |
If you build PCRE with a locale setting that is not the standard C locale, the
|
486 |
character tables may be different (see next paragraph). In some cases, this may
|
487 |
cause failures in the second set of tests. For example, in a locale where the
|
488 |
isprint() function yields TRUE for characters in the range 128-255, the use of
|
489 |
[:isascii:] inside a character class defines a different set of characters, and
|
490 |
this shows up in this test as a difference in the compiled code, which is being
|
491 |
listed for checking. Where the comparison test output contains [\x00-\x7f] the
|
492 |
test will contain [\x00-\xff], and similarly in some other cases. This is not a
|
493 |
bug in PCRE.
|
494 |
|
495 |
The third set of tests checks pcre_maketables(), the facility for building a
|
496 |
set of character tables for a specific locale and using them instead of the
|
497 |
default tables. The tests make use of the "fr_FR" (French) locale. Before
|
498 |
running the test, the script checks for the presence of this locale by running
|
499 |
the "locale" command. If that command fails, or if it doesn't include "fr_FR"
|
500 |
in the list of available locales, the third test cannot be run, and a comment
|
501 |
is output to say why. If running this test produces instances of the error
|
502 |
|
503 |
** Failed to set locale "fr_FR"
|
504 |
|
505 |
in the comparison output, it means that locale is not available on your system,
|
506 |
despite being listed by "locale". This does not mean that PCRE is broken.
|
507 |
|
508 |
[If you are trying to run this test on Windows, you may be able to get it to
|
509 |
work by changing "fr_FR" to "french" everywhere it occurs.]
|
510 |
|
511 |
The fourth test checks the UTF-8 support. It is not run automatically unless
|
512 |
PCRE is built with UTF-8 support. To do this you must set --enable-utf8 when
|
513 |
running "configure". This file can be also fed directly to the perltest script,
|
514 |
provided you are running Perl 5.8 or higher. (For Perl 5.6, a small patch,
|
515 |
commented in the script, can be be used.)
|
516 |
|
517 |
The fifth test checks error handling with UTF-8 encoding, and internal UTF-8
|
518 |
features of PCRE that are not relevant to Perl.
|
519 |
|
520 |
The sixth test checks the support for Unicode character properties. It it not
|
521 |
run automatically unless PCRE is built with Unicode property support. To to
|
522 |
this you must set --enable-unicode-properties when running "configure".
|
523 |
|
524 |
The seventh, eighth, and ninth tests check the pcre_dfa_exec() alternative
|
525 |
matching function, in non-UTF-8 mode, UTF-8 mode, and UTF-8 mode with Unicode
|
526 |
property support, respectively. The eighth and ninth tests are not run
|
527 |
automatically unless PCRE is build with the relevant support.
|
528 |
|
529 |
|
530 |
Character tables
|
531 |
----------------
|
532 |
|
533 |
For speed, PCRE uses four tables for manipulating and identifying characters
|
534 |
whose code point values are less than 256. The final argument of the
|
535 |
pcre_compile() function is a pointer to a block of memory containing the
|
536 |
concatenated tables. A call to pcre_maketables() can be used to generate a set
|
537 |
of tables in the current locale. If the final argument for pcre_compile() is
|
538 |
passed as NULL, a set of default tables that is built into the binary is used.
|
539 |
|
540 |
The source file called pcre_chartables.c contains the default set of tables. By
|
541 |
default, this is created as a copy of pcre_chartables.c.dist, which contains
|
542 |
tables for ASCII coding. However, if --enable-rebuild-chartables is specified
|
543 |
for ./configure, a different version of pcre_chartables.c is built by the
|
544 |
program dftables (compiled from dftables.c), which uses the ANSI C character
|
545 |
handling functions such as isalnum(), isalpha(), isupper(), islower(), etc. to
|
546 |
build the table sources. This means that the default C locale which is set for
|
547 |
your system will control the contents of these default tables. You can change
|
548 |
the default tables by editing pcre_chartables.c and then re-building PCRE. If
|
549 |
you do this, you should take care to ensure that the file does not get
|
550 |
automatically re-generated. The best way to do this is to move
|
551 |
pcre_chartables.c.dist out of the way and replace it with your customized
|
552 |
tables.
|
553 |
|
554 |
When the dftables program is run as a result of --enable-rebuild-chartables,
|
555 |
it uses the default C locale that is set on your system. It does not pay
|
556 |
attention to the LC_xxx environment variables. In other words, it uses the
|
557 |
system's default locale rather than whatever the compiling user happens to have
|
558 |
set. If you really do want to build a source set of character tables in a
|
559 |
locale that is specified by the LC_xxx variables, you can run the dftables
|
560 |
program by hand with the -L option. For example:
|
561 |
|
562 |
./dftables -L pcre_chartables.c.special
|
563 |
|
564 |
The first two 256-byte tables provide lower casing and case flipping functions,
|
565 |
respectively. The next table consists of three 32-byte bit maps which identify
|
566 |
digits, "word" characters, and white space, respectively. These are used when
|
567 |
building 32-byte bit maps that represent character classes for code points less
|
568 |
than 256.
|
569 |
|
570 |
The final 256-byte table has bits indicating various character types, as
|
571 |
follows:
|
572 |
|
573 |
1 white space character
|
574 |
2 letter
|
575 |
4 decimal digit
|
576 |
8 hexadecimal digit
|
577 |
16 alphanumeric or '_'
|
578 |
128 regular expression metacharacter or binary zero
|
579 |
|
580 |
You should not alter the set of characters that contain the 128 bit, as that
|
581 |
will cause PCRE to malfunction.
|
582 |
|
583 |
|
584 |
File manifest
|
585 |
-------------
|
586 |
|
587 |
The distribution should contain the following files:
|
588 |
|
589 |
(A) Source files of the PCRE library functions and their headers:
|
590 |
|
591 |
dftables.c auxiliary program for building pcre_chartables.c
|
592 |
when --enable-rebuild-chartables is specified
|
593 |
|
594 |
pcre_chartables.c.dist a default set of character tables that assume ASCII
|
595 |
coding; used, unless --enable-rebuild-chartables is
|
596 |
specified, by copying to pcre_chartables.c
|
597 |
|
598 |
pcreposix.c )
|
599 |
pcre_compile.c )
|
600 |
pcre_config.c )
|
601 |
pcre_dfa_exec.c )
|
602 |
pcre_exec.c )
|
603 |
pcre_fullinfo.c )
|
604 |
pcre_get.c ) sources for the functions in the library,
|
605 |
pcre_globals.c ) and some internal functions that they use
|
606 |
pcre_info.c )
|
607 |
pcre_maketables.c )
|
608 |
pcre_newline.c )
|
609 |
pcre_ord2utf8.c )
|
610 |
pcre_refcount.c )
|
611 |
pcre_study.c )
|
612 |
pcre_tables.c )
|
613 |
pcre_try_flipped.c )
|
614 |
pcre_ucp_searchfuncs.c )
|
615 |
pcre_valid_utf8.c )
|
616 |
pcre_version.c )
|
617 |
pcre_xclass.c )
|
618 |
pcre_printint.src ) debugging function that is #included in pcretest,
|
619 |
) and can also be #included in pcre_compile()
|
620 |
pcre.h.in template for pcre.h when built by "configure"
|
621 |
pcreposix.h header for the external POSIX wrapper API
|
622 |
pcre_internal.h header for internal use
|
623 |
ucp.h ) headers concerned with
|
624 |
ucpinternal.h ) Unicode property handling
|
625 |
ucptable.h ) (this one is the data table)
|
626 |
|
627 |
config.h.in template for config.h, which is built by "configure"
|
628 |
|
629 |
pcrecpp.h public header file for the C++ wrapper
|
630 |
pcrecpparg.h.in template for another C++ header file
|
631 |
pcre_scanner.h public header file for C++ scanner functions
|
632 |
pcrecpp.cc )
|
633 |
pcre_scanner.cc ) source for the C++ wrapper library
|
634 |
|
635 |
pcre_stringpiece.h.in template for pcre_stringpiece.h, the header for the
|
636 |
C++ stringpiece functions
|
637 |
pcre_stringpiece.cc source for the C++ stringpiece functions
|
638 |
|
639 |
(B) Source files for programs that use PCRE:
|
640 |
|
641 |
pcredemo.c simple demonstration of coding calls to PCRE
|
642 |
pcregrep.c source of a grep utility that uses PCRE
|
643 |
pcretest.c comprehensive test program
|
644 |
|
645 |
(C) Auxiliary files:
|
646 |
|
647 |
132html script to turn "man" pages into HTML
|
648 |
AUTHORS information about the author of PCRE
|
649 |
ChangeLog log of changes to the code
|
650 |
CleanTxt script to clean nroff output for txt man pages
|
651 |
Detrail script to remove trailing spaces
|
652 |
HACKING some notes about the internals of PCRE
|
653 |
INSTALL generic installation instructions
|
654 |
LICENCE conditions for the use of PCRE
|
655 |
COPYING the same, using GNU's standard name
|
656 |
Makefile.in ) template for Unix Makefile, which is built by
|
657 |
) "configure"
|
658 |
Makefile.am ) the automake input that was used to create
|
659 |
) Makefile.in
|
660 |
NEWS important changes in this release
|
661 |
NON-UNIX-USE notes on building PCRE on non-Unix systems
|
662 |
PrepareRelease script to make preparations for "make dist"
|
663 |
README this file
|
664 |
RunTest a Unix shell script for running tests
|
665 |
RunGrepTest a Unix shell script for pcregrep tests
|
666 |
aclocal.m4 m4 macros (generated by "aclocal")
|
667 |
config.guess ) files used by libtool,
|
668 |
config.sub ) used only when building a shared library
|
669 |
configure a configuring shell script (built by autoconf)
|
670 |
configure.ac ) the autoconf input that was used to build
|
671 |
) "configure" and config.h
|
672 |
depcomp ) script to find program dependencies, generated by
|
673 |
) automake
|
674 |
doc/*.3 man page sources for the PCRE functions
|
675 |
doc/*.1 man page sources for pcregrep and pcretest
|
676 |
doc/index.html.src the base HTML page
|
677 |
doc/html/* HTML documentation
|
678 |
doc/pcre.txt plain text version of the man pages
|
679 |
doc/pcretest.txt plain text documentation of test program
|
680 |
doc/perltest.txt plain text documentation of Perl test program
|
681 |
install-sh a shell script for installing files
|
682 |
libpcre.pc.in template for libpcre.pc for pkg-config
|
683 |
libpcrecpp.pc.in template for libpcrecpp.pc for pkg-config
|
684 |
ltmain.sh file used to build a libtool script
|
685 |
missing ) common stub for a few missing GNU programs while
|
686 |
) installing, generated by automake
|
687 |
mkinstalldirs script for making install directories
|
688 |
perltest.pl Perl test program
|
689 |
pcre-config.in source of script which retains PCRE information
|
690 |
pcrecpp_unittest.cc )
|
691 |
pcre_scanner_unittest.cc ) test programs for the C++ wrapper
|
692 |
pcre_stringpiece_unittest.cc )
|
693 |
testdata/testinput* test data for main library tests
|
694 |
testdata/testoutput* expected test results
|
695 |
testdata/grep* input and output for pcregrep tests
|
696 |
|
697 |
(D) Auxiliary files for cmake support
|
698 |
|
699 |
CMakeLists.txt
|
700 |
config-cmake.h.in
|
701 |
|
702 |
(E) Auxiliary files for VPASCAL
|
703 |
|
704 |
makevp.bat
|
705 |
makevp_c.txt
|
706 |
makevp_l.txt
|
707 |
pcregexp.pas
|
708 |
|
709 |
(F) Auxiliary files for building PCRE "by hand"
|
710 |
|
711 |
pcre.h.generic ) a version of the public PCRE header file
|
712 |
) for use in non-"configure" environments
|
713 |
config.h.generic ) a version of config.h for use in non-"configure"
|
714 |
) environments
|
715 |
|
716 |
(F) Miscellaneous
|
717 |
|
718 |
RunTest.bat a script for running tests under Windows
|
719 |
|
720 |
Philip Hazel
|
721 |
Email local part: ph10
|
722 |
Email domain: cam.ac.uk
|
723 |
Last updated: 17 September 2007
|