1 |
.TH PCRE 3
|
2 |
.SH NAME
|
3 |
pcre - Perl-compatible regular expressions.
|
4 |
.SH SYNOPSIS
|
5 |
.B #include <pcre.h>
|
6 |
.PP
|
7 |
.SM
|
8 |
.br
|
9 |
.B pcre *pcre_compile(const char *\fIpattern\fR, int \fIoptions\fR,
|
10 |
.ti +5n
|
11 |
.B const char **\fIerrptr\fR, int *\fIerroffset\fR,
|
12 |
.ti +5n
|
13 |
.B const unsigned char *\fItableptr\fR);
|
14 |
.PP
|
15 |
.br
|
16 |
.B pcre_extra *pcre_study(const pcre *\fIcode\fR, int \fIoptions\fR,
|
17 |
.ti +5n
|
18 |
.B const char **\fIerrptr\fR);
|
19 |
.PP
|
20 |
.br
|
21 |
.B int pcre_exec(const pcre *\fIcode\fR, "const pcre_extra *\fIextra\fR,"
|
22 |
.ti +5n
|
23 |
.B "const char *\fIsubject\fR," int \fIlength\fR, int \fIstartoffset\fR,
|
24 |
.ti +5n
|
25 |
.B int \fIoptions\fR, int *\fIovector\fR, int \fIovecsize\fR);
|
26 |
.PP
|
27 |
.br
|
28 |
.B int pcre_copy_substring(const char *\fIsubject\fR, int *\fIovector\fR,
|
29 |
.ti +5n
|
30 |
.B int \fIstringcount\fR, int \fIstringnumber\fR, char *\fIbuffer\fR,
|
31 |
.ti +5n
|
32 |
.B int \fIbuffersize\fR);
|
33 |
.PP
|
34 |
.br
|
35 |
.B int pcre_get_substring(const char *\fIsubject\fR, int *\fIovector\fR,
|
36 |
.ti +5n
|
37 |
.B int \fIstringcount\fR, int \fIstringnumber\fR,
|
38 |
.ti +5n
|
39 |
.B const char **\fIstringptr\fR);
|
40 |
.PP
|
41 |
.br
|
42 |
.B int pcre_get_substring_list(const char *\fIsubject\fR,
|
43 |
.ti +5n
|
44 |
.B int *\fIovector\fR, int \fIstringcount\fR, "const char ***\fIlistptr\fR);"
|
45 |
.PP
|
46 |
.br
|
47 |
.B const unsigned char *pcre_maketables(void);
|
48 |
.PP
|
49 |
.br
|
50 |
.B int pcre_fullinfo(const pcre *\fIcode\fR, "const pcre_extra *\fIextra\fR,"
|
51 |
.ti +5n
|
52 |
.B int \fIwhat\fR, void *\fIwhere\fR);
|
53 |
.PP
|
54 |
.br
|
55 |
.B int pcre_info(const pcre *\fIcode\fR, int *\fIoptptr\fR, int
|
56 |
.B *\fIfirstcharptr\fR);
|
57 |
.PP
|
58 |
.br
|
59 |
.B char *pcre_version(void);
|
60 |
.PP
|
61 |
.br
|
62 |
.B void *(*pcre_malloc)(size_t);
|
63 |
.PP
|
64 |
.br
|
65 |
.B void (*pcre_free)(void *);
|
66 |
|
67 |
|
68 |
|
69 |
.SH DESCRIPTION
|
70 |
The PCRE library is a set of functions that implement regular expression
|
71 |
pattern matching using the same syntax and semantics as Perl 5, with just a few
|
72 |
differences (see below). The current implementation corresponds to Perl 5.005,
|
73 |
with some additional features from the Perl development release.
|
74 |
|
75 |
PCRE has its own native API, which is described in this document. There is also
|
76 |
a set of wrapper functions that correspond to the POSIX regular expression API.
|
77 |
These are described in the \fBpcreposix\fR documentation.
|
78 |
|
79 |
The native API function prototypes are defined in the header file \fBpcre.h\fR,
|
80 |
and on Unix systems the library itself is called \fBlibpcre.a\fR, so can be
|
81 |
accessed by adding \fB-lpcre\fR to the command for linking an application which
|
82 |
calls it. The header file defines the macros PCRE_MAJOR and PCRE_MINOR to
|
83 |
contain the major and minor release numbers for the library. Applications can
|
84 |
use these to include support for different releases.
|
85 |
|
86 |
The functions \fBpcre_compile()\fR, \fBpcre_study()\fR, and \fBpcre_exec()\fR
|
87 |
are used for compiling and matching regular expressions, while
|
88 |
\fBpcre_copy_substring()\fR, \fBpcre_get_substring()\fR, and
|
89 |
\fBpcre_get_substring_list()\fR are convenience functions for extracting
|
90 |
captured substrings from a matched subject string. The function
|
91 |
\fBpcre_maketables()\fR is used (optionally) to build a set of character tables
|
92 |
in the current locale for passing to \fBpcre_compile()\fR.
|
93 |
|
94 |
The function \fBpcre_fullinfo()\fR is used to find out information about a
|
95 |
compiled pattern; \fBpcre_info()\fR is an obsolete version which returns only
|
96 |
some of the available information, but is retained for backwards compatibility.
|
97 |
The function \fBpcre_version()\fR returns a pointer to a string containing the
|
98 |
version of PCRE and its date of release.
|
99 |
|
100 |
The global variables \fBpcre_malloc\fR and \fBpcre_free\fR initially contain
|
101 |
the entry points of the standard \fBmalloc()\fR and \fBfree()\fR functions
|
102 |
respectively. PCRE calls the memory management functions via these variables,
|
103 |
so a calling program can replace them if it wishes to intercept the calls. This
|
104 |
should be done before calling any PCRE functions.
|
105 |
|
106 |
|
107 |
.SH MULTI-THREADING
|
108 |
The PCRE functions can be used in multi-threading applications, with the
|
109 |
proviso that the memory management functions pointed to by \fBpcre_malloc\fR
|
110 |
and \fBpcre_free\fR are shared by all threads.
|
111 |
|
112 |
The compiled form of a regular expression is not altered during matching, so
|
113 |
the same compiled pattern can safely be used by several threads at once.
|
114 |
|
115 |
|
116 |
.SH COMPILING A PATTERN
|
117 |
The function \fBpcre_compile()\fR is called to compile a pattern into an
|
118 |
internal form. The pattern is a C string terminated by a binary zero, and
|
119 |
is passed in the argument \fIpattern\fR. A pointer to a single block of memory
|
120 |
that is obtained via \fBpcre_malloc\fR is returned. This contains the
|
121 |
compiled code and related data. The \fBpcre\fR type is defined for this for
|
122 |
convenience, but in fact \fBpcre\fR is just a typedef for \fBvoid\fR, since the
|
123 |
contents of the block are not externally defined. It is up to the caller to
|
124 |
free the memory when it is no longer required.
|
125 |
.PP
|
126 |
The size of a compiled pattern is roughly proportional to the length of the
|
127 |
pattern string, except that each character class (other than those containing
|
128 |
just a single character, negated or not) requires 33 bytes, and repeat
|
129 |
quantifiers with a minimum greater than one or a bounded maximum cause the
|
130 |
relevant portions of the compiled pattern to be replicated.
|
131 |
.PP
|
132 |
The \fIoptions\fR argument contains independent bits that affect the
|
133 |
compilation. It should be zero if no options are required. Some of the options,
|
134 |
in particular, those that are compatible with Perl, can also be set and unset
|
135 |
from within the pattern (see the detailed description of regular expressions
|
136 |
below). For these options, the contents of the \fIoptions\fR argument specifies
|
137 |
their initial settings at the start of compilation and execution. The
|
138 |
PCRE_ANCHORED option can be set at the time of matching as well as at compile
|
139 |
time.
|
140 |
.PP
|
141 |
If \fIerrptr\fR is NULL, \fBpcre_compile()\fR returns NULL immediately.
|
142 |
Otherwise, if compilation of a pattern fails, \fBpcre_compile()\fR returns
|
143 |
NULL, and sets the variable pointed to by \fIerrptr\fR to point to a textual
|
144 |
error message. The offset from the start of the pattern to the character where
|
145 |
the error was discovered is placed in the variable pointed to by
|
146 |
\fIerroffset\fR, which must not be NULL. If it is, an immediate error is given.
|
147 |
.PP
|
148 |
If the final argument, \fItableptr\fR, is NULL, PCRE uses a default set of
|
149 |
character tables which are built when it is compiled, using the default C
|
150 |
locale. Otherwise, \fItableptr\fR must be the result of a call to
|
151 |
\fBpcre_maketables()\fR. See the section on locale support below.
|
152 |
.PP
|
153 |
The following option bits are defined in the header file:
|
154 |
|
155 |
PCRE_ANCHORED
|
156 |
|
157 |
If this bit is set, the pattern is forced to be "anchored", that is, it is
|
158 |
constrained to match only at the start of the string which is being searched
|
159 |
(the "subject string"). This effect can also be achieved by appropriate
|
160 |
constructs in the pattern itself, which is the only way to do it in Perl.
|
161 |
|
162 |
PCRE_CASELESS
|
163 |
|
164 |
If this bit is set, letters in the pattern match both upper and lower case
|
165 |
letters. It is equivalent to Perl's /i option.
|
166 |
|
167 |
PCRE_DOLLAR_ENDONLY
|
168 |
|
169 |
If this bit is set, a dollar metacharacter in the pattern matches only at the
|
170 |
end of the subject string. Without this option, a dollar also matches
|
171 |
immediately before the final character if it is a newline (but not before any
|
172 |
other newlines). The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is
|
173 |
set. There is no equivalent to this option in Perl.
|
174 |
|
175 |
PCRE_DOTALL
|
176 |
|
177 |
If this bit is set, a dot metacharater in the pattern matches all characters,
|
178 |
including newlines. Without it, newlines are excluded. This option is
|
179 |
equivalent to Perl's /s option. A negative class such as [^a] always matches a
|
180 |
newline character, independent of the setting of this option.
|
181 |
|
182 |
PCRE_EXTENDED
|
183 |
|
184 |
If this bit is set, whitespace data characters in the pattern are totally
|
185 |
ignored except when escaped or inside a character class, and characters between
|
186 |
an unescaped # outside a character class and the next newline character,
|
187 |
inclusive, are also ignored. This is equivalent to Perl's /x option, and makes
|
188 |
it possible to include comments inside complicated patterns. Note, however,
|
189 |
that this applies only to data characters. Whitespace characters may never
|
190 |
appear within special character sequences in a pattern, for example within the
|
191 |
sequence (?( which introduces a conditional subpattern.
|
192 |
|
193 |
PCRE_EXTRA
|
194 |
|
195 |
This option was invented in order to turn on additional functionality of PCRE
|
196 |
that is incompatible with Perl, but it is currently of very little use. When
|
197 |
set, any backslash in a pattern that is followed by a letter that has no
|
198 |
special meaning causes an error, thus reserving these combinations for future
|
199 |
expansion. By default, as in Perl, a backslash followed by a letter with no
|
200 |
special meaning is treated as a literal. There are at present no other features
|
201 |
controlled by this option. It can also be set by a (?X) option setting within a
|
202 |
pattern.
|
203 |
|
204 |
PCRE_MULTILINE
|
205 |
|
206 |
By default, PCRE treats the subject string as consisting of a single "line" of
|
207 |
characters (even if it actually contains several newlines). The "start of line"
|
208 |
metacharacter (^) matches only at the start of the string, while the "end of
|
209 |
line" metacharacter ($) matches only at the end of the string, or before a
|
210 |
terminating newline (unless PCRE_DOLLAR_ENDONLY is set). This is the same as
|
211 |
Perl.
|
212 |
|
213 |
When PCRE_MULTILINE it is set, the "start of line" and "end of line" constructs
|
214 |
match immediately following or immediately before any newline in the subject
|
215 |
string, respectively, as well as at the very start and end. This is equivalent
|
216 |
to Perl's /m option. If there are no "\\n" characters in a subject string, or
|
217 |
no occurrences of ^ or $ in a pattern, setting PCRE_MULTILINE has no
|
218 |
effect.
|
219 |
|
220 |
PCRE_UNGREEDY
|
221 |
|
222 |
This option inverts the "greediness" of the quantifiers so that they are not
|
223 |
greedy by default, but become greedy if followed by "?". It is not compatible
|
224 |
with Perl. It can also be set by a (?U) option setting within the pattern.
|
225 |
|
226 |
|
227 |
.SH STUDYING A PATTERN
|
228 |
When a pattern is going to be used several times, it is worth spending more
|
229 |
time analyzing it in order to speed up the time taken for matching. The
|
230 |
function \fBpcre_study()\fR takes a pointer to a compiled pattern as its first
|
231 |
argument, and returns a pointer to a \fBpcre_extra\fR block (another \fBvoid\fR
|
232 |
typedef) containing additional information about the pattern; this can be
|
233 |
passed to \fBpcre_exec()\fR. If no additional information is available, NULL
|
234 |
is returned.
|
235 |
|
236 |
The second argument contains option bits. At present, no options are defined
|
237 |
for \fBpcre_study()\fR, and this argument should always be zero.
|
238 |
|
239 |
The third argument for \fBpcre_study()\fR is a pointer to an error message. If
|
240 |
studying succeeds (even if no data is returned), the variable it points to is
|
241 |
set to NULL. Otherwise it points to a textual error message.
|
242 |
|
243 |
At present, studying a pattern is useful only for non-anchored patterns that do
|
244 |
not have a single fixed starting character. A bitmap of possible starting
|
245 |
characters is created.
|
246 |
|
247 |
|
248 |
.SH LOCALE SUPPORT
|
249 |
PCRE handles caseless matching, and determines whether characters are letters,
|
250 |
digits, or whatever, by reference to a set of tables. The library contains a
|
251 |
default set of tables which is created in the default C locale when PCRE is
|
252 |
compiled. This is used when the final argument of \fBpcre_compile()\fR is NULL,
|
253 |
and is sufficient for many applications.
|
254 |
|
255 |
An alternative set of tables can, however, be supplied. Such tables are built
|
256 |
by calling the \fBpcre_maketables()\fR function, which has no arguments, in the
|
257 |
relevant locale. The result can then be passed to \fBpcre_compile()\fR as often
|
258 |
as necessary. For example, to build and use tables that are appropriate for the
|
259 |
French locale (where accented characters with codes greater than 128 are
|
260 |
treated as letters), the following code could be used:
|
261 |
|
262 |
setlocale(LC_CTYPE, "fr");
|
263 |
tables = pcre_maketables();
|
264 |
re = pcre_compile(..., tables);
|
265 |
|
266 |
The tables are built in memory that is obtained via \fBpcre_malloc\fR. The
|
267 |
pointer that is passed to \fBpcre_compile\fR is saved with the compiled
|
268 |
pattern, and the same tables are used via this pointer by \fBpcre_study()\fR
|
269 |
and \fBpcre_exec()\fR. Thus for any single pattern, compilation, studying and
|
270 |
matching all happen in the same locale, but different patterns can be compiled
|
271 |
in different locales. It is the caller's responsibility to ensure that the
|
272 |
memory containing the tables remains available for as long as it is needed.
|
273 |
|
274 |
|
275 |
.SH INFORMATION ABOUT A PATTERN
|
276 |
The \fBpcre_fullinfo()\fR function returns information about a compiled
|
277 |
pattern. It replaces the obsolete \fBpcre_info()\fR function, which is
|
278 |
nevertheless retained for backwards compability (and is documented below).
|
279 |
|
280 |
The first argument for \fBpcre_fullinfo()\fR is a pointer to the compiled
|
281 |
pattern. The second argument is the result of \fBpcre_study()\fR, or NULL if
|
282 |
the pattern was not studied. The third argument specifies which piece of
|
283 |
information is required, while the fourth argument is a pointer to a variable
|
284 |
to receive the data. The yield of the function is zero for success, or one of
|
285 |
the following negative numbers:
|
286 |
|
287 |
PCRE_ERROR_NULL the argument \fIcode\fR was NULL
|
288 |
the argument \fIwhere\fR was NULL
|
289 |
PCRE_ERROR_BADMAGIC the "magic number" was not found
|
290 |
PCRE_ERROR_BADOPTION the value of \fIwhat\fR was invalid
|
291 |
|
292 |
The possible values for the third argument are defined in \fBpcre.h\fR, and are
|
293 |
as follows:
|
294 |
|
295 |
PCRE_INFO_OPTIONS
|
296 |
|
297 |
Return a copy of the options with which the pattern was compiled. The fourth
|
298 |
argument should point to au \fBunsigned long int\fR variable. These option bits
|
299 |
are those specified in the call to \fBpcre_compile()\fR, modified by any
|
300 |
top-level option settings within the pattern itself, and with the PCRE_ANCHORED
|
301 |
bit forcibly set if the form of the pattern implies that it can match only at
|
302 |
the start of a subject string.
|
303 |
|
304 |
PCRE_INFO_SIZE
|
305 |
|
306 |
Return the size of the compiled pattern, that is, the value that was passed as
|
307 |
the argument to \fBpcre_malloc()\fR when PCRE was getting memory in which to
|
308 |
place the compiled data. The fourth argument should point to a \fBsize_t\fR
|
309 |
variable.
|
310 |
|
311 |
PCRE_INFO_CAPTURECOUNT
|
312 |
|
313 |
Return the number of capturing subpatterns in the pattern. The fourth argument
|
314 |
should point to an \fbint\fR variable.
|
315 |
|
316 |
PCRE_INFO_BACKREFMAX
|
317 |
|
318 |
Return the number of the highest back reference in the pattern. The fourth
|
319 |
argument should point to an \fBint\fR variable. Zero is returned if there are
|
320 |
no back references.
|
321 |
|
322 |
PCRE_INFO_FIRSTCHAR
|
323 |
|
324 |
Return information about the first character of any matched string, for a
|
325 |
non-anchored pattern. If there is a fixed first character, e.g. from a pattern
|
326 |
such as (cat|cow|coyote), it is returned in the integer pointed to by
|
327 |
\fIwhere\fR. Otherwise, if either
|
328 |
|
329 |
(a) the pattern was compiled with the PCRE_MULTILINE option, and every branch
|
330 |
starts with "^", or
|
331 |
|
332 |
(b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set
|
333 |
(if it were set, the pattern would be anchored),
|
334 |
|
335 |
-1 is returned, indicating that the pattern matches only at the start of a
|
336 |
subject string or after any "\\n" within the string. Otherwise -2 is returned.
|
337 |
For anchored patterns, -2 is returned.
|
338 |
|
339 |
PCRE_INFO_FIRSTTABLE
|
340 |
|
341 |
If the pattern was studied, and this resulted in the construction of a 256-bit
|
342 |
table indicating a fixed set of characters for the first character in any
|
343 |
matching string, a pointer to the table is returned. Otherwise NULL is
|
344 |
returned. The fourth argument should point to an \fBunsigned char *\fR
|
345 |
variable.
|
346 |
|
347 |
PCRE_INFO_LASTLITERAL
|
348 |
|
349 |
For a non-anchored pattern, return the value of the rightmost literal character
|
350 |
which must exist in any matched string, other than at its start. The fourth
|
351 |
argument should point to an \fBint\fR variable. If there is no such character,
|
352 |
or if the pattern is anchored, -1 is returned. For example, for the pattern
|
353 |
/a\\d+z\\d+/ the returned value is 'z'.
|
354 |
|
355 |
The \fBpcre_info()\fR function is now obsolete because its interface is too
|
356 |
restrictive to return all the available data about a compiled pattern. New
|
357 |
programs should use \fBpcre_fullinfo()\fR instead. The yield of
|
358 |
\fBpcre_info()\fR is the number of capturing subpatterns, or one of the
|
359 |
following negative numbers:
|
360 |
|
361 |
PCRE_ERROR_NULL the argument \fIcode\fR was NULL
|
362 |
PCRE_ERROR_BADMAGIC the "magic number" was not found
|
363 |
|
364 |
If the \fIoptptr\fR argument is not NULL, a copy of the options with which the
|
365 |
pattern was compiled is placed in the integer it points to (see
|
366 |
PCRE_INFO_OPTIONS above).
|
367 |
|
368 |
If the pattern is not anchored and the \fIfirstcharptr\fR argument is not NULL,
|
369 |
it is used to pass back information about the first character of any matched
|
370 |
string (see PCRE_INFO_FIRSTCHAR above).
|
371 |
|
372 |
|
373 |
.SH MATCHING A PATTERN
|
374 |
The function \fBpcre_exec()\fR is called to match a subject string against a
|
375 |
pre-compiled pattern, which is passed in the \fIcode\fR argument. If the
|
376 |
pattern has been studied, the result of the study should be passed in the
|
377 |
\fIextra\fR argument. Otherwise this must be NULL.
|
378 |
|
379 |
The PCRE_ANCHORED option can be passed in the \fIoptions\fR argument, whose
|
380 |
unused bits must be zero. However, if a pattern was compiled with
|
381 |
PCRE_ANCHORED, or turned out to be anchored by virtue of its contents, it
|
382 |
cannot be made unachored at matching time.
|
383 |
|
384 |
There are also three further options that can be set only at matching time:
|
385 |
|
386 |
PCRE_NOTBOL
|
387 |
|
388 |
The first character of the string is not the beginning of a line, so the
|
389 |
circumflex metacharacter should not match before it. Setting this without
|
390 |
PCRE_MULTILINE (at compile time) causes circumflex never to match.
|
391 |
|
392 |
PCRE_NOTEOL
|
393 |
|
394 |
The end of the string is not the end of a line, so the dollar metacharacter
|
395 |
should not match it nor (except in multiline mode) a newline immediately before
|
396 |
it. Setting this without PCRE_MULTILINE (at compile time) causes dollar never
|
397 |
to match.
|
398 |
|
399 |
PCRE_NOTEMPTY
|
400 |
|
401 |
An empty string is not considered to be a valid match if this option is set. If
|
402 |
there are alternatives in the pattern, they are tried. If all the alternatives
|
403 |
match the empty string, the entire match fails. For example, if the pattern
|
404 |
|
405 |
a?b?
|
406 |
|
407 |
is applied to a string not beginning with "a" or "b", it matches the empty
|
408 |
string at the start of the subject. With PCRE_NOTEMPTY set, this match is not
|
409 |
valid, so PCRE searches further into the string for occurrences of "a" or "b".
|
410 |
|
411 |
Perl has no direct equivalent of PCRE_NOTEMPTY, but it does make a special case
|
412 |
of a pattern match of the empty string within its \fBsplit()\fR function, and
|
413 |
when using the /g modifier. It is possible to emulate Perl's behaviour after
|
414 |
matching a null string by first trying the match again at the same offset with
|
415 |
PCRE_NOTEMPTY set, and then if that fails by advancing the starting offset (see
|
416 |
below) and trying an ordinary match again.
|
417 |
|
418 |
The subject string is passed as a pointer in \fIsubject\fR, a length in
|
419 |
\fIlength\fR, and a starting offset in \fIstartoffset\fR. Unlike the pattern
|
420 |
string, it may contain binary zero characters. When the starting offset is
|
421 |
zero, the search for a match starts at the beginning of the subject, and this
|
422 |
is by far the most common case.
|
423 |
|
424 |
A non-zero starting offset is useful when searching for another match in the
|
425 |
same subject by calling \fBpcre_exec()\fR again after a previous success.
|
426 |
Setting \fIstartoffset\fR differs from just passing over a shortened string and
|
427 |
setting PCRE_NOTBOL in the case of a pattern that begins with any kind of
|
428 |
lookbehind. For example, consider the pattern
|
429 |
|
430 |
\\Biss\\B
|
431 |
|
432 |
which finds occurrences of "iss" in the middle of words. (\\B matches only if
|
433 |
the current position in the subject is not a word boundary.) When applied to
|
434 |
the string "Mississipi" the first call to \fBpcre_exec()\fR finds the first
|
435 |
occurrence. If \fBpcre_exec()\fR is called again with just the remainder of the
|
436 |
subject, namely "issipi", it does not match, because \\B is always false at the
|
437 |
start of the subject, which is deemed to be a word boundary. However, if
|
438 |
\fBpcre_exec()\fR is passed the entire string again, but with \fIstartoffset\fR
|
439 |
set to 4, it finds the second occurrence of "iss" because it is able to look
|
440 |
behind the starting point to discover that it is preceded by a letter.
|
441 |
|
442 |
If a non-zero starting offset is passed when the pattern is anchored, one
|
443 |
attempt to match at the given offset is tried. This can only succeed if the
|
444 |
pattern does not require the match to be at the start of the subject.
|
445 |
|
446 |
In general, a pattern matches a certain portion of the subject, and in
|
447 |
addition, further substrings from the subject may be picked out by parts of the
|
448 |
pattern. Following the usage in Jeffrey Friedl's book, this is called
|
449 |
"capturing" in what follows, and the phrase "capturing subpattern" is used for
|
450 |
a fragment of a pattern that picks out a substring. PCRE supports several other
|
451 |
kinds of parenthesized subpattern that do not cause substrings to be captured.
|
452 |
|
453 |
Captured substrings are returned to the caller via a vector of integer offsets
|
454 |
whose address is passed in \fIovector\fR. The number of elements in the vector
|
455 |
is passed in \fIovecsize\fR. The first two-thirds of the vector is used to pass
|
456 |
back captured substrings, each substring using a pair of integers. The
|
457 |
remaining third of the vector is used as workspace by \fBpcre_exec()\fR while
|
458 |
matching capturing subpatterns, and is not available for passing back
|
459 |
information. The length passed in \fIovecsize\fR should always be a multiple of
|
460 |
three. If it is not, it is rounded down.
|
461 |
|
462 |
When a match has been successful, information about captured substrings is
|
463 |
returned in pairs of integers, starting at the beginning of \fIovector\fR, and
|
464 |
continuing up to two-thirds of its length at the most. The first element of a
|
465 |
pair is set to the offset of the first character in a substring, and the second
|
466 |
is set to the offset of the first character after the end of a substring. The
|
467 |
first pair, \fIovector[0]\fR and \fIovector[1]\fR, identify the portion of the
|
468 |
subject string matched by the entire pattern. The next pair is used for the
|
469 |
first capturing subpattern, and so on. The value returned by \fBpcre_exec()\fR
|
470 |
is the number of pairs that have been set. If there are no capturing
|
471 |
subpatterns, the return value from a successful match is 1, indicating that
|
472 |
just the first pair of offsets has been set.
|
473 |
|
474 |
Some convenience functions are provided for extracting the captured substrings
|
475 |
as separate strings. These are described in the following section.
|
476 |
|
477 |
It is possible for an capturing subpattern number \fIn+1\fR to match some
|
478 |
part of the subject when subpattern \fIn\fR has not been used at all. For
|
479 |
example, if the string "abc" is matched against the pattern (a|(z))(bc)
|
480 |
subpatterns 1 and 3 are matched, but 2 is not. When this happens, both offset
|
481 |
values corresponding to the unused subpattern are set to -1.
|
482 |
|
483 |
If a capturing subpattern is matched repeatedly, it is the last portion of the
|
484 |
string that it matched that gets returned.
|
485 |
|
486 |
If the vector is too small to hold all the captured substrings, it is used as
|
487 |
far as possible (up to two-thirds of its length), and the function returns a
|
488 |
value of zero. In particular, if the substring offsets are not of interest,
|
489 |
\fBpcre_exec()\fR may be called with \fIovector\fR passed as NULL and
|
490 |
\fIovecsize\fR as zero. However, if the pattern contains back references and
|
491 |
the \fIovector\fR isn't big enough to remember the related substrings, PCRE has
|
492 |
to get additional memory for use during matching. Thus it is usually advisable
|
493 |
to supply an \fIovector\fR.
|
494 |
|
495 |
Note that \fBpcre_info()\fR can be used to find out how many capturing
|
496 |
subpatterns there are in a compiled pattern. The smallest size for
|
497 |
\fIovector\fR that will allow for \fIn\fR captured substrings in addition to
|
498 |
the offsets of the substring matched by the whole pattern is (\fIn\fR+1)*3.
|
499 |
|
500 |
If \fBpcre_exec()\fR fails, it returns a negative number. The following are
|
501 |
defined in the header file:
|
502 |
|
503 |
PCRE_ERROR_NOMATCH (-1)
|
504 |
|
505 |
The subject string did not match the pattern.
|
506 |
|
507 |
PCRE_ERROR_NULL (-2)
|
508 |
|
509 |
Either \fIcode\fR or \fIsubject\fR was passed as NULL, or \fIovector\fR was
|
510 |
NULL and \fIovecsize\fR was not zero.
|
511 |
|
512 |
PCRE_ERROR_BADOPTION (-3)
|
513 |
|
514 |
An unrecognized bit was set in the \fIoptions\fR argument.
|
515 |
|
516 |
PCRE_ERROR_BADMAGIC (-4)
|
517 |
|
518 |
PCRE stores a 4-byte "magic number" at the start of the compiled code, to catch
|
519 |
the case when it is passed a junk pointer. This is the error it gives when the
|
520 |
magic number isn't present.
|
521 |
|
522 |
PCRE_ERROR_UNKNOWN_NODE (-5)
|
523 |
|
524 |
While running the pattern match, an unknown item was encountered in the
|
525 |
compiled pattern. This error could be caused by a bug in PCRE or by overwriting
|
526 |
of the compiled pattern.
|
527 |
|
528 |
PCRE_ERROR_NOMEMORY (-6)
|
529 |
|
530 |
If a pattern contains back references, but the \fIovector\fR that is passed to
|
531 |
\fBpcre_exec()\fR is not big enough to remember the referenced substrings, PCRE
|
532 |
gets a block of memory at the start of matching to use for this purpose. If the
|
533 |
call via \fBpcre_malloc()\fR fails, this error is given. The memory is freed at
|
534 |
the end of matching.
|
535 |
|
536 |
|
537 |
.SH EXTRACTING CAPTURED SUBSTRINGS
|
538 |
Captured substrings can be accessed directly by using the offsets returned by
|
539 |
\fBpcre_exec()\fR in \fIovector\fR. For convenience, the functions
|
540 |
\fBpcre_copy_substring()\fR, \fBpcre_get_substring()\fR, and
|
541 |
\fBpcre_get_substring_list()\fR are provided for extracting captured substrings
|
542 |
as new, separate, zero-terminated strings. A substring that contains a binary
|
543 |
zero is correctly extracted and has a further zero added on the end, but the
|
544 |
result does not, of course, function as a C string.
|
545 |
|
546 |
The first three arguments are the same for all three functions: \fIsubject\fR
|
547 |
is the subject string which has just been successfully matched, \fIovector\fR
|
548 |
is a pointer to the vector of integer offsets that was passed to
|
549 |
\fBpcre_exec()\fR, and \fIstringcount\fR is the number of substrings that
|
550 |
were captured by the match, including the substring that matched the entire
|
551 |
regular expression. This is the value returned by \fBpcre_exec\fR if it
|
552 |
is greater than zero. If \fBpcre_exec()\fR returned zero, indicating that it
|
553 |
ran out of space in \fIovector\fR, the value passed as \fIstringcount\fR should
|
554 |
be the size of the vector divided by three.
|
555 |
|
556 |
The functions \fBpcre_copy_substring()\fR and \fBpcre_get_substring()\fR
|
557 |
extract a single substring, whose number is given as \fIstringnumber\fR. A
|
558 |
value of zero extracts the substring that matched the entire pattern, while
|
559 |
higher values extract the captured substrings. For \fBpcre_copy_substring()\fR,
|
560 |
the string is placed in \fIbuffer\fR, whose length is given by
|
561 |
\fIbuffersize\fR, while for \fBpcre_get_substring()\fR a new block of store is
|
562 |
obtained via \fBpcre_malloc\fR, and its address is returned via
|
563 |
\fIstringptr\fR. The yield of the function is the length of the string, not
|
564 |
including the terminating zero, or one of
|
565 |
|
566 |
PCRE_ERROR_NOMEMORY (-6)
|
567 |
|
568 |
The buffer was too small for \fBpcre_copy_substring()\fR, or the attempt to get
|
569 |
memory failed for \fBpcre_get_substring()\fR.
|
570 |
|
571 |
PCRE_ERROR_NOSUBSTRING (-7)
|
572 |
|
573 |
There is no substring whose number is \fIstringnumber\fR.
|
574 |
|
575 |
The \fBpcre_get_substring_list()\fR function extracts all available substrings
|
576 |
and builds a list of pointers to them. All this is done in a single block of
|
577 |
memory which is obtained via \fBpcre_malloc\fR. The address of the memory block
|
578 |
is returned via \fIlistptr\fR, which is also the start of the list of string
|
579 |
pointers. The end of the list is marked by a NULL pointer. The yield of the
|
580 |
function is zero if all went well, or
|
581 |
|
582 |
PCRE_ERROR_NOMEMORY (-6)
|
583 |
|
584 |
if the attempt to get the memory block failed.
|
585 |
|
586 |
When any of these functions encounter a substring that is unset, which can
|
587 |
happen when capturing subpattern number \fIn+1\fR matches some part of the
|
588 |
subject, but subpattern \fIn\fR has not been used at all, they return an empty
|
589 |
string. This can be distinguished from a genuine zero-length substring by
|
590 |
inspecting the appropriate offset in \fIovector\fR, which is negative for unset
|
591 |
substrings.
|
592 |
|
593 |
|
594 |
|
595 |
.SH LIMITATIONS
|
596 |
There are some size limitations in PCRE but it is hoped that they will never in
|
597 |
practice be relevant.
|
598 |
The maximum length of a compiled pattern is 65539 (sic) bytes.
|
599 |
All values in repeating quantifiers must be less than 65536.
|
600 |
The maximum number of capturing subpatterns is 99.
|
601 |
The maximum number of all parenthesized subpatterns, including capturing
|
602 |
subpatterns, assertions, and other types of subpattern, is 200.
|
603 |
|
604 |
The maximum length of a subject string is the largest positive number that an
|
605 |
integer variable can hold. However, PCRE uses recursion to handle subpatterns
|
606 |
and indefinite repetition. This means that the available stack space may limit
|
607 |
the size of a subject string that can be processed by certain patterns.
|
608 |
|
609 |
|
610 |
.SH DIFFERENCES FROM PERL
|
611 |
The differences described here are with respect to Perl 5.005.
|
612 |
|
613 |
1. By default, a whitespace character is any character that the C library
|
614 |
function \fBisspace()\fR recognizes, though it is possible to compile PCRE with
|
615 |
alternative character type tables. Normally \fBisspace()\fR matches space,
|
616 |
formfeed, newline, carriage return, horizontal tab, and vertical tab. Perl 5
|
617 |
no longer includes vertical tab in its set of whitespace characters. The \\v
|
618 |
escape that was in the Perl documentation for a long time was never in fact
|
619 |
recognized. However, the character itself was treated as whitespace at least
|
620 |
up to 5.002. In 5.004 and 5.005 it does not match \\s.
|
621 |
|
622 |
2. PCRE does not allow repeat quantifiers on lookahead assertions. Perl permits
|
623 |
them, but they do not mean what you might think. For example, (?!a){3} does
|
624 |
not assert that the next three characters are not "a". It just asserts that the
|
625 |
next character is not "a" three times.
|
626 |
|
627 |
3. Capturing subpatterns that occur inside negative lookahead assertions are
|
628 |
counted, but their entries in the offsets vector are never set. Perl sets its
|
629 |
numerical variables from any such patterns that are matched before the
|
630 |
assertion fails to match something (thereby succeeding), but only if the
|
631 |
negative lookahead assertion contains just one branch.
|
632 |
|
633 |
4. Though binary zero characters are supported in the subject string, they are
|
634 |
not allowed in a pattern string because it is passed as a normal C string,
|
635 |
terminated by zero. The escape sequence "\\0" can be used in the pattern to
|
636 |
represent a binary zero.
|
637 |
|
638 |
5. The following Perl escape sequences are not supported: \\l, \\u, \\L, \\U,
|
639 |
\\E, \\Q. In fact these are implemented by Perl's general string-handling and
|
640 |
are not part of its pattern matching engine.
|
641 |
|
642 |
6. The Perl \\G assertion is not supported as it is not relevant to single
|
643 |
pattern matches.
|
644 |
|
645 |
7. Fairly obviously, PCRE does not support the (?{code}) and (?p{code})
|
646 |
constructions. However, there is some experimental support for recursive
|
647 |
patterns using the non-Perl item (?R).
|
648 |
|
649 |
8. There are at the time of writing some oddities in Perl 5.005_02 concerned
|
650 |
with the settings of captured strings when part of a pattern is repeated. For
|
651 |
example, matching "aba" against the pattern /^(a(b)?)+$/ sets $2 to the value
|
652 |
"b", but matching "aabbaa" against /^(aa(bb)?)+$/ leaves $2 unset. However, if
|
653 |
the pattern is changed to /^(aa(b(b))?)+$/ then $2 (and $3) are set.
|
654 |
|
655 |
In Perl 5.004 $2 is set in both cases, and that is also true of PCRE. If in the
|
656 |
future Perl changes to a consistent state that is different, PCRE may change to
|
657 |
follow.
|
658 |
|
659 |
9. Another as yet unresolved discrepancy is that in Perl 5.005_02 the pattern
|
660 |
/^(a)?(?(1)a|b)+$/ matches the string "a", whereas in PCRE it does not.
|
661 |
However, in both Perl and PCRE /^(a)?a/ matched against "a" leaves $1 unset.
|
662 |
|
663 |
10. PCRE provides some extensions to the Perl regular expression facilities:
|
664 |
|
665 |
(a) Although lookbehind assertions must match fixed length strings, each
|
666 |
alternative branch of a lookbehind assertion can match a different length of
|
667 |
string. Perl 5.005 requires them all to have the same length.
|
668 |
|
669 |
(b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ meta-
|
670 |
character matches only at the very end of the string.
|
671 |
|
672 |
(c) If PCRE_EXTRA is set, a backslash followed by a letter with no special
|
673 |
meaning is faulted.
|
674 |
|
675 |
(d) If PCRE_UNGREEDY is set, the greediness of the repetition quantifiers is
|
676 |
inverted, that is, by default they are not greedy, but if followed by a
|
677 |
question mark they are.
|
678 |
|
679 |
(e) PCRE_ANCHORED can be used to force a pattern to be tried only at the start
|
680 |
of the subject.
|
681 |
|
682 |
(f) The PCRE_NOTBOL, PCRE_NOTEOL, and PCRE_NOTEMPTY options for
|
683 |
\fBpcre_exec()\fR have no Perl equivalents.
|
684 |
|
685 |
(g) The (?R) construct allows for recursive pattern matching (Perl 5.6 can do
|
686 |
this using the (?p{code}) construct, which PCRE cannot of course support.)
|
687 |
|
688 |
|
689 |
.SH REGULAR EXPRESSION DETAILS
|
690 |
The syntax and semantics of the regular expressions supported by PCRE are
|
691 |
described below. Regular expressions are also described in the Perl
|
692 |
documentation and in a number of other books, some of which have copious
|
693 |
examples. Jeffrey Friedl's "Mastering Regular Expressions", published by
|
694 |
O'Reilly (ISBN 1-56592-257), covers them in great detail. The description
|
695 |
here is intended as reference documentation.
|
696 |
|
697 |
A regular expression is a pattern that is matched against a subject string from
|
698 |
left to right. Most characters stand for themselves in a pattern, and match the
|
699 |
corresponding characters in the subject. As a trivial example, the pattern
|
700 |
|
701 |
The quick brown fox
|
702 |
|
703 |
matches a portion of a subject string that is identical to itself. The power of
|
704 |
regular expressions comes from the ability to include alternatives and
|
705 |
repetitions in the pattern. These are encoded in the pattern by the use of
|
706 |
\fImeta-characters\fR, which do not stand for themselves but instead are
|
707 |
interpreted in some special way.
|
708 |
|
709 |
There are two different sets of meta-characters: those that are recognized
|
710 |
anywhere in the pattern except within square brackets, and those that are
|
711 |
recognized in square brackets. Outside square brackets, the meta-characters are
|
712 |
as follows:
|
713 |
|
714 |
\\ general escape character with several uses
|
715 |
^ assert start of subject (or line, in multiline mode)
|
716 |
$ assert end of subject (or line, in multiline mode)
|
717 |
. match any character except newline (by default)
|
718 |
[ start character class definition
|
719 |
| start of alternative branch
|
720 |
( start subpattern
|
721 |
) end subpattern
|
722 |
? extends the meaning of (
|
723 |
also 0 or 1 quantifier
|
724 |
also quantifier minimizer
|
725 |
* 0 or more quantifier
|
726 |
+ 1 or more quantifier
|
727 |
{ start min/max quantifier
|
728 |
|
729 |
Part of a pattern that is in square brackets is called a "character class". In
|
730 |
a character class the only meta-characters are:
|
731 |
|
732 |
\\ general escape character
|
733 |
^ negate the class, but only if the first character
|
734 |
- indicates character range
|
735 |
] terminates the character class
|
736 |
|
737 |
The following sections describe the use of each of the meta-characters.
|
738 |
|
739 |
|
740 |
.SH BACKSLASH
|
741 |
The backslash character has several uses. Firstly, if it is followed by a
|
742 |
non-alphameric character, it takes away any special meaning that character may
|
743 |
have. This use of backslash as an escape character applies both inside and
|
744 |
outside character classes.
|
745 |
|
746 |
For example, if you want to match a "*" character, you write "\\*" in the
|
747 |
pattern. This applies whether or not the following character would otherwise be
|
748 |
interpreted as a meta-character, so it is always safe to precede a
|
749 |
non-alphameric with "\\" to specify that it stands for itself. In particular,
|
750 |
if you want to match a backslash, you write "\\\\".
|
751 |
|
752 |
If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the
|
753 |
pattern (other than in a character class) and characters between a "#" outside
|
754 |
a character class and the next newline character are ignored. An escaping
|
755 |
backslash can be used to include a whitespace or "#" character as part of the
|
756 |
pattern.
|
757 |
|
758 |
A second use of backslash provides a way of encoding non-printing characters
|
759 |
in patterns in a visible manner. There is no restriction on the appearance of
|
760 |
non-printing characters, apart from the binary zero that terminates a pattern,
|
761 |
but when a pattern is being prepared by text editing, it is usually easier to
|
762 |
use one of the following escape sequences than the binary character it
|
763 |
represents:
|
764 |
|
765 |
\\a alarm, that is, the BEL character (hex 07)
|
766 |
\\cx "control-x", where x is any character
|
767 |
\\e escape (hex 1B)
|
768 |
\\f formfeed (hex 0C)
|
769 |
\\n newline (hex 0A)
|
770 |
\\r carriage return (hex 0D)
|
771 |
\\t tab (hex 09)
|
772 |
\\xhh character with hex code hh
|
773 |
\\ddd character with octal code ddd, or backreference
|
774 |
|
775 |
The precise effect of "\\cx" is as follows: if "x" is a lower case letter, it
|
776 |
is converted to upper case. Then bit 6 of the character (hex 40) is inverted.
|
777 |
Thus "\\cz" becomes hex 1A, but "\\c{" becomes hex 3B, while "\\c;" becomes hex
|
778 |
7B.
|
779 |
|
780 |
After "\\x", up to two hexadecimal digits are read (letters can be in upper or
|
781 |
lower case).
|
782 |
|
783 |
After "\\0" up to two further octal digits are read. In both cases, if there
|
784 |
are fewer than two digits, just those that are present are used. Thus the
|
785 |
sequence "\\0\\x\\07" specifies two binary zeros followed by a BEL character.
|
786 |
Make sure you supply two digits after the initial zero if the character that
|
787 |
follows is itself an octal digit.
|
788 |
|
789 |
The handling of a backslash followed by a digit other than 0 is complicated.
|
790 |
Outside a character class, PCRE reads it and any following digits as a decimal
|
791 |
number. If the number is less than 10, or if there have been at least that many
|
792 |
previous capturing left parentheses in the expression, the entire sequence is
|
793 |
taken as a \fIback reference\fR. A description of how this works is given
|
794 |
later, following the discussion of parenthesized subpatterns.
|
795 |
|
796 |
Inside a character class, or if the decimal number is greater than 9 and there
|
797 |
have not been that many capturing subpatterns, PCRE re-reads up to three octal
|
798 |
digits following the backslash, and generates a single byte from the least
|
799 |
significant 8 bits of the value. Any subsequent digits stand for themselves.
|
800 |
For example:
|
801 |
|
802 |
\\040 is another way of writing a space
|
803 |
\\40 is the same, provided there are fewer than 40
|
804 |
previous capturing subpatterns
|
805 |
\\7 is always a back reference
|
806 |
\\11 might be a back reference, or another way of
|
807 |
writing a tab
|
808 |
\\011 is always a tab
|
809 |
\\0113 is a tab followed by the character "3"
|
810 |
\\113 is the character with octal code 113 (since there
|
811 |
can be no more than 99 back references)
|
812 |
\\377 is a byte consisting entirely of 1 bits
|
813 |
\\81 is either a back reference, or a binary zero
|
814 |
followed by the two characters "8" and "1"
|
815 |
|
816 |
Note that octal values of 100 or greater must not be introduced by a leading
|
817 |
zero, because no more than three octal digits are ever read.
|
818 |
|
819 |
All the sequences that define a single byte value can be used both inside and
|
820 |
outside character classes. In addition, inside a character class, the sequence
|
821 |
"\\b" is interpreted as the backspace character (hex 08). Outside a character
|
822 |
class it has a different meaning (see below).
|
823 |
|
824 |
The third use of backslash is for specifying generic character types:
|
825 |
|
826 |
\\d any decimal digit
|
827 |
\\D any character that is not a decimal digit
|
828 |
\\s any whitespace character
|
829 |
\\S any character that is not a whitespace character
|
830 |
\\w any "word" character
|
831 |
\\W any "non-word" character
|
832 |
|
833 |
Each pair of escape sequences partitions the complete set of characters into
|
834 |
two disjoint sets. Any given character matches one, and only one, of each pair.
|
835 |
|
836 |
A "word" character is any letter or digit or the underscore character, that is,
|
837 |
any character which can be part of a Perl "word". The definition of letters and
|
838 |
digits is controlled by PCRE's character tables, and may vary if locale-
|
839 |
specific matching is taking place (see "Locale support" above). For example, in
|
840 |
the "fr" (French) locale, some character codes greater than 128 are used for
|
841 |
accented letters, and these are matched by \\w.
|
842 |
|
843 |
These character type sequences can appear both inside and outside character
|
844 |
classes. They each match one character of the appropriate type. If the current
|
845 |
matching point is at the end of the subject string, all of them fail, since
|
846 |
there is no character to match.
|
847 |
|
848 |
The fourth use of backslash is for certain simple assertions. An assertion
|
849 |
specifies a condition that has to be met at a particular point in a match,
|
850 |
without consuming any characters from the subject string. The use of
|
851 |
subpatterns for more complicated assertions is described below. The backslashed
|
852 |
assertions are
|
853 |
|
854 |
\\b word boundary
|
855 |
\\B not a word boundary
|
856 |
\\A start of subject (independent of multiline mode)
|
857 |
\\Z end of subject or newline at end (independent of multiline mode)
|
858 |
\\z end of subject (independent of multiline mode)
|
859 |
|
860 |
These assertions may not appear in character classes (but note that "\\b" has a
|
861 |
different meaning, namely the backspace character, inside a character class).
|
862 |
|
863 |
A word boundary is a position in the subject string where the current character
|
864 |
and the previous character do not both match \\w or \\W (i.e. one matches
|
865 |
\\w and the other matches \\W), or the start or end of the string if the
|
866 |
first or last character matches \\w, respectively.
|
867 |
|
868 |
The \\A, \\Z, and \\z assertions differ from the traditional circumflex and
|
869 |
dollar (described below) in that they only ever match at the very start and end
|
870 |
of the subject string, whatever options are set. They are not affected by the
|
871 |
PCRE_NOTBOL or PCRE_NOTEOL options. If the \fIstartoffset\fR argument of
|
872 |
\fBpcre_exec()\fR is non-zero, \\A can never match. The difference between \\Z
|
873 |
and \\z is that \\Z matches before a newline that is the last character of the
|
874 |
string as well as at the end of the string, whereas \\z matches only at the
|
875 |
end.
|
876 |
|
877 |
|
878 |
.SH CIRCUMFLEX AND DOLLAR
|
879 |
Outside a character class, in the default matching mode, the circumflex
|
880 |
character is an assertion which is true only if the current matching point is
|
881 |
at the start of the subject string. If the \fIstartoffset\fR argument of
|
882 |
\fBpcre_exec()\fR is non-zero, circumflex can never match. Inside a character
|
883 |
class, circumflex has an entirely different meaning (see below).
|
884 |
|
885 |
Circumflex need not be the first character of the pattern if a number of
|
886 |
alternatives are involved, but it should be the first thing in each alternative
|
887 |
in which it appears if the pattern is ever to match that branch. If all
|
888 |
possible alternatives start with a circumflex, that is, if the pattern is
|
889 |
constrained to match only at the start of the subject, it is said to be an
|
890 |
"anchored" pattern. (There are also other constructs that can cause a pattern
|
891 |
to be anchored.)
|
892 |
|
893 |
A dollar character is an assertion which is true only if the current matching
|
894 |
point is at the end of the subject string, or immediately before a newline
|
895 |
character that is the last character in the string (by default). Dollar need
|
896 |
not be the last character of the pattern if a number of alternatives are
|
897 |
involved, but it should be the last item in any branch in which it appears.
|
898 |
Dollar has no special meaning in a character class.
|
899 |
|
900 |
The meaning of dollar can be changed so that it matches only at the very end of
|
901 |
the string, by setting the PCRE_DOLLAR_ENDONLY option at compile or matching
|
902 |
time. This does not affect the \\Z assertion.
|
903 |
|
904 |
The meanings of the circumflex and dollar characters are changed if the
|
905 |
PCRE_MULTILINE option is set. When this is the case, they match immediately
|
906 |
after and immediately before an internal "\\n" character, respectively, in
|
907 |
addition to matching at the start and end of the subject string. For example,
|
908 |
the pattern /^abc$/ matches the subject string "def\\nabc" in multiline mode,
|
909 |
but not otherwise. Consequently, patterns that are anchored in single line mode
|
910 |
because all branches start with "^" are not anchored in multiline mode, and a
|
911 |
match for circumflex is possible when the \fIstartoffset\fR argument of
|
912 |
\fBpcre_exec()\fR is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if
|
913 |
PCRE_MULTILINE is set.
|
914 |
|
915 |
Note that the sequences \\A, \\Z, and \\z can be used to match the start and
|
916 |
end of the subject in both modes, and if all branches of a pattern start with
|
917 |
\\A is it always anchored, whether PCRE_MULTILINE is set or not.
|
918 |
|
919 |
|
920 |
.SH FULL STOP (PERIOD, DOT)
|
921 |
Outside a character class, a dot in the pattern matches any one character in
|
922 |
the subject, including a non-printing character, but not (by default) newline.
|
923 |
If the PCRE_DOTALL option is set, dots match newlines as well. The handling of
|
924 |
dot is entirely independent of the handling of circumflex and dollar, the only
|
925 |
relationship being that they both involve newline characters. Dot has no
|
926 |
special meaning in a character class.
|
927 |
|
928 |
|
929 |
.SH SQUARE BRACKETS
|
930 |
An opening square bracket introduces a character class, terminated by a closing
|
931 |
square bracket. A closing square bracket on its own is not special. If a
|
932 |
closing square bracket is required as a member of the class, it should be the
|
933 |
first data character in the class (after an initial circumflex, if present) or
|
934 |
escaped with a backslash.
|
935 |
|
936 |
A character class matches a single character in the subject; the character must
|
937 |
be in the set of characters defined by the class, unless the first character in
|
938 |
the class is a circumflex, in which case the subject character must not be in
|
939 |
the set defined by the class. If a circumflex is actually required as a member
|
940 |
of the class, ensure it is not the first character, or escape it with a
|
941 |
backslash.
|
942 |
|
943 |
For example, the character class [aeiou] matches any lower case vowel, while
|
944 |
[^aeiou] matches any character that is not a lower case vowel. Note that a
|
945 |
circumflex is just a convenient notation for specifying the characters which
|
946 |
are in the class by enumerating those that are not. It is not an assertion: it
|
947 |
still consumes a character from the subject string, and fails if the current
|
948 |
pointer is at the end of the string.
|
949 |
|
950 |
When caseless matching is set, any letters in a class represent both their
|
951 |
upper case and lower case versions, so for example, a caseless [aeiou] matches
|
952 |
"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a
|
953 |
caseful version would.
|
954 |
|
955 |
The newline character is never treated in any special way in character classes,
|
956 |
whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class
|
957 |
such as [^a] will always match a newline.
|
958 |
|
959 |
The minus (hyphen) character can be used to specify a range of characters in a
|
960 |
character class. For example, [d-m] matches any letter between d and m,
|
961 |
inclusive. If a minus character is required in a class, it must be escaped with
|
962 |
a backslash or appear in a position where it cannot be interpreted as
|
963 |
indicating a range, typically as the first or last character in the class.
|
964 |
|
965 |
It is not possible to have the literal character "]" as the end character of a
|
966 |
range. A pattern such as [W-]46] is interpreted as a class of two characters
|
967 |
("W" and "-") followed by a literal string "46]", so it would match "W46]" or
|
968 |
"-46]". However, if the "]" is escaped with a backslash it is interpreted as
|
969 |
the end of range, so [W-\\]46] is interpreted as a single class containing a
|
970 |
range followed by two separate characters. The octal or hexadecimal
|
971 |
representation of "]" can also be used to end a range.
|
972 |
|
973 |
Ranges operate in ASCII collating sequence. They can also be used for
|
974 |
characters specified numerically, for example [\\000-\\037]. If a range that
|
975 |
includes letters is used when caseless matching is set, it matches the letters
|
976 |
in either case. For example, [W-c] is equivalent to [][\\^_`wxyzabc], matched
|
977 |
caselessly, and if character tables for the "fr" locale are in use,
|
978 |
[\\xc8-\\xcb] matches accented E characters in both cases.
|
979 |
|
980 |
The character types \\d, \\D, \\s, \\S, \\w, and \\W may also appear in a
|
981 |
character class, and add the characters that they match to the class. For
|
982 |
example, [\\dABCDEF] matches any hexadecimal digit. A circumflex can
|
983 |
conveniently be used with the upper case character types to specify a more
|
984 |
restricted set of characters than the matching lower case type. For example,
|
985 |
the class [^\\W_] matches any letter or digit, but not underscore.
|
986 |
|
987 |
All non-alphameric characters other than \\, -, ^ (at the start) and the
|
988 |
terminating ] are non-special in character classes, but it does no harm if they
|
989 |
are escaped.
|
990 |
|
991 |
|
992 |
.SH POSIX CHARACTER CLASSES
|
993 |
Perl 5.6 (not yet released at the time of writing) is going to support the
|
994 |
POSIX notation for character classes, which uses names enclosed by [: and :]
|
995 |
within the enclosing square brackets. PCRE supports this notation. For example,
|
996 |
|
997 |
[01[:alpha:]%]
|
998 |
|
999 |
matches "0", "1", any alphabetic character, or "%". The supported class names
|
1000 |
are
|
1001 |
|
1002 |
alnum letters and digits
|
1003 |
alpha letters
|
1004 |
ascii character codes 0 - 127
|
1005 |
cntrl control characters
|
1006 |
digit decimal digits (same as \\d)
|
1007 |
graph printing characters, excluding space
|
1008 |
lower lower case letters
|
1009 |
print printing characters, including space
|
1010 |
punct printing characters, excluding letters and digits
|
1011 |
space white space (same as \\s)
|
1012 |
upper upper case letters
|
1013 |
word "word" characters (same as \\w)
|
1014 |
xdigit hexadecimal digits
|
1015 |
|
1016 |
The names "ascii" and "word" are Perl extensions. Another Perl extension is
|
1017 |
negation, which is indicated by a ^ character after the colon. For example,
|
1018 |
|
1019 |
[12[:^digit:]]
|
1020 |
|
1021 |
matches "1", "2", or any non-digit. PCRE (and Perl) also recogize the POSIX
|
1022 |
syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not
|
1023 |
supported, and an error is given if they are encountered.
|
1024 |
|
1025 |
|
1026 |
.SH VERTICAL BAR
|
1027 |
Vertical bar characters are used to separate alternative patterns. For example,
|
1028 |
the pattern
|
1029 |
|
1030 |
gilbert|sullivan
|
1031 |
|
1032 |
matches either "gilbert" or "sullivan". Any number of alternatives may appear,
|
1033 |
and an empty alternative is permitted (matching the empty string).
|
1034 |
The matching process tries each alternative in turn, from left to right,
|
1035 |
and the first one that succeeds is used. If the alternatives are within a
|
1036 |
subpattern (defined below), "succeeds" means matching the rest of the main
|
1037 |
pattern as well as the alternative in the subpattern.
|
1038 |
|
1039 |
|
1040 |
.SH INTERNAL OPTION SETTING
|
1041 |
The settings of PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and PCRE_EXTENDED
|
1042 |
can be changed from within the pattern by a sequence of Perl option letters
|
1043 |
enclosed between "(?" and ")". The option letters are
|
1044 |
|
1045 |
i for PCRE_CASELESS
|
1046 |
m for PCRE_MULTILINE
|
1047 |
s for PCRE_DOTALL
|
1048 |
x for PCRE_EXTENDED
|
1049 |
|
1050 |
For example, (?im) sets caseless, multiline matching. It is also possible to
|
1051 |
unset these options by preceding the letter with a hyphen, and a combined
|
1052 |
setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and
|
1053 |
PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also
|
1054 |
permitted. If a letter appears both before and after the hyphen, the option is
|
1055 |
unset.
|
1056 |
|
1057 |
The scope of these option changes depends on where in the pattern the setting
|
1058 |
occurs. For settings that are outside any subpattern (defined below), the
|
1059 |
effect is the same as if the options were set or unset at the start of
|
1060 |
matching. The following patterns all behave in exactly the same way:
|
1061 |
|
1062 |
(?i)abc
|
1063 |
a(?i)bc
|
1064 |
ab(?i)c
|
1065 |
abc(?i)
|
1066 |
|
1067 |
which in turn is the same as compiling the pattern abc with PCRE_CASELESS set.
|
1068 |
In other words, such "top level" settings apply to the whole pattern (unless
|
1069 |
there are other changes inside subpatterns). If there is more than one setting
|
1070 |
of the same option at top level, the rightmost setting is used.
|
1071 |
|
1072 |
If an option change occurs inside a subpattern, the effect is different. This
|
1073 |
is a change of behaviour in Perl 5.005. An option change inside a subpattern
|
1074 |
affects only that part of the subpattern that follows it, so
|
1075 |
|
1076 |
(a(?i)b)c
|
1077 |
|
1078 |
matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used).
|
1079 |
By this means, options can be made to have different settings in different
|
1080 |
parts of the pattern. Any changes made in one alternative do carry on
|
1081 |
into subsequent branches within the same subpattern. For example,
|
1082 |
|
1083 |
(a(?i)b|c)
|
1084 |
|
1085 |
matches "ab", "aB", "c", and "C", even though when matching "C" the first
|
1086 |
branch is abandoned before the option setting. This is because the effects of
|
1087 |
option settings happen at compile time. There would be some very weird
|
1088 |
behaviour otherwise.
|
1089 |
|
1090 |
The PCRE-specific options PCRE_UNGREEDY and PCRE_EXTRA can be changed in the
|
1091 |
same way as the Perl-compatible options by using the characters U and X
|
1092 |
respectively. The (?X) flag setting is special in that it must always occur
|
1093 |
earlier in the pattern than any of the additional features it turns on, even
|
1094 |
when it is at top level. It is best put at the start.
|
1095 |
|
1096 |
|
1097 |
.SH SUBPATTERNS
|
1098 |
Subpatterns are delimited by parentheses (round brackets), which can be nested.
|
1099 |
Marking part of a pattern as a subpattern does two things:
|
1100 |
|
1101 |
1. It localizes a set of alternatives. For example, the pattern
|
1102 |
|
1103 |
cat(aract|erpillar|)
|
1104 |
|
1105 |
matches one of the words "cat", "cataract", or "caterpillar". Without the
|
1106 |
parentheses, it would match "cataract", "erpillar" or the empty string.
|
1107 |
|
1108 |
2. It sets up the subpattern as a capturing subpattern (as defined above).
|
1109 |
When the whole pattern matches, that portion of the subject string that matched
|
1110 |
the subpattern is passed back to the caller via the \fIovector\fR argument of
|
1111 |
\fBpcre_exec()\fR. Opening parentheses are counted from left to right (starting
|
1112 |
from 1) to obtain the numbers of the capturing subpatterns.
|
1113 |
|
1114 |
For example, if the string "the red king" is matched against the pattern
|
1115 |
|
1116 |
the ((red|white) (king|queen))
|
1117 |
|
1118 |
the captured substrings are "red king", "red", and "king", and are numbered 1,
|
1119 |
2, and 3.
|
1120 |
|
1121 |
The fact that plain parentheses fulfil two functions is not always helpful.
|
1122 |
There are often times when a grouping subpattern is required without a
|
1123 |
capturing requirement. If an opening parenthesis is followed by "?:", the
|
1124 |
subpattern does not do any capturing, and is not counted when computing the
|
1125 |
number of any subsequent capturing subpatterns. For example, if the string "the
|
1126 |
white queen" is matched against the pattern
|
1127 |
|
1128 |
the ((?:red|white) (king|queen))
|
1129 |
|
1130 |
the captured substrings are "white queen" and "queen", and are numbered 1 and
|
1131 |
2. The maximum number of captured substrings is 99, and the maximum number of
|
1132 |
all subpatterns, both capturing and non-capturing, is 200.
|
1133 |
|
1134 |
As a convenient shorthand, if any option settings are required at the start of
|
1135 |
a non-capturing subpattern, the option letters may appear between the "?" and
|
1136 |
the ":". Thus the two patterns
|
1137 |
|
1138 |
(?i:saturday|sunday)
|
1139 |
(?:(?i)saturday|sunday)
|
1140 |
|
1141 |
match exactly the same set of strings. Because alternative branches are tried
|
1142 |
from left to right, and options are not reset until the end of the subpattern
|
1143 |
is reached, an option setting in one branch does affect subsequent branches, so
|
1144 |
the above patterns match "SUNDAY" as well as "Saturday".
|
1145 |
|
1146 |
|
1147 |
.SH REPETITION
|
1148 |
Repetition is specified by quantifiers, which can follow any of the following
|
1149 |
items:
|
1150 |
|
1151 |
a single character, possibly escaped
|
1152 |
the . metacharacter
|
1153 |
a character class
|
1154 |
a back reference (see next section)
|
1155 |
a parenthesized subpattern (unless it is an assertion - see below)
|
1156 |
|
1157 |
The general repetition quantifier specifies a minimum and maximum number of
|
1158 |
permitted matches, by giving the two numbers in curly brackets (braces),
|
1159 |
separated by a comma. The numbers must be less than 65536, and the first must
|
1160 |
be less than or equal to the second. For example:
|
1161 |
|
1162 |
z{2,4}
|
1163 |
|
1164 |
matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special
|
1165 |
character. If the second number is omitted, but the comma is present, there is
|
1166 |
no upper limit; if the second number and the comma are both omitted, the
|
1167 |
quantifier specifies an exact number of required matches. Thus
|
1168 |
|
1169 |
[aeiou]{3,}
|
1170 |
|
1171 |
matches at least 3 successive vowels, but may match many more, while
|
1172 |
|
1173 |
\\d{8}
|
1174 |
|
1175 |
matches exactly 8 digits. An opening curly bracket that appears in a position
|
1176 |
where a quantifier is not allowed, or one that does not match the syntax of a
|
1177 |
quantifier, is taken as a literal character. For example, {,6} is not a
|
1178 |
quantifier, but a literal string of four characters.
|
1179 |
|
1180 |
The quantifier {0} is permitted, causing the expression to behave as if the
|
1181 |
previous item and the quantifier were not present.
|
1182 |
|
1183 |
For convenience (and historical compatibility) the three most common
|
1184 |
quantifiers have single-character abbreviations:
|
1185 |
|
1186 |
* is equivalent to {0,}
|
1187 |
+ is equivalent to {1,}
|
1188 |
? is equivalent to {0,1}
|
1189 |
|
1190 |
It is possible to construct infinite loops by following a subpattern that can
|
1191 |
match no characters with a quantifier that has no upper limit, for example:
|
1192 |
|
1193 |
(a?)*
|
1194 |
|
1195 |
Earlier versions of Perl and PCRE used to give an error at compile time for
|
1196 |
such patterns. However, because there are cases where this can be useful, such
|
1197 |
patterns are now accepted, but if any repetition of the subpattern does in fact
|
1198 |
match no characters, the loop is forcibly broken.
|
1199 |
|
1200 |
By default, the quantifiers are "greedy", that is, they match as much as
|
1201 |
possible (up to the maximum number of permitted times), without causing the
|
1202 |
rest of the pattern to fail. The classic example of where this gives problems
|
1203 |
is in trying to match comments in C programs. These appear between the
|
1204 |
sequences /* and */ and within the sequence, individual * and / characters may
|
1205 |
appear. An attempt to match C comments by applying the pattern
|
1206 |
|
1207 |
/\\*.*\\*/
|
1208 |
|
1209 |
to the string
|
1210 |
|
1211 |
/* first command */ not comment /* second comment */
|
1212 |
|
1213 |
fails, because it matches the entire string due to the greediness of the .*
|
1214 |
item.
|
1215 |
|
1216 |
However, if a quantifier is followed by a question mark, it ceases to be
|
1217 |
greedy, and instead matches the minimum number of times possible, so the
|
1218 |
pattern
|
1219 |
|
1220 |
/\\*.*?\\*/
|
1221 |
|
1222 |
does the right thing with the C comments. The meaning of the various
|
1223 |
quantifiers is not otherwise changed, just the preferred number of matches.
|
1224 |
Do not confuse this use of question mark with its use as a quantifier in its
|
1225 |
own right. Because it has two uses, it can sometimes appear doubled, as in
|
1226 |
|
1227 |
\\d??\\d
|
1228 |
|
1229 |
which matches one digit by preference, but can match two if that is the only
|
1230 |
way the rest of the pattern matches.
|
1231 |
|
1232 |
If the PCRE_UNGREEDY option is set (an option which is not available in Perl),
|
1233 |
the quantifiers are not greedy by default, but individual ones can be made
|
1234 |
greedy by following them with a question mark. In other words, it inverts the
|
1235 |
default behaviour.
|
1236 |
|
1237 |
When a parenthesized subpattern is quantified with a minimum repeat count that
|
1238 |
is greater than 1 or with a limited maximum, more store is required for the
|
1239 |
compiled pattern, in proportion to the size of the minimum or maximum.
|
1240 |
|
1241 |
If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent
|
1242 |
to Perl's /s) is set, thus allowing the . to match newlines, the pattern is
|
1243 |
implicitly anchored, because whatever follows will be tried against every
|
1244 |
character position in the subject string, so there is no point in retrying the
|
1245 |
overall match at any position after the first. PCRE treats such a pattern as
|
1246 |
though it were preceded by \\A. In cases where it is known that the subject
|
1247 |
string contains no newlines, it is worth setting PCRE_DOTALL when the pattern
|
1248 |
begins with .* in order to obtain this optimization, or alternatively using ^
|
1249 |
to indicate anchoring explicitly.
|
1250 |
|
1251 |
When a capturing subpattern is repeated, the value captured is the substring
|
1252 |
that matched the final iteration. For example, after
|
1253 |
|
1254 |
(tweedle[dume]{3}\\s*)+
|
1255 |
|
1256 |
has matched "tweedledum tweedledee" the value of the captured substring is
|
1257 |
"tweedledee". However, if there are nested capturing subpatterns, the
|
1258 |
corresponding captured values may have been set in previous iterations. For
|
1259 |
example, after
|
1260 |
|
1261 |
/(a|(b))+/
|
1262 |
|
1263 |
matches "aba" the value of the second captured substring is "b".
|
1264 |
|
1265 |
|
1266 |
.SH BACK REFERENCES
|
1267 |
Outside a character class, a backslash followed by a digit greater than 0 (and
|
1268 |
possibly further digits) is a back reference to a capturing subpattern earlier
|
1269 |
(i.e. to its left) in the pattern, provided there have been that many previous
|
1270 |
capturing left parentheses.
|
1271 |
|
1272 |
However, if the decimal number following the backslash is less than 10, it is
|
1273 |
always taken as a back reference, and causes an error only if there are not
|
1274 |
that many capturing left parentheses in the entire pattern. In other words, the
|
1275 |
parentheses that are referenced need not be to the left of the reference for
|
1276 |
numbers less than 10. See the section entitled "Backslash" above for further
|
1277 |
details of the handling of digits following a backslash.
|
1278 |
|
1279 |
A back reference matches whatever actually matched the capturing subpattern in
|
1280 |
the current subject string, rather than anything matching the subpattern
|
1281 |
itself. So the pattern
|
1282 |
|
1283 |
(sens|respons)e and \\1ibility
|
1284 |
|
1285 |
matches "sense and sensibility" and "response and responsibility", but not
|
1286 |
"sense and responsibility". If caseful matching is in force at the time of the
|
1287 |
back reference, the case of letters is relevant. For example,
|
1288 |
|
1289 |
((?i)rah)\\s+\\1
|
1290 |
|
1291 |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original
|
1292 |
capturing subpattern is matched caselessly.
|
1293 |
|
1294 |
There may be more than one back reference to the same subpattern. If a
|
1295 |
subpattern has not actually been used in a particular match, any back
|
1296 |
references to it always fail. For example, the pattern
|
1297 |
|
1298 |
(a|(bc))\\2
|
1299 |
|
1300 |
always fails if it starts to match "a" rather than "bc". Because there may be
|
1301 |
up to 99 back references, all digits following the backslash are taken
|
1302 |
as part of a potential back reference number. If the pattern continues with a
|
1303 |
digit character, some delimiter must be used to terminate the back reference.
|
1304 |
If the PCRE_EXTENDED option is set, this can be whitespace. Otherwise an empty
|
1305 |
comment can be used.
|
1306 |
|
1307 |
A back reference that occurs inside the parentheses to which it refers fails
|
1308 |
when the subpattern is first used, so, for example, (a\\1) never matches.
|
1309 |
However, such references can be useful inside repeated subpatterns. For
|
1310 |
example, the pattern
|
1311 |
|
1312 |
(a|b\\1)+
|
1313 |
|
1314 |
matches any number of "a"s and also "aba", "ababaa" etc. At each iteration of
|
1315 |
the subpattern, the back reference matches the character string corresponding
|
1316 |
to the previous iteration. In order for this to work, the pattern must be such
|
1317 |
that the first iteration does not need to match the back reference. This can be
|
1318 |
done using alternation, as in the example above, or by a quantifier with a
|
1319 |
minimum of zero.
|
1320 |
|
1321 |
|
1322 |
.SH ASSERTIONS
|
1323 |
An assertion is a test on the characters following or preceding the current
|
1324 |
matching point that does not actually consume any characters. The simple
|
1325 |
assertions coded as \\b, \\B, \\A, \\Z, \\z, ^ and $ are described above. More
|
1326 |
complicated assertions are coded as subpatterns. There are two kinds: those
|
1327 |
that look ahead of the current position in the subject string, and those that
|
1328 |
look behind it.
|
1329 |
|
1330 |
An assertion subpattern is matched in the normal way, except that it does not
|
1331 |
cause the current matching position to be changed. Lookahead assertions start
|
1332 |
with (?= for positive assertions and (?! for negative assertions. For example,
|
1333 |
|
1334 |
\\w+(?=;)
|
1335 |
|
1336 |
matches a word followed by a semicolon, but does not include the semicolon in
|
1337 |
the match, and
|
1338 |
|
1339 |
foo(?!bar)
|
1340 |
|
1341 |
matches any occurrence of "foo" that is not followed by "bar". Note that the
|
1342 |
apparently similar pattern
|
1343 |
|
1344 |
(?!foo)bar
|
1345 |
|
1346 |
does not find an occurrence of "bar" that is preceded by something other than
|
1347 |
"foo"; it finds any occurrence of "bar" whatsoever, because the assertion
|
1348 |
(?!foo) is always true when the next three characters are "bar". A
|
1349 |
lookbehind assertion is needed to achieve this effect.
|
1350 |
|
1351 |
Lookbehind assertions start with (?<= for positive assertions and (?<! for
|
1352 |
negative assertions. For example,
|
1353 |
|
1354 |
(?<!foo)bar
|
1355 |
|
1356 |
does find an occurrence of "bar" that is not preceded by "foo". The contents of
|
1357 |
a lookbehind assertion are restricted such that all the strings it matches must
|
1358 |
have a fixed length. However, if there are several alternatives, they do not
|
1359 |
all have to have the same fixed length. Thus
|
1360 |
|
1361 |
(?<=bullock|donkey)
|
1362 |
|
1363 |
is permitted, but
|
1364 |
|
1365 |
(?<!dogs?|cats?)
|
1366 |
|
1367 |
causes an error at compile time. Branches that match different length strings
|
1368 |
are permitted only at the top level of a lookbehind assertion. This is an
|
1369 |
extension compared with Perl 5.005, which requires all branches to match the
|
1370 |
same length of string. An assertion such as
|
1371 |
|
1372 |
(?<=ab(c|de))
|
1373 |
|
1374 |
is not permitted, because its single top-level branch can match two different
|
1375 |
lengths, but it is acceptable if rewritten to use two top-level branches:
|
1376 |
|
1377 |
(?<=abc|abde)
|
1378 |
|
1379 |
The implementation of lookbehind assertions is, for each alternative, to
|
1380 |
temporarily move the current position back by the fixed width and then try to
|
1381 |
match. If there are insufficient characters before the current position, the
|
1382 |
match is deemed to fail. Lookbehinds in conjunction with once-only subpatterns
|
1383 |
can be particularly useful for matching at the ends of strings; an example is
|
1384 |
given at the end of the section on once-only subpatterns.
|
1385 |
|
1386 |
Several assertions (of any sort) may occur in succession. For example,
|
1387 |
|
1388 |
(?<=\\d{3})(?<!999)foo
|
1389 |
|
1390 |
matches "foo" preceded by three digits that are not "999". Notice that each of
|
1391 |
the assertions is applied independently at the same point in the subject
|
1392 |
string. First there is a check that the previous three characters are all
|
1393 |
digits, and then there is a check that the same three characters are not "999".
|
1394 |
This pattern does \fInot\fR match "foo" preceded by six characters, the first
|
1395 |
of which are digits and the last three of which are not "999". For example, it
|
1396 |
doesn't match "123abcfoo". A pattern to do that is
|
1397 |
|
1398 |
(?<=\\d{3}...)(?<!999)foo
|
1399 |
|
1400 |
This time the first assertion looks at the preceding six characters, checking
|
1401 |
that the first three are digits, and then the second assertion checks that the
|
1402 |
preceding three characters are not "999".
|
1403 |
|
1404 |
Assertions can be nested in any combination. For example,
|
1405 |
|
1406 |
(?<=(?<!foo)bar)baz
|
1407 |
|
1408 |
matches an occurrence of "baz" that is preceded by "bar" which in turn is not
|
1409 |
preceded by "foo", while
|
1410 |
|
1411 |
(?<=\\d{3}(?!999)...)foo
|
1412 |
|
1413 |
is another pattern which matches "foo" preceded by three digits and any three
|
1414 |
characters that are not "999".
|
1415 |
|
1416 |
Assertion subpatterns are not capturing subpatterns, and may not be repeated,
|
1417 |
because it makes no sense to assert the same thing several times. If any kind
|
1418 |
of assertion contains capturing subpatterns within it, these are counted for
|
1419 |
the purposes of numbering the capturing subpatterns in the whole pattern.
|
1420 |
However, substring capturing is carried out only for positive assertions,
|
1421 |
because it does not make sense for negative assertions.
|
1422 |
|
1423 |
Assertions count towards the maximum of 200 parenthesized subpatterns.
|
1424 |
|
1425 |
|
1426 |
.SH ONCE-ONLY SUBPATTERNS
|
1427 |
With both maximizing and minimizing repetition, failure of what follows
|
1428 |
normally causes the repeated item to be re-evaluated to see if a different
|
1429 |
number of repeats allows the rest of the pattern to match. Sometimes it is
|
1430 |
useful to prevent this, either to change the nature of the match, or to cause
|
1431 |
it fail earlier than it otherwise might, when the author of the pattern knows
|
1432 |
there is no point in carrying on.
|
1433 |
|
1434 |
Consider, for example, the pattern \\d+foo when applied to the subject line
|
1435 |
|
1436 |
123456bar
|
1437 |
|
1438 |
After matching all 6 digits and then failing to match "foo", the normal
|
1439 |
action of the matcher is to try again with only 5 digits matching the \\d+
|
1440 |
item, and then with 4, and so on, before ultimately failing. Once-only
|
1441 |
subpatterns provide the means for specifying that once a portion of the pattern
|
1442 |
has matched, it is not to be re-evaluated in this way, so the matcher would
|
1443 |
give up immediately on failing to match "foo" the first time. The notation is
|
1444 |
another kind of special parenthesis, starting with (?> as in this example:
|
1445 |
|
1446 |
(?>\\d+)bar
|
1447 |
|
1448 |
This kind of parenthesis "locks up" the part of the pattern it contains once
|
1449 |
it has matched, and a failure further into the pattern is prevented from
|
1450 |
backtracking into it. Backtracking past it to previous items, however, works as
|
1451 |
normal.
|
1452 |
|
1453 |
An alternative description is that a subpattern of this type matches the string
|
1454 |
of characters that an identical standalone pattern would match, if anchored at
|
1455 |
the current point in the subject string.
|
1456 |
|
1457 |
Once-only subpatterns are not capturing subpatterns. Simple cases such as the
|
1458 |
above example can be thought of as a maximizing repeat that must swallow
|
1459 |
everything it can. So, while both \\d+ and \\d+? are prepared to adjust the
|
1460 |
number of digits they match in order to make the rest of the pattern match,
|
1461 |
(?>\\d+) can only match an entire sequence of digits.
|
1462 |
|
1463 |
This construction can of course contain arbitrarily complicated subpatterns,
|
1464 |
and it can be nested.
|
1465 |
|
1466 |
Once-only subpatterns can be used in conjunction with lookbehind assertions to
|
1467 |
specify efficient matching at the end of the subject string. Consider a simple
|
1468 |
pattern such as
|
1469 |
|
1470 |
abcd$
|
1471 |
|
1472 |
when applied to a long string which does not match. Because matching proceeds
|
1473 |
from left to right, PCRE will look for each "a" in the subject and then see if
|
1474 |
what follows matches the rest of the pattern. If the pattern is specified as
|
1475 |
|
1476 |
^.*abcd$
|
1477 |
|
1478 |
the initial .* matches the entire string at first, but when this fails (because
|
1479 |
there is no following "a"), it backtracks to match all but the last character,
|
1480 |
then all but the last two characters, and so on. Once again the search for "a"
|
1481 |
covers the entire string, from right to left, so we are no better off. However,
|
1482 |
if the pattern is written as
|
1483 |
|
1484 |
^(?>.*)(?<=abcd)
|
1485 |
|
1486 |
there can be no backtracking for the .* item; it can match only the entire
|
1487 |
string. The subsequent lookbehind assertion does a single test on the last four
|
1488 |
characters. If it fails, the match fails immediately. For long strings, this
|
1489 |
approach makes a significant difference to the processing time.
|
1490 |
|
1491 |
When a pattern contains an unlimited repeat inside a subpattern that can itself
|
1492 |
be repeated an unlimited number of times, the use of a once-only subpattern is
|
1493 |
the only way to avoid some failing matches taking a very long time indeed.
|
1494 |
The pattern
|
1495 |
|
1496 |
(\\D+|<\\d+>)*[!?]
|
1497 |
|
1498 |
matches an unlimited number of substrings that either consist of non-digits, or
|
1499 |
digits enclosed in <>, followed by either ! or ?. When it matches, it runs
|
1500 |
quickly. However, if it is applied to
|
1501 |
|
1502 |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
|
1503 |
|
1504 |
it takes a long time before reporting failure. This is because the string can
|
1505 |
be divided between the two repeats in a large number of ways, and all have to
|
1506 |
be tried. (The example used [!?] rather than a single character at the end,
|
1507 |
because both PCRE and Perl have an optimization that allows for fast failure
|
1508 |
when a single character is used. They remember the last single character that
|
1509 |
is required for a match, and fail early if it is not present in the string.)
|
1510 |
If the pattern is changed to
|
1511 |
|
1512 |
((?>\\D+)|<\\d+>)*[!?]
|
1513 |
|
1514 |
sequences of non-digits cannot be broken, and failure happens quickly.
|
1515 |
|
1516 |
|
1517 |
.SH CONDITIONAL SUBPATTERNS
|
1518 |
It is possible to cause the matching process to obey a subpattern
|
1519 |
conditionally or to choose between two alternative subpatterns, depending on
|
1520 |
the result of an assertion, or whether a previous capturing subpattern matched
|
1521 |
or not. The two possible forms of conditional subpattern are
|
1522 |
|
1523 |
(?(condition)yes-pattern)
|
1524 |
(?(condition)yes-pattern|no-pattern)
|
1525 |
|
1526 |
If the condition is satisfied, the yes-pattern is used; otherwise the
|
1527 |
no-pattern (if present) is used. If there are more than two alternatives in the
|
1528 |
subpattern, a compile-time error occurs.
|
1529 |
|
1530 |
There are two kinds of condition. If the text between the parentheses consists
|
1531 |
of a sequence of digits, the condition is satisfied if the capturing subpattern
|
1532 |
of that number has previously matched. Consider the following pattern, which
|
1533 |
contains non-significant white space to make it more readable (assume the
|
1534 |
PCRE_EXTENDED option) and to divide it into three parts for ease of discussion:
|
1535 |
|
1536 |
( \\( )? [^()]+ (?(1) \\) )
|
1537 |
|
1538 |
The first part matches an optional opening parenthesis, and if that
|
1539 |
character is present, sets it as the first captured substring. The second part
|
1540 |
matches one or more characters that are not parentheses. The third part is a
|
1541 |
conditional subpattern that tests whether the first set of parentheses matched
|
1542 |
or not. If they did, that is, if subject started with an opening parenthesis,
|
1543 |
the condition is true, and so the yes-pattern is executed and a closing
|
1544 |
parenthesis is required. Otherwise, since no-pattern is not present, the
|
1545 |
subpattern matches nothing. In other words, this pattern matches a sequence of
|
1546 |
non-parentheses, optionally enclosed in parentheses.
|
1547 |
|
1548 |
If the condition is not a sequence of digits, it must be an assertion. This may
|
1549 |
be a positive or negative lookahead or lookbehind assertion. Consider this
|
1550 |
pattern, again containing non-significant white space, and with the two
|
1551 |
alternatives on the second line:
|
1552 |
|
1553 |
(?(?=[^a-z]*[a-z])
|
1554 |
\\d{2}-[a-z]{3}-\\d{2} | \\d{2}-\\d{2}-\\d{2} )
|
1555 |
|
1556 |
The condition is a positive lookahead assertion that matches an optional
|
1557 |
sequence of non-letters followed by a letter. In other words, it tests for the
|
1558 |
presence of at least one letter in the subject. If a letter is found, the
|
1559 |
subject is matched against the first alternative; otherwise it is matched
|
1560 |
against the second. This pattern matches strings in one of the two forms
|
1561 |
dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits.
|
1562 |
|
1563 |
|
1564 |
.SH COMMENTS
|
1565 |
The sequence (?# marks the start of a comment which continues up to the next
|
1566 |
closing parenthesis. Nested parentheses are not permitted. The characters
|
1567 |
that make up a comment play no part in the pattern matching at all.
|
1568 |
|
1569 |
If the PCRE_EXTENDED option is set, an unescaped # character outside a
|
1570 |
character class introduces a comment that continues up to the next newline
|
1571 |
character in the pattern.
|
1572 |
|
1573 |
|
1574 |
.SH RECURSIVE PATTERNS
|
1575 |
Consider the problem of matching a string in parentheses, allowing for
|
1576 |
unlimited nested parentheses. Without the use of recursion, the best that can
|
1577 |
be done is to use a pattern that matches up to some fixed depth of nesting. It
|
1578 |
is not possible to handle an arbitrary nesting depth. Perl 5.6 has provided an
|
1579 |
experimental facility that allows regular expressions to recurse (amongst other
|
1580 |
things). It does this by interpolating Perl code in the expression at run time,
|
1581 |
and the code can refer to the expression itself. A Perl pattern to solve the
|
1582 |
parentheses problem can be created like this:
|
1583 |
|
1584 |
$re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x;
|
1585 |
|
1586 |
The (?p{...}) item interpolates Perl code at run time, and in this case refers
|
1587 |
recursively to the pattern in which it appears. Obviously, PCRE cannot support
|
1588 |
the interpolation of Perl code. Instead, the special item (?R) is provided for
|
1589 |
the specific case of recursion. This PCRE pattern solves the parentheses
|
1590 |
problem (assume the PCRE_EXTENDED option is set so that white space is
|
1591 |
ignored):
|
1592 |
|
1593 |
\\( ( (?>[^()]+) | (?R) )* \\)
|
1594 |
|
1595 |
First it matches an opening parenthesis. Then it matches any number of
|
1596 |
substrings which can either be a sequence of non-parentheses, or a recursive
|
1597 |
match of the pattern itself (i.e. a correctly parenthesized substring). Finally
|
1598 |
there is a closing parenthesis.
|
1599 |
|
1600 |
This particular example pattern contains nested unlimited repeats, and so the
|
1601 |
use of a once-only subpattern for matching strings of non-parentheses is
|
1602 |
important when applying the pattern to strings that do not match. For example,
|
1603 |
when it is applied to
|
1604 |
|
1605 |
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()
|
1606 |
|
1607 |
it yields "no match" quickly. However, if a once-only subpattern is not used,
|
1608 |
the match runs for a very long time indeed because there are so many different
|
1609 |
ways the + and * repeats can carve up the subject, and all have to be tested
|
1610 |
before failure can be reported.
|
1611 |
|
1612 |
The values set for any capturing subpatterns are those from the outermost level
|
1613 |
of the recursion at which the subpattern value is set. If the pattern above is
|
1614 |
matched against
|
1615 |
|
1616 |
(ab(cd)ef)
|
1617 |
|
1618 |
the value for the capturing parentheses is "ef", which is the last value taken
|
1619 |
on at the top level. If additional parentheses are added, giving
|
1620 |
|
1621 |
\\( ( ( (?>[^()]+) | (?R) )* ) \\)
|
1622 |
^ ^
|
1623 |
^ ^
|
1624 |
the string they capture is "ab(cd)ef", the contents of the top level
|
1625 |
parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE
|
1626 |
has to obtain extra memory to store data during a recursion, which it does by
|
1627 |
using \fBpcre_malloc\fR, freeing it via \fBpcre_free\fR afterwards. If no
|
1628 |
memory can be obtained, it saves data for the first 15 capturing parentheses
|
1629 |
only, as there is no way to give an out-of-memory error from within a
|
1630 |
recursion.
|
1631 |
|
1632 |
|
1633 |
.SH PERFORMANCE
|
1634 |
Certain items that may appear in patterns are more efficient than others. It is
|
1635 |
more efficient to use a character class like [aeiou] than a set of alternatives
|
1636 |
such as (a|e|i|o|u). In general, the simplest construction that provides the
|
1637 |
required behaviour is usually the most efficient. Jeffrey Friedl's book
|
1638 |
contains a lot of discussion about optimizing regular expressions for efficient
|
1639 |
performance.
|
1640 |
|
1641 |
When a pattern begins with .* and the PCRE_DOTALL option is set, the pattern is
|
1642 |
implicitly anchored by PCRE, since it can match only at the start of a subject
|
1643 |
string. However, if PCRE_DOTALL is not set, PCRE cannot make this optimization,
|
1644 |
because the . metacharacter does not then match a newline, and if the subject
|
1645 |
string contains newlines, the pattern may match from the character immediately
|
1646 |
following one of them instead of from the very start. For example, the pattern
|
1647 |
|
1648 |
(.*) second
|
1649 |
|
1650 |
matches the subject "first\\nand second" (where \\n stands for a newline
|
1651 |
character) with the first captured substring being "and". In order to do this,
|
1652 |
PCRE has to retry the match starting after every newline in the subject.
|
1653 |
|
1654 |
If you are using such a pattern with subject strings that do not contain
|
1655 |
newlines, the best performance is obtained by setting PCRE_DOTALL, or starting
|
1656 |
the pattern with ^.* to indicate explicit anchoring. That saves PCRE from
|
1657 |
having to scan along the subject looking for a newline to restart at.
|
1658 |
|
1659 |
Beware of patterns that contain nested indefinite repeats. These can take a
|
1660 |
long time to run when applied to a string that does not match. Consider the
|
1661 |
pattern fragment
|
1662 |
|
1663 |
(a+)*
|
1664 |
|
1665 |
This can match "aaaa" in 33 different ways, and this number increases very
|
1666 |
rapidly as the string gets longer. (The * repeat can match 0, 1, 2, 3, or 4
|
1667 |
times, and for each of those cases other than 0, the + repeats can match
|
1668 |
different numbers of times.) When the remainder of the pattern is such that the
|
1669 |
entire match is going to fail, PCRE has in principle to try every possible
|
1670 |
variation, and this can take an extremely long time.
|
1671 |
|
1672 |
An optimization catches some of the more simple cases such as
|
1673 |
|
1674 |
(a+)*b
|
1675 |
|
1676 |
where a literal character follows. Before embarking on the standard matching
|
1677 |
procedure, PCRE checks that there is a "b" later in the subject string, and if
|
1678 |
there is not, it fails the match immediately. However, when there is no
|
1679 |
following literal this optimization cannot be used. You can see the difference
|
1680 |
by comparing the behaviour of
|
1681 |
|
1682 |
(a+)*\\d
|
1683 |
|
1684 |
with the pattern above. The former gives a failure almost instantly when
|
1685 |
applied to a whole line of "a" characters, whereas the latter takes an
|
1686 |
appreciable time with strings longer than about 20 characters.
|
1687 |
|
1688 |
.SH AUTHOR
|
1689 |
Philip Hazel <ph10@cam.ac.uk>
|
1690 |
.br
|
1691 |
University Computing Service,
|
1692 |
.br
|
1693 |
New Museums Site,
|
1694 |
.br
|
1695 |
Cambridge CB2 3QG, England.
|
1696 |
.br
|
1697 |
Phone: +44 1223 334714
|
1698 |
|
1699 |
Last updated: 27 January 2000
|
1700 |
.br
|
1701 |
Copyright (c) 1997-2000 University of Cambridge.
|