1 |
.TH PCRE 3 |
.TH PCREPATTERN 3 |
2 |
.SH NAME |
.SH NAME |
3 |
PCRE - Perl-compatible regular expressions |
PCRE - Perl-compatible regular expressions |
4 |
.SH PCRE REGULAR EXPRESSION DETAILS |
.SH "PCRE REGULAR EXPRESSION DETAILS" |
5 |
.rs |
.rs |
6 |
.sp |
.sp |
7 |
The syntax and semantics of the regular expressions supported by PCRE are |
The syntax and semantics of the regular expressions supported by PCRE are |
8 |
described below. Regular expressions are also described in the Perl |
described below. Regular expressions are also described in the Perl |
9 |
documentation and in a number of other books, some of which have copious |
documentation and in a number of books, some of which have copious examples. |
10 |
examples. Jeffrey Friedl's "Mastering Regular Expressions", published by |
Jeffrey Friedl's "Mastering Regular Expressions", published by O'Reilly, covers |
11 |
O'Reilly, covers them in great detail. The description here is intended as |
regular expressions in great detail. This description of PCRE's regular |
12 |
reference documentation. |
expressions is intended as reference material. |
13 |
|
.P |
14 |
The basic operation of PCRE is on strings of bytes. However, there is also |
The original operation of PCRE was on strings of one-byte characters. However, |
15 |
support for UTF-8 character strings. To use this support you must build PCRE to |
there is now also support for UTF-8 character strings. To use this, you must |
16 |
include UTF-8 support, and then call \fBpcre_compile()\fR with the PCRE_UTF8 |
build PCRE to include UTF-8 support, and then call \fBpcre_compile()\fP with |
17 |
option. How this affects the pattern matching is mentioned in several places |
the PCRE_UTF8 option. How this affects pattern matching is mentioned in several |
18 |
below. There is also a summary of UTF-8 features in the |
places below. There is also a summary of UTF-8 features in the |
19 |
.\" HTML <a href="pcre.html#utf8support"> |
.\" HTML <a href="pcre.html#utf8support"> |
20 |
.\" </a> |
.\" </a> |
21 |
section on UTF-8 support |
section on UTF-8 support |
22 |
.\" |
.\" |
23 |
in the main |
in the main |
24 |
.\" HREF |
.\" HREF |
25 |
\fBpcre\fR |
\fBpcre\fP |
26 |
.\" |
.\" |
27 |
page. |
page. |
28 |
|
.P |
29 |
|
The remainder of this document discusses the patterns that are supported by |
30 |
|
PCRE when its main matching function, \fBpcre_exec()\fP, is used. |
31 |
|
From release 6.0, PCRE offers a second matching function, |
32 |
|
\fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not |
33 |
|
Perl-compatible. Some of the features discussed below are not available when |
34 |
|
\fBpcre_dfa_exec()\fP is used. The advantages and disadvantages of the |
35 |
|
alternative function, and how it differs from the normal function, are |
36 |
|
discussed in the |
37 |
|
.\" HREF |
38 |
|
\fBpcrematching\fP |
39 |
|
.\" |
40 |
|
page. |
41 |
|
. |
42 |
|
. |
43 |
|
.SH "CHARACTERS AND METACHARACTERS" |
44 |
|
.rs |
45 |
|
.sp |
46 |
A regular expression is a pattern that is matched against a subject string from |
A regular expression is a pattern that is matched against a subject string from |
47 |
left to right. Most characters stand for themselves in a pattern, and match the |
left to right. Most characters stand for themselves in a pattern, and match the |
48 |
corresponding characters in the subject. As a trivial example, the pattern |
corresponding characters in the subject. As a trivial example, the pattern |
49 |
|
.sp |
50 |
The quick brown fox |
The quick brown fox |
51 |
|
.sp |
52 |
matches a portion of a subject string that is identical to itself. The power of |
matches a portion of a subject string that is identical to itself. When |
53 |
regular expressions comes from the ability to include alternatives and |
caseless matching is specified (the PCRE_CASELESS option), letters are matched |
54 |
repetitions in the pattern. These are encoded in the pattern by the use of |
independently of case. In UTF-8 mode, PCRE always understands the concept of |
55 |
\fImeta-characters\fR, which do not stand for themselves but instead are |
case for characters whose values are less than 128, so caseless matching is |
56 |
|
always possible. For characters with higher values, the concept of case is |
57 |
|
supported if PCRE is compiled with Unicode property support, but not otherwise. |
58 |
|
If you want to use caseless matching for characters 128 and above, you must |
59 |
|
ensure that PCRE is compiled with Unicode property support as well as with |
60 |
|
UTF-8 support. |
61 |
|
.P |
62 |
|
The power of regular expressions comes from the ability to include alternatives |
63 |
|
and repetitions in the pattern. These are encoded in the pattern by the use of |
64 |
|
\fImetacharacters\fP, which do not stand for themselves but instead are |
65 |
interpreted in some special way. |
interpreted in some special way. |
66 |
|
.P |
67 |
There are two different sets of meta-characters: those that are recognized |
There are two different sets of metacharacters: those that are recognized |
68 |
anywhere in the pattern except within square brackets, and those that are |
anywhere in the pattern except within square brackets, and those that are |
69 |
recognized in square brackets. Outside square brackets, the meta-characters are |
recognized within square brackets. Outside square brackets, the metacharacters |
70 |
as follows: |
are as follows: |
71 |
|
.sp |
72 |
\\ general escape character with several uses |
\e general escape character with several uses |
73 |
^ assert start of string (or line, in multiline mode) |
^ assert start of string (or line, in multiline mode) |
74 |
$ assert end of string (or line, in multiline mode) |
$ assert end of string (or line, in multiline mode) |
75 |
. match any character except newline (by default) |
. match any character except newline (by default) |
84 |
+ 1 or more quantifier |
+ 1 or more quantifier |
85 |
also "possessive quantifier" |
also "possessive quantifier" |
86 |
{ start min/max quantifier |
{ start min/max quantifier |
87 |
|
.sp |
88 |
Part of a pattern that is in square brackets is called a "character class". In |
Part of a pattern that is in square brackets is called a "character class". In |
89 |
a character class the only meta-characters are: |
a character class the only metacharacters are: |
90 |
|
.sp |
91 |
\\ general escape character |
\e general escape character |
92 |
^ negate the class, but only if the first character |
^ negate the class, but only if the first character |
93 |
- indicates character range |
- indicates character range |
94 |
|
.\" JOIN |
95 |
[ POSIX character class (only if followed by POSIX |
[ POSIX character class (only if followed by POSIX |
96 |
syntax) |
syntax) |
97 |
] terminates the character class |
] terminates the character class |
98 |
|
.sp |
99 |
The following sections describe the use of each of the meta-characters. |
The following sections describe the use of each of the metacharacters. |
100 |
|
. |
101 |
|
. |
102 |
.SH BACKSLASH |
.SH BACKSLASH |
103 |
.rs |
.rs |
104 |
.sp |
.sp |
105 |
The backslash character has several uses. Firstly, if it is followed by a |
The backslash character has several uses. Firstly, if it is followed by a |
106 |
non-alphameric character, it takes away any special meaning that character may |
non-alphanumeric character, it takes away any special meaning that character |
107 |
have. This use of backslash as an escape character applies both inside and |
may have. This use of backslash as an escape character applies both inside and |
108 |
outside character classes. |
outside character classes. |
109 |
|
.P |
110 |
For example, if you want to match a * character, you write \\* in the pattern. |
For example, if you want to match a * character, you write \e* in the pattern. |
111 |
This escaping action applies whether or not the following character would |
This escaping action applies whether or not the following character would |
112 |
otherwise be interpreted as a meta-character, so it is always safe to precede a |
otherwise be interpreted as a metacharacter, so it is always safe to precede a |
113 |
non-alphameric with backslash to specify that it stands for itself. In |
non-alphanumeric with backslash to specify that it stands for itself. In |
114 |
particular, if you want to match a backslash, you write \\\\. |
particular, if you want to match a backslash, you write \e\e. |
115 |
|
.P |
116 |
If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the |
If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the |
117 |
pattern (other than in a character class) and characters between a # outside |
pattern (other than in a character class) and characters between a # outside |
118 |
a character class and the next newline character are ignored. An escaping |
a character class and the next newline are ignored. An escaping backslash can |
119 |
backslash can be used to include a whitespace or # character as part of the |
be used to include a whitespace or # character as part of the pattern. |
120 |
pattern. |
.P |
|
|
|
121 |
If you want to remove the special meaning from a sequence of characters, you |
If you want to remove the special meaning from a sequence of characters, you |
122 |
can do so by putting them between \\Q and \\E. This is different from Perl in |
can do so by putting them between \eQ and \eE. This is different from Perl in |
123 |
that $ and @ are handled as literals in \\Q...\\E sequences in PCRE, whereas in |
that $ and @ are handled as literals in \eQ...\eE sequences in PCRE, whereas in |
124 |
Perl, $ and @ cause variable interpolation. Note the following examples: |
Perl, $ and @ cause variable interpolation. Note the following examples: |
125 |
|
.sp |
126 |
Pattern PCRE matches Perl matches |
Pattern PCRE matches Perl matches |
127 |
|
.sp |
128 |
\\Qabc$xyz\\E abc$xyz abc followed by the |
.\" JOIN |
129 |
|
\eQabc$xyz\eE abc$xyz abc followed by the |
130 |
contents of $xyz |
contents of $xyz |
131 |
\\Qabc\\$xyz\\E abc\\$xyz abc\\$xyz |
\eQabc\e$xyz\eE abc\e$xyz abc\e$xyz |
132 |
\\Qabc\\E\\$\\Qxyz\\E abc$xyz abc$xyz |
\eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz |
133 |
|
.sp |
134 |
The \\Q...\\E sequence is recognized both inside and outside character classes. |
The \eQ...\eE sequence is recognized both inside and outside character classes. |
135 |
|
. |
136 |
|
. |
137 |
|
.\" HTML <a name="digitsafterbackslash"></a> |
138 |
|
.SS "Non-printing characters" |
139 |
|
.rs |
140 |
|
.sp |
141 |
A second use of backslash provides a way of encoding non-printing characters |
A second use of backslash provides a way of encoding non-printing characters |
142 |
in patterns in a visible manner. There is no restriction on the appearance of |
in patterns in a visible manner. There is no restriction on the appearance of |
143 |
non-printing characters, apart from the binary zero that terminates a pattern, |
non-printing characters, apart from the binary zero that terminates a pattern, |
144 |
but when a pattern is being prepared by text editing, it is usually easier to |
but when a pattern is being prepared by text editing, it is usually easier to |
145 |
use one of the following escape sequences than the binary character it |
use one of the following escape sequences than the binary character it |
146 |
represents: |
represents: |
147 |
|
.sp |
148 |
\\a alarm, that is, the BEL character (hex 07) |
\ea alarm, that is, the BEL character (hex 07) |
149 |
\\cx "control-x", where x is any character |
\ecx "control-x", where x is any character |
150 |
\\e escape (hex 1B) |
\ee escape (hex 1B) |
151 |
\\f formfeed (hex 0C) |
\ef formfeed (hex 0C) |
152 |
\\n newline (hex 0A) |
\en newline (hex 0A) |
153 |
\\r carriage return (hex 0D) |
\er carriage return (hex 0D) |
154 |
\\t tab (hex 09) |
\et tab (hex 09) |
155 |
\\ddd character with octal code ddd, or backreference |
\eddd character with octal code ddd, or backreference |
156 |
\\xhh character with hex code hh |
\exhh character with hex code hh |
157 |
\\x{hhh..} character with hex code hhh... (UTF-8 mode only) |
\ex{hhh..} character with hex code hhh.. |
158 |
|
.sp |
159 |
The precise effect of \\cx is as follows: if x is a lower case letter, it |
The precise effect of \ecx is as follows: if x is a lower case letter, it |
160 |
is converted to upper case. Then bit 6 of the character (hex 40) is inverted. |
is converted to upper case. Then bit 6 of the character (hex 40) is inverted. |
161 |
Thus \\cz becomes hex 1A, but \\c{ becomes hex 3B, while \\c; becomes hex |
Thus \ecz becomes hex 1A, but \ec{ becomes hex 3B, while \ec; becomes hex |
162 |
7B. |
7B. |
163 |
|
.P |
164 |
After \\x, from zero to two hexadecimal digits are read (letters can be in |
After \ex, from zero to two hexadecimal digits are read (letters can be in |
165 |
upper or lower case). In UTF-8 mode, any number of hexadecimal digits may |
upper or lower case). Any number of hexadecimal digits may appear between \ex{ |
166 |
appear between \\x{ and }, but the value of the character code must be less |
and }, but the value of the character code must be less than 256 in non-UTF-8 |
167 |
than 2**31 (that is, the maximum hexadecimal value is 7FFFFFFF). If characters |
mode, and less than 2**31 in UTF-8 mode (that is, the maximum hexadecimal value |
168 |
other than hexadecimal digits appear between \\x{ and }, or if there is no |
is 7FFFFFFF). If characters other than hexadecimal digits appear between \ex{ |
169 |
terminating }, this form of escape is not recognized. Instead, the initial |
and }, or if there is no terminating }, this form of escape is not recognized. |
170 |
\\x will be interpreted as a basic hexadecimal escape, with no following |
Instead, the initial \ex will be interpreted as a basic hexadecimal escape, |
171 |
digits, giving a byte whose value is zero. |
with no following digits, giving a character whose value is zero. |
172 |
|
.P |
173 |
Characters whose value is less than 256 can be defined by either of the two |
Characters whose value is less than 256 can be defined by either of the two |
174 |
syntaxes for \\x when PCRE is in UTF-8 mode. There is no difference in the |
syntaxes for \ex. There is no difference in the way they are handled. For |
175 |
way they are handled. For example, \\xdc is exactly the same as \\x{dc}. |
example, \exdc is exactly the same as \ex{dc}. |
176 |
|
.P |
177 |
After \\0 up to two further octal digits are read. In both cases, if there |
After \e0 up to two further octal digits are read. If there are fewer than two |
178 |
are fewer than two digits, just those that are present are used. Thus the |
digits, just those that are present are used. Thus the sequence \e0\ex\e07 |
179 |
sequence \\0\\x\\07 specifies two binary zeros followed by a BEL character |
specifies two binary zeros followed by a BEL character (code value 7). Make |
180 |
(code value 7). Make sure you supply two digits after the initial zero if the |
sure you supply two digits after the initial zero if the pattern character that |
181 |
character that follows is itself an octal digit. |
follows is itself an octal digit. |
182 |
|
.P |
183 |
The handling of a backslash followed by a digit other than 0 is complicated. |
The handling of a backslash followed by a digit other than 0 is complicated. |
184 |
Outside a character class, PCRE reads it and any following digits as a decimal |
Outside a character class, PCRE reads it and any following digits as a decimal |
185 |
number. If the number is less than 10, or if there have been at least that many |
number. If the number is less than 10, or if there have been at least that many |
186 |
previous capturing left parentheses in the expression, the entire sequence is |
previous capturing left parentheses in the expression, the entire sequence is |
187 |
taken as a \fIback reference\fR. A description of how this works is given |
taken as a \fIback reference\fP. A description of how this works is given |
188 |
later, following the discussion of parenthesized subpatterns. |
.\" HTML <a href="#backreferences"> |
189 |
|
.\" </a> |
190 |
|
later, |
191 |
|
.\" |
192 |
|
following the discussion of |
193 |
|
.\" HTML <a href="#subpattern"> |
194 |
|
.\" </a> |
195 |
|
parenthesized subpatterns. |
196 |
|
.\" |
197 |
|
.P |
198 |
Inside a character class, or if the decimal number is greater than 9 and there |
Inside a character class, or if the decimal number is greater than 9 and there |
199 |
have not been that many capturing subpatterns, PCRE re-reads up to three octal |
have not been that many capturing subpatterns, PCRE re-reads up to three octal |
200 |
digits following the backslash, and generates a single byte from the least |
digits following the backslash, and uses them to generate a data character. Any |
201 |
significant 8 bits of the value. Any subsequent digits stand for themselves. |
subsequent digits stand for themselves. In non-UTF-8 mode, the value of a |
202 |
For example: |
character specified in octal must be less than \e400. In UTF-8 mode, values up |
203 |
|
to \e777 are permitted. For example: |
204 |
\\040 is another way of writing a space |
.sp |
205 |
\\40 is the same, provided there are fewer than 40 |
\e040 is another way of writing a space |
206 |
|
.\" JOIN |
207 |
|
\e40 is the same, provided there are fewer than 40 |
208 |
previous capturing subpatterns |
previous capturing subpatterns |
209 |
\\7 is always a back reference |
\e7 is always a back reference |
210 |
\\11 might be a back reference, or another way of |
.\" JOIN |
211 |
|
\e11 might be a back reference, or another way of |
212 |
writing a tab |
writing a tab |
213 |
\\011 is always a tab |
\e011 is always a tab |
214 |
\\0113 is a tab followed by the character "3" |
\e0113 is a tab followed by the character "3" |
215 |
\\113 might be a back reference, otherwise the |
.\" JOIN |
216 |
|
\e113 might be a back reference, otherwise the |
217 |
character with octal code 113 |
character with octal code 113 |
218 |
\\377 might be a back reference, otherwise |
.\" JOIN |
219 |
|
\e377 might be a back reference, otherwise |
220 |
the byte consisting entirely of 1 bits |
the byte consisting entirely of 1 bits |
221 |
\\81 is either a back reference, or a binary zero |
.\" JOIN |
222 |
|
\e81 is either a back reference, or a binary zero |
223 |
followed by the two characters "8" and "1" |
followed by the two characters "8" and "1" |
224 |
|
.sp |
225 |
Note that octal values of 100 or greater must not be introduced by a leading |
Note that octal values of 100 or greater must not be introduced by a leading |
226 |
zero, because no more than three octal digits are ever read. |
zero, because no more than three octal digits are ever read. |
227 |
|
.P |
228 |
All the sequences that define a single byte value or a single UTF-8 character |
All the sequences that define a single character value can be used both inside |
229 |
(in UTF-8 mode) can be used both inside and outside character classes. In |
and outside character classes. In addition, inside a character class, the |
230 |
addition, inside a character class, the sequence \\b is interpreted as the |
sequence \eb is interpreted as the backspace character (hex 08), and the |
231 |
backspace character (hex 08). Outside a character class it has a different |
sequences \eR and \eX are interpreted as the characters "R" and "X", |
232 |
meaning (see below). |
respectively. Outside a character class, these sequences have different |
233 |
|
meanings |
234 |
The third use of backslash is for specifying generic character types: |
.\" HTML <a href="#uniextseq"> |
235 |
|
.\" </a> |
236 |
\\d any decimal digit |
(see below). |
237 |
\\D any character that is not a decimal digit |
.\" |
238 |
\\s any whitespace character |
. |
239 |
\\S any character that is not a whitespace character |
. |
240 |
\\w any "word" character |
.SS "Absolute and relative back references" |
241 |
\\W any "non-word" character |
.rs |
242 |
|
.sp |
243 |
|
The sequence \eg followed by a positive or negative number, optionally enclosed |
244 |
|
in braces, is an absolute or relative back reference. A named back reference |
245 |
|
can be coded as \eg{name}. Back references are discussed |
246 |
|
.\" HTML <a href="#backreferences"> |
247 |
|
.\" </a> |
248 |
|
later, |
249 |
|
.\" |
250 |
|
following the discussion of |
251 |
|
.\" HTML <a href="#subpattern"> |
252 |
|
.\" </a> |
253 |
|
parenthesized subpatterns. |
254 |
|
.\" |
255 |
|
. |
256 |
|
. |
257 |
|
.SS "Generic character types" |
258 |
|
.rs |
259 |
|
.sp |
260 |
|
Another use of backslash is for specifying generic character types. The |
261 |
|
following are always recognized: |
262 |
|
.sp |
263 |
|
\ed any decimal digit |
264 |
|
\eD any character that is not a decimal digit |
265 |
|
\eh any horizontal whitespace character |
266 |
|
\eH any character that is not a horizontal whitespace character |
267 |
|
\es any whitespace character |
268 |
|
\eS any character that is not a whitespace character |
269 |
|
\ev any vertical whitespace character |
270 |
|
\eV any character that is not a vertical whitespace character |
271 |
|
\ew any "word" character |
272 |
|
\eW any "non-word" character |
273 |
|
.sp |
274 |
Each pair of escape sequences partitions the complete set of characters into |
Each pair of escape sequences partitions the complete set of characters into |
275 |
two disjoint sets. Any given character matches one, and only one, of each pair. |
two disjoint sets. Any given character matches one, and only one, of each pair. |
276 |
|
.P |
277 |
In UTF-8 mode, characters with values greater than 255 never match \\d, \\s, or |
These character type sequences can appear both inside and outside character |
278 |
\\w, and always match \\D, \\S, and \\W. |
classes. They each match one character of the appropriate type. If the current |
279 |
|
matching point is at the end of the subject string, all of them fail, since |
280 |
For compatibility with Perl, \\s does not match the VT character (code 11). |
there is no character to match. |
281 |
This makes it different from the the POSIX "space" class. The \\s characters |
.P |
282 |
are HT (9), LF (10), FF (12), CR (13), and space (32). |
For compatibility with Perl, \es does not match the VT character (code 11). |
283 |
|
This makes it different from the the POSIX "space" class. The \es characters |
284 |
A "word" character is any letter or digit or the underscore character, that is, |
are HT (9), LF (10), FF (12), CR (13), and space (32). If "use locale;" is |
285 |
any character which can be part of a Perl "word". The definition of letters and |
included in a Perl script, \es may match the VT character. In PCRE, it never |
286 |
digits is controlled by PCRE's character tables, and may vary if locale- |
does. |
287 |
specific matching is taking place (see |
.P |
288 |
|
In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or |
289 |
|
\ew, and always match \eD, \eS, and \eW. This is true even when Unicode |
290 |
|
character property support is available. These sequences retain their original |
291 |
|
meanings from before UTF-8 support was available, mainly for efficiency |
292 |
|
reasons. |
293 |
|
.P |
294 |
|
The sequences \eh, \eH, \ev, and \eV are Perl 5.10 features. In contrast to the |
295 |
|
other sequences, these do match certain high-valued codepoints in UTF-8 mode. |
296 |
|
The horizontal space characters are: |
297 |
|
.sp |
298 |
|
U+0009 Horizontal tab |
299 |
|
U+0020 Space |
300 |
|
U+00A0 Non-break space |
301 |
|
U+1680 Ogham space mark |
302 |
|
U+180E Mongolian vowel separator |
303 |
|
U+2000 En quad |
304 |
|
U+2001 Em quad |
305 |
|
U+2002 En space |
306 |
|
U+2003 Em space |
307 |
|
U+2004 Three-per-em space |
308 |
|
U+2005 Four-per-em space |
309 |
|
U+2006 Six-per-em space |
310 |
|
U+2007 Figure space |
311 |
|
U+2008 Punctuation space |
312 |
|
U+2009 Thin space |
313 |
|
U+200A Hair space |
314 |
|
U+202F Narrow no-break space |
315 |
|
U+205F Medium mathematical space |
316 |
|
U+3000 Ideographic space |
317 |
|
.sp |
318 |
|
The vertical space characters are: |
319 |
|
.sp |
320 |
|
U+000A Linefeed |
321 |
|
U+000B Vertical tab |
322 |
|
U+000C Formfeed |
323 |
|
U+000D Carriage return |
324 |
|
U+0085 Next line |
325 |
|
U+2028 Line separator |
326 |
|
U+2029 Paragraph separator |
327 |
|
.P |
328 |
|
A "word" character is an underscore or any character less than 256 that is a |
329 |
|
letter or digit. The definition of letters and digits is controlled by PCRE's |
330 |
|
low-valued character tables, and may vary if locale-specific matching is taking |
331 |
|
place (see |
332 |
.\" HTML <a href="pcreapi.html#localesupport"> |
.\" HTML <a href="pcreapi.html#localesupport"> |
333 |
.\" </a> |
.\" </a> |
334 |
"Locale support" |
"Locale support" |
335 |
.\" |
.\" |
336 |
in the |
in the |
337 |
.\" HREF |
.\" HREF |
338 |
\fBpcreapi\fR |
\fBpcreapi\fP |
339 |
.\" |
.\" |
340 |
page). For example, in the "fr" (French) locale, some character codes greater |
page). For example, in a French locale such as "fr_FR" in Unix-like systems, |
341 |
than 128 are used for accented letters, and these are matched by \\w. |
or "french" in Windows, some character codes greater than 128 are used for |
342 |
|
accented letters, and these are matched by \ew. The use of locales with Unicode |
343 |
These character type sequences can appear both inside and outside character |
is discouraged. |
344 |
classes. They each match one character of the appropriate type. If the current |
. |
345 |
matching point is at the end of the subject string, all of them fail, since |
. |
346 |
there is no character to match. |
.SS "Newline sequences" |
347 |
|
.rs |
348 |
The fourth use of backslash is for certain simple assertions. An assertion |
.sp |
349 |
|
Outside a character class, the escape sequence \eR matches any Unicode newline |
350 |
|
sequence. This is a Perl 5.10 feature. In non-UTF-8 mode \eR is equivalent to |
351 |
|
the following: |
352 |
|
.sp |
353 |
|
(?>\er\en|\en|\ex0b|\ef|\er|\ex85) |
354 |
|
.sp |
355 |
|
This is an example of an "atomic group", details of which are given |
356 |
|
.\" HTML <a href="#atomicgroup"> |
357 |
|
.\" </a> |
358 |
|
below. |
359 |
|
.\" |
360 |
|
This particular group matches either the two-character sequence CR followed by |
361 |
|
LF, or one of the single characters LF (linefeed, U+000A), VT (vertical tab, |
362 |
|
U+000B), FF (formfeed, U+000C), CR (carriage return, U+000D), or NEL (next |
363 |
|
line, U+0085). The two-character sequence is treated as a single unit that |
364 |
|
cannot be split. |
365 |
|
.P |
366 |
|
In UTF-8 mode, two additional characters whose codepoints are greater than 255 |
367 |
|
are added: LS (line separator, U+2028) and PS (paragraph separator, U+2029). |
368 |
|
Unicode character property support is not needed for these characters to be |
369 |
|
recognized. |
370 |
|
.P |
371 |
|
Inside a character class, \eR matches the letter "R". |
372 |
|
. |
373 |
|
. |
374 |
|
.\" HTML <a name="uniextseq"></a> |
375 |
|
.SS Unicode character properties |
376 |
|
.rs |
377 |
|
.sp |
378 |
|
When PCRE is built with Unicode character property support, three additional |
379 |
|
escape sequences to match character properties are available when UTF-8 mode |
380 |
|
is selected. They are: |
381 |
|
.sp |
382 |
|
\ep{\fIxx\fP} a character with the \fIxx\fP property |
383 |
|
\eP{\fIxx\fP} a character without the \fIxx\fP property |
384 |
|
\eX an extended Unicode sequence |
385 |
|
.sp |
386 |
|
The property names represented by \fIxx\fP above are limited to the Unicode |
387 |
|
script names, the general category properties, and "Any", which matches any |
388 |
|
character (including newline). Other properties such as "InMusicalSymbols" are |
389 |
|
not currently supported by PCRE. Note that \eP{Any} does not match any |
390 |
|
characters, so always causes a match failure. |
391 |
|
.P |
392 |
|
Sets of Unicode characters are defined as belonging to certain scripts. A |
393 |
|
character from one of these sets can be matched using a script name. For |
394 |
|
example: |
395 |
|
.sp |
396 |
|
\ep{Greek} |
397 |
|
\eP{Han} |
398 |
|
.sp |
399 |
|
Those that are not part of an identified script are lumped together as |
400 |
|
"Common". The current list of scripts is: |
401 |
|
.P |
402 |
|
Arabic, |
403 |
|
Armenian, |
404 |
|
Balinese, |
405 |
|
Bengali, |
406 |
|
Bopomofo, |
407 |
|
Braille, |
408 |
|
Buginese, |
409 |
|
Buhid, |
410 |
|
Canadian_Aboriginal, |
411 |
|
Cherokee, |
412 |
|
Common, |
413 |
|
Coptic, |
414 |
|
Cuneiform, |
415 |
|
Cypriot, |
416 |
|
Cyrillic, |
417 |
|
Deseret, |
418 |
|
Devanagari, |
419 |
|
Ethiopic, |
420 |
|
Georgian, |
421 |
|
Glagolitic, |
422 |
|
Gothic, |
423 |
|
Greek, |
424 |
|
Gujarati, |
425 |
|
Gurmukhi, |
426 |
|
Han, |
427 |
|
Hangul, |
428 |
|
Hanunoo, |
429 |
|
Hebrew, |
430 |
|
Hiragana, |
431 |
|
Inherited, |
432 |
|
Kannada, |
433 |
|
Katakana, |
434 |
|
Kharoshthi, |
435 |
|
Khmer, |
436 |
|
Lao, |
437 |
|
Latin, |
438 |
|
Limbu, |
439 |
|
Linear_B, |
440 |
|
Malayalam, |
441 |
|
Mongolian, |
442 |
|
Myanmar, |
443 |
|
New_Tai_Lue, |
444 |
|
Nko, |
445 |
|
Ogham, |
446 |
|
Old_Italic, |
447 |
|
Old_Persian, |
448 |
|
Oriya, |
449 |
|
Osmanya, |
450 |
|
Phags_Pa, |
451 |
|
Phoenician, |
452 |
|
Runic, |
453 |
|
Shavian, |
454 |
|
Sinhala, |
455 |
|
Syloti_Nagri, |
456 |
|
Syriac, |
457 |
|
Tagalog, |
458 |
|
Tagbanwa, |
459 |
|
Tai_Le, |
460 |
|
Tamil, |
461 |
|
Telugu, |
462 |
|
Thaana, |
463 |
|
Thai, |
464 |
|
Tibetan, |
465 |
|
Tifinagh, |
466 |
|
Ugaritic, |
467 |
|
Yi. |
468 |
|
.P |
469 |
|
Each character has exactly one general category property, specified by a |
470 |
|
two-letter abbreviation. For compatibility with Perl, negation can be specified |
471 |
|
by including a circumflex between the opening brace and the property name. For |
472 |
|
example, \ep{^Lu} is the same as \eP{Lu}. |
473 |
|
.P |
474 |
|
If only one letter is specified with \ep or \eP, it includes all the general |
475 |
|
category properties that start with that letter. In this case, in the absence |
476 |
|
of negation, the curly brackets in the escape sequence are optional; these two |
477 |
|
examples have the same effect: |
478 |
|
.sp |
479 |
|
\ep{L} |
480 |
|
\epL |
481 |
|
.sp |
482 |
|
The following general category property codes are supported: |
483 |
|
.sp |
484 |
|
C Other |
485 |
|
Cc Control |
486 |
|
Cf Format |
487 |
|
Cn Unassigned |
488 |
|
Co Private use |
489 |
|
Cs Surrogate |
490 |
|
.sp |
491 |
|
L Letter |
492 |
|
Ll Lower case letter |
493 |
|
Lm Modifier letter |
494 |
|
Lo Other letter |
495 |
|
Lt Title case letter |
496 |
|
Lu Upper case letter |
497 |
|
.sp |
498 |
|
M Mark |
499 |
|
Mc Spacing mark |
500 |
|
Me Enclosing mark |
501 |
|
Mn Non-spacing mark |
502 |
|
.sp |
503 |
|
N Number |
504 |
|
Nd Decimal number |
505 |
|
Nl Letter number |
506 |
|
No Other number |
507 |
|
.sp |
508 |
|
P Punctuation |
509 |
|
Pc Connector punctuation |
510 |
|
Pd Dash punctuation |
511 |
|
Pe Close punctuation |
512 |
|
Pf Final punctuation |
513 |
|
Pi Initial punctuation |
514 |
|
Po Other punctuation |
515 |
|
Ps Open punctuation |
516 |
|
.sp |
517 |
|
S Symbol |
518 |
|
Sc Currency symbol |
519 |
|
Sk Modifier symbol |
520 |
|
Sm Mathematical symbol |
521 |
|
So Other symbol |
522 |
|
.sp |
523 |
|
Z Separator |
524 |
|
Zl Line separator |
525 |
|
Zp Paragraph separator |
526 |
|
Zs Space separator |
527 |
|
.sp |
528 |
|
The special property L& is also supported: it matches a character that has |
529 |
|
the Lu, Ll, or Lt property, in other words, a letter that is not classified as |
530 |
|
a modifier or "other". |
531 |
|
.P |
532 |
|
The long synonyms for these properties that Perl supports (such as \ep{Letter}) |
533 |
|
are not supported by PCRE, nor is it permitted to prefix any of these |
534 |
|
properties with "Is". |
535 |
|
.P |
536 |
|
No character that is in the Unicode table has the Cn (unassigned) property. |
537 |
|
Instead, this property is assumed for any code point that is not in the |
538 |
|
Unicode table. |
539 |
|
.P |
540 |
|
Specifying caseless matching does not affect these escape sequences. For |
541 |
|
example, \ep{Lu} always matches only upper case letters. |
542 |
|
.P |
543 |
|
The \eX escape matches any number of Unicode characters that form an extended |
544 |
|
Unicode sequence. \eX is equivalent to |
545 |
|
.sp |
546 |
|
(?>\ePM\epM*) |
547 |
|
.sp |
548 |
|
That is, it matches a character without the "mark" property, followed by zero |
549 |
|
or more characters with the "mark" property, and treats the sequence as an |
550 |
|
atomic group |
551 |
|
.\" HTML <a href="#atomicgroup"> |
552 |
|
.\" </a> |
553 |
|
(see below). |
554 |
|
.\" |
555 |
|
Characters with the "mark" property are typically accents that affect the |
556 |
|
preceding character. |
557 |
|
.P |
558 |
|
Matching characters by Unicode property is not fast, because PCRE has to search |
559 |
|
a structure that contains data for over fifteen thousand characters. That is |
560 |
|
why the traditional escape sequences such as \ed and \ew do not use Unicode |
561 |
|
properties in PCRE. |
562 |
|
. |
563 |
|
. |
564 |
|
.\" HTML <a name="resetmatchstart"></a> |
565 |
|
.SS "Resetting the match start" |
566 |
|
.rs |
567 |
|
.sp |
568 |
|
The escape sequence \eK, which is a Perl 5.10 feature, causes any previously |
569 |
|
matched characters not to be included in the final matched sequence. For |
570 |
|
example, the pattern: |
571 |
|
.sp |
572 |
|
foo\eKbar |
573 |
|
.sp |
574 |
|
matches "foobar", but reports that it has matched "bar". This feature is |
575 |
|
similar to a lookbehind assertion |
576 |
|
.\" HTML <a href="#lookbehind"> |
577 |
|
.\" </a> |
578 |
|
(described below). |
579 |
|
.\" |
580 |
|
However, in this case, the part of the subject before the real match does not |
581 |
|
have to be of fixed length, as lookbehind assertions do. The use of \eK does |
582 |
|
not interfere with the setting of |
583 |
|
.\" HTML <a href="#subpattern"> |
584 |
|
.\" </a> |
585 |
|
captured substrings. |
586 |
|
.\" |
587 |
|
For example, when the pattern |
588 |
|
.sp |
589 |
|
(foo)\eKbar |
590 |
|
.sp |
591 |
|
matches "foobar", the first substring is still set to "foo". |
592 |
|
. |
593 |
|
. |
594 |
|
.\" HTML <a name="smallassertions"></a> |
595 |
|
.SS "Simple assertions" |
596 |
|
.rs |
597 |
|
.sp |
598 |
|
The final use of backslash is for certain simple assertions. An assertion |
599 |
specifies a condition that has to be met at a particular point in a match, |
specifies a condition that has to be met at a particular point in a match, |
600 |
without consuming any characters from the subject string. The use of |
without consuming any characters from the subject string. The use of |
601 |
subpatterns for more complicated assertions is described below. The backslashed |
subpatterns for more complicated assertions is described |
602 |
assertions are |
.\" HTML <a href="#bigassertions"> |
603 |
|
.\" </a> |
604 |
\\b matches at a word boundary |
below. |
605 |
\\B matches when not at a word boundary |
.\" |
606 |
\\A matches at start of subject |
The backslashed assertions are: |
607 |
\\Z matches at end of subject or before newline at end |
.sp |
608 |
\\z matches at end of subject |
\eb matches at a word boundary |
609 |
\\G matches at first matching position in subject |
\eB matches when not at a word boundary |
610 |
|
\eA matches at the start of the subject |
611 |
These assertions may not appear in character classes (but note that \\b has a |
\eZ matches at the end of the subject |
612 |
|
also matches before a newline at the end of the subject |
613 |
|
\ez matches only at the end of the subject |
614 |
|
\eG matches at the first matching position in the subject |
615 |
|
.sp |
616 |
|
These assertions may not appear in character classes (but note that \eb has a |
617 |
different meaning, namely the backspace character, inside a character class). |
different meaning, namely the backspace character, inside a character class). |
618 |
|
.P |
619 |
A word boundary is a position in the subject string where the current character |
A word boundary is a position in the subject string where the current character |
620 |
and the previous character do not both match \\w or \\W (i.e. one matches |
and the previous character do not both match \ew or \eW (i.e. one matches |
621 |
\\w and the other matches \\W), or the start or end of the string if the |
\ew and the other matches \eW), or the start or end of the string if the |
622 |
first or last character matches \\w, respectively. |
first or last character matches \ew, respectively. |
623 |
|
.P |
624 |
The \\A, \\Z, and \\z assertions differ from the traditional circumflex and |
The \eA, \eZ, and \ez assertions differ from the traditional circumflex and |
625 |
dollar (described below) in that they only ever match at the very start and end |
dollar (described in the next section) in that they only ever match at the very |
626 |
of the subject string, whatever options are set. Thus, they are independent of |
start and end of the subject string, whatever options are set. Thus, they are |
627 |
multiline mode. |
independent of multiline mode. These three assertions are not affected by the |
628 |
|
PCRE_NOTBOL or PCRE_NOTEOL options, which affect only the behaviour of the |
629 |
They are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options. If the |
circumflex and dollar metacharacters. However, if the \fIstartoffset\fP |
630 |
\fIstartoffset\fR argument of \fBpcre_exec()\fR is non-zero, indicating that |
argument of \fBpcre_exec()\fP is non-zero, indicating that matching is to start |
631 |
matching is to start at a point other than the beginning of the subject, \\A |
at a point other than the beginning of the subject, \eA can never match. The |
632 |
can never match. The difference between \\Z and \\z is that \\Z matches before |
difference between \eZ and \ez is that \eZ matches before a newline at the end |
633 |
a newline that is the last character of the string as well as at the end of the |
of the string as well as at the very end, whereas \ez matches only at the end. |
634 |
string, whereas \\z matches only at the end. |
.P |
635 |
|
The \eG assertion is true only when the current matching position is at the |
636 |
The \\G assertion is true only when the current matching position is at the |
start point of the match, as specified by the \fIstartoffset\fP argument of |
637 |
start point of the match, as specified by the \fIstartoffset\fR argument of |
\fBpcre_exec()\fP. It differs from \eA when the value of \fIstartoffset\fP is |
638 |
\fBpcre_exec()\fR. It differs from \\A when the value of \fIstartoffset\fR is |
non-zero. By calling \fBpcre_exec()\fP multiple times with appropriate |
|
non-zero. By calling \fBpcre_exec()\fR multiple times with appropriate |
|
639 |
arguments, you can mimic Perl's /g option, and it is in this kind of |
arguments, you can mimic Perl's /g option, and it is in this kind of |
640 |
implementation where \\G can be useful. |
implementation where \eG can be useful. |
641 |
|
.P |
642 |
Note, however, that PCRE's interpretation of \\G, as the start of the current |
Note, however, that PCRE's interpretation of \eG, as the start of the current |
643 |
match, is subtly different from Perl's, which defines it as the end of the |
match, is subtly different from Perl's, which defines it as the end of the |
644 |
previous match. In Perl, these can be different when the previously matched |
previous match. In Perl, these can be different when the previously matched |
645 |
string was empty. Because PCRE does just one match at a time, it cannot |
string was empty. Because PCRE does just one match at a time, it cannot |
646 |
reproduce this behaviour. |
reproduce this behaviour. |
647 |
|
.P |
648 |
If all the alternatives of a pattern begin with \\G, the expression is anchored |
If all the alternatives of a pattern begin with \eG, the expression is anchored |
649 |
to the starting match position, and the "anchored" flag is set in the compiled |
to the starting match position, and the "anchored" flag is set in the compiled |
650 |
regular expression. |
regular expression. |
651 |
|
. |
652 |
.SH CIRCUMFLEX AND DOLLAR |
. |
653 |
|
.SH "CIRCUMFLEX AND DOLLAR" |
654 |
.rs |
.rs |
655 |
.sp |
.sp |
656 |
Outside a character class, in the default matching mode, the circumflex |
Outside a character class, in the default matching mode, the circumflex |
657 |
character is an assertion which is true only if the current matching point is |
character is an assertion that is true only if the current matching point is |
658 |
at the start of the subject string. If the \fIstartoffset\fR argument of |
at the start of the subject string. If the \fIstartoffset\fP argument of |
659 |
\fBpcre_exec()\fR is non-zero, circumflex can never match if the PCRE_MULTILINE |
\fBpcre_exec()\fP is non-zero, circumflex can never match if the PCRE_MULTILINE |
660 |
option is unset. Inside a character class, circumflex has an entirely different |
option is unset. Inside a character class, circumflex has an entirely different |
661 |
meaning (see below). |
meaning |
662 |
|
.\" HTML <a href="#characterclass"> |
663 |
|
.\" </a> |
664 |
|
(see below). |
665 |
|
.\" |
666 |
|
.P |
667 |
Circumflex need not be the first character of the pattern if a number of |
Circumflex need not be the first character of the pattern if a number of |
668 |
alternatives are involved, but it should be the first thing in each alternative |
alternatives are involved, but it should be the first thing in each alternative |
669 |
in which it appears if the pattern is ever to match that branch. If all |
in which it appears if the pattern is ever to match that branch. If all |
671 |
constrained to match only at the start of the subject, it is said to be an |
constrained to match only at the start of the subject, it is said to be an |
672 |
"anchored" pattern. (There are also other constructs that can cause a pattern |
"anchored" pattern. (There are also other constructs that can cause a pattern |
673 |
to be anchored.) |
to be anchored.) |
674 |
|
.P |
675 |
A dollar character is an assertion which is true only if the current matching |
A dollar character is an assertion that is true only if the current matching |
676 |
point is at the end of the subject string, or immediately before a newline |
point is at the end of the subject string, or immediately before a newline |
677 |
character that is the last character in the string (by default). Dollar need |
at the end of the string (by default). Dollar need not be the last character of |
678 |
not be the last character of the pattern if a number of alternatives are |
the pattern if a number of alternatives are involved, but it should be the last |
679 |
involved, but it should be the last item in any branch in which it appears. |
item in any branch in which it appears. Dollar has no special meaning in a |
680 |
Dollar has no special meaning in a character class. |
character class. |
681 |
|
.P |
682 |
The meaning of dollar can be changed so that it matches only at the very end of |
The meaning of dollar can be changed so that it matches only at the very end of |
683 |
the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This |
the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This |
684 |
does not affect the \\Z assertion. |
does not affect the \eZ assertion. |
685 |
|
.P |
686 |
The meanings of the circumflex and dollar characters are changed if the |
The meanings of the circumflex and dollar characters are changed if the |
687 |
PCRE_MULTILINE option is set. When this is the case, they match immediately |
PCRE_MULTILINE option is set. When this is the case, a circumflex matches |
688 |
after and immediately before an internal newline character, respectively, in |
immediately after internal newlines as well as at the start of the subject |
689 |
addition to matching at the start and end of the subject string. For example, |
string. It does not match after a newline that ends the string. A dollar |
690 |
the pattern /^abc$/ matches the subject string "def\\nabc" in multiline mode, |
matches before any newlines in the string, as well as at the very end, when |
691 |
but not otherwise. Consequently, patterns that are anchored in single line mode |
PCRE_MULTILINE is set. When newline is specified as the two-character |
692 |
because all branches start with ^ are not anchored in multiline mode, and a |
sequence CRLF, isolated CR and LF characters do not indicate newlines. |
693 |
match for circumflex is possible when the \fIstartoffset\fR argument of |
.P |
694 |
\fBpcre_exec()\fR is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if |
For example, the pattern /^abc$/ matches the subject string "def\enabc" (where |
695 |
PCRE_MULTILINE is set. |
\en represents a newline) in multiline mode, but not otherwise. Consequently, |
696 |
|
patterns that are anchored in single line mode because all branches start with |
697 |
Note that the sequences \\A, \\Z, and \\z can be used to match the start and |
^ are not anchored in multiline mode, and a match for circumflex is possible |
698 |
|
when the \fIstartoffset\fP argument of \fBpcre_exec()\fP is non-zero. The |
699 |
|
PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is set. |
700 |
|
.P |
701 |
|
Note that the sequences \eA, \eZ, and \ez can be used to match the start and |
702 |
end of the subject in both modes, and if all branches of a pattern start with |
end of the subject in both modes, and if all branches of a pattern start with |
703 |
\\A it is always anchored, whether PCRE_MULTILINE is set or not. |
\eA it is always anchored, whether or not PCRE_MULTILINE is set. |
704 |
|
. |
705 |
.SH FULL STOP (PERIOD, DOT) |
. |
706 |
|
.SH "FULL STOP (PERIOD, DOT)" |
707 |
.rs |
.rs |
708 |
.sp |
.sp |
709 |
Outside a character class, a dot in the pattern matches any one character in |
Outside a character class, a dot in the pattern matches any one character in |
710 |
the subject, including a non-printing character, but not (by default) newline. |
the subject string except (by default) a character that signifies the end of a |
711 |
In UTF-8 mode, a dot matches any UTF-8 character, which might be more than one |
line. In UTF-8 mode, the matched character may be more than one byte long. |
712 |
byte long, except (by default) for newline. If the PCRE_DOTALL option is set, |
.P |
713 |
dots match newlines as well. The handling of dot is entirely independent of the |
When a line ending is defined as a single character, dot never matches that |
714 |
handling of circumflex and dollar, the only relationship being that they both |
character; when the two-character sequence CRLF is used, dot does not match CR |
715 |
involve newline characters. Dot has no special meaning in a character class. |
if it is immediately followed by LF, but otherwise it matches all characters |
716 |
|
(including isolated CRs and LFs). When any Unicode line endings are being |
717 |
.SH MATCHING A SINGLE BYTE |
recognized, dot does not match CR or LF or any of the other line ending |
718 |
.rs |
characters. |
719 |
.sp |
.P |
720 |
Outside a character class, the escape sequence \\C matches any one byte, both |
The behaviour of dot with regard to newlines can be changed. If the PCRE_DOTALL |
721 |
in and out of UTF-8 mode. Unlike a dot, it always matches a newline. The |
option is set, a dot matches any one character, without exception. If the |
722 |
feature is provided in Perl in order to match individual bytes in UTF-8 mode. |
two-character sequence CRLF is present in the subject string, it takes two dots |
723 |
Because it breaks up UTF-8 characters into individual bytes, what remains in |
to match it. |
724 |
the string may be a malformed UTF-8 string. For this reason it is best avoided. |
.P |
725 |
|
The handling of dot is entirely independent of the handling of circumflex and |
726 |
PCRE does not allow \\C to appear in lookbehind assertions (see below), because |
dollar, the only relationship being that they both involve newlines. Dot has no |
727 |
in UTF-8 mode it makes it impossible to calculate the length of the lookbehind. |
special meaning in a character class. |
728 |
|
. |
729 |
.SH SQUARE BRACKETS |
. |
730 |
|
.SH "MATCHING A SINGLE BYTE" |
731 |
|
.rs |
732 |
|
.sp |
733 |
|
Outside a character class, the escape sequence \eC matches any one byte, both |
734 |
|
in and out of UTF-8 mode. Unlike a dot, it always matches any line-ending |
735 |
|
characters. The feature is provided in Perl in order to match individual bytes |
736 |
|
in UTF-8 mode. Because it breaks up UTF-8 characters into individual bytes, |
737 |
|
what remains in the string may be a malformed UTF-8 string. For this reason, |
738 |
|
the \eC escape sequence is best avoided. |
739 |
|
.P |
740 |
|
PCRE does not allow \eC to appear in lookbehind assertions |
741 |
|
.\" HTML <a href="#lookbehind"> |
742 |
|
.\" </a> |
743 |
|
(described below), |
744 |
|
.\" |
745 |
|
because in UTF-8 mode this would make it impossible to calculate the length of |
746 |
|
the lookbehind. |
747 |
|
. |
748 |
|
. |
749 |
|
.\" HTML <a name="characterclass"></a> |
750 |
|
.SH "SQUARE BRACKETS AND CHARACTER CLASSES" |
751 |
.rs |
.rs |
752 |
.sp |
.sp |
753 |
An opening square bracket introduces a character class, terminated by a closing |
An opening square bracket introduces a character class, terminated by a closing |
755 |
closing square bracket is required as a member of the class, it should be the |
closing square bracket is required as a member of the class, it should be the |
756 |
first data character in the class (after an initial circumflex, if present) or |
first data character in the class (after an initial circumflex, if present) or |
757 |
escaped with a backslash. |
escaped with a backslash. |
758 |
|
.P |
759 |
A character class matches a single character in the subject. In UTF-8 mode, the |
A character class matches a single character in the subject. In UTF-8 mode, the |
760 |
character may occupy more than one byte. A matched character must be in the set |
character may occupy more than one byte. A matched character must be in the set |
761 |
of characters defined by the class, unless the first character in the class |
of characters defined by the class, unless the first character in the class |
763 |
the set defined by the class. If a circumflex is actually required as a member |
the set defined by the class. If a circumflex is actually required as a member |
764 |
of the class, ensure it is not the first character, or escape it with a |
of the class, ensure it is not the first character, or escape it with a |
765 |
backslash. |
backslash. |
766 |
|
.P |
767 |
For example, the character class [aeiou] matches any lower case vowel, while |
For example, the character class [aeiou] matches any lower case vowel, while |
768 |
[^aeiou] matches any character that is not a lower case vowel. Note that a |
[^aeiou] matches any character that is not a lower case vowel. Note that a |
769 |
circumflex is just a convenient notation for specifying the characters which |
circumflex is just a convenient notation for specifying the characters that |
770 |
are in the class by enumerating those that are not. It is not an assertion: it |
are in the class by enumerating those that are not. A class that starts with a |
771 |
still consumes a character from the subject string, and fails if the current |
circumflex is not an assertion: it still consumes a character from the subject |
772 |
pointer is at the end of the string. |
string, and therefore it fails if the current pointer is at the end of the |
773 |
|
string. |
774 |
|
.P |
775 |
In UTF-8 mode, characters with values greater than 255 can be included in a |
In UTF-8 mode, characters with values greater than 255 can be included in a |
776 |
class as a literal string of bytes, or by using the \\x{ escaping mechanism. |
class as a literal string of bytes, or by using the \ex{ escaping mechanism. |
777 |
|
.P |
778 |
When caseless matching is set, any letters in a class represent both their |
When caseless matching is set, any letters in a class represent both their |
779 |
upper case and lower case versions, so for example, a caseless [aeiou] matches |
upper case and lower case versions, so for example, a caseless [aeiou] matches |
780 |
"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a |
"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a |
781 |
caseful version would. PCRE does not support the concept of case for characters |
caseful version would. In UTF-8 mode, PCRE always understands the concept of |
782 |
with values greater than 255. |
case for characters whose values are less than 128, so caseless matching is |
783 |
|
always possible. For characters with higher values, the concept of case is |
784 |
The newline character is never treated in any special way in character classes, |
supported if PCRE is compiled with Unicode property support, but not otherwise. |
785 |
whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class |
If you want to use caseless matching for characters 128 and above, you must |
786 |
such as [^a] will always match a newline. |
ensure that PCRE is compiled with Unicode property support as well as with |
787 |
|
UTF-8 support. |
788 |
|
.P |
789 |
|
Characters that might indicate line breaks are never treated in any special way |
790 |
|
when matching character classes, whatever line-ending sequence is in use, and |
791 |
|
whatever setting of the PCRE_DOTALL and PCRE_MULTILINE options is used. A class |
792 |
|
such as [^a] always matches one of these characters. |
793 |
|
.P |
794 |
The minus (hyphen) character can be used to specify a range of characters in a |
The minus (hyphen) character can be used to specify a range of characters in a |
795 |
character class. For example, [d-m] matches any letter between d and m, |
character class. For example, [d-m] matches any letter between d and m, |
796 |
inclusive. If a minus character is required in a class, it must be escaped with |
inclusive. If a minus character is required in a class, it must be escaped with |
797 |
a backslash or appear in a position where it cannot be interpreted as |
a backslash or appear in a position where it cannot be interpreted as |
798 |
indicating a range, typically as the first or last character in the class. |
indicating a range, typically as the first or last character in the class. |
799 |
|
.P |
800 |
It is not possible to have the literal character "]" as the end character of a |
It is not possible to have the literal character "]" as the end character of a |
801 |
range. A pattern such as [W-]46] is interpreted as a class of two characters |
range. A pattern such as [W-]46] is interpreted as a class of two characters |
802 |
("W" and "-") followed by a literal string "46]", so it would match "W46]" or |
("W" and "-") followed by a literal string "46]", so it would match "W46]" or |
803 |
"-46]". However, if the "]" is escaped with a backslash it is interpreted as |
"-46]". However, if the "]" is escaped with a backslash it is interpreted as |
804 |
the end of range, so [W-\\]46] is interpreted as a single class containing a |
the end of range, so [W-\e]46] is interpreted as a class containing a range |
805 |
range followed by two separate characters. The octal or hexadecimal |
followed by two other characters. The octal or hexadecimal representation of |
806 |
representation of "]" can also be used to end a range. |
"]" can also be used to end a range. |
807 |
|
.P |
808 |
Ranges operate in the collating sequence of character values. They can also be |
Ranges operate in the collating sequence of character values. They can also be |
809 |
used for characters specified numerically, for example [\\000-\\037]. In UTF-8 |
used for characters specified numerically, for example [\e000-\e037]. In UTF-8 |
810 |
mode, ranges can include characters whose values are greater than 255, for |
mode, ranges can include characters whose values are greater than 255, for |
811 |
example [\\x{100}-\\x{2ff}]. |
example [\ex{100}-\ex{2ff}]. |
812 |
|
.P |
813 |
If a range that includes letters is used when caseless matching is set, it |
If a range that includes letters is used when caseless matching is set, it |
814 |
matches the letters in either case. For example, [W-c] is equivalent to |
matches the letters in either case. For example, [W-c] is equivalent to |
815 |
[][\\^_`wxyzabc], matched caselessly, and if character tables for the "fr" |
[][\e\e^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if character |
816 |
locale are in use, [\\xc8-\\xcb] matches accented E characters in both cases. |
tables for a French locale are in use, [\exc8-\excb] matches accented E |
817 |
|
characters in both cases. In UTF-8 mode, PCRE supports the concept of case for |
818 |
The character types \\d, \\D, \\s, \\S, \\w, and \\W may also appear in a |
characters with values greater than 128 only when it is compiled with Unicode |
819 |
character class, and add the characters that they match to the class. For |
property support. |
820 |
example, [\\dABCDEF] matches any hexadecimal digit. A circumflex can |
.P |
821 |
|
The character types \ed, \eD, \ep, \eP, \es, \eS, \ew, and \eW may also appear |
822 |
|
in a character class, and add the characters that they match to the class. For |
823 |
|
example, [\edABCDEF] matches any hexadecimal digit. A circumflex can |
824 |
conveniently be used with the upper case character types to specify a more |
conveniently be used with the upper case character types to specify a more |
825 |
restricted set of characters than the matching lower case type. For example, |
restricted set of characters than the matching lower case type. For example, |
826 |
the class [^\\W_] matches any letter or digit, but not underscore. |
the class [^\eW_] matches any letter or digit, but not underscore. |
827 |
|
.P |
828 |
All non-alphameric characters other than \\, -, ^ (at the start) and the |
The only metacharacters that are recognized in character classes are backslash, |
829 |
terminating ] are non-special in character classes, but it does no harm if they |
hyphen (only where it can be interpreted as specifying a range), circumflex |
830 |
are escaped. |
(only at the start), opening square bracket (only when it can be interpreted as |
831 |
|
introducing a POSIX class name - see the next section), and the terminating |
832 |
.SH POSIX CHARACTER CLASSES |
closing square bracket. However, escaping other non-alphanumeric characters |
833 |
|
does no harm. |
834 |
|
. |
835 |
|
. |
836 |
|
.SH "POSIX CHARACTER CLASSES" |
837 |
.rs |
.rs |
838 |
.sp |
.sp |
839 |
Perl supports the POSIX notation for character classes, which uses names |
Perl supports the POSIX notation for character classes. This uses names |
840 |
enclosed by [: and :] within the enclosing square brackets. PCRE also supports |
enclosed by [: and :] within the enclosing square brackets. PCRE also supports |
841 |
this notation. For example, |
this notation. For example, |
842 |
|
.sp |
843 |
[01[:alpha:]%] |
[01[:alpha:]%] |
844 |
|
.sp |
845 |
matches "0", "1", any alphabetic character, or "%". The supported class names |
matches "0", "1", any alphabetic character, or "%". The supported class names |
846 |
are |
are |
847 |
|
.sp |
848 |
alnum letters and digits |
alnum letters and digits |
849 |
alpha letters |
alpha letters |
850 |
ascii character codes 0 - 127 |
ascii character codes 0 - 127 |
851 |
blank space or tab only |
blank space or tab only |
852 |
cntrl control characters |
cntrl control characters |
853 |
digit decimal digits (same as \\d) |
digit decimal digits (same as \ed) |
854 |
graph printing characters, excluding space |
graph printing characters, excluding space |
855 |
lower lower case letters |
lower lower case letters |
856 |
print printing characters, including space |
print printing characters, including space |
857 |
punct printing characters, excluding letters and digits |
punct printing characters, excluding letters and digits |
858 |
space white space (not quite the same as \\s) |
space white space (not quite the same as \es) |
859 |
upper upper case letters |
upper upper case letters |
860 |
word "word" characters (same as \\w) |
word "word" characters (same as \ew) |
861 |
xdigit hexadecimal digits |
xdigit hexadecimal digits |
862 |
|
.sp |
863 |
The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and |
The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and |
864 |
space (32). Notice that this list includes the VT character (code 11). This |
space (32). Notice that this list includes the VT character (code 11). This |
865 |
makes "space" different to \\s, which does not include VT (for Perl |
makes "space" different to \es, which does not include VT (for Perl |
866 |
compatibility). |
compatibility). |
867 |
|
.P |
868 |
The name "word" is a Perl extension, and "blank" is a GNU extension from Perl |
The name "word" is a Perl extension, and "blank" is a GNU extension from Perl |
869 |
5.8. Another Perl extension is negation, which is indicated by a ^ character |
5.8. Another Perl extension is negation, which is indicated by a ^ character |
870 |
after the colon. For example, |
after the colon. For example, |
871 |
|
.sp |
872 |
[12[:^digit:]] |
[12[:^digit:]] |
873 |
|
.sp |
874 |
matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX |
matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX |
875 |
syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not |
syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not |
876 |
supported, and an error is given if they are encountered. |
supported, and an error is given if they are encountered. |
877 |
|
.P |
878 |
In UTF-8 mode, characters with values greater than 255 do not match any of |
In UTF-8 mode, characters with values greater than 128 do not match any of |
879 |
the POSIX character classes. |
the POSIX character classes. |
880 |
|
. |
881 |
.SH VERTICAL BAR |
. |
882 |
|
.SH "VERTICAL BAR" |
883 |
.rs |
.rs |
884 |
.sp |
.sp |
885 |
Vertical bar characters are used to separate alternative patterns. For example, |
Vertical bar characters are used to separate alternative patterns. For example, |
886 |
the pattern |
the pattern |
887 |
|
.sp |
888 |
gilbert|sullivan |
gilbert|sullivan |
889 |
|
.sp |
890 |
matches either "gilbert" or "sullivan". Any number of alternatives may appear, |
matches either "gilbert" or "sullivan". Any number of alternatives may appear, |
891 |
and an empty alternative is permitted (matching the empty string). |
and an empty alternative is permitted (matching the empty string). The matching |
892 |
The matching process tries each alternative in turn, from left to right, |
process tries each alternative in turn, from left to right, and the first one |
893 |
and the first one that succeeds is used. If the alternatives are within a |
that succeeds is used. If the alternatives are within a subpattern |
894 |
subpattern (defined below), "succeeds" means matching the rest of the main |
.\" HTML <a href="#subpattern"> |
895 |
pattern as well as the alternative in the subpattern. |
.\" </a> |
896 |
|
(defined below), |
897 |
.SH INTERNAL OPTION SETTING |
.\" |
898 |
|
"succeeds" means matching the rest of the main pattern as well as the |
899 |
|
alternative in the subpattern. |
900 |
|
. |
901 |
|
. |
902 |
|
.SH "INTERNAL OPTION SETTING" |
903 |
.rs |
.rs |
904 |
.sp |
.sp |
905 |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
906 |
PCRE_EXTENDED options can be changed from within the pattern by a sequence of |
PCRE_EXTENDED options can be changed from within the pattern by a sequence of |
907 |
Perl option letters enclosed between "(?" and ")". The option letters are |
Perl option letters enclosed between "(?" and ")". The option letters are |
908 |
|
.sp |
909 |
i for PCRE_CASELESS |
i for PCRE_CASELESS |
910 |
m for PCRE_MULTILINE |
m for PCRE_MULTILINE |
911 |
s for PCRE_DOTALL |
s for PCRE_DOTALL |
912 |
x for PCRE_EXTENDED |
x for PCRE_EXTENDED |
913 |
|
.sp |
914 |
For example, (?im) sets caseless, multiline matching. It is also possible to |
For example, (?im) sets caseless, multiline matching. It is also possible to |
915 |
unset these options by preceding the letter with a hyphen, and a combined |
unset these options by preceding the letter with a hyphen, and a combined |
916 |
setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and |
setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and |
917 |
PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also |
PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also |
918 |
permitted. If a letter appears both before and after the hyphen, the option is |
permitted. If a letter appears both before and after the hyphen, the option is |
919 |
unset. |
unset. |
920 |
|
.P |
921 |
When an option change occurs at top level (that is, not inside subpattern |
When an option change occurs at top level (that is, not inside subpattern |
922 |
parentheses), the change applies to the remainder of the pattern that follows. |
parentheses), the change applies to the remainder of the pattern that follows. |
923 |
If the change is placed right at the start of a pattern, PCRE extracts it into |
If the change is placed right at the start of a pattern, PCRE extracts it into |
924 |
the global options (and it will therefore show up in data extracted by the |
the global options (and it will therefore show up in data extracted by the |
925 |
\fBpcre_fullinfo()\fR function). |
\fBpcre_fullinfo()\fP function). |
926 |
|
.P |
927 |
An option change within a subpattern affects only that part of the current |
An option change within a subpattern (see below for a description of |
928 |
pattern that follows it, so |
subpatterns) affects only that part of the current pattern that follows it, so |
929 |
|
.sp |
930 |
(a(?i)b)c |
(a(?i)b)c |
931 |
|
.sp |
932 |
matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). |
matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). |
933 |
By this means, options can be made to have different settings in different |
By this means, options can be made to have different settings in different |
934 |
parts of the pattern. Any changes made in one alternative do carry on |
parts of the pattern. Any changes made in one alternative do carry on |
935 |
into subsequent branches within the same subpattern. For example, |
into subsequent branches within the same subpattern. For example, |
936 |
|
.sp |
937 |
(a(?i)b|c) |
(a(?i)b|c) |
938 |
|
.sp |
939 |
matches "ab", "aB", "c", and "C", even though when matching "C" the first |
matches "ab", "aB", "c", and "C", even though when matching "C" the first |
940 |
branch is abandoned before the option setting. This is because the effects of |
branch is abandoned before the option setting. This is because the effects of |
941 |
option settings happen at compile time. There would be some very weird |
option settings happen at compile time. There would be some very weird |
942 |
behaviour otherwise. |
behaviour otherwise. |
943 |
|
.P |
944 |
The PCRE-specific options PCRE_UNGREEDY and PCRE_EXTRA can be changed in the |
The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be |
945 |
same way as the Perl-compatible options by using the characters U and X |
changed in the same way as the Perl-compatible options by using the characters |
946 |
respectively. The (?X) flag setting is special in that it must always occur |
J, U and X respectively. |
947 |
earlier in the pattern than any of the additional features it turns on, even |
. |
948 |
when it is at top level. It is best put at the start. |
. |
949 |
|
.\" HTML <a name="subpattern"></a> |
950 |
.SH SUBPATTERNS |
.SH SUBPATTERNS |
951 |
.rs |
.rs |
952 |
.sp |
.sp |
953 |
Subpatterns are delimited by parentheses (round brackets), which can be nested. |
Subpatterns are delimited by parentheses (round brackets), which can be nested. |
954 |
Marking part of a pattern as a subpattern does two things: |
Turning part of a pattern into a subpattern does two things: |
955 |
|
.sp |
956 |
1. It localizes a set of alternatives. For example, the pattern |
1. It localizes a set of alternatives. For example, the pattern |
957 |
|
.sp |
958 |
cat(aract|erpillar|) |
cat(aract|erpillar|) |
959 |
|
.sp |
960 |
matches one of the words "cat", "cataract", or "caterpillar". Without the |
matches one of the words "cat", "cataract", or "caterpillar". Without the |
961 |
parentheses, it would match "cataract", "erpillar" or the empty string. |
parentheses, it would match "cataract", "erpillar" or an empty string. |
962 |
|
.sp |
963 |
2. It sets up the subpattern as a capturing subpattern (as defined above). |
2. It sets up the subpattern as a capturing subpattern. This means that, when |
964 |
When the whole pattern matches, that portion of the subject string that matched |
the whole pattern matches, that portion of the subject string that matched the |
965 |
the subpattern is passed back to the caller via the \fIovector\fR argument of |
subpattern is passed back to the caller via the \fIovector\fP argument of |
966 |
\fBpcre_exec()\fR. Opening parentheses are counted from left to right (starting |
\fBpcre_exec()\fP. Opening parentheses are counted from left to right (starting |
967 |
from 1) to obtain the numbers of the capturing subpatterns. |
from 1) to obtain numbers for the capturing subpatterns. |
968 |
|
.P |
969 |
For example, if the string "the red king" is matched against the pattern |
For example, if the string "the red king" is matched against the pattern |
970 |
|
.sp |
971 |
the ((red|white) (king|queen)) |
the ((red|white) (king|queen)) |
972 |
|
.sp |
973 |
the captured substrings are "red king", "red", and "king", and are numbered 1, |
the captured substrings are "red king", "red", and "king", and are numbered 1, |
974 |
2, and 3, respectively. |
2, and 3, respectively. |
975 |
|
.P |
976 |
The fact that plain parentheses fulfil two functions is not always helpful. |
The fact that plain parentheses fulfil two functions is not always helpful. |
977 |
There are often times when a grouping subpattern is required without a |
There are often times when a grouping subpattern is required without a |
978 |
capturing requirement. If an opening parenthesis is followed by a question mark |
capturing requirement. If an opening parenthesis is followed by a question mark |
979 |
and a colon, the subpattern does not do any capturing, and is not counted when |
and a colon, the subpattern does not do any capturing, and is not counted when |
980 |
computing the number of any subsequent capturing subpatterns. For example, if |
computing the number of any subsequent capturing subpatterns. For example, if |
981 |
the string "the white queen" is matched against the pattern |
the string "the white queen" is matched against the pattern |
982 |
|
.sp |
983 |
the ((?:red|white) (king|queen)) |
the ((?:red|white) (king|queen)) |
984 |
|
.sp |
985 |
the captured substrings are "white queen" and "queen", and are numbered 1 and |
the captured substrings are "white queen" and "queen", and are numbered 1 and |
986 |
2. The maximum number of capturing subpatterns is 65535, and the maximum depth |
2. The maximum number of capturing subpatterns is 65535. |
987 |
of nesting of all subpatterns, both capturing and non-capturing, is 200. |
.P |
|
|
|
988 |
As a convenient shorthand, if any option settings are required at the start of |
As a convenient shorthand, if any option settings are required at the start of |
989 |
a non-capturing subpattern, the option letters may appear between the "?" and |
a non-capturing subpattern, the option letters may appear between the "?" and |
990 |
the ":". Thus the two patterns |
the ":". Thus the two patterns |
991 |
|
.sp |
992 |
(?i:saturday|sunday) |
(?i:saturday|sunday) |
993 |
(?:(?i)saturday|sunday) |
(?:(?i)saturday|sunday) |
994 |
|
.sp |
995 |
match exactly the same set of strings. Because alternative branches are tried |
match exactly the same set of strings. Because alternative branches are tried |
996 |
from left to right, and options are not reset until the end of the subpattern |
from left to right, and options are not reset until the end of the subpattern |
997 |
is reached, an option setting in one branch does affect subsequent branches, so |
is reached, an option setting in one branch does affect subsequent branches, so |
998 |
the above patterns match "SUNDAY" as well as "Saturday". |
the above patterns match "SUNDAY" as well as "Saturday". |
999 |
|
. |
1000 |
.SH NAMED SUBPATTERNS |
. |
1001 |
|
.SH "DUPLICATE SUBPATTERN NUMBERS" |
1002 |
|
.rs |
1003 |
|
.sp |
1004 |
|
Perl 5.10 introduced a feature whereby each alternative in a subpattern uses |
1005 |
|
the same numbers for its capturing parentheses. Such a subpattern starts with |
1006 |
|
(?| and is itself a non-capturing subpattern. For example, consider this |
1007 |
|
pattern: |
1008 |
|
.sp |
1009 |
|
(?|(Sat)ur|(Sun))day |
1010 |
|
.sp |
1011 |
|
Because the two alternatives are inside a (?| group, both sets of capturing |
1012 |
|
parentheses are numbered one. Thus, when the pattern matches, you can look |
1013 |
|
at captured substring number one, whichever alternative matched. This construct |
1014 |
|
is useful when you want to capture part, but not all, of one of a number of |
1015 |
|
alternatives. Inside a (?| group, parentheses are numbered as usual, but the |
1016 |
|
number is reset at the start of each branch. The numbers of any capturing |
1017 |
|
buffers that follow the subpattern start after the highest number used in any |
1018 |
|
branch. The following example is taken from the Perl documentation. |
1019 |
|
The numbers underneath show in which buffer the captured content will be |
1020 |
|
stored. |
1021 |
|
.sp |
1022 |
|
# before ---------------branch-reset----------- after |
1023 |
|
/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x |
1024 |
|
# 1 2 2 3 2 3 4 |
1025 |
|
.sp |
1026 |
|
A backreference or a recursive call to a numbered subpattern always refers to |
1027 |
|
the first one in the pattern with the given number. |
1028 |
|
.P |
1029 |
|
An alternative approach to using this "branch reset" feature is to use |
1030 |
|
duplicate named subpatterns, as described in the next section. |
1031 |
|
. |
1032 |
|
. |
1033 |
|
.SH "NAMED SUBPATTERNS" |
1034 |
.rs |
.rs |
1035 |
.sp |
.sp |
1036 |
Identifying capturing parentheses by number is simple, but it can be very hard |
Identifying capturing parentheses by number is simple, but it can be very hard |
1037 |
to keep track of the numbers in complicated regular expressions. Furthermore, |
to keep track of the numbers in complicated regular expressions. Furthermore, |
1038 |
if an expression is modified, the numbers may change. To help with the |
if an expression is modified, the numbers may change. To help with this |
1039 |
difficulty, PCRE supports the naming of subpatterns, something that Perl does |
difficulty, PCRE supports the naming of subpatterns. This feature was not |
1040 |
not provide. The Python syntax (?P<name>...) is used. Names consist of |
added to Perl until release 5.10. Python had the feature earlier, and PCRE |
1041 |
alphanumeric characters and underscores, and must be unique within a pattern. |
introduced it at release 4.0, using the Python syntax. PCRE now supports both |
1042 |
|
the Perl and the Python syntax. |
1043 |
Named capturing parentheses are still allocated numbers as well as names. The |
.P |
1044 |
PCRE API provides function calls for extracting the name-to-number translation |
In PCRE, a subpattern can be named in one of three ways: (?<name>...) or |
1045 |
table from a compiled pattern. For further details see the |
(?'name'...) as in Perl, or (?P<name>...) as in Python. References to capturing |
1046 |
|
parentheses from other parts of the pattern, such as |
1047 |
|
.\" HTML <a href="#backreferences"> |
1048 |
|
.\" </a> |
1049 |
|
backreferences, |
1050 |
|
.\" |
1051 |
|
.\" HTML <a href="#recursion"> |
1052 |
|
.\" </a> |
1053 |
|
recursion, |
1054 |
|
.\" |
1055 |
|
and |
1056 |
|
.\" HTML <a href="#conditions"> |
1057 |
|
.\" </a> |
1058 |
|
conditions, |
1059 |
|
.\" |
1060 |
|
can be made by name as well as by number. |
1061 |
|
.P |
1062 |
|
Names consist of up to 32 alphanumeric characters and underscores. Named |
1063 |
|
capturing parentheses are still allocated numbers as well as names, exactly as |
1064 |
|
if the names were not present. The PCRE API provides function calls for |
1065 |
|
extracting the name-to-number translation table from a compiled pattern. There |
1066 |
|
is also a convenience function for extracting a captured substring by name. |
1067 |
|
.P |
1068 |
|
By default, a name must be unique within a pattern, but it is possible to relax |
1069 |
|
this constraint by setting the PCRE_DUPNAMES option at compile time. This can |
1070 |
|
be useful for patterns where only one instance of the named parentheses can |
1071 |
|
match. Suppose you want to match the name of a weekday, either as a 3-letter |
1072 |
|
abbreviation or as the full name, and in both cases you want to extract the |
1073 |
|
abbreviation. This pattern (ignoring the line breaks) does the job: |
1074 |
|
.sp |
1075 |
|
(?<DN>Mon|Fri|Sun)(?:day)?| |
1076 |
|
(?<DN>Tue)(?:sday)?| |
1077 |
|
(?<DN>Wed)(?:nesday)?| |
1078 |
|
(?<DN>Thu)(?:rsday)?| |
1079 |
|
(?<DN>Sat)(?:urday)? |
1080 |
|
.sp |
1081 |
|
There are five capturing substrings, but only one is ever set after a match. |
1082 |
|
(An alternative way of solving this problem is to use a "branch reset" |
1083 |
|
subpattern, as described in the previous section.) |
1084 |
|
.P |
1085 |
|
The convenience function for extracting the data by name returns the substring |
1086 |
|
for the first (and in this example, the only) subpattern of that name that |
1087 |
|
matched. This saves searching to find which numbered subpattern it was. If you |
1088 |
|
make a reference to a non-unique named subpattern from elsewhere in the |
1089 |
|
pattern, the one that corresponds to the lowest number is used. For further |
1090 |
|
details of the interfaces for handling named subpatterns, see the |
1091 |
.\" HREF |
.\" HREF |
1092 |
\fBpcreapi\fR |
\fBpcreapi\fP |
1093 |
.\" |
.\" |
1094 |
documentation. |
documentation. |
1095 |
|
. |
1096 |
|
. |
1097 |
.SH REPETITION |
.SH REPETITION |
1098 |
.rs |
.rs |
1099 |
.sp |
.sp |
1100 |
Repetition is specified by quantifiers, which can follow any of the following |
Repetition is specified by quantifiers, which can follow any of the following |
1101 |
items: |
items: |
1102 |
|
.sp |
1103 |
a literal data character |
a literal data character |
1104 |
the . metacharacter |
the dot metacharacter |
1105 |
the \\C escape sequence |
the \eC escape sequence |
1106 |
escapes such as \\d that match single characters |
the \eX escape sequence (in UTF-8 mode with Unicode properties) |
1107 |
|
the \eR escape sequence |
1108 |
|
an escape such as \ed that matches a single character |
1109 |
a character class |
a character class |
1110 |
a back reference (see next section) |
a back reference (see next section) |
1111 |
a parenthesized subpattern (unless it is an assertion) |
a parenthesized subpattern (unless it is an assertion) |
1112 |
|
.sp |
1113 |
The general repetition quantifier specifies a minimum and maximum number of |
The general repetition quantifier specifies a minimum and maximum number of |
1114 |
permitted matches, by giving the two numbers in curly brackets (braces), |
permitted matches, by giving the two numbers in curly brackets (braces), |
1115 |
separated by a comma. The numbers must be less than 65536, and the first must |
separated by a comma. The numbers must be less than 65536, and the first must |
1116 |
be less than or equal to the second. For example: |
be less than or equal to the second. For example: |
1117 |
|
.sp |
1118 |
z{2,4} |
z{2,4} |
1119 |
|
.sp |
1120 |
matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special |
matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special |
1121 |
character. If the second number is omitted, but the comma is present, there is |
character. If the second number is omitted, but the comma is present, there is |
1122 |
no upper limit; if the second number and the comma are both omitted, the |
no upper limit; if the second number and the comma are both omitted, the |
1123 |
quantifier specifies an exact number of required matches. Thus |
quantifier specifies an exact number of required matches. Thus |
1124 |
|
.sp |
1125 |
[aeiou]{3,} |
[aeiou]{3,} |
1126 |
|
.sp |
1127 |
matches at least 3 successive vowels, but may match many more, while |
matches at least 3 successive vowels, but may match many more, while |
1128 |
|
.sp |
1129 |
\\d{8} |
\ed{8} |
1130 |
|
.sp |
1131 |
matches exactly 8 digits. An opening curly bracket that appears in a position |
matches exactly 8 digits. An opening curly bracket that appears in a position |
1132 |
where a quantifier is not allowed, or one that does not match the syntax of a |
where a quantifier is not allowed, or one that does not match the syntax of a |
1133 |
quantifier, is taken as a literal character. For example, {,6} is not a |
quantifier, is taken as a literal character. For example, {,6} is not a |
1134 |
quantifier, but a literal string of four characters. |
quantifier, but a literal string of four characters. |
1135 |
|
.P |
1136 |
In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual |
In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual |
1137 |
bytes. Thus, for example, \\x{100}{2} matches two UTF-8 characters, each of |
bytes. Thus, for example, \ex{100}{2} matches two UTF-8 characters, each of |
1138 |
which is represented by a two-byte sequence. |
which is represented by a two-byte sequence. Similarly, when Unicode property |
1139 |
|
support is available, \eX{3} matches three Unicode extended sequences, each of |
1140 |
|
which may be several bytes long (and they may be of different lengths). |
1141 |
|
.P |
1142 |
The quantifier {0} is permitted, causing the expression to behave as if the |
The quantifier {0} is permitted, causing the expression to behave as if the |
1143 |
previous item and the quantifier were not present. |
previous item and the quantifier were not present. |
1144 |
|
.P |
1145 |
For convenience (and historical compatibility) the three most common |
For convenience, the three most common quantifiers have single-character |
1146 |
quantifiers have single-character abbreviations: |
abbreviations: |
1147 |
|
.sp |
1148 |
* is equivalent to {0,} |
* is equivalent to {0,} |
1149 |
+ is equivalent to {1,} |
+ is equivalent to {1,} |
1150 |
? is equivalent to {0,1} |
? is equivalent to {0,1} |
1151 |
|
.sp |
1152 |
It is possible to construct infinite loops by following a subpattern that can |
It is possible to construct infinite loops by following a subpattern that can |
1153 |
match no characters with a quantifier that has no upper limit, for example: |
match no characters with a quantifier that has no upper limit, for example: |
1154 |
|
.sp |
1155 |
(a?)* |
(a?)* |
1156 |
|
.sp |
1157 |
Earlier versions of Perl and PCRE used to give an error at compile time for |
Earlier versions of Perl and PCRE used to give an error at compile time for |
1158 |
such patterns. However, because there are cases where this can be useful, such |
such patterns. However, because there are cases where this can be useful, such |
1159 |
patterns are now accepted, but if any repetition of the subpattern does in fact |
patterns are now accepted, but if any repetition of the subpattern does in fact |
1160 |
match no characters, the loop is forcibly broken. |
match no characters, the loop is forcibly broken. |
1161 |
|
.P |
1162 |
By default, the quantifiers are "greedy", that is, they match as much as |
By default, the quantifiers are "greedy", that is, they match as much as |
1163 |
possible (up to the maximum number of permitted times), without causing the |
possible (up to the maximum number of permitted times), without causing the |
1164 |
rest of the pattern to fail. The classic example of where this gives problems |
rest of the pattern to fail. The classic example of where this gives problems |
1165 |
is in trying to match comments in C programs. These appear between the |
is in trying to match comments in C programs. These appear between /* and */ |
1166 |
sequences /* and */ and within the sequence, individual * and / characters may |
and within the comment, individual * and / characters may appear. An attempt to |
1167 |
appear. An attempt to match C comments by applying the pattern |
match C comments by applying the pattern |
1168 |
|
.sp |
1169 |
/\\*.*\\*/ |
/\e*.*\e*/ |
1170 |
|
.sp |
1171 |
to the string |
to the string |
1172 |
|
.sp |
1173 |
/* first command */ not comment /* second comment */ |
/* first comment */ not comment /* second comment */ |
1174 |
|
.sp |
1175 |
fails, because it matches the entire string owing to the greediness of the .* |
fails, because it matches the entire string owing to the greediness of the .* |
1176 |
item. |
item. |
1177 |
|
.P |
1178 |
However, if a quantifier is followed by a question mark, it ceases to be |
However, if a quantifier is followed by a question mark, it ceases to be |
1179 |
greedy, and instead matches the minimum number of times possible, so the |
greedy, and instead matches the minimum number of times possible, so the |
1180 |
pattern |
pattern |
1181 |
|
.sp |
1182 |
/\\*.*?\\*/ |
/\e*.*?\e*/ |
1183 |
|
.sp |
1184 |
does the right thing with the C comments. The meaning of the various |
does the right thing with the C comments. The meaning of the various |
1185 |
quantifiers is not otherwise changed, just the preferred number of matches. |
quantifiers is not otherwise changed, just the preferred number of matches. |
1186 |
Do not confuse this use of question mark with its use as a quantifier in its |
Do not confuse this use of question mark with its use as a quantifier in its |
1187 |
own right. Because it has two uses, it can sometimes appear doubled, as in |
own right. Because it has two uses, it can sometimes appear doubled, as in |
1188 |
|
.sp |
1189 |
\\d??\\d |
\ed??\ed |
1190 |
|
.sp |
1191 |
which matches one digit by preference, but can match two if that is the only |
which matches one digit by preference, but can match two if that is the only |
1192 |
way the rest of the pattern matches. |
way the rest of the pattern matches. |
1193 |
|
.P |
1194 |
If the PCRE_UNGREEDY option is set (an option which is not available in Perl), |
If the PCRE_UNGREEDY option is set (an option that is not available in Perl), |
1195 |
the quantifiers are not greedy by default, but individual ones can be made |
the quantifiers are not greedy by default, but individual ones can be made |
1196 |
greedy by following them with a question mark. In other words, it inverts the |
greedy by following them with a question mark. In other words, it inverts the |
1197 |
default behaviour. |
default behaviour. |
1198 |
|
.P |
1199 |
When a parenthesized subpattern is quantified with a minimum repeat count that |
When a parenthesized subpattern is quantified with a minimum repeat count that |
1200 |
is greater than 1 or with a limited maximum, more store is required for the |
is greater than 1 or with a limited maximum, more memory is required for the |
1201 |
compiled pattern, in proportion to the size of the minimum or maximum. |
compiled pattern, in proportion to the size of the minimum or maximum. |
1202 |
|
.P |
1203 |
If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent |
If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent |
1204 |
to Perl's /s) is set, thus allowing the . to match newlines, the pattern is |
to Perl's /s) is set, thus allowing the dot to match newlines, the pattern is |
1205 |
implicitly anchored, because whatever follows will be tried against every |
implicitly anchored, because whatever follows will be tried against every |
1206 |
character position in the subject string, so there is no point in retrying the |
character position in the subject string, so there is no point in retrying the |
1207 |
overall match at any position after the first. PCRE normally treats such a |
overall match at any position after the first. PCRE normally treats such a |
1208 |
pattern as though it were preceded by \\A. |
pattern as though it were preceded by \eA. |
1209 |
|
.P |
1210 |
In cases where it is known that the subject string contains no newlines, it is |
In cases where it is known that the subject string contains no newlines, it is |
1211 |
worth setting PCRE_DOTALL in order to obtain this optimization, or |
worth setting PCRE_DOTALL in order to obtain this optimization, or |
1212 |
alternatively using ^ to indicate anchoring explicitly. |
alternatively using ^ to indicate anchoring explicitly. |
1213 |
|
.P |
1214 |
However, there is one situation where the optimization cannot be used. When .* |
However, there is one situation where the optimization cannot be used. When .* |
1215 |
is inside capturing parentheses that are the subject of a backreference |
is inside capturing parentheses that are the subject of a backreference |
1216 |
elsewhere in the pattern, a match at the start may fail, and a later one |
elsewhere in the pattern, a match at the start may fail where a later one |
1217 |
succeed. Consider, for example: |
succeeds. Consider, for example: |
1218 |
|
.sp |
1219 |
(.*)abc\\1 |
(.*)abc\e1 |
1220 |
|
.sp |
1221 |
If the subject is "xyz123abc123" the match point is the fourth character. For |
If the subject is "xyz123abc123" the match point is the fourth character. For |
1222 |
this reason, such a pattern is not implicitly anchored. |
this reason, such a pattern is not implicitly anchored. |
1223 |
|
.P |
1224 |
When a capturing subpattern is repeated, the value captured is the substring |
When a capturing subpattern is repeated, the value captured is the substring |
1225 |
that matched the final iteration. For example, after |
that matched the final iteration. For example, after |
1226 |
|
.sp |
1227 |
(tweedle[dume]{3}\\s*)+ |
(tweedle[dume]{3}\es*)+ |
1228 |
|
.sp |
1229 |
has matched "tweedledum tweedledee" the value of the captured substring is |
has matched "tweedledum tweedledee" the value of the captured substring is |
1230 |
"tweedledee". However, if there are nested capturing subpatterns, the |
"tweedledee". However, if there are nested capturing subpatterns, the |
1231 |
corresponding captured values may have been set in previous iterations. For |
corresponding captured values may have been set in previous iterations. For |
1232 |
example, after |
example, after |
1233 |
|
.sp |
1234 |
/(a|(b))+/ |
/(a|(b))+/ |
1235 |
|
.sp |
1236 |
matches "aba" the value of the second captured substring is "b". |
matches "aba" the value of the second captured substring is "b". |
1237 |
|
. |
1238 |
.SH ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS |
. |
1239 |
|
.\" HTML <a name="atomicgroup"></a> |
1240 |
|
.SH "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS" |
1241 |
.rs |
.rs |
1242 |
.sp |
.sp |
1243 |
With both maximizing and minimizing repetition, failure of what follows |
With both maximizing ("greedy") and minimizing ("ungreedy" or "lazy") |
1244 |
normally causes the repeated item to be re-evaluated to see if a different |
repetition, failure of what follows normally causes the repeated item to be |
1245 |
number of repeats allows the rest of the pattern to match. Sometimes it is |
re-evaluated to see if a different number of repeats allows the rest of the |
1246 |
useful to prevent this, either to change the nature of the match, or to cause |
pattern to match. Sometimes it is useful to prevent this, either to change the |
1247 |
it fail earlier than it otherwise might, when the author of the pattern knows |
nature of the match, or to cause it fail earlier than it otherwise might, when |
1248 |
there is no point in carrying on. |
the author of the pattern knows there is no point in carrying on. |
1249 |
|
.P |
1250 |
Consider, for example, the pattern \\d+foo when applied to the subject line |
Consider, for example, the pattern \ed+foo when applied to the subject line |
1251 |
|
.sp |
1252 |
123456bar |
123456bar |
1253 |
|
.sp |
1254 |
After matching all 6 digits and then failing to match "foo", the normal |
After matching all 6 digits and then failing to match "foo", the normal |
1255 |
action of the matcher is to try again with only 5 digits matching the \\d+ |
action of the matcher is to try again with only 5 digits matching the \ed+ |
1256 |
item, and then with 4, and so on, before ultimately failing. "Atomic grouping" |
item, and then with 4, and so on, before ultimately failing. "Atomic grouping" |
1257 |
(a term taken from Jeffrey Friedl's book) provides the means for specifying |
(a term taken from Jeffrey Friedl's book) provides the means for specifying |
1258 |
that once a subpattern has matched, it is not to be re-evaluated in this way. |
that once a subpattern has matched, it is not to be re-evaluated in this way. |
1259 |
|
.P |
1260 |
If we use atomic grouping for the previous example, the matcher would give up |
If we use atomic grouping for the previous example, the matcher gives up |
1261 |
immediately on failing to match "foo" the first time. The notation is a kind of |
immediately on failing to match "foo" the first time. The notation is a kind of |
1262 |
special parenthesis, starting with (?> as in this example: |
special parenthesis, starting with (?> as in this example: |
1263 |
|
.sp |
1264 |
(?>\\d+)foo |
(?>\ed+)foo |
1265 |
|
.sp |
1266 |
This kind of parenthesis "locks up" the part of the pattern it contains once |
This kind of parenthesis "locks up" the part of the pattern it contains once |
1267 |
it has matched, and a failure further into the pattern is prevented from |
it has matched, and a failure further into the pattern is prevented from |
1268 |
backtracking into it. Backtracking past it to previous items, however, works as |
backtracking into it. Backtracking past it to previous items, however, works as |
1269 |
normal. |
normal. |
1270 |
|
.P |
1271 |
An alternative description is that a subpattern of this type matches the string |
An alternative description is that a subpattern of this type matches the string |
1272 |
of characters that an identical standalone pattern would match, if anchored at |
of characters that an identical standalone pattern would match, if anchored at |
1273 |
the current point in the subject string. |
the current point in the subject string. |
1274 |
|
.P |
1275 |
Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as |
Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as |
1276 |
the above example can be thought of as a maximizing repeat that must swallow |
the above example can be thought of as a maximizing repeat that must swallow |
1277 |
everything it can. So, while both \\d+ and \\d+? are prepared to adjust the |
everything it can. So, while both \ed+ and \ed+? are prepared to adjust the |
1278 |
number of digits they match in order to make the rest of the pattern match, |
number of digits they match in order to make the rest of the pattern match, |
1279 |
(?>\\d+) can only match an entire sequence of digits. |
(?>\ed+) can only match an entire sequence of digits. |
1280 |
|
.P |
1281 |
Atomic groups in general can of course contain arbitrarily complicated |
Atomic groups in general can of course contain arbitrarily complicated |
1282 |
subpatterns, and can be nested. However, when the subpattern for an atomic |
subpatterns, and can be nested. However, when the subpattern for an atomic |
1283 |
group is just a single repeated item, as in the example above, a simpler |
group is just a single repeated item, as in the example above, a simpler |
1284 |
notation, called a "possessive quantifier" can be used. This consists of an |
notation, called a "possessive quantifier" can be used. This consists of an |
1285 |
additional + character following a quantifier. Using this notation, the |
additional + character following a quantifier. Using this notation, the |
1286 |
previous example can be rewritten as |
previous example can be rewritten as |
1287 |
|
.sp |
1288 |
\\d++bar |
\ed++foo |
1289 |
|
.sp |
1290 |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
1291 |
option is ignored. They are a convenient notation for the simpler forms of |
option is ignored. They are a convenient notation for the simpler forms of |
1292 |
atomic group. However, there is no difference in the meaning or processing of a |
atomic group. However, there is no difference in the meaning of a possessive |
1293 |
possessive quantifier and the equivalent atomic group. |
quantifier and the equivalent atomic group, though there may be a performance |
1294 |
|
difference; possessive quantifiers should be slightly faster. |
1295 |
The possessive quantifier syntax is an extension to the Perl syntax. It |
.P |
1296 |
originates in Sun's Java package. |
The possessive quantifier syntax is an extension to the Perl 5.8 syntax. |
1297 |
|
Jeffrey Friedl originated the idea (and the name) in the first edition of his |
1298 |
|
book. Mike McCloskey liked it, so implemented it when he built Sun's Java |
1299 |
|
package, and PCRE copied it from there. It ultimately found its way into Perl |
1300 |
|
at release 5.10. |
1301 |
|
.P |
1302 |
|
PCRE has an optimization that automatically "possessifies" certain simple |
1303 |
|
pattern constructs. For example, the sequence A+B is treated as A++B because |
1304 |
|
there is no point in backtracking into a sequence of A's when B must follow. |
1305 |
|
.P |
1306 |
When a pattern contains an unlimited repeat inside a subpattern that can itself |
When a pattern contains an unlimited repeat inside a subpattern that can itself |
1307 |
be repeated an unlimited number of times, the use of an atomic group is the |
be repeated an unlimited number of times, the use of an atomic group is the |
1308 |
only way to avoid some failing matches taking a very long time indeed. The |
only way to avoid some failing matches taking a very long time indeed. The |
1309 |
pattern |
pattern |
1310 |
|
.sp |
1311 |
(\\D+|<\\d+>)*[!?] |
(\eD+|<\ed+>)*[!?] |
1312 |
|
.sp |
1313 |
matches an unlimited number of substrings that either consist of non-digits, or |
matches an unlimited number of substrings that either consist of non-digits, or |
1314 |
digits enclosed in <>, followed by either ! or ?. When it matches, it runs |
digits enclosed in <>, followed by either ! or ?. When it matches, it runs |
1315 |
quickly. However, if it is applied to |
quickly. However, if it is applied to |
1316 |
|
.sp |
1317 |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
1318 |
|
.sp |
1319 |
it takes a long time before reporting failure. This is because the string can |
it takes a long time before reporting failure. This is because the string can |
1320 |
be divided between the two repeats in a large number of ways, and all have to |
be divided between the internal \eD+ repeat and the external * repeat in a |
1321 |
be tried. (The example used [!?] rather than a single character at the end, |
large number of ways, and all have to be tried. (The example uses [!?] rather |
1322 |
because both PCRE and Perl have an optimization that allows for fast failure |
than a single character at the end, because both PCRE and Perl have an |
1323 |
when a single character is used. They remember the last single character that |
optimization that allows for fast failure when a single character is used. They |
1324 |
is required for a match, and fail early if it is not present in the string.) |
remember the last single character that is required for a match, and fail early |
1325 |
If the pattern is changed to |
if it is not present in the string.) If the pattern is changed so that it uses |
1326 |
|
an atomic group, like this: |
1327 |
((?>\\D+)|<\\d+>)*[!?] |
.sp |
1328 |
|
((?>\eD+)|<\ed+>)*[!?] |
1329 |
|
.sp |
1330 |
sequences of non-digits cannot be broken, and failure happens quickly. |
sequences of non-digits cannot be broken, and failure happens quickly. |
1331 |
|
. |
1332 |
.SH BACK REFERENCES |
. |
1333 |
|
.\" HTML <a name="backreferences"></a> |
1334 |
|
.SH "BACK REFERENCES" |
1335 |
.rs |
.rs |
1336 |
.sp |
.sp |
1337 |
Outside a character class, a backslash followed by a digit greater than 0 (and |
Outside a character class, a backslash followed by a digit greater than 0 (and |
1338 |
possibly further digits) is a back reference to a capturing subpattern earlier |
possibly further digits) is a back reference to a capturing subpattern earlier |
1339 |
(that is, to its left) in the pattern, provided there have been that many |
(that is, to its left) in the pattern, provided there have been that many |
1340 |
previous capturing left parentheses. |
previous capturing left parentheses. |
1341 |
|
.P |
1342 |
However, if the decimal number following the backslash is less than 10, it is |
However, if the decimal number following the backslash is less than 10, it is |
1343 |
always taken as a back reference, and causes an error only if there are not |
always taken as a back reference, and causes an error only if there are not |
1344 |
that many capturing left parentheses in the entire pattern. In other words, the |
that many capturing left parentheses in the entire pattern. In other words, the |
1345 |
parentheses that are referenced need not be to the left of the reference for |
parentheses that are referenced need not be to the left of the reference for |
1346 |
numbers less than 10. See the section entitled "Backslash" above for further |
numbers less than 10. A "forward back reference" of this type can make sense |
1347 |
details of the handling of digits following a backslash. |
when a repetition is involved and the subpattern to the right has participated |
1348 |
|
in an earlier iteration. |
1349 |
|
.P |
1350 |
|
It is not possible to have a numerical "forward back reference" to a subpattern |
1351 |
|
whose number is 10 or more using this syntax because a sequence such as \e50 is |
1352 |
|
interpreted as a character defined in octal. See the subsection entitled |
1353 |
|
"Non-printing characters" |
1354 |
|
.\" HTML <a href="#digitsafterbackslash"> |
1355 |
|
.\" </a> |
1356 |
|
above |
1357 |
|
.\" |
1358 |
|
for further details of the handling of digits following a backslash. There is |
1359 |
|
no such problem when named parentheses are used. A back reference to any |
1360 |
|
subpattern is possible using named parentheses (see below). |
1361 |
|
.P |
1362 |
|
Another way of avoiding the ambiguity inherent in the use of digits following a |
1363 |
|
backslash is to use the \eg escape sequence, which is a feature introduced in |
1364 |
|
Perl 5.10. This escape must be followed by a positive or a negative number, |
1365 |
|
optionally enclosed in braces. These examples are all identical: |
1366 |
|
.sp |
1367 |
|
(ring), \e1 |
1368 |
|
(ring), \eg1 |
1369 |
|
(ring), \eg{1} |
1370 |
|
.sp |
1371 |
|
A positive number specifies an absolute reference without the ambiguity that is |
1372 |
|
present in the older syntax. It is also useful when literal digits follow the |
1373 |
|
reference. A negative number is a relative reference. Consider this example: |
1374 |
|
.sp |
1375 |
|
(abc(def)ghi)\eg{-1} |
1376 |
|
.sp |
1377 |
|
The sequence \eg{-1} is a reference to the most recently started capturing |
1378 |
|
subpattern before \eg, that is, is it equivalent to \e2. Similarly, \eg{-2} |
1379 |
|
would be equivalent to \e1. The use of relative references can be helpful in |
1380 |
|
long patterns, and also in patterns that are created by joining together |
1381 |
|
fragments that contain references within themselves. |
1382 |
|
.P |
1383 |
A back reference matches whatever actually matched the capturing subpattern in |
A back reference matches whatever actually matched the capturing subpattern in |
1384 |
the current subject string, rather than anything matching the subpattern |
the current subject string, rather than anything matching the subpattern |
1385 |
itself (see |
itself (see |
1388 |
"Subpatterns as subroutines" |
"Subpatterns as subroutines" |
1389 |
.\" |
.\" |
1390 |
below for a way of doing that). So the pattern |
below for a way of doing that). So the pattern |
1391 |
|
.sp |
1392 |
(sens|respons)e and \\1ibility |
(sens|respons)e and \e1ibility |
1393 |
|
.sp |
1394 |
matches "sense and sensibility" and "response and responsibility", but not |
matches "sense and sensibility" and "response and responsibility", but not |
1395 |
"sense and responsibility". If caseful matching is in force at the time of the |
"sense and responsibility". If caseful matching is in force at the time of the |
1396 |
back reference, the case of letters is relevant. For example, |
back reference, the case of letters is relevant. For example, |
1397 |
|
.sp |
1398 |
((?i)rah)\\s+\\1 |
((?i)rah)\es+\e1 |
1399 |
|
.sp |
1400 |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
1401 |
capturing subpattern is matched caselessly. |
capturing subpattern is matched caselessly. |
1402 |
|
.P |
1403 |
Back references to named subpatterns use the Python syntax (?P=name). We could |
There are several different ways of writing back references to named |
1404 |
rewrite the above example as follows: |
subpatterns. The .NET syntax \ek{name} and the Perl syntax \ek<name> or |
1405 |
|
\ek'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified |
1406 |
(?<p1>(?i)rah)\\s+(?P=p1) |
back reference syntax, in which \eg can be used for both numeric and named |
1407 |
|
references, is also supported. We could rewrite the above example in any of |
1408 |
|
the following ways: |
1409 |
|
.sp |
1410 |
|
(?<p1>(?i)rah)\es+\ek<p1> |
1411 |
|
(?'p1'(?i)rah)\es+\ek{p1} |
1412 |
|
(?P<p1>(?i)rah)\es+(?P=p1) |
1413 |
|
(?<p1>(?i)rah)\es+\eg{p1} |
1414 |
|
.sp |
1415 |
|
A subpattern that is referenced by name may appear in the pattern before or |
1416 |
|
after the reference. |
1417 |
|
.P |
1418 |
There may be more than one back reference to the same subpattern. If a |
There may be more than one back reference to the same subpattern. If a |
1419 |
subpattern has not actually been used in a particular match, any back |
subpattern has not actually been used in a particular match, any back |
1420 |
references to it always fail. For example, the pattern |
references to it always fail. For example, the pattern |
1421 |
|
.sp |
1422 |
(a|(bc))\\2 |
(a|(bc))\e2 |
1423 |
|
.sp |
1424 |
always fails if it starts to match "a" rather than "bc". Because there may be |
always fails if it starts to match "a" rather than "bc". Because there may be |
1425 |
many capturing parentheses in a pattern, all digits following the backslash are |
many capturing parentheses in a pattern, all digits following the backslash are |
1426 |
taken as part of a potential back reference number. If the pattern continues |
taken as part of a potential back reference number. If the pattern continues |
1427 |
with a digit character, some delimiter must be used to terminate the back |
with a digit character, some delimiter must be used to terminate the back |
1428 |
reference. If the PCRE_EXTENDED option is set, this can be whitespace. |
reference. If the PCRE_EXTENDED option is set, this can be whitespace. |
1429 |
Otherwise an empty comment can be used. |
Otherwise an empty comment (see |
1430 |
|
.\" HTML <a href="#comments"> |
1431 |
|
.\" </a> |
1432 |
|
"Comments" |
1433 |
|
.\" |
1434 |
|
below) can be used. |
1435 |
|
.P |
1436 |
A back reference that occurs inside the parentheses to which it refers fails |
A back reference that occurs inside the parentheses to which it refers fails |
1437 |
when the subpattern is first used, so, for example, (a\\1) never matches. |
when the subpattern is first used, so, for example, (a\e1) never matches. |
1438 |
However, such references can be useful inside repeated subpatterns. For |
However, such references can be useful inside repeated subpatterns. For |
1439 |
example, the pattern |
example, the pattern |
1440 |
|
.sp |
1441 |
(a|b\\1)+ |
(a|b\e1)+ |
1442 |
|
.sp |
1443 |
matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of |
matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of |
1444 |
the subpattern, the back reference matches the character string corresponding |
the subpattern, the back reference matches the character string corresponding |
1445 |
to the previous iteration. In order for this to work, the pattern must be such |
to the previous iteration. In order for this to work, the pattern must be such |
1446 |
that the first iteration does not need to match the back reference. This can be |
that the first iteration does not need to match the back reference. This can be |
1447 |
done using alternation, as in the example above, or by a quantifier with a |
done using alternation, as in the example above, or by a quantifier with a |
1448 |
minimum of zero. |
minimum of zero. |
1449 |
|
. |
1450 |
|
. |
1451 |
|
.\" HTML <a name="bigassertions"></a> |
1452 |
.SH ASSERTIONS |
.SH ASSERTIONS |
1453 |
.rs |
.rs |
1454 |
.sp |
.sp |
1455 |
An assertion is a test on the characters following or preceding the current |
An assertion is a test on the characters following or preceding the current |
1456 |
matching point that does not actually consume any characters. The simple |
matching point that does not actually consume any characters. The simple |
1457 |
assertions coded as \\b, \\B, \\A, \\G, \\Z, \\z, ^ and $ are described above. |
assertions coded as \eb, \eB, \eA, \eG, \eZ, \ez, ^ and $ are described |
1458 |
|
.\" HTML <a href="#smallassertions"> |
1459 |
|
.\" </a> |
1460 |
|
above. |
1461 |
|
.\" |
1462 |
|
.P |
1463 |
More complicated assertions are coded as subpatterns. There are two kinds: |
More complicated assertions are coded as subpatterns. There are two kinds: |
1464 |
those that look ahead of the current position in the subject string, and those |
those that look ahead of the current position in the subject string, and those |
1465 |
that look behind it. |
that look behind it. An assertion subpattern is matched in the normal way, |
1466 |
|
except that it does not cause the current matching position to be changed. |
1467 |
An assertion subpattern is matched in the normal way, except that it does not |
.P |
1468 |
cause the current matching position to be changed. Lookahead assertions start |
Assertion subpatterns are not capturing subpatterns, and may not be repeated, |
1469 |
with (?= for positive assertions and (?! for negative assertions. For example, |
because it makes no sense to assert the same thing several times. If any kind |
1470 |
|
of assertion contains capturing subpatterns within it, these are counted for |
1471 |
\\w+(?=;) |
the purposes of numbering the capturing subpatterns in the whole pattern. |
1472 |
|
However, substring capturing is carried out only for positive assertions, |
1473 |
|
because it does not make sense for negative assertions. |
1474 |
|
. |
1475 |
|
. |
1476 |
|
.SS "Lookahead assertions" |
1477 |
|
.rs |
1478 |
|
.sp |
1479 |
|
Lookahead assertions start with (?= for positive assertions and (?! for |
1480 |
|
negative assertions. For example, |
1481 |
|
.sp |
1482 |
|
\ew+(?=;) |
1483 |
|
.sp |
1484 |
matches a word followed by a semicolon, but does not include the semicolon in |
matches a word followed by a semicolon, but does not include the semicolon in |
1485 |
the match, and |
the match, and |
1486 |
|
.sp |
1487 |
foo(?!bar) |
foo(?!bar) |
1488 |
|
.sp |
1489 |
matches any occurrence of "foo" that is not followed by "bar". Note that the |
matches any occurrence of "foo" that is not followed by "bar". Note that the |
1490 |
apparently similar pattern |
apparently similar pattern |
1491 |
|
.sp |
1492 |
(?!foo)bar |
(?!foo)bar |
1493 |
|
.sp |
1494 |
does not find an occurrence of "bar" that is preceded by something other than |
does not find an occurrence of "bar" that is preceded by something other than |
1495 |
"foo"; it finds any occurrence of "bar" whatsoever, because the assertion |
"foo"; it finds any occurrence of "bar" whatsoever, because the assertion |
1496 |
(?!foo) is always true when the next three characters are "bar". A |
(?!foo) is always true when the next three characters are "bar". A |
1497 |
lookbehind assertion is needed to achieve this effect. |
lookbehind assertion is needed to achieve the other effect. |
1498 |
|
.P |
1499 |
If you want to force a matching failure at some point in a pattern, the most |
If you want to force a matching failure at some point in a pattern, the most |
1500 |
convenient way to do it is with (?!) because an empty string always matches, so |
convenient way to do it is with (?!) because an empty string always matches, so |
1501 |
an assertion that requires there not to be an empty string must always fail. |
an assertion that requires there not to be an empty string must always fail. |
1502 |
|
. |
1503 |
|
. |
1504 |
|
.\" HTML <a name="lookbehind"></a> |
1505 |
|
.SS "Lookbehind assertions" |
1506 |
|
.rs |
1507 |
|
.sp |
1508 |
Lookbehind assertions start with (?<= for positive assertions and (?<! for |
Lookbehind assertions start with (?<= for positive assertions and (?<! for |
1509 |
negative assertions. For example, |
negative assertions. For example, |
1510 |
|
.sp |
1511 |
(?<!foo)bar |
(?<!foo)bar |
1512 |
|
.sp |
1513 |
does find an occurrence of "bar" that is not preceded by "foo". The contents of |
does find an occurrence of "bar" that is not preceded by "foo". The contents of |
1514 |
a lookbehind assertion are restricted such that all the strings it matches must |
a lookbehind assertion are restricted such that all the strings it matches must |
1515 |
have a fixed length. However, if there are several alternatives, they do not |
have a fixed length. However, if there are several top-level alternatives, they |
1516 |
all have to have the same fixed length. Thus |
do not all have to have the same fixed length. Thus |
1517 |
|
.sp |
1518 |
(?<=bullock|donkey) |
(?<=bullock|donkey) |
1519 |
|
.sp |
1520 |
is permitted, but |
is permitted, but |
1521 |
|
.sp |
1522 |
(?<!dogs?|cats?) |
(?<!dogs?|cats?) |
1523 |
|
.sp |
1524 |
causes an error at compile time. Branches that match different length strings |
causes an error at compile time. Branches that match different length strings |
1525 |
are permitted only at the top level of a lookbehind assertion. This is an |
are permitted only at the top level of a lookbehind assertion. This is an |
1526 |
extension compared with Perl (at least for 5.8), which requires all branches to |
extension compared with Perl (at least for 5.8), which requires all branches to |
1527 |
match the same length of string. An assertion such as |
match the same length of string. An assertion such as |
1528 |
|
.sp |
1529 |
(?<=ab(c|de)) |
(?<=ab(c|de)) |
1530 |
|
.sp |
1531 |
is not permitted, because its single top-level branch can match two different |
is not permitted, because its single top-level branch can match two different |
1532 |
lengths, but it is acceptable if rewritten to use two top-level branches: |
lengths, but it is acceptable if rewritten to use two top-level branches: |
1533 |
|
.sp |
1534 |
(?<=abc|abde) |
(?<=abc|abde) |
1535 |
|
.sp |
1536 |
|
In some cases, the Perl 5.10 escape sequence \eK |
1537 |
|
.\" HTML <a href="#resetmatchstart"> |
1538 |
|
.\" </a> |
1539 |
|
(see above) |
1540 |
|
.\" |
1541 |
|
can be used instead of a lookbehind assertion; this is not restricted to a |
1542 |
|
fixed-length. |
1543 |
|
.P |
1544 |
The implementation of lookbehind assertions is, for each alternative, to |
The implementation of lookbehind assertions is, for each alternative, to |
1545 |
temporarily move the current position back by the fixed width and then try to |
temporarily move the current position back by the fixed length and then try to |
1546 |
match. If there are insufficient characters before the current position, the |
match. If there are insufficient characters before the current position, the |
1547 |
match is deemed to fail. |
assertion fails. |
1548 |
|
.P |
1549 |
PCRE does not allow the \\C escape (which matches a single byte in UTF-8 mode) |
PCRE does not allow the \eC escape (which matches a single byte in UTF-8 mode) |
1550 |
to appear in lookbehind assertions, because it makes it impossible to calculate |
to appear in lookbehind assertions, because it makes it impossible to calculate |
1551 |
the length of the lookbehind. |
the length of the lookbehind. The \eX and \eR escapes, which can match |
1552 |
|
different numbers of bytes, are also not permitted. |
1553 |
Atomic groups can be used in conjunction with lookbehind assertions to specify |
.P |
1554 |
efficient matching at the end of the subject string. Consider a simple pattern |
Possessive quantifiers can be used in conjunction with lookbehind assertions to |
1555 |
such as |
specify efficient matching at the end of the subject string. Consider a simple |
1556 |
|
pattern such as |
1557 |
|
.sp |
1558 |
abcd$ |
abcd$ |
1559 |
|
.sp |
1560 |
when applied to a long string that does not match. Because matching proceeds |
when applied to a long string that does not match. Because matching proceeds |
1561 |
from left to right, PCRE will look for each "a" in the subject and then see if |
from left to right, PCRE will look for each "a" in the subject and then see if |
1562 |
what follows matches the rest of the pattern. If the pattern is specified as |
what follows matches the rest of the pattern. If the pattern is specified as |
1563 |
|
.sp |
1564 |
^.*abcd$ |
^.*abcd$ |
1565 |
|
.sp |
1566 |
the initial .* matches the entire string at first, but when this fails (because |
the initial .* matches the entire string at first, but when this fails (because |
1567 |
there is no following "a"), it backtracks to match all but the last character, |
there is no following "a"), it backtracks to match all but the last character, |
1568 |
then all but the last two characters, and so on. Once again the search for "a" |
then all but the last two characters, and so on. Once again the search for "a" |
1569 |
covers the entire string, from right to left, so we are no better off. However, |
covers the entire string, from right to left, so we are no better off. However, |
1570 |
if the pattern is written as |
if the pattern is written as |
1571 |
|
.sp |
|
^(?>.*)(?<=abcd) |
|
|
|
|
|
or, equivalently, |
|
|
|
|
1572 |
^.*+(?<=abcd) |
^.*+(?<=abcd) |
1573 |
|
.sp |
1574 |
there can be no backtracking for the .* item; it can match only the entire |
there can be no backtracking for the .*+ item; it can match only the entire |
1575 |
string. The subsequent lookbehind assertion does a single test on the last four |
string. The subsequent lookbehind assertion does a single test on the last four |
1576 |
characters. If it fails, the match fails immediately. For long strings, this |
characters. If it fails, the match fails immediately. For long strings, this |
1577 |
approach makes a significant difference to the processing time. |
approach makes a significant difference to the processing time. |
1578 |
|
. |
1579 |
|
. |
1580 |
|
.SS "Using multiple assertions" |
1581 |
|
.rs |
1582 |
|
.sp |
1583 |
Several assertions (of any sort) may occur in succession. For example, |
Several assertions (of any sort) may occur in succession. For example, |
1584 |
|
.sp |
1585 |
(?<=\\d{3})(?<!999)foo |
(?<=\ed{3})(?<!999)foo |
1586 |
|
.sp |
1587 |
matches "foo" preceded by three digits that are not "999". Notice that each of |
matches "foo" preceded by three digits that are not "999". Notice that each of |
1588 |
the assertions is applied independently at the same point in the subject |
the assertions is applied independently at the same point in the subject |
1589 |
string. First there is a check that the previous three characters are all |
string. First there is a check that the previous three characters are all |
1590 |
digits, and then there is a check that the same three characters are not "999". |
digits, and then there is a check that the same three characters are not "999". |
1591 |
This pattern does \fInot\fR match "foo" preceded by six characters, the first |
This pattern does \fInot\fP match "foo" preceded by six characters, the first |
1592 |
of which are digits and the last three of which are not "999". For example, it |
of which are digits and the last three of which are not "999". For example, it |
1593 |
doesn't match "123abcfoo". A pattern to do that is |
doesn't match "123abcfoo". A pattern to do that is |
1594 |
|
.sp |
1595 |
(?<=\\d{3}...)(?<!999)foo |
(?<=\ed{3}...)(?<!999)foo |
1596 |
|
.sp |
1597 |
This time the first assertion looks at the preceding six characters, checking |
This time the first assertion looks at the preceding six characters, checking |
1598 |
that the first three are digits, and then the second assertion checks that the |
that the first three are digits, and then the second assertion checks that the |
1599 |
preceding three characters are not "999". |
preceding three characters are not "999". |
1600 |
|
.P |
1601 |
Assertions can be nested in any combination. For example, |
Assertions can be nested in any combination. For example, |
1602 |
|
.sp |
1603 |
(?<=(?<!foo)bar)baz |
(?<=(?<!foo)bar)baz |
1604 |
|
.sp |
1605 |
matches an occurrence of "baz" that is preceded by "bar" which in turn is not |
matches an occurrence of "baz" that is preceded by "bar" which in turn is not |
1606 |
preceded by "foo", while |
preceded by "foo", while |
1607 |
|
.sp |
1608 |
(?<=\\d{3}(?!999)...)foo |
(?<=\ed{3}(?!999)...)foo |
1609 |
|
.sp |
1610 |
is another pattern which matches "foo" preceded by three digits and any three |
is another pattern that matches "foo" preceded by three digits and any three |
1611 |
characters that are not "999". |
characters that are not "999". |
1612 |
|
. |
1613 |
Assertion subpatterns are not capturing subpatterns, and may not be repeated, |
. |
1614 |
because it makes no sense to assert the same thing several times. If any kind |
.\" HTML <a name="conditions"></a> |
1615 |
of assertion contains capturing subpatterns within it, these are counted for |
.SH "CONDITIONAL SUBPATTERNS" |
|
the purposes of numbering the capturing subpatterns in the whole pattern. |
|
|
However, substring capturing is carried out only for positive assertions, |
|
|
because it does not make sense for negative assertions. |
|
|
|
|
|
.SH CONDITIONAL SUBPATTERNS |
|
1616 |
.rs |
.rs |
1617 |
.sp |
.sp |
1618 |
It is possible to cause the matching process to obey a subpattern |
It is possible to cause the matching process to obey a subpattern |
1619 |
conditionally or to choose between two alternative subpatterns, depending on |
conditionally or to choose between two alternative subpatterns, depending on |
1620 |
the result of an assertion, or whether a previous capturing subpattern matched |
the result of an assertion, or whether a previous capturing subpattern matched |
1621 |
or not. The two possible forms of conditional subpattern are |
or not. The two possible forms of conditional subpattern are |
1622 |
|
.sp |
1623 |
(?(condition)yes-pattern) |
(?(condition)yes-pattern) |
1624 |
(?(condition)yes-pattern|no-pattern) |
(?(condition)yes-pattern|no-pattern) |
1625 |
|
.sp |
1626 |
If the condition is satisfied, the yes-pattern is used; otherwise the |
If the condition is satisfied, the yes-pattern is used; otherwise the |
1627 |
no-pattern (if present) is used. If there are more than two alternatives in the |
no-pattern (if present) is used. If there are more than two alternatives in the |
1628 |
subpattern, a compile-time error occurs. |
subpattern, a compile-time error occurs. |
1629 |
|
.P |
1630 |
There are three kinds of condition. If the text between the parentheses |
There are four kinds of condition: references to subpatterns, references to |
1631 |
consists of a sequence of digits, the condition is satisfied if the capturing |
recursion, a pseudo-condition called DEFINE, and assertions. |
1632 |
subpattern of that number has previously matched. The number must be greater |
. |
1633 |
than zero. Consider the following pattern, which contains non-significant white |
.SS "Checking for a used subpattern by number" |
1634 |
space to make it more readable (assume the PCRE_EXTENDED option) and to divide |
.rs |
1635 |
it into three parts for ease of discussion: |
.sp |
1636 |
|
If the text between the parentheses consists of a sequence of digits, the |
1637 |
( \\( )? [^()]+ (?(1) \\) ) |
condition is true if the capturing subpattern of that number has previously |
1638 |
|
matched. An alternative notation is to precede the digits with a plus or minus |
1639 |
|
sign. In this case, the subpattern number is relative rather than absolute. |
1640 |
|
The most recently opened parentheses can be referenced by (?(-1), the next most |
1641 |
|
recent by (?(-2), and so on. In looping constructs it can also make sense to |
1642 |
|
refer to subsequent groups with constructs such as (?(+2). |
1643 |
|
.P |
1644 |
|
Consider the following pattern, which contains non-significant white space to |
1645 |
|
make it more readable (assume the PCRE_EXTENDED option) and to divide it into |
1646 |
|
three parts for ease of discussion: |
1647 |
|
.sp |
1648 |
|
( \e( )? [^()]+ (?(1) \e) ) |
1649 |
|
.sp |
1650 |
The first part matches an optional opening parenthesis, and if that |
The first part matches an optional opening parenthesis, and if that |
1651 |
character is present, sets it as the first captured substring. The second part |
character is present, sets it as the first captured substring. The second part |
1652 |
matches one or more characters that are not parentheses. The third part is a |
matches one or more characters that are not parentheses. The third part is a |
1656 |
parenthesis is required. Otherwise, since no-pattern is not present, the |
parenthesis is required. Otherwise, since no-pattern is not present, the |
1657 |
subpattern matches nothing. In other words, this pattern matches a sequence of |
subpattern matches nothing. In other words, this pattern matches a sequence of |
1658 |
non-parentheses, optionally enclosed in parentheses. |
non-parentheses, optionally enclosed in parentheses. |
1659 |
|
.P |
1660 |
If the condition is the string (R), it is satisfied if a recursive call to the |
If you were embedding this pattern in a larger one, you could use a relative |
1661 |
pattern or subpattern has been made. At "top level", the condition is false. |
reference: |
1662 |
This is a PCRE extension. Recursive patterns are described in the next section. |
.sp |
1663 |
|
...other stuff... ( \e( )? [^()]+ (?(-1) \e) ) ... |
1664 |
If the condition is not a sequence of digits or (R), it must be an assertion. |
.sp |
1665 |
|
This makes the fragment independent of the parentheses in the larger pattern. |
1666 |
|
. |
1667 |
|
.SS "Checking for a used subpattern by name" |
1668 |
|
.rs |
1669 |
|
.sp |
1670 |
|
Perl uses the syntax (?(<name>)...) or (?('name')...) to test for a used |
1671 |
|
subpattern by name. For compatibility with earlier versions of PCRE, which had |
1672 |
|
this facility before Perl, the syntax (?(name)...) is also recognized. However, |
1673 |
|
there is a possible ambiguity with this syntax, because subpattern names may |
1674 |
|
consist entirely of digits. PCRE looks first for a named subpattern; if it |
1675 |
|
cannot find one and the name consists entirely of digits, PCRE looks for a |
1676 |
|
subpattern of that number, which must be greater than zero. Using subpattern |
1677 |
|
names that consist entirely of digits is not recommended. |
1678 |
|
.P |
1679 |
|
Rewriting the above example to use a named subpattern gives this: |
1680 |
|
.sp |
1681 |
|
(?<OPEN> \e( )? [^()]+ (?(<OPEN>) \e) ) |
1682 |
|
.sp |
1683 |
|
. |
1684 |
|
.SS "Checking for pattern recursion" |
1685 |
|
.rs |
1686 |
|
.sp |
1687 |
|
If the condition is the string (R), and there is no subpattern with the name R, |
1688 |
|
the condition is true if a recursive call to the whole pattern or any |
1689 |
|
subpattern has been made. If digits or a name preceded by ampersand follow the |
1690 |
|
letter R, for example: |
1691 |
|
.sp |
1692 |
|
(?(R3)...) or (?(R&name)...) |
1693 |
|
.sp |
1694 |
|
the condition is true if the most recent recursion is into the subpattern whose |
1695 |
|
number or name is given. This condition does not check the entire recursion |
1696 |
|
stack. |
1697 |
|
.P |
1698 |
|
At "top level", all these recursion test conditions are false. Recursive |
1699 |
|
patterns are described below. |
1700 |
|
. |
1701 |
|
.SS "Defining subpatterns for use by reference only" |
1702 |
|
.rs |
1703 |
|
.sp |
1704 |
|
If the condition is the string (DEFINE), and there is no subpattern with the |
1705 |
|
name DEFINE, the condition is always false. In this case, there may be only one |
1706 |
|
alternative in the subpattern. It is always skipped if control reaches this |
1707 |
|
point in the pattern; the idea of DEFINE is that it can be used to define |
1708 |
|
"subroutines" that can be referenced from elsewhere. (The use of "subroutines" |
1709 |
|
is described below.) For example, a pattern to match an IPv4 address could be |
1710 |
|
written like this (ignore whitespace and line breaks): |
1711 |
|
.sp |
1712 |
|
(?(DEFINE) (?<byte> 2[0-4]\ed | 25[0-5] | 1\ed\ed | [1-9]?\ed) ) |
1713 |
|
\eb (?&byte) (\e.(?&byte)){3} \eb |
1714 |
|
.sp |
1715 |
|
The first part of the pattern is a DEFINE group inside which a another group |
1716 |
|
named "byte" is defined. This matches an individual component of an IPv4 |
1717 |
|
address (a number less than 256). When matching takes place, this part of the |
1718 |
|
pattern is skipped because DEFINE acts like a false condition. |
1719 |
|
.P |
1720 |
|
The rest of the pattern uses references to the named group to match the four |
1721 |
|
dot-separated components of an IPv4 address, insisting on a word boundary at |
1722 |
|
each end. |
1723 |
|
. |
1724 |
|
.SS "Assertion conditions" |
1725 |
|
.rs |
1726 |
|
.sp |
1727 |
|
If the condition is not in any of the above formats, it must be an assertion. |
1728 |
This may be a positive or negative lookahead or lookbehind assertion. Consider |
This may be a positive or negative lookahead or lookbehind assertion. Consider |
1729 |
this pattern, again containing non-significant white space, and with the two |
this pattern, again containing non-significant white space, and with the two |
1730 |
alternatives on the second line: |
alternatives on the second line: |
1731 |
|
.sp |
1732 |
(?(?=[^a-z]*[a-z]) |
(?(?=[^a-z]*[a-z]) |
1733 |
\\d{2}-[a-z]{3}-\\d{2} | \\d{2}-\\d{2}-\\d{2} ) |
\ed{2}-[a-z]{3}-\ed{2} | \ed{2}-\ed{2}-\ed{2} ) |
1734 |
|
.sp |
1735 |
The condition is a positive lookahead assertion that matches an optional |
The condition is a positive lookahead assertion that matches an optional |
1736 |
sequence of non-letters followed by a letter. In other words, it tests for the |
sequence of non-letters followed by a letter. In other words, it tests for the |
1737 |
presence of at least one letter in the subject. If a letter is found, the |
presence of at least one letter in the subject. If a letter is found, the |
1738 |
subject is matched against the first alternative; otherwise it is matched |
subject is matched against the first alternative; otherwise it is matched |
1739 |
against the second. This pattern matches strings in one of the two forms |
against the second. This pattern matches strings in one of the two forms |
1740 |
dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. |
dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. |
1741 |
|
. |
1742 |
|
. |
1743 |
|
.\" HTML <a name="comments"></a> |
1744 |
.SH COMMENTS |
.SH COMMENTS |
1745 |
.rs |
.rs |
1746 |
.sp |
.sp |
1747 |
The sequence (?# marks the start of a comment which continues up to the next |
The sequence (?# marks the start of a comment that continues up to the next |
1748 |
closing parenthesis. Nested parentheses are not permitted. The characters |
closing parenthesis. Nested parentheses are not permitted. The characters |
1749 |
that make up a comment play no part in the pattern matching at all. |
that make up a comment play no part in the pattern matching at all. |
1750 |
|
.P |
1751 |
If the PCRE_EXTENDED option is set, an unescaped # character outside a |
If the PCRE_EXTENDED option is set, an unescaped # character outside a |
1752 |
character class introduces a comment that continues up to the next newline |
character class introduces a comment that continues to immediately after the |
1753 |
character in the pattern. |
next newline in the pattern. |
1754 |
|
. |
1755 |
.SH RECURSIVE PATTERNS |
. |
1756 |
|
.\" HTML <a name="recursion"></a> |
1757 |
|
.SH "RECURSIVE PATTERNS" |
1758 |
.rs |
.rs |
1759 |
.sp |
.sp |
1760 |
Consider the problem of matching a string in parentheses, allowing for |
Consider the problem of matching a string in parentheses, allowing for |
1761 |
unlimited nested parentheses. Without the use of recursion, the best that can |
unlimited nested parentheses. Without the use of recursion, the best that can |
1762 |
be done is to use a pattern that matches up to some fixed depth of nesting. It |
be done is to use a pattern that matches up to some fixed depth of nesting. It |
1763 |
is not possible to handle an arbitrary nesting depth. Perl has provided an |
is not possible to handle an arbitrary nesting depth. |
1764 |
experimental facility that allows regular expressions to recurse (amongst other |
.P |
1765 |
things). It does this by interpolating Perl code in the expression at run time, |
For some time, Perl has provided a facility that allows regular expressions to |
1766 |
and the code can refer to the expression itself. A Perl pattern to solve the |
recurse (amongst other things). It does this by interpolating Perl code in the |
1767 |
parentheses problem can be created like this: |
expression at run time, and the code can refer to the expression itself. A Perl |
1768 |
|
pattern using code interpolation to solve the parentheses problem can be |
1769 |
$re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x; |
created like this: |
1770 |
|
.sp |
1771 |
|
$re = qr{\e( (?: (?>[^()]+) | (?p{$re}) )* \e)}x; |
1772 |
|
.sp |
1773 |
The (?p{...}) item interpolates Perl code at run time, and in this case refers |
The (?p{...}) item interpolates Perl code at run time, and in this case refers |
1774 |
recursively to the pattern in which it appears. Obviously, PCRE cannot support |
recursively to the pattern in which it appears. |
1775 |
the interpolation of Perl code. Instead, it supports some special syntax for |
.P |
1776 |
recursion of the entire pattern, and also for individual subpattern recursion. |
Obviously, PCRE cannot support the interpolation of Perl code. Instead, it |
1777 |
|
supports special syntax for recursion of the entire pattern, and also for |
1778 |
The special item that consists of (? followed by a number greater than zero and |
individual subpattern recursion. After its introduction in PCRE and Python, |
1779 |
a closing parenthesis is a recursive call of the subpattern of the given |
this kind of recursion was introduced into Perl at release 5.10. |
1780 |
number, provided that it occurs inside that subpattern. (If not, it is a |
.P |
1781 |
"subroutine" call, which is described in the next section.) The special item |
A special item that consists of (? followed by a number greater than zero and a |
1782 |
(?R) is a recursive call of the entire regular expression. |
closing parenthesis is a recursive call of the subpattern of the given number, |
1783 |
|
provided that it occurs inside that subpattern. (If not, it is a "subroutine" |
1784 |
For example, this PCRE pattern solves the nested parentheses problem (assume |
call, which is described in the next section.) The special item (?R) or (?0) is |
1785 |
the PCRE_EXTENDED option is set so that white space is ignored): |
a recursive call of the entire regular expression. |
1786 |
|
.P |
1787 |
\\( ( (?>[^()]+) | (?R) )* \\) |
In PCRE (like Python, but unlike Perl), a recursive subpattern call is always |
1788 |
|
treated as an atomic group. That is, once it has matched some of the subject |
1789 |
|
string, it is never re-entered, even if it contains untried alternatives and |
1790 |
|
there is a subsequent matching failure. |
1791 |
|
.P |
1792 |
|
This PCRE pattern solves the nested parentheses problem (assume the |
1793 |
|
PCRE_EXTENDED option is set so that white space is ignored): |
1794 |
|
.sp |
1795 |
|
\e( ( (?>[^()]+) | (?R) )* \e) |
1796 |
|
.sp |
1797 |
First it matches an opening parenthesis. Then it matches any number of |
First it matches an opening parenthesis. Then it matches any number of |
1798 |
substrings which can either be a sequence of non-parentheses, or a recursive |
substrings which can either be a sequence of non-parentheses, or a recursive |
1799 |
match of the pattern itself (that is a correctly parenthesized substring). |
match of the pattern itself (that is, a correctly parenthesized substring). |
1800 |
Finally there is a closing parenthesis. |
Finally there is a closing parenthesis. |
1801 |
|
.P |
1802 |
If this were part of a larger pattern, you would not want to recurse the entire |
If this were part of a larger pattern, you would not want to recurse the entire |
1803 |
pattern, so instead you could use this: |
pattern, so instead you could use this: |
1804 |
|
.sp |
1805 |
( \\( ( (?>[^()]+) | (?1) )* \\) ) |
( \e( ( (?>[^()]+) | (?1) )* \e) ) |
1806 |
|
.sp |
1807 |
We have put the pattern into parentheses, and caused the recursion to refer to |
We have put the pattern into parentheses, and caused the recursion to refer to |
1808 |
them instead of the whole pattern. In a larger pattern, keeping track of |
them instead of the whole pattern. |
1809 |
parenthesis numbers can be tricky. It may be more convenient to use named |
.P |
1810 |
parentheses instead. For this, PCRE uses (?P>name), which is an extension to |
In a larger pattern, keeping track of parenthesis numbers can be tricky. This |
1811 |
the Python syntax that PCRE uses for named parentheses (Perl does not provide |
is made easier by the use of relative references. (A Perl 5.10 feature.) |
1812 |
named parentheses). We could rewrite the above example as follows: |
Instead of (?1) in the pattern above you can write (?-2) to refer to the second |
1813 |
|
most recently opened parentheses preceding the recursion. In other words, a |
1814 |
(?P<pn> \\( ( (?>[^()]+) | (?P>pn) )* \\) ) |
negative number counts capturing parentheses leftwards from the point at which |
1815 |
|
it is encountered. |
1816 |
This particular example pattern contains nested unlimited repeats, and so the |
.P |
1817 |
use of atomic grouping for matching strings of non-parentheses is important |
It is also possible to refer to subsequently opened parentheses, by writing |
1818 |
when applying the pattern to strings that do not match. For example, when this |
references such as (?+2). However, these cannot be recursive because the |
1819 |
pattern is applied to |
reference is not inside the parentheses that are referenced. They are always |
1820 |
|
"subroutine" calls, as described in the next section. |
1821 |
|
.P |
1822 |
|
An alternative approach is to use named parentheses instead. The Perl syntax |
1823 |
|
for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We |
1824 |
|
could rewrite the above example as follows: |
1825 |
|
.sp |
1826 |
|
(?<pn> \e( ( (?>[^()]+) | (?&pn) )* \e) ) |
1827 |
|
.sp |
1828 |
|
If there is more than one subpattern with the same name, the earliest one is |
1829 |
|
used. |
1830 |
|
.P |
1831 |
|
This particular example pattern that we have been looking at contains nested |
1832 |
|
unlimited repeats, and so the use of atomic grouping for matching strings of |
1833 |
|
non-parentheses is important when applying the pattern to strings that do not |
1834 |
|
match. For example, when this pattern is applied to |
1835 |
|
.sp |
1836 |
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
1837 |
|
.sp |
1838 |
it yields "no match" quickly. However, if atomic grouping is not used, |
it yields "no match" quickly. However, if atomic grouping is not used, |
1839 |
the match runs for a very long time indeed because there are so many different |
the match runs for a very long time indeed because there are so many different |
1840 |
ways the + and * repeats can carve up the subject, and all have to be tested |
ways the + and * repeats can carve up the subject, and all have to be tested |
1841 |
before failure can be reported. |
before failure can be reported. |
1842 |
|
.P |
1843 |
At the end of a match, the values set for any capturing subpatterns are those |
At the end of a match, the values set for any capturing subpatterns are those |
1844 |
from the outermost level of the recursion at which the subpattern value is set. |
from the outermost level of the recursion at which the subpattern value is set. |
1845 |
If you want to obtain intermediate values, a callout function can be used (see |
If you want to obtain intermediate values, a callout function can be used (see |
1846 |
below and the |
below and the |
1847 |
.\" HREF |
.\" HREF |
1848 |
\fBpcrecallout\fR |
\fBpcrecallout\fP |
1849 |
.\" |
.\" |
1850 |
documentation). If the pattern above is matched against |
documentation). If the pattern above is matched against |
1851 |
|
.sp |
1852 |
(ab(cd)ef) |
(ab(cd)ef) |
1853 |
|
.sp |
1854 |
the value for the capturing parentheses is "ef", which is the last value taken |
the value for the capturing parentheses is "ef", which is the last value taken |
1855 |
on at the top level. If additional parentheses are added, giving |
on at the top level. If additional parentheses are added, giving |
1856 |
|
.sp |
1857 |
\\( ( ( (?>[^()]+) | (?R) )* ) \\) |
\e( ( ( (?>[^()]+) | (?R) )* ) \e) |
1858 |
^ ^ |
^ ^ |
1859 |
^ ^ |
^ ^ |
1860 |
|
.sp |
1861 |
the string they capture is "ab(cd)ef", the contents of the top level |
the string they capture is "ab(cd)ef", the contents of the top level |
1862 |
parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE |
parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE |
1863 |
has to obtain extra memory to store data during a recursion, which it does by |
has to obtain extra memory to store data during a recursion, which it does by |
1864 |
using \fBpcre_malloc\fR, freeing it via \fBpcre_free\fR afterwards. If no |
using \fBpcre_malloc\fP, freeing it via \fBpcre_free\fP afterwards. If no |
1865 |
memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
1866 |
|
.P |
1867 |
Do not confuse the (?R) item with the condition (R), which tests for recursion. |
Do not confuse the (?R) item with the condition (R), which tests for recursion. |
1868 |
Consider this pattern, which matches text in angle brackets, allowing for |
Consider this pattern, which matches text in angle brackets, allowing for |
1869 |
arbitrary nesting. Only digits are allowed in nested brackets (that is, when |
arbitrary nesting. Only digits are allowed in nested brackets (that is, when |
1870 |
recursing), whereas any characters are permitted at the outer level. |
recursing), whereas any characters are permitted at the outer level. |
1871 |
|
.sp |
1872 |
< (?: (?(R) \\d++ | [^<>]*+) | (?R)) * > |
< (?: (?(R) \ed++ | [^<>]*+) | (?R)) * > |
1873 |
|
.sp |
1874 |
In this pattern, (?(R) is the start of a conditional subpattern, with two |
In this pattern, (?(R) is the start of a conditional subpattern, with two |
1875 |
different alternatives for the recursive and non-recursive cases. The (?R) item |
different alternatives for the recursive and non-recursive cases. The (?R) item |
1876 |
is the actual recursive call. |
is the actual recursive call. |
1877 |
|
. |
1878 |
|
. |
1879 |
.\" HTML <a name="subpatternsassubroutines"></a> |
.\" HTML <a name="subpatternsassubroutines"></a> |
1880 |
.SH SUBPATTERNS AS SUBROUTINES |
.SH "SUBPATTERNS AS SUBROUTINES" |
1881 |
.rs |
.rs |
1882 |
.sp |
.sp |
1883 |
If the syntax for a recursive subpattern reference (either by number or by |
If the syntax for a recursive subpattern reference (either by number or by |
1884 |
name) is used outside the parentheses to which it refers, it operates like a |
name) is used outside the parentheses to which it refers, it operates like a |
1885 |
subroutine in a programming language. An earlier example pointed out that the |
subroutine in a programming language. The "called" subpattern may be defined |
1886 |
pattern |
before or after the reference. A numbered reference can be absolute or |
1887 |
|
relative, as in these examples: |
1888 |
(sens|respons)e and \\1ibility |
.sp |
1889 |
|
(...(absolute)...)...(?2)... |
1890 |
|
(...(relative)...)...(?-1)... |
1891 |
|
(...(?+1)...(relative)... |
1892 |
|
.sp |
1893 |
|
An earlier example pointed out that the pattern |
1894 |
|
.sp |
1895 |
|
(sens|respons)e and \e1ibility |
1896 |
|
.sp |
1897 |
matches "sense and sensibility" and "response and responsibility", but not |
matches "sense and sensibility" and "response and responsibility", but not |
1898 |
"sense and responsibility". If instead the pattern |
"sense and responsibility". If instead the pattern |
1899 |
|
.sp |
1900 |
(sens|respons)e and (?1)ibility |
(sens|respons)e and (?1)ibility |
1901 |
|
.sp |
1902 |
is used, it does match "sense and responsibility" as well as the other two |
is used, it does match "sense and responsibility" as well as the other two |
1903 |
strings. Such references must, however, follow the subpattern to which they |
strings. Another example is given in the discussion of DEFINE above. |
1904 |
refer. |
.P |
1905 |
|
Like recursive subpatterns, a "subroutine" call is always treated as an atomic |
1906 |
|
group. That is, once it has matched some of the subject string, it is never |
1907 |
|
re-entered, even if it contains untried alternatives and there is a subsequent |
1908 |
|
matching failure. |
1909 |
|
.P |
1910 |
|
When a subpattern is used as a subroutine, processing options such as |
1911 |
|
case-independence are fixed when the subpattern is defined. They cannot be |
1912 |
|
changed for different calls. For example, consider this pattern: |
1913 |
|
.sp |
1914 |
|
(abc)(?i:(?-1)) |
1915 |
|
.sp |
1916 |
|
It matches "abcabc". It does not match "abcABC" because the change of |
1917 |
|
processing option does not affect the called subpattern. |
1918 |
|
. |
1919 |
|
. |
1920 |
.SH CALLOUTS |
.SH CALLOUTS |
1921 |
.rs |
.rs |
1922 |
.sp |
.sp |
1924 |
code to be obeyed in the middle of matching a regular expression. This makes it |
code to be obeyed in the middle of matching a regular expression. This makes it |
1925 |
possible, amongst other things, to extract different substrings that match the |
possible, amongst other things, to extract different substrings that match the |
1926 |
same pair of parentheses when there is a repetition. |
same pair of parentheses when there is a repetition. |
1927 |
|
.P |
1928 |
PCRE provides a similar feature, but of course it cannot obey arbitrary Perl |
PCRE provides a similar feature, but of course it cannot obey arbitrary Perl |
1929 |
code. The feature is called "callout". The caller of PCRE provides an external |
code. The feature is called "callout". The caller of PCRE provides an external |
1930 |
function by putting its entry point in the global variable \fIpcre_callout\fR. |
function by putting its entry point in the global variable \fIpcre_callout\fP. |
1931 |
By default, this variable contains NULL, which disables all calling out. |
By default, this variable contains NULL, which disables all calling out. |
1932 |
|
.P |
1933 |
Within a regular expression, (?C) indicates the points at which the external |
Within a regular expression, (?C) indicates the points at which the external |
1934 |
function is to be called. If you want to identify different callout points, you |
function is to be called. If you want to identify different callout points, you |
1935 |
can put a number less than 256 after the letter C. The default value is zero. |
can put a number less than 256 after the letter C. The default value is zero. |
1936 |
For example, this pattern has two callout points: |
For example, this pattern has two callout points: |
1937 |
|
.sp |
1938 |
(?C1)\dabc(?C2)def |
(?C1)abc(?C2)def |
1939 |
|
.sp |
1940 |
During matching, when PCRE reaches a callout point (and \fIpcre_callout\fR is |
If the PCRE_AUTO_CALLOUT flag is passed to \fBpcre_compile()\fP, callouts are |
1941 |
|
automatically installed before each item in the pattern. They are all numbered |
1942 |
|
255. |
1943 |
|
.P |
1944 |
|
During matching, when PCRE reaches a callout point (and \fIpcre_callout\fP is |
1945 |
set), the external function is called. It is provided with the number of the |
set), the external function is called. It is provided with the number of the |
1946 |
callout, and, optionally, one item of data originally supplied by the caller of |
callout, the position in the pattern, and, optionally, one item of data |
1947 |
\fBpcre_exec()\fR. The callout function may cause matching to backtrack, or to |
originally supplied by the caller of \fBpcre_exec()\fP. The callout function |
1948 |
fail altogether. A complete description of the interface to the callout |
may cause matching to proceed, to backtrack, or to fail altogether. A complete |
1949 |
function is given in the |
description of the interface to the callout function is given in the |
1950 |
.\" HREF |
.\" HREF |
1951 |
\fBpcrecallout\fR |
\fBpcrecallout\fP |
1952 |
.\" |
.\" |
1953 |
documentation. |
documentation. |
1954 |
|
. |
1955 |
.in 0 |
. |
1956 |
Last updated: 03 February 2003 |
.SH "SEE ALSO" |
1957 |
.br |
.rs |
1958 |
Copyright (c) 1997-2003 University of Cambridge. |
.sp |
1959 |
|
\fBpcreapi\fP(3), \fBpcrecallout\fP(3), \fBpcrematching\fP(3), \fBpcre\fP(3). |
1960 |
|
. |
1961 |
|
. |
1962 |
|
.SH AUTHOR |
1963 |
|
.rs |
1964 |
|
.sp |
1965 |
|
.nf |
1966 |
|
Philip Hazel |
1967 |
|
University Computing Service |
1968 |
|
Cambridge CB2 3QH, England. |
1969 |
|
.fi |
1970 |
|
. |
1971 |
|
. |
1972 |
|
.SH REVISION |
1973 |
|
.rs |
1974 |
|
.sp |
1975 |
|
.nf |
1976 |
|
Last updated: 13 June 2007 |
1977 |
|
Copyright (c) 1997-2007 University of Cambridge. |
1978 |
|
.fi |