1 |
.TH PCRE 3 |
.TH PCRE 3 |
2 |
.SH NAME |
.SH NAME |
3 |
PCRE - Perl-compatible regular expressions |
PCRE - Perl-compatible regular expressions |
4 |
.SH PCRE REGULAR EXPRESSION DETAILS |
.SH "PCRE REGULAR EXPRESSION DETAILS" |
5 |
.rs |
.rs |
6 |
.sp |
.sp |
7 |
The syntax and semantics of the regular expressions supported by PCRE are |
The syntax and semantics of the regular expressions supported by PCRE are |
8 |
described below. Regular expressions are also described in the Perl |
described below. Regular expressions are also described in the Perl |
9 |
documentation and in a number of other books, some of which have copious |
documentation and in a number of books, some of which have copious examples. |
10 |
examples. Jeffrey Friedl's "Mastering Regular Expressions", published by |
Jeffrey Friedl's "Mastering Regular Expressions", published by O'Reilly, covers |
11 |
O'Reilly, covers them in great detail. The description here is intended as |
regular expressions in great detail. This description of PCRE's regular |
12 |
reference documentation. |
expressions is intended as reference material. |
13 |
|
.P |
14 |
The basic operation of PCRE is on strings of bytes. However, there is also |
The original operation of PCRE was on strings of one-byte characters. However, |
15 |
support for UTF-8 character strings. To use this support you must build PCRE to |
there is now also support for UTF-8 character strings. To use this, you must |
16 |
include UTF-8 support, and then call \fBpcre_compile()\fR with the PCRE_UTF8 |
build PCRE to include UTF-8 support, and then call \fBpcre_compile()\fP with |
17 |
option. How this affects the pattern matching is mentioned in several places |
the PCRE_UTF8 option. How this affects pattern matching is mentioned in several |
18 |
below. There is also a summary of UTF-8 features in the |
places below. There is also a summary of UTF-8 features in the |
19 |
.\" HTML <a href="pcre.html#utf8support"> |
.\" HTML <a href="pcre.html#utf8support"> |
20 |
.\" </a> |
.\" </a> |
21 |
section on UTF-8 support |
section on UTF-8 support |
22 |
.\" |
.\" |
23 |
in the main |
in the main |
24 |
.\" HREF |
.\" HREF |
25 |
\fBpcre\fR |
\fBpcre\fP |
26 |
.\" |
.\" |
27 |
page. |
page. |
28 |
|
.P |
29 |
|
The remainder of this document discusses the patterns that are supported by |
30 |
|
PCRE when its main matching function, \fBpcre_exec()\fP, is used. |
31 |
|
From release 6.0, PCRE offers a second matching function, |
32 |
|
\fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not |
33 |
|
Perl-compatible. The advantages and disadvantages of the alternative function, |
34 |
|
and how it differs from the normal function, are discussed in the |
35 |
|
.\" HREF |
36 |
|
\fBpcrematching\fP |
37 |
|
.\" |
38 |
|
page. |
39 |
|
.P |
40 |
A regular expression is a pattern that is matched against a subject string from |
A regular expression is a pattern that is matched against a subject string from |
41 |
left to right. Most characters stand for themselves in a pattern, and match the |
left to right. Most characters stand for themselves in a pattern, and match the |
42 |
corresponding characters in the subject. As a trivial example, the pattern |
corresponding characters in the subject. As a trivial example, the pattern |
43 |
|
.sp |
44 |
The quick brown fox |
The quick brown fox |
45 |
|
.sp |
46 |
matches a portion of a subject string that is identical to itself. The power of |
matches a portion of a subject string that is identical to itself. When |
47 |
regular expressions comes from the ability to include alternatives and |
caseless matching is specified (the PCRE_CASELESS option), letters are matched |
48 |
repetitions in the pattern. These are encoded in the pattern by the use of |
independently of case. In UTF-8 mode, PCRE always understands the concept of |
49 |
\fImeta-characters\fR, which do not stand for themselves but instead are |
case for characters whose values are less than 128, so caseless matching is |
50 |
|
always possible. For characters with higher values, the concept of case is |
51 |
|
supported if PCRE is compiled with Unicode property support, but not otherwise. |
52 |
|
If you want to use caseless matching for characters 128 and above, you must |
53 |
|
ensure that PCRE is compiled with Unicode property support as well as with |
54 |
|
UTF-8 support. |
55 |
|
.P |
56 |
|
The power of regular expressions comes from the ability to include alternatives |
57 |
|
and repetitions in the pattern. These are encoded in the pattern by the use of |
58 |
|
\fImetacharacters\fP, which do not stand for themselves but instead are |
59 |
interpreted in some special way. |
interpreted in some special way. |
60 |
|
.P |
61 |
There are two different sets of meta-characters: those that are recognized |
There are two different sets of metacharacters: those that are recognized |
62 |
anywhere in the pattern except within square brackets, and those that are |
anywhere in the pattern except within square brackets, and those that are |
63 |
recognized in square brackets. Outside square brackets, the meta-characters are |
recognized in square brackets. Outside square brackets, the metacharacters are |
64 |
as follows: |
as follows: |
65 |
|
.sp |
66 |
\\ general escape character with several uses |
\e general escape character with several uses |
67 |
^ assert start of string (or line, in multiline mode) |
^ assert start of string (or line, in multiline mode) |
68 |
$ assert end of string (or line, in multiline mode) |
$ assert end of string (or line, in multiline mode) |
69 |
. match any character except newline (by default) |
. match any character except newline (by default) |
78 |
+ 1 or more quantifier |
+ 1 or more quantifier |
79 |
also "possessive quantifier" |
also "possessive quantifier" |
80 |
{ start min/max quantifier |
{ start min/max quantifier |
81 |
|
.sp |
82 |
Part of a pattern that is in square brackets is called a "character class". In |
Part of a pattern that is in square brackets is called a "character class". In |
83 |
a character class the only meta-characters are: |
a character class the only metacharacters are: |
84 |
|
.sp |
85 |
\\ general escape character |
\e general escape character |
86 |
^ negate the class, but only if the first character |
^ negate the class, but only if the first character |
87 |
- indicates character range |
- indicates character range |
88 |
|
.\" JOIN |
89 |
[ POSIX character class (only if followed by POSIX |
[ POSIX character class (only if followed by POSIX |
90 |
syntax) |
syntax) |
91 |
] terminates the character class |
] terminates the character class |
92 |
|
.sp |
93 |
The following sections describe the use of each of the meta-characters. |
The following sections describe the use of each of the metacharacters. |
94 |
|
. |
95 |
.SH BACKSLASH |
.SH BACKSLASH |
96 |
.rs |
.rs |
97 |
.sp |
.sp |
98 |
The backslash character has several uses. Firstly, if it is followed by a |
The backslash character has several uses. Firstly, if it is followed by a |
99 |
non-alphameric character, it takes away any special meaning that character may |
non-alphanumeric character, it takes away any special meaning that character may |
100 |
have. This use of backslash as an escape character applies both inside and |
have. This use of backslash as an escape character applies both inside and |
101 |
outside character classes. |
outside character classes. |
102 |
|
.P |
103 |
For example, if you want to match a * character, you write \\* in the pattern. |
For example, if you want to match a * character, you write \e* in the pattern. |
104 |
This escaping action applies whether or not the following character would |
This escaping action applies whether or not the following character would |
105 |
otherwise be interpreted as a meta-character, so it is always safe to precede a |
otherwise be interpreted as a metacharacter, so it is always safe to precede a |
106 |
non-alphameric with backslash to specify that it stands for itself. In |
non-alphanumeric with backslash to specify that it stands for itself. In |
107 |
particular, if you want to match a backslash, you write \\\\. |
particular, if you want to match a backslash, you write \e\e. |
108 |
|
.P |
109 |
If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the |
If a pattern is compiled with the PCRE_EXTENDED option, whitespace in the |
110 |
pattern (other than in a character class) and characters between a # outside |
pattern (other than in a character class) and characters between a # outside |
111 |
a character class and the next newline character are ignored. An escaping |
a character class and the next newline character are ignored. An escaping |
112 |
backslash can be used to include a whitespace or # character as part of the |
backslash can be used to include a whitespace or # character as part of the |
113 |
pattern. |
pattern. |
114 |
|
.P |
115 |
If you want to remove the special meaning from a sequence of characters, you |
If you want to remove the special meaning from a sequence of characters, you |
116 |
can do so by putting them between \\Q and \\E. This is different from Perl in |
can do so by putting them between \eQ and \eE. This is different from Perl in |
117 |
that $ and @ are handled as literals in \\Q...\\E sequences in PCRE, whereas in |
that $ and @ are handled as literals in \eQ...\eE sequences in PCRE, whereas in |
118 |
Perl, $ and @ cause variable interpolation. Note the following examples: |
Perl, $ and @ cause variable interpolation. Note the following examples: |
119 |
|
.sp |
120 |
Pattern PCRE matches Perl matches |
Pattern PCRE matches Perl matches |
121 |
|
.sp |
122 |
\\Qabc$xyz\\E abc$xyz abc followed by the |
.\" JOIN |
123 |
|
\eQabc$xyz\eE abc$xyz abc followed by the |
124 |
contents of $xyz |
contents of $xyz |
125 |
\\Qabc\\$xyz\\E abc\\$xyz abc\\$xyz |
\eQabc\e$xyz\eE abc\e$xyz abc\e$xyz |
126 |
\\Qabc\\E\\$\\Qxyz\\E abc$xyz abc$xyz |
\eQabc\eE\e$\eQxyz\eE abc$xyz abc$xyz |
127 |
|
.sp |
128 |
The \\Q...\\E sequence is recognized both inside and outside character classes. |
The \eQ...\eE sequence is recognized both inside and outside character classes. |
129 |
|
. |
130 |
|
. |
131 |
|
.\" HTML <a name="digitsafterbackslash"></a> |
132 |
|
.SS "Non-printing characters" |
133 |
|
.rs |
134 |
|
.sp |
135 |
A second use of backslash provides a way of encoding non-printing characters |
A second use of backslash provides a way of encoding non-printing characters |
136 |
in patterns in a visible manner. There is no restriction on the appearance of |
in patterns in a visible manner. There is no restriction on the appearance of |
137 |
non-printing characters, apart from the binary zero that terminates a pattern, |
non-printing characters, apart from the binary zero that terminates a pattern, |
138 |
but when a pattern is being prepared by text editing, it is usually easier to |
but when a pattern is being prepared by text editing, it is usually easier to |
139 |
use one of the following escape sequences than the binary character it |
use one of the following escape sequences than the binary character it |
140 |
represents: |
represents: |
141 |
|
.sp |
142 |
\\a alarm, that is, the BEL character (hex 07) |
\ea alarm, that is, the BEL character (hex 07) |
143 |
\\cx "control-x", where x is any character |
\ecx "control-x", where x is any character |
144 |
\\e escape (hex 1B) |
\ee escape (hex 1B) |
145 |
\\f formfeed (hex 0C) |
\ef formfeed (hex 0C) |
146 |
\\n newline (hex 0A) |
\en newline (hex 0A) |
147 |
\\r carriage return (hex 0D) |
\er carriage return (hex 0D) |
148 |
\\t tab (hex 09) |
\et tab (hex 09) |
149 |
\\ddd character with octal code ddd, or backreference |
\eddd character with octal code ddd, or backreference |
150 |
\\xhh character with hex code hh |
\exhh character with hex code hh |
151 |
\\x{hhh..} character with hex code hhh... (UTF-8 mode only) |
\ex{hhh..} character with hex code hhh... (UTF-8 mode only) |
152 |
|
.sp |
153 |
The precise effect of \\cx is as follows: if x is a lower case letter, it |
The precise effect of \ecx is as follows: if x is a lower case letter, it |
154 |
is converted to upper case. Then bit 6 of the character (hex 40) is inverted. |
is converted to upper case. Then bit 6 of the character (hex 40) is inverted. |
155 |
Thus \\cz becomes hex 1A, but \\c{ becomes hex 3B, while \\c; becomes hex |
Thus \ecz becomes hex 1A, but \ec{ becomes hex 3B, while \ec; becomes hex |
156 |
7B. |
7B. |
157 |
|
.P |
158 |
After \\x, from zero to two hexadecimal digits are read (letters can be in |
After \ex, from zero to two hexadecimal digits are read (letters can be in |
159 |
upper or lower case). In UTF-8 mode, any number of hexadecimal digits may |
upper or lower case). In UTF-8 mode, any number of hexadecimal digits may |
160 |
appear between \\x{ and }, but the value of the character code must be less |
appear between \ex{ and }, but the value of the character code must be less |
161 |
than 2**31 (that is, the maximum hexadecimal value is 7FFFFFFF). If characters |
than 2**31 (that is, the maximum hexadecimal value is 7FFFFFFF). If characters |
162 |
other than hexadecimal digits appear between \\x{ and }, or if there is no |
other than hexadecimal digits appear between \ex{ and }, or if there is no |
163 |
terminating }, this form of escape is not recognized. Instead, the initial |
terminating }, this form of escape is not recognized. Instead, the initial |
164 |
\\x will be interpreted as a basic hexadecimal escape, with no following |
\ex will be interpreted as a basic hexadecimal escape, with no following |
165 |
digits, giving a byte whose value is zero. |
digits, giving a character whose value is zero. |
166 |
|
.P |
167 |
Characters whose value is less than 256 can be defined by either of the two |
Characters whose value is less than 256 can be defined by either of the two |
168 |
syntaxes for \\x when PCRE is in UTF-8 mode. There is no difference in the |
syntaxes for \ex when PCRE is in UTF-8 mode. There is no difference in the |
169 |
way they are handled. For example, \\xdc is exactly the same as \\x{dc}. |
way they are handled. For example, \exdc is exactly the same as \ex{dc}. |
170 |
|
.P |
171 |
After \\0 up to two further octal digits are read. In both cases, if there |
After \e0 up to two further octal digits are read. In both cases, if there |
172 |
are fewer than two digits, just those that are present are used. Thus the |
are fewer than two digits, just those that are present are used. Thus the |
173 |
sequence \\0\\x\\07 specifies two binary zeros followed by a BEL character |
sequence \e0\ex\e07 specifies two binary zeros followed by a BEL character |
174 |
(code value 7). Make sure you supply two digits after the initial zero if the |
(code value 7). Make sure you supply two digits after the initial zero if the |
175 |
character that follows is itself an octal digit. |
pattern character that follows is itself an octal digit. |
176 |
|
.P |
177 |
The handling of a backslash followed by a digit other than 0 is complicated. |
The handling of a backslash followed by a digit other than 0 is complicated. |
178 |
Outside a character class, PCRE reads it and any following digits as a decimal |
Outside a character class, PCRE reads it and any following digits as a decimal |
179 |
number. If the number is less than 10, or if there have been at least that many |
number. If the number is less than 10, or if there have been at least that many |
180 |
previous capturing left parentheses in the expression, the entire sequence is |
previous capturing left parentheses in the expression, the entire sequence is |
181 |
taken as a \fIback reference\fR. A description of how this works is given |
taken as a \fIback reference\fP. A description of how this works is given |
182 |
later, following the discussion of parenthesized subpatterns. |
.\" HTML <a href="#backreferences"> |
183 |
|
.\" </a> |
184 |
|
later, |
185 |
|
.\" |
186 |
|
following the discussion of |
187 |
|
.\" HTML <a href="#subpattern"> |
188 |
|
.\" </a> |
189 |
|
parenthesized subpatterns. |
190 |
|
.\" |
191 |
|
.P |
192 |
Inside a character class, or if the decimal number is greater than 9 and there |
Inside a character class, or if the decimal number is greater than 9 and there |
193 |
have not been that many capturing subpatterns, PCRE re-reads up to three octal |
have not been that many capturing subpatterns, PCRE re-reads up to three octal |
194 |
digits following the backslash, and generates a single byte from the least |
digits following the backslash, and generates a single byte from the least |
195 |
significant 8 bits of the value. Any subsequent digits stand for themselves. |
significant 8 bits of the value. Any subsequent digits stand for themselves. |
196 |
For example: |
For example: |
197 |
|
.sp |
198 |
\\040 is another way of writing a space |
\e040 is another way of writing a space |
199 |
\\40 is the same, provided there are fewer than 40 |
.\" JOIN |
200 |
|
\e40 is the same, provided there are fewer than 40 |
201 |
previous capturing subpatterns |
previous capturing subpatterns |
202 |
\\7 is always a back reference |
\e7 is always a back reference |
203 |
\\11 might be a back reference, or another way of |
.\" JOIN |
204 |
|
\e11 might be a back reference, or another way of |
205 |
writing a tab |
writing a tab |
206 |
\\011 is always a tab |
\e011 is always a tab |
207 |
\\0113 is a tab followed by the character "3" |
\e0113 is a tab followed by the character "3" |
208 |
\\113 might be a back reference, otherwise the |
.\" JOIN |
209 |
|
\e113 might be a back reference, otherwise the |
210 |
character with octal code 113 |
character with octal code 113 |
211 |
\\377 might be a back reference, otherwise |
.\" JOIN |
212 |
|
\e377 might be a back reference, otherwise |
213 |
the byte consisting entirely of 1 bits |
the byte consisting entirely of 1 bits |
214 |
\\81 is either a back reference, or a binary zero |
.\" JOIN |
215 |
|
\e81 is either a back reference, or a binary zero |
216 |
followed by the two characters "8" and "1" |
followed by the two characters "8" and "1" |
217 |
|
.sp |
218 |
Note that octal values of 100 or greater must not be introduced by a leading |
Note that octal values of 100 or greater must not be introduced by a leading |
219 |
zero, because no more than three octal digits are ever read. |
zero, because no more than three octal digits are ever read. |
220 |
|
.P |
221 |
All the sequences that define a single byte value or a single UTF-8 character |
All the sequences that define a single byte value or a single UTF-8 character |
222 |
(in UTF-8 mode) can be used both inside and outside character classes. In |
(in UTF-8 mode) can be used both inside and outside character classes. In |
223 |
addition, inside a character class, the sequence \\b is interpreted as the |
addition, inside a character class, the sequence \eb is interpreted as the |
224 |
backspace character (hex 08). Outside a character class it has a different |
backspace character (hex 08), and the sequence \eX is interpreted as the |
225 |
meaning (see below). |
character "X". Outside a character class, these sequences have different |
226 |
|
meanings |
227 |
The third use of backslash is for specifying generic character types: |
.\" HTML <a href="#uniextseq"> |
228 |
|
.\" </a> |
229 |
\\d any decimal digit |
(see below). |
230 |
\\D any character that is not a decimal digit |
.\" |
231 |
\\s any whitespace character |
. |
232 |
\\S any character that is not a whitespace character |
. |
233 |
\\w any "word" character |
.SS "Generic character types" |
234 |
\\W any "non-word" character |
.rs |
235 |
|
.sp |
236 |
|
The third use of backslash is for specifying generic character types. The |
237 |
|
following are always recognized: |
238 |
|
.sp |
239 |
|
\ed any decimal digit |
240 |
|
\eD any character that is not a decimal digit |
241 |
|
\es any whitespace character |
242 |
|
\eS any character that is not a whitespace character |
243 |
|
\ew any "word" character |
244 |
|
\eW any "non-word" character |
245 |
|
.sp |
246 |
Each pair of escape sequences partitions the complete set of characters into |
Each pair of escape sequences partitions the complete set of characters into |
247 |
two disjoint sets. Any given character matches one, and only one, of each pair. |
two disjoint sets. Any given character matches one, and only one, of each pair. |
248 |
|
.P |
249 |
In UTF-8 mode, characters with values greater than 255 never match \\d, \\s, or |
These character type sequences can appear both inside and outside character |
250 |
\\w, and always match \\D, \\S, and \\W. |
classes. They each match one character of the appropriate type. If the current |
251 |
|
matching point is at the end of the subject string, all of them fail, since |
252 |
For compatibility with Perl, \\s does not match the VT character (code 11). |
there is no character to match. |
253 |
This makes it different from the the POSIX "space" class. The \\s characters |
.P |
254 |
|
For compatibility with Perl, \es does not match the VT character (code 11). |
255 |
|
This makes it different from the the POSIX "space" class. The \es characters |
256 |
are HT (9), LF (10), FF (12), CR (13), and space (32). |
are HT (9), LF (10), FF (12), CR (13), and space (32). |
257 |
|
.P |
258 |
A "word" character is any letter or digit or the underscore character, that is, |
A "word" character is an underscore or any character less than 256 that is a |
259 |
any character which can be part of a Perl "word". The definition of letters and |
letter or digit. The definition of letters and digits is controlled by PCRE's |
260 |
digits is controlled by PCRE's character tables, and may vary if locale- |
low-valued character tables, and may vary if locale-specific matching is taking |
261 |
specific matching is taking place (see |
place (see |
262 |
.\" HTML <a href="pcreapi.html#localesupport"> |
.\" HTML <a href="pcreapi.html#localesupport"> |
263 |
.\" </a> |
.\" </a> |
264 |
"Locale support" |
"Locale support" |
265 |
.\" |
.\" |
266 |
in the |
in the |
267 |
.\" HREF |
.\" HREF |
268 |
\fBpcreapi\fR |
\fBpcreapi\fP |
269 |
.\" |
.\" |
270 |
page). For example, in the "fr" (French) locale, some character codes greater |
page). For example, in the "fr_FR" (French) locale, some character codes |
271 |
than 128 are used for accented letters, and these are matched by \\w. |
greater than 128 are used for accented letters, and these are matched by \ew. |
272 |
|
.P |
273 |
These character type sequences can appear both inside and outside character |
In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or |
274 |
classes. They each match one character of the appropriate type. If the current |
\ew, and always match \eD, \eS, and \eW. This is true even when Unicode |
275 |
matching point is at the end of the subject string, all of them fail, since |
character property support is available. |
276 |
there is no character to match. |
. |
277 |
|
. |
278 |
|
.\" HTML <a name="uniextseq"></a> |
279 |
|
.SS Unicode character properties |
280 |
|
.rs |
281 |
|
.sp |
282 |
|
When PCRE is built with Unicode character property support, three additional |
283 |
|
escape sequences to match generic character types are available when UTF-8 mode |
284 |
|
is selected. They are: |
285 |
|
.sp |
286 |
|
\ep{\fIxx\fP} a character with the \fIxx\fP property |
287 |
|
\eP{\fIxx\fP} a character without the \fIxx\fP property |
288 |
|
\eX an extended Unicode sequence |
289 |
|
.sp |
290 |
|
The property names represented by \fIxx\fP above are limited to the |
291 |
|
Unicode general category properties. Each character has exactly one such |
292 |
|
property, specified by a two-letter abbreviation. For compatibility with Perl, |
293 |
|
negation can be specified by including a circumflex between the opening brace |
294 |
|
and the property name. For example, \ep{^Lu} is the same as \eP{Lu}. |
295 |
|
.P |
296 |
|
If only one letter is specified with \ep or \eP, it includes all the properties |
297 |
|
that start with that letter. In this case, in the absence of negation, the |
298 |
|
curly brackets in the escape sequence are optional; these two examples have |
299 |
|
the same effect: |
300 |
|
.sp |
301 |
|
\ep{L} |
302 |
|
\epL |
303 |
|
.sp |
304 |
|
The following property codes are supported: |
305 |
|
.sp |
306 |
|
C Other |
307 |
|
Cc Control |
308 |
|
Cf Format |
309 |
|
Cn Unassigned |
310 |
|
Co Private use |
311 |
|
Cs Surrogate |
312 |
|
.sp |
313 |
|
L Letter |
314 |
|
Ll Lower case letter |
315 |
|
Lm Modifier letter |
316 |
|
Lo Other letter |
317 |
|
Lt Title case letter |
318 |
|
Lu Upper case letter |
319 |
|
.sp |
320 |
|
M Mark |
321 |
|
Mc Spacing mark |
322 |
|
Me Enclosing mark |
323 |
|
Mn Non-spacing mark |
324 |
|
.sp |
325 |
|
N Number |
326 |
|
Nd Decimal number |
327 |
|
Nl Letter number |
328 |
|
No Other number |
329 |
|
.sp |
330 |
|
P Punctuation |
331 |
|
Pc Connector punctuation |
332 |
|
Pd Dash punctuation |
333 |
|
Pe Close punctuation |
334 |
|
Pf Final punctuation |
335 |
|
Pi Initial punctuation |
336 |
|
Po Other punctuation |
337 |
|
Ps Open punctuation |
338 |
|
.sp |
339 |
|
S Symbol |
340 |
|
Sc Currency symbol |
341 |
|
Sk Modifier symbol |
342 |
|
Sm Mathematical symbol |
343 |
|
So Other symbol |
344 |
|
.sp |
345 |
|
Z Separator |
346 |
|
Zl Line separator |
347 |
|
Zp Paragraph separator |
348 |
|
Zs Space separator |
349 |
|
.sp |
350 |
|
Extended properties such as "Greek" or "InMusicalSymbols" are not supported by |
351 |
|
PCRE. |
352 |
|
.P |
353 |
|
Specifying caseless matching does not affect these escape sequences. For |
354 |
|
example, \ep{Lu} always matches only upper case letters. |
355 |
|
.P |
356 |
|
The \eX escape matches any number of Unicode characters that form an extended |
357 |
|
Unicode sequence. \eX is equivalent to |
358 |
|
.sp |
359 |
|
(?>\ePM\epM*) |
360 |
|
.sp |
361 |
|
That is, it matches a character without the "mark" property, followed by zero |
362 |
|
or more characters with the "mark" property, and treats the sequence as an |
363 |
|
atomic group |
364 |
|
.\" HTML <a href="#atomicgroup"> |
365 |
|
.\" </a> |
366 |
|
(see below). |
367 |
|
.\" |
368 |
|
Characters with the "mark" property are typically accents that affect the |
369 |
|
preceding character. |
370 |
|
.P |
371 |
|
Matching characters by Unicode property is not fast, because PCRE has to search |
372 |
|
a structure that contains data for over fifteen thousand characters. That is |
373 |
|
why the traditional escape sequences such as \ed and \ew do not use Unicode |
374 |
|
properties in PCRE. |
375 |
|
. |
376 |
|
. |
377 |
|
.\" HTML <a name="smallassertions"></a> |
378 |
|
.SS "Simple assertions" |
379 |
|
.rs |
380 |
|
.sp |
381 |
The fourth use of backslash is for certain simple assertions. An assertion |
The fourth use of backslash is for certain simple assertions. An assertion |
382 |
specifies a condition that has to be met at a particular point in a match, |
specifies a condition that has to be met at a particular point in a match, |
383 |
without consuming any characters from the subject string. The use of |
without consuming any characters from the subject string. The use of |
384 |
subpatterns for more complicated assertions is described below. The backslashed |
subpatterns for more complicated assertions is described |
385 |
assertions are |
.\" HTML <a href="#bigassertions"> |
386 |
|
.\" </a> |
387 |
\\b matches at a word boundary |
below. |
388 |
\\B matches when not at a word boundary |
.\" |
389 |
\\A matches at start of subject |
The backslashed |
390 |
\\Z matches at end of subject or before newline at end |
assertions are: |
391 |
\\z matches at end of subject |
.sp |
392 |
\\G matches at first matching position in subject |
\eb matches at a word boundary |
393 |
|
\eB matches when not at a word boundary |
394 |
These assertions may not appear in character classes (but note that \\b has a |
\eA matches at start of subject |
395 |
|
\eZ matches at end of subject or before newline at end |
396 |
|
\ez matches at end of subject |
397 |
|
\eG matches at first matching position in subject |
398 |
|
.sp |
399 |
|
These assertions may not appear in character classes (but note that \eb has a |
400 |
different meaning, namely the backspace character, inside a character class). |
different meaning, namely the backspace character, inside a character class). |
401 |
|
.P |
402 |
A word boundary is a position in the subject string where the current character |
A word boundary is a position in the subject string where the current character |
403 |
and the previous character do not both match \\w or \\W (i.e. one matches |
and the previous character do not both match \ew or \eW (i.e. one matches |
404 |
\\w and the other matches \\W), or the start or end of the string if the |
\ew and the other matches \eW), or the start or end of the string if the |
405 |
first or last character matches \\w, respectively. |
first or last character matches \ew, respectively. |
406 |
|
.P |
407 |
The \\A, \\Z, and \\z assertions differ from the traditional circumflex and |
The \eA, \eZ, and \ez assertions differ from the traditional circumflex and |
408 |
dollar (described below) in that they only ever match at the very start and end |
dollar (described in the next section) in that they only ever match at the very |
409 |
of the subject string, whatever options are set. Thus, they are independent of |
start and end of the subject string, whatever options are set. Thus, they are |
410 |
multiline mode. |
independent of multiline mode. These three assertions are not affected by the |
411 |
|
PCRE_NOTBOL or PCRE_NOTEOL options, which affect only the behaviour of the |
412 |
They are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options. If the |
circumflex and dollar metacharacters. However, if the \fIstartoffset\fP |
413 |
\fIstartoffset\fR argument of \fBpcre_exec()\fR is non-zero, indicating that |
argument of \fBpcre_exec()\fP is non-zero, indicating that matching is to start |
414 |
matching is to start at a point other than the beginning of the subject, \\A |
at a point other than the beginning of the subject, \eA can never match. The |
415 |
can never match. The difference between \\Z and \\z is that \\Z matches before |
difference between \eZ and \ez is that \eZ matches before a newline that is the |
416 |
a newline that is the last character of the string as well as at the end of the |
last character of the string as well as at the end of the string, whereas \ez |
417 |
string, whereas \\z matches only at the end. |
matches only at the end. |
418 |
|
.P |
419 |
The \\G assertion is true only when the current matching position is at the |
The \eG assertion is true only when the current matching position is at the |
420 |
start point of the match, as specified by the \fIstartoffset\fR argument of |
start point of the match, as specified by the \fIstartoffset\fP argument of |
421 |
\fBpcre_exec()\fR. It differs from \\A when the value of \fIstartoffset\fR is |
\fBpcre_exec()\fP. It differs from \eA when the value of \fIstartoffset\fP is |
422 |
non-zero. By calling \fBpcre_exec()\fR multiple times with appropriate |
non-zero. By calling \fBpcre_exec()\fP multiple times with appropriate |
423 |
arguments, you can mimic Perl's /g option, and it is in this kind of |
arguments, you can mimic Perl's /g option, and it is in this kind of |
424 |
implementation where \\G can be useful. |
implementation where \eG can be useful. |
425 |
|
.P |
426 |
Note, however, that PCRE's interpretation of \\G, as the start of the current |
Note, however, that PCRE's interpretation of \eG, as the start of the current |
427 |
match, is subtly different from Perl's, which defines it as the end of the |
match, is subtly different from Perl's, which defines it as the end of the |
428 |
previous match. In Perl, these can be different when the previously matched |
previous match. In Perl, these can be different when the previously matched |
429 |
string was empty. Because PCRE does just one match at a time, it cannot |
string was empty. Because PCRE does just one match at a time, it cannot |
430 |
reproduce this behaviour. |
reproduce this behaviour. |
431 |
|
.P |
432 |
If all the alternatives of a pattern begin with \\G, the expression is anchored |
If all the alternatives of a pattern begin with \eG, the expression is anchored |
433 |
to the starting match position, and the "anchored" flag is set in the compiled |
to the starting match position, and the "anchored" flag is set in the compiled |
434 |
regular expression. |
regular expression. |
435 |
|
. |
436 |
.SH CIRCUMFLEX AND DOLLAR |
. |
437 |
|
.SH "CIRCUMFLEX AND DOLLAR" |
438 |
.rs |
.rs |
439 |
.sp |
.sp |
440 |
Outside a character class, in the default matching mode, the circumflex |
Outside a character class, in the default matching mode, the circumflex |
441 |
character is an assertion which is true only if the current matching point is |
character is an assertion that is true only if the current matching point is |
442 |
at the start of the subject string. If the \fIstartoffset\fR argument of |
at the start of the subject string. If the \fIstartoffset\fP argument of |
443 |
\fBpcre_exec()\fR is non-zero, circumflex can never match if the PCRE_MULTILINE |
\fBpcre_exec()\fP is non-zero, circumflex can never match if the PCRE_MULTILINE |
444 |
option is unset. Inside a character class, circumflex has an entirely different |
option is unset. Inside a character class, circumflex has an entirely different |
445 |
meaning (see below). |
meaning |
446 |
|
.\" HTML <a href="#characterclass"> |
447 |
|
.\" </a> |
448 |
|
(see below). |
449 |
|
.\" |
450 |
|
.P |
451 |
Circumflex need not be the first character of the pattern if a number of |
Circumflex need not be the first character of the pattern if a number of |
452 |
alternatives are involved, but it should be the first thing in each alternative |
alternatives are involved, but it should be the first thing in each alternative |
453 |
in which it appears if the pattern is ever to match that branch. If all |
in which it appears if the pattern is ever to match that branch. If all |
455 |
constrained to match only at the start of the subject, it is said to be an |
constrained to match only at the start of the subject, it is said to be an |
456 |
"anchored" pattern. (There are also other constructs that can cause a pattern |
"anchored" pattern. (There are also other constructs that can cause a pattern |
457 |
to be anchored.) |
to be anchored.) |
458 |
|
.P |
459 |
A dollar character is an assertion which is true only if the current matching |
A dollar character is an assertion that is true only if the current matching |
460 |
point is at the end of the subject string, or immediately before a newline |
point is at the end of the subject string, or immediately before a newline |
461 |
character that is the last character in the string (by default). Dollar need |
character that is the last character in the string (by default). Dollar need |
462 |
not be the last character of the pattern if a number of alternatives are |
not be the last character of the pattern if a number of alternatives are |
463 |
involved, but it should be the last item in any branch in which it appears. |
involved, but it should be the last item in any branch in which it appears. |
464 |
Dollar has no special meaning in a character class. |
Dollar has no special meaning in a character class. |
465 |
|
.P |
466 |
The meaning of dollar can be changed so that it matches only at the very end of |
The meaning of dollar can be changed so that it matches only at the very end of |
467 |
the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This |
the string, by setting the PCRE_DOLLAR_ENDONLY option at compile time. This |
468 |
does not affect the \\Z assertion. |
does not affect the \eZ assertion. |
469 |
|
.P |
470 |
The meanings of the circumflex and dollar characters are changed if the |
The meanings of the circumflex and dollar characters are changed if the |
471 |
PCRE_MULTILINE option is set. When this is the case, they match immediately |
PCRE_MULTILINE option is set. When this is the case, they match immediately |
472 |
after and immediately before an internal newline character, respectively, in |
after and immediately before an internal newline character, respectively, in |
473 |
addition to matching at the start and end of the subject string. For example, |
addition to matching at the start and end of the subject string. For example, |
474 |
the pattern /^abc$/ matches the subject string "def\\nabc" in multiline mode, |
the pattern /^abc$/ matches the subject string "def\enabc" (where \en |
475 |
but not otherwise. Consequently, patterns that are anchored in single line mode |
represents a newline character) in multiline mode, but not otherwise. |
476 |
because all branches start with ^ are not anchored in multiline mode, and a |
Consequently, patterns that are anchored in single line mode because all |
477 |
match for circumflex is possible when the \fIstartoffset\fR argument of |
branches start with ^ are not anchored in multiline mode, and a match for |
478 |
\fBpcre_exec()\fR is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if |
circumflex is possible when the \fIstartoffset\fP argument of \fBpcre_exec()\fP |
479 |
PCRE_MULTILINE is set. |
is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if PCRE_MULTILINE is |
480 |
|
set. |
481 |
Note that the sequences \\A, \\Z, and \\z can be used to match the start and |
.P |
482 |
|
Note that the sequences \eA, \eZ, and \ez can be used to match the start and |
483 |
end of the subject in both modes, and if all branches of a pattern start with |
end of the subject in both modes, and if all branches of a pattern start with |
484 |
\\A it is always anchored, whether PCRE_MULTILINE is set or not. |
\eA it is always anchored, whether PCRE_MULTILINE is set or not. |
485 |
|
. |
486 |
.SH FULL STOP (PERIOD, DOT) |
. |
487 |
|
.SH "FULL STOP (PERIOD, DOT)" |
488 |
.rs |
.rs |
489 |
.sp |
.sp |
490 |
Outside a character class, a dot in the pattern matches any one character in |
Outside a character class, a dot in the pattern matches any one character in |
491 |
the subject, including a non-printing character, but not (by default) newline. |
the subject, including a non-printing character, but not (by default) newline. |
492 |
In UTF-8 mode, a dot matches any UTF-8 character, which might be more than one |
In UTF-8 mode, a dot matches any UTF-8 character, which might be more than one |
493 |
byte long, except (by default) for newline. If the PCRE_DOTALL option is set, |
byte long, except (by default) newline. If the PCRE_DOTALL option is set, |
494 |
dots match newlines as well. The handling of dot is entirely independent of the |
dots match newlines as well. The handling of dot is entirely independent of the |
495 |
handling of circumflex and dollar, the only relationship being that they both |
handling of circumflex and dollar, the only relationship being that they both |
496 |
involve newline characters. Dot has no special meaning in a character class. |
involve newline characters. Dot has no special meaning in a character class. |
497 |
|
. |
498 |
.SH MATCHING A SINGLE BYTE |
. |
499 |
|
.SH "MATCHING A SINGLE BYTE" |
500 |
.rs |
.rs |
501 |
.sp |
.sp |
502 |
Outside a character class, the escape sequence \\C matches any one byte, both |
Outside a character class, the escape sequence \eC matches any one byte, both |
503 |
in and out of UTF-8 mode. Unlike a dot, it always matches a newline. The |
in and out of UTF-8 mode. Unlike a dot, it can match a newline. The feature is |
504 |
feature is provided in Perl in order to match individual bytes in UTF-8 mode. |
provided in Perl in order to match individual bytes in UTF-8 mode. Because it |
505 |
Because it breaks up UTF-8 characters into individual bytes, what remains in |
breaks up UTF-8 characters into individual bytes, what remains in the string |
506 |
the string may be a malformed UTF-8 string. For this reason it is best avoided. |
may be a malformed UTF-8 string. For this reason, the \eC escape sequence is |
507 |
|
best avoided. |
508 |
PCRE does not allow \\C to appear in lookbehind assertions (see below), because |
.P |
509 |
in UTF-8 mode it makes it impossible to calculate the length of the lookbehind. |
PCRE does not allow \eC to appear in lookbehind assertions |
510 |
|
.\" HTML <a href="#lookbehind"> |
511 |
.SH SQUARE BRACKETS |
.\" </a> |
512 |
|
(described below), |
513 |
|
.\" |
514 |
|
because in UTF-8 mode this would make it impossible to calculate the length of |
515 |
|
the lookbehind. |
516 |
|
. |
517 |
|
. |
518 |
|
.\" HTML <a name="characterclass"></a> |
519 |
|
.SH "SQUARE BRACKETS AND CHARACTER CLASSES" |
520 |
.rs |
.rs |
521 |
.sp |
.sp |
522 |
An opening square bracket introduces a character class, terminated by a closing |
An opening square bracket introduces a character class, terminated by a closing |
524 |
closing square bracket is required as a member of the class, it should be the |
closing square bracket is required as a member of the class, it should be the |
525 |
first data character in the class (after an initial circumflex, if present) or |
first data character in the class (after an initial circumflex, if present) or |
526 |
escaped with a backslash. |
escaped with a backslash. |
527 |
|
.P |
528 |
A character class matches a single character in the subject. In UTF-8 mode, the |
A character class matches a single character in the subject. In UTF-8 mode, the |
529 |
character may occupy more than one byte. A matched character must be in the set |
character may occupy more than one byte. A matched character must be in the set |
530 |
of characters defined by the class, unless the first character in the class |
of characters defined by the class, unless the first character in the class |
532 |
the set defined by the class. If a circumflex is actually required as a member |
the set defined by the class. If a circumflex is actually required as a member |
533 |
of the class, ensure it is not the first character, or escape it with a |
of the class, ensure it is not the first character, or escape it with a |
534 |
backslash. |
backslash. |
535 |
|
.P |
536 |
For example, the character class [aeiou] matches any lower case vowel, while |
For example, the character class [aeiou] matches any lower case vowel, while |
537 |
[^aeiou] matches any character that is not a lower case vowel. Note that a |
[^aeiou] matches any character that is not a lower case vowel. Note that a |
538 |
circumflex is just a convenient notation for specifying the characters which |
circumflex is just a convenient notation for specifying the characters that |
539 |
are in the class by enumerating those that are not. It is not an assertion: it |
are in the class by enumerating those that are not. A class that starts with a |
540 |
still consumes a character from the subject string, and fails if the current |
circumflex is not an assertion: it still consumes a character from the subject |
541 |
pointer is at the end of the string. |
string, and therefore it fails if the current pointer is at the end of the |
542 |
|
string. |
543 |
|
.P |
544 |
In UTF-8 mode, characters with values greater than 255 can be included in a |
In UTF-8 mode, characters with values greater than 255 can be included in a |
545 |
class as a literal string of bytes, or by using the \\x{ escaping mechanism. |
class as a literal string of bytes, or by using the \ex{ escaping mechanism. |
546 |
|
.P |
547 |
When caseless matching is set, any letters in a class represent both their |
When caseless matching is set, any letters in a class represent both their |
548 |
upper case and lower case versions, so for example, a caseless [aeiou] matches |
upper case and lower case versions, so for example, a caseless [aeiou] matches |
549 |
"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a |
"A" as well as "a", and a caseless [^aeiou] does not match "A", whereas a |
550 |
caseful version would. PCRE does not support the concept of case for characters |
caseful version would. In UTF-8 mode, PCRE always understands the concept of |
551 |
with values greater than 255. |
case for characters whose values are less than 128, so caseless matching is |
552 |
|
always possible. For characters with higher values, the concept of case is |
553 |
|
supported if PCRE is compiled with Unicode property support, but not otherwise. |
554 |
|
If you want to use caseless matching for characters 128 and above, you must |
555 |
|
ensure that PCRE is compiled with Unicode property support as well as with |
556 |
|
UTF-8 support. |
557 |
|
.P |
558 |
The newline character is never treated in any special way in character classes, |
The newline character is never treated in any special way in character classes, |
559 |
whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class |
whatever the setting of the PCRE_DOTALL or PCRE_MULTILINE options is. A class |
560 |
such as [^a] will always match a newline. |
such as [^a] will always match a newline. |
561 |
|
.P |
562 |
The minus (hyphen) character can be used to specify a range of characters in a |
The minus (hyphen) character can be used to specify a range of characters in a |
563 |
character class. For example, [d-m] matches any letter between d and m, |
character class. For example, [d-m] matches any letter between d and m, |
564 |
inclusive. If a minus character is required in a class, it must be escaped with |
inclusive. If a minus character is required in a class, it must be escaped with |
565 |
a backslash or appear in a position where it cannot be interpreted as |
a backslash or appear in a position where it cannot be interpreted as |
566 |
indicating a range, typically as the first or last character in the class. |
indicating a range, typically as the first or last character in the class. |
567 |
|
.P |
568 |
It is not possible to have the literal character "]" as the end character of a |
It is not possible to have the literal character "]" as the end character of a |
569 |
range. A pattern such as [W-]46] is interpreted as a class of two characters |
range. A pattern such as [W-]46] is interpreted as a class of two characters |
570 |
("W" and "-") followed by a literal string "46]", so it would match "W46]" or |
("W" and "-") followed by a literal string "46]", so it would match "W46]" or |
571 |
"-46]". However, if the "]" is escaped with a backslash it is interpreted as |
"-46]". However, if the "]" is escaped with a backslash it is interpreted as |
572 |
the end of range, so [W-\\]46] is interpreted as a single class containing a |
the end of range, so [W-\e]46] is interpreted as a class containing a range |
573 |
range followed by two separate characters. The octal or hexadecimal |
followed by two other characters. The octal or hexadecimal representation of |
574 |
representation of "]" can also be used to end a range. |
"]" can also be used to end a range. |
575 |
|
.P |
576 |
Ranges operate in the collating sequence of character values. They can also be |
Ranges operate in the collating sequence of character values. They can also be |
577 |
used for characters specified numerically, for example [\\000-\\037]. In UTF-8 |
used for characters specified numerically, for example [\e000-\e037]. In UTF-8 |
578 |
mode, ranges can include characters whose values are greater than 255, for |
mode, ranges can include characters whose values are greater than 255, for |
579 |
example [\\x{100}-\\x{2ff}]. |
example [\ex{100}-\ex{2ff}]. |
580 |
|
.P |
581 |
If a range that includes letters is used when caseless matching is set, it |
If a range that includes letters is used when caseless matching is set, it |
582 |
matches the letters in either case. For example, [W-c] is equivalent to |
matches the letters in either case. For example, [W-c] is equivalent to |
583 |
[][\\^_`wxyzabc], matched caselessly, and if character tables for the "fr" |
[][\e\e^_`wxyzabc], matched caselessly, and in non-UTF-8 mode, if character |
584 |
locale are in use, [\\xc8-\\xcb] matches accented E characters in both cases. |
tables for the "fr_FR" locale are in use, [\exc8-\excb] matches accented E |
585 |
|
characters in both cases. In UTF-8 mode, PCRE supports the concept of case for |
586 |
The character types \\d, \\D, \\s, \\S, \\w, and \\W may also appear in a |
characters with values greater than 128 only when it is compiled with Unicode |
587 |
character class, and add the characters that they match to the class. For |
property support. |
588 |
example, [\\dABCDEF] matches any hexadecimal digit. A circumflex can |
.P |
589 |
|
The character types \ed, \eD, \ep, \eP, \es, \eS, \ew, and \eW may also appear |
590 |
|
in a character class, and add the characters that they match to the class. For |
591 |
|
example, [\edABCDEF] matches any hexadecimal digit. A circumflex can |
592 |
conveniently be used with the upper case character types to specify a more |
conveniently be used with the upper case character types to specify a more |
593 |
restricted set of characters than the matching lower case type. For example, |
restricted set of characters than the matching lower case type. For example, |
594 |
the class [^\\W_] matches any letter or digit, but not underscore. |
the class [^\eW_] matches any letter or digit, but not underscore. |
595 |
|
.P |
596 |
All non-alphameric characters other than \\, -, ^ (at the start) and the |
The only metacharacters that are recognized in character classes are backslash, |
597 |
terminating ] are non-special in character classes, but it does no harm if they |
hyphen (only where it can be interpreted as specifying a range), circumflex |
598 |
are escaped. |
(only at the start), opening square bracket (only when it can be interpreted as |
599 |
|
introducing a POSIX class name - see the next section), and the terminating |
600 |
.SH POSIX CHARACTER CLASSES |
closing square bracket. However, escaping other non-alphanumeric characters |
601 |
|
does no harm. |
602 |
|
. |
603 |
|
. |
604 |
|
.SH "POSIX CHARACTER CLASSES" |
605 |
.rs |
.rs |
606 |
.sp |
.sp |
607 |
Perl supports the POSIX notation for character classes, which uses names |
Perl supports the POSIX notation for character classes. This uses names |
608 |
enclosed by [: and :] within the enclosing square brackets. PCRE also supports |
enclosed by [: and :] within the enclosing square brackets. PCRE also supports |
609 |
this notation. For example, |
this notation. For example, |
610 |
|
.sp |
611 |
[01[:alpha:]%] |
[01[:alpha:]%] |
612 |
|
.sp |
613 |
matches "0", "1", any alphabetic character, or "%". The supported class names |
matches "0", "1", any alphabetic character, or "%". The supported class names |
614 |
are |
are |
615 |
|
.sp |
616 |
alnum letters and digits |
alnum letters and digits |
617 |
alpha letters |
alpha letters |
618 |
ascii character codes 0 - 127 |
ascii character codes 0 - 127 |
619 |
blank space or tab only |
blank space or tab only |
620 |
cntrl control characters |
cntrl control characters |
621 |
digit decimal digits (same as \\d) |
digit decimal digits (same as \ed) |
622 |
graph printing characters, excluding space |
graph printing characters, excluding space |
623 |
lower lower case letters |
lower lower case letters |
624 |
print printing characters, including space |
print printing characters, including space |
625 |
punct printing characters, excluding letters and digits |
punct printing characters, excluding letters and digits |
626 |
space white space (not quite the same as \\s) |
space white space (not quite the same as \es) |
627 |
upper upper case letters |
upper upper case letters |
628 |
word "word" characters (same as \\w) |
word "word" characters (same as \ew) |
629 |
xdigit hexadecimal digits |
xdigit hexadecimal digits |
630 |
|
.sp |
631 |
The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and |
The "space" characters are HT (9), LF (10), VT (11), FF (12), CR (13), and |
632 |
space (32). Notice that this list includes the VT character (code 11). This |
space (32). Notice that this list includes the VT character (code 11). This |
633 |
makes "space" different to \\s, which does not include VT (for Perl |
makes "space" different to \es, which does not include VT (for Perl |
634 |
compatibility). |
compatibility). |
635 |
|
.P |
636 |
The name "word" is a Perl extension, and "blank" is a GNU extension from Perl |
The name "word" is a Perl extension, and "blank" is a GNU extension from Perl |
637 |
5.8. Another Perl extension is negation, which is indicated by a ^ character |
5.8. Another Perl extension is negation, which is indicated by a ^ character |
638 |
after the colon. For example, |
after the colon. For example, |
639 |
|
.sp |
640 |
[12[:^digit:]] |
[12[:^digit:]] |
641 |
|
.sp |
642 |
matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX |
matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the POSIX |
643 |
syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not |
syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not |
644 |
supported, and an error is given if they are encountered. |
supported, and an error is given if they are encountered. |
645 |
|
.P |
646 |
In UTF-8 mode, characters with values greater than 255 do not match any of |
In UTF-8 mode, characters with values greater than 128 do not match any of |
647 |
the POSIX character classes. |
the POSIX character classes. |
648 |
|
. |
649 |
.SH VERTICAL BAR |
. |
650 |
|
.SH "VERTICAL BAR" |
651 |
.rs |
.rs |
652 |
.sp |
.sp |
653 |
Vertical bar characters are used to separate alternative patterns. For example, |
Vertical bar characters are used to separate alternative patterns. For example, |
654 |
the pattern |
the pattern |
655 |
|
.sp |
656 |
gilbert|sullivan |
gilbert|sullivan |
657 |
|
.sp |
658 |
matches either "gilbert" or "sullivan". Any number of alternatives may appear, |
matches either "gilbert" or "sullivan". Any number of alternatives may appear, |
659 |
and an empty alternative is permitted (matching the empty string). |
and an empty alternative is permitted (matching the empty string). |
660 |
The matching process tries each alternative in turn, from left to right, |
The matching process tries each alternative in turn, from left to right, |
661 |
and the first one that succeeds is used. If the alternatives are within a |
and the first one that succeeds is used. If the alternatives are within a |
662 |
subpattern (defined below), "succeeds" means matching the rest of the main |
subpattern |
663 |
pattern as well as the alternative in the subpattern. |
.\" HTML <a href="#subpattern"> |
664 |
|
.\" </a> |
665 |
.SH INTERNAL OPTION SETTING |
(defined below), |
666 |
|
.\" |
667 |
|
"succeeds" means matching the rest of the main pattern as well as the |
668 |
|
alternative in the subpattern. |
669 |
|
. |
670 |
|
. |
671 |
|
.SH "INTERNAL OPTION SETTING" |
672 |
.rs |
.rs |
673 |
.sp |
.sp |
674 |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
675 |
PCRE_EXTENDED options can be changed from within the pattern by a sequence of |
PCRE_EXTENDED options can be changed from within the pattern by a sequence of |
676 |
Perl option letters enclosed between "(?" and ")". The option letters are |
Perl option letters enclosed between "(?" and ")". The option letters are |
677 |
|
.sp |
678 |
i for PCRE_CASELESS |
i for PCRE_CASELESS |
679 |
m for PCRE_MULTILINE |
m for PCRE_MULTILINE |
680 |
s for PCRE_DOTALL |
s for PCRE_DOTALL |
681 |
x for PCRE_EXTENDED |
x for PCRE_EXTENDED |
682 |
|
.sp |
683 |
For example, (?im) sets caseless, multiline matching. It is also possible to |
For example, (?im) sets caseless, multiline matching. It is also possible to |
684 |
unset these options by preceding the letter with a hyphen, and a combined |
unset these options by preceding the letter with a hyphen, and a combined |
685 |
setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and |
setting and unsetting such as (?im-sx), which sets PCRE_CASELESS and |
686 |
PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also |
PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, is also |
687 |
permitted. If a letter appears both before and after the hyphen, the option is |
permitted. If a letter appears both before and after the hyphen, the option is |
688 |
unset. |
unset. |
689 |
|
.P |
690 |
When an option change occurs at top level (that is, not inside subpattern |
When an option change occurs at top level (that is, not inside subpattern |
691 |
parentheses), the change applies to the remainder of the pattern that follows. |
parentheses), the change applies to the remainder of the pattern that follows. |
692 |
If the change is placed right at the start of a pattern, PCRE extracts it into |
If the change is placed right at the start of a pattern, PCRE extracts it into |
693 |
the global options (and it will therefore show up in data extracted by the |
the global options (and it will therefore show up in data extracted by the |
694 |
\fBpcre_fullinfo()\fR function). |
\fBpcre_fullinfo()\fP function). |
695 |
|
.P |
696 |
An option change within a subpattern affects only that part of the current |
An option change within a subpattern affects only that part of the current |
697 |
pattern that follows it, so |
pattern that follows it, so |
698 |
|
.sp |
699 |
(a(?i)b)c |
(a(?i)b)c |
700 |
|
.sp |
701 |
matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). |
matches abc and aBc and no other strings (assuming PCRE_CASELESS is not used). |
702 |
By this means, options can be made to have different settings in different |
By this means, options can be made to have different settings in different |
703 |
parts of the pattern. Any changes made in one alternative do carry on |
parts of the pattern. Any changes made in one alternative do carry on |
704 |
into subsequent branches within the same subpattern. For example, |
into subsequent branches within the same subpattern. For example, |
705 |
|
.sp |
706 |
(a(?i)b|c) |
(a(?i)b|c) |
707 |
|
.sp |
708 |
matches "ab", "aB", "c", and "C", even though when matching "C" the first |
matches "ab", "aB", "c", and "C", even though when matching "C" the first |
709 |
branch is abandoned before the option setting. This is because the effects of |
branch is abandoned before the option setting. This is because the effects of |
710 |
option settings happen at compile time. There would be some very weird |
option settings happen at compile time. There would be some very weird |
711 |
behaviour otherwise. |
behaviour otherwise. |
712 |
|
.P |
713 |
The PCRE-specific options PCRE_UNGREEDY and PCRE_EXTRA can be changed in the |
The PCRE-specific options PCRE_UNGREEDY and PCRE_EXTRA can be changed in the |
714 |
same way as the Perl-compatible options by using the characters U and X |
same way as the Perl-compatible options by using the characters U and X |
715 |
respectively. The (?X) flag setting is special in that it must always occur |
respectively. The (?X) flag setting is special in that it must always occur |
716 |
earlier in the pattern than any of the additional features it turns on, even |
earlier in the pattern than any of the additional features it turns on, even |
717 |
when it is at top level. It is best put at the start. |
when it is at top level. It is best to put it at the start. |
718 |
|
. |
719 |
|
. |
720 |
|
.\" HTML <a name="subpattern"></a> |
721 |
.SH SUBPATTERNS |
.SH SUBPATTERNS |
722 |
.rs |
.rs |
723 |
.sp |
.sp |
724 |
Subpatterns are delimited by parentheses (round brackets), which can be nested. |
Subpatterns are delimited by parentheses (round brackets), which can be nested. |
725 |
Marking part of a pattern as a subpattern does two things: |
Turning part of a pattern into a subpattern does two things: |
726 |
|
.sp |
727 |
1. It localizes a set of alternatives. For example, the pattern |
1. It localizes a set of alternatives. For example, the pattern |
728 |
|
.sp |
729 |
cat(aract|erpillar|) |
cat(aract|erpillar|) |
730 |
|
.sp |
731 |
matches one of the words "cat", "cataract", or "caterpillar". Without the |
matches one of the words "cat", "cataract", or "caterpillar". Without the |
732 |
parentheses, it would match "cataract", "erpillar" or the empty string. |
parentheses, it would match "cataract", "erpillar" or the empty string. |
733 |
|
.sp |
734 |
2. It sets up the subpattern as a capturing subpattern (as defined above). |
2. It sets up the subpattern as a capturing subpattern. This means that, when |
735 |
When the whole pattern matches, that portion of the subject string that matched |
the whole pattern matches, that portion of the subject string that matched the |
736 |
the subpattern is passed back to the caller via the \fIovector\fR argument of |
subpattern is passed back to the caller via the \fIovector\fP argument of |
737 |
\fBpcre_exec()\fR. Opening parentheses are counted from left to right (starting |
\fBpcre_exec()\fP. Opening parentheses are counted from left to right (starting |
738 |
from 1) to obtain the numbers of the capturing subpatterns. |
from 1) to obtain numbers for the capturing subpatterns. |
739 |
|
.P |
740 |
For example, if the string "the red king" is matched against the pattern |
For example, if the string "the red king" is matched against the pattern |
741 |
|
.sp |
742 |
the ((red|white) (king|queen)) |
the ((red|white) (king|queen)) |
743 |
|
.sp |
744 |
the captured substrings are "red king", "red", and "king", and are numbered 1, |
the captured substrings are "red king", "red", and "king", and are numbered 1, |
745 |
2, and 3, respectively. |
2, and 3, respectively. |
746 |
|
.P |
747 |
The fact that plain parentheses fulfil two functions is not always helpful. |
The fact that plain parentheses fulfil two functions is not always helpful. |
748 |
There are often times when a grouping subpattern is required without a |
There are often times when a grouping subpattern is required without a |
749 |
capturing requirement. If an opening parenthesis is followed by a question mark |
capturing requirement. If an opening parenthesis is followed by a question mark |
750 |
and a colon, the subpattern does not do any capturing, and is not counted when |
and a colon, the subpattern does not do any capturing, and is not counted when |
751 |
computing the number of any subsequent capturing subpatterns. For example, if |
computing the number of any subsequent capturing subpatterns. For example, if |
752 |
the string "the white queen" is matched against the pattern |
the string "the white queen" is matched against the pattern |
753 |
|
.sp |
754 |
the ((?:red|white) (king|queen)) |
the ((?:red|white) (king|queen)) |
755 |
|
.sp |
756 |
the captured substrings are "white queen" and "queen", and are numbered 1 and |
the captured substrings are "white queen" and "queen", and are numbered 1 and |
757 |
2. The maximum number of capturing subpatterns is 65535, and the maximum depth |
2. The maximum number of capturing subpatterns is 65535, and the maximum depth |
758 |
of nesting of all subpatterns, both capturing and non-capturing, is 200. |
of nesting of all subpatterns, both capturing and non-capturing, is 200. |
759 |
|
.P |
760 |
As a convenient shorthand, if any option settings are required at the start of |
As a convenient shorthand, if any option settings are required at the start of |
761 |
a non-capturing subpattern, the option letters may appear between the "?" and |
a non-capturing subpattern, the option letters may appear between the "?" and |
762 |
the ":". Thus the two patterns |
the ":". Thus the two patterns |
763 |
|
.sp |
764 |
(?i:saturday|sunday) |
(?i:saturday|sunday) |
765 |
(?:(?i)saturday|sunday) |
(?:(?i)saturday|sunday) |
766 |
|
.sp |
767 |
match exactly the same set of strings. Because alternative branches are tried |
match exactly the same set of strings. Because alternative branches are tried |
768 |
from left to right, and options are not reset until the end of the subpattern |
from left to right, and options are not reset until the end of the subpattern |
769 |
is reached, an option setting in one branch does affect subsequent branches, so |
is reached, an option setting in one branch does affect subsequent branches, so |
770 |
the above patterns match "SUNDAY" as well as "Saturday". |
the above patterns match "SUNDAY" as well as "Saturday". |
771 |
|
. |
772 |
.SH NAMED SUBPATTERNS |
. |
773 |
|
.SH "NAMED SUBPATTERNS" |
774 |
.rs |
.rs |
775 |
.sp |
.sp |
776 |
Identifying capturing parentheses by number is simple, but it can be very hard |
Identifying capturing parentheses by number is simple, but it can be very hard |
777 |
to keep track of the numbers in complicated regular expressions. Furthermore, |
to keep track of the numbers in complicated regular expressions. Furthermore, |
778 |
if an expression is modified, the numbers may change. To help with the |
if an expression is modified, the numbers may change. To help with this |
779 |
difficulty, PCRE supports the naming of subpatterns, something that Perl does |
difficulty, PCRE supports the naming of subpatterns, something that Perl does |
780 |
not provide. The Python syntax (?P<name>...) is used. Names consist of |
not provide. The Python syntax (?P<name>...) is used. Names consist of |
781 |
alphanumeric characters and underscores, and must be unique within a pattern. |
alphanumeric characters and underscores, and must be unique within a pattern. |
782 |
|
.P |
783 |
Named capturing parentheses are still allocated numbers as well as names. The |
Named capturing parentheses are still allocated numbers as well as names. The |
784 |
PCRE API provides function calls for extracting the name-to-number translation |
PCRE API provides function calls for extracting the name-to-number translation |
785 |
table from a compiled pattern. For further details see the |
table from a compiled pattern. There is also a convenience function for |
786 |
|
extracting a captured substring by name. For further details see the |
787 |
.\" HREF |
.\" HREF |
788 |
\fBpcreapi\fR |
\fBpcreapi\fP |
789 |
.\" |
.\" |
790 |
documentation. |
documentation. |
791 |
|
. |
792 |
|
. |
793 |
.SH REPETITION |
.SH REPETITION |
794 |
.rs |
.rs |
795 |
.sp |
.sp |
796 |
Repetition is specified by quantifiers, which can follow any of the following |
Repetition is specified by quantifiers, which can follow any of the following |
797 |
items: |
items: |
798 |
|
.sp |
799 |
a literal data character |
a literal data character |
800 |
the . metacharacter |
the . metacharacter |
801 |
the \\C escape sequence |
the \eC escape sequence |
802 |
escapes such as \\d that match single characters |
the \eX escape sequence (in UTF-8 mode with Unicode properties) |
803 |
|
an escape such as \ed that matches a single character |
804 |
a character class |
a character class |
805 |
a back reference (see next section) |
a back reference (see next section) |
806 |
a parenthesized subpattern (unless it is an assertion) |
a parenthesized subpattern (unless it is an assertion) |
807 |
|
.sp |
808 |
The general repetition quantifier specifies a minimum and maximum number of |
The general repetition quantifier specifies a minimum and maximum number of |
809 |
permitted matches, by giving the two numbers in curly brackets (braces), |
permitted matches, by giving the two numbers in curly brackets (braces), |
810 |
separated by a comma. The numbers must be less than 65536, and the first must |
separated by a comma. The numbers must be less than 65536, and the first must |
811 |
be less than or equal to the second. For example: |
be less than or equal to the second. For example: |
812 |
|
.sp |
813 |
z{2,4} |
z{2,4} |
814 |
|
.sp |
815 |
matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special |
matches "zz", "zzz", or "zzzz". A closing brace on its own is not a special |
816 |
character. If the second number is omitted, but the comma is present, there is |
character. If the second number is omitted, but the comma is present, there is |
817 |
no upper limit; if the second number and the comma are both omitted, the |
no upper limit; if the second number and the comma are both omitted, the |
818 |
quantifier specifies an exact number of required matches. Thus |
quantifier specifies an exact number of required matches. Thus |
819 |
|
.sp |
820 |
[aeiou]{3,} |
[aeiou]{3,} |
821 |
|
.sp |
822 |
matches at least 3 successive vowels, but may match many more, while |
matches at least 3 successive vowels, but may match many more, while |
823 |
|
.sp |
824 |
\\d{8} |
\ed{8} |
825 |
|
.sp |
826 |
matches exactly 8 digits. An opening curly bracket that appears in a position |
matches exactly 8 digits. An opening curly bracket that appears in a position |
827 |
where a quantifier is not allowed, or one that does not match the syntax of a |
where a quantifier is not allowed, or one that does not match the syntax of a |
828 |
quantifier, is taken as a literal character. For example, {,6} is not a |
quantifier, is taken as a literal character. For example, {,6} is not a |
829 |
quantifier, but a literal string of four characters. |
quantifier, but a literal string of four characters. |
830 |
|
.P |
831 |
In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual |
In UTF-8 mode, quantifiers apply to UTF-8 characters rather than to individual |
832 |
bytes. Thus, for example, \\x{100}{2} matches two UTF-8 characters, each of |
bytes. Thus, for example, \ex{100}{2} matches two UTF-8 characters, each of |
833 |
which is represented by a two-byte sequence. |
which is represented by a two-byte sequence. Similarly, when Unicode property |
834 |
|
support is available, \eX{3} matches three Unicode extended sequences, each of |
835 |
|
which may be several bytes long (and they may be of different lengths). |
836 |
|
.P |
837 |
The quantifier {0} is permitted, causing the expression to behave as if the |
The quantifier {0} is permitted, causing the expression to behave as if the |
838 |
previous item and the quantifier were not present. |
previous item and the quantifier were not present. |
839 |
|
.P |
840 |
For convenience (and historical compatibility) the three most common |
For convenience (and historical compatibility) the three most common |
841 |
quantifiers have single-character abbreviations: |
quantifiers have single-character abbreviations: |
842 |
|
.sp |
843 |
* is equivalent to {0,} |
* is equivalent to {0,} |
844 |
+ is equivalent to {1,} |
+ is equivalent to {1,} |
845 |
? is equivalent to {0,1} |
? is equivalent to {0,1} |
846 |
|
.sp |
847 |
It is possible to construct infinite loops by following a subpattern that can |
It is possible to construct infinite loops by following a subpattern that can |
848 |
match no characters with a quantifier that has no upper limit, for example: |
match no characters with a quantifier that has no upper limit, for example: |
849 |
|
.sp |
850 |
(a?)* |
(a?)* |
851 |
|
.sp |
852 |
Earlier versions of Perl and PCRE used to give an error at compile time for |
Earlier versions of Perl and PCRE used to give an error at compile time for |
853 |
such patterns. However, because there are cases where this can be useful, such |
such patterns. However, because there are cases where this can be useful, such |
854 |
patterns are now accepted, but if any repetition of the subpattern does in fact |
patterns are now accepted, but if any repetition of the subpattern does in fact |
855 |
match no characters, the loop is forcibly broken. |
match no characters, the loop is forcibly broken. |
856 |
|
.P |
857 |
By default, the quantifiers are "greedy", that is, they match as much as |
By default, the quantifiers are "greedy", that is, they match as much as |
858 |
possible (up to the maximum number of permitted times), without causing the |
possible (up to the maximum number of permitted times), without causing the |
859 |
rest of the pattern to fail. The classic example of where this gives problems |
rest of the pattern to fail. The classic example of where this gives problems |
860 |
is in trying to match comments in C programs. These appear between the |
is in trying to match comments in C programs. These appear between /* and */ |
861 |
sequences /* and */ and within the sequence, individual * and / characters may |
and within the comment, individual * and / characters may appear. An attempt to |
862 |
appear. An attempt to match C comments by applying the pattern |
match C comments by applying the pattern |
863 |
|
.sp |
864 |
/\\*.*\\*/ |
/\e*.*\e*/ |
865 |
|
.sp |
866 |
to the string |
to the string |
867 |
|
.sp |
868 |
/* first command */ not comment /* second comment */ |
/* first comment */ not comment /* second comment */ |
869 |
|
.sp |
870 |
fails, because it matches the entire string owing to the greediness of the .* |
fails, because it matches the entire string owing to the greediness of the .* |
871 |
item. |
item. |
872 |
|
.P |
873 |
However, if a quantifier is followed by a question mark, it ceases to be |
However, if a quantifier is followed by a question mark, it ceases to be |
874 |
greedy, and instead matches the minimum number of times possible, so the |
greedy, and instead matches the minimum number of times possible, so the |
875 |
pattern |
pattern |
876 |
|
.sp |
877 |
/\\*.*?\\*/ |
/\e*.*?\e*/ |
878 |
|
.sp |
879 |
does the right thing with the C comments. The meaning of the various |
does the right thing with the C comments. The meaning of the various |
880 |
quantifiers is not otherwise changed, just the preferred number of matches. |
quantifiers is not otherwise changed, just the preferred number of matches. |
881 |
Do not confuse this use of question mark with its use as a quantifier in its |
Do not confuse this use of question mark with its use as a quantifier in its |
882 |
own right. Because it has two uses, it can sometimes appear doubled, as in |
own right. Because it has two uses, it can sometimes appear doubled, as in |
883 |
|
.sp |
884 |
\\d??\\d |
\ed??\ed |
885 |
|
.sp |
886 |
which matches one digit by preference, but can match two if that is the only |
which matches one digit by preference, but can match two if that is the only |
887 |
way the rest of the pattern matches. |
way the rest of the pattern matches. |
888 |
|
.P |
889 |
If the PCRE_UNGREEDY option is set (an option which is not available in Perl), |
If the PCRE_UNGREEDY option is set (an option which is not available in Perl), |
890 |
the quantifiers are not greedy by default, but individual ones can be made |
the quantifiers are not greedy by default, but individual ones can be made |
891 |
greedy by following them with a question mark. In other words, it inverts the |
greedy by following them with a question mark. In other words, it inverts the |
892 |
default behaviour. |
default behaviour. |
893 |
|
.P |
894 |
When a parenthesized subpattern is quantified with a minimum repeat count that |
When a parenthesized subpattern is quantified with a minimum repeat count that |
895 |
is greater than 1 or with a limited maximum, more store is required for the |
is greater than 1 or with a limited maximum, more memory is required for the |
896 |
compiled pattern, in proportion to the size of the minimum or maximum. |
compiled pattern, in proportion to the size of the minimum or maximum. |
897 |
|
.P |
898 |
If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent |
If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent |
899 |
to Perl's /s) is set, thus allowing the . to match newlines, the pattern is |
to Perl's /s) is set, thus allowing the . to match newlines, the pattern is |
900 |
implicitly anchored, because whatever follows will be tried against every |
implicitly anchored, because whatever follows will be tried against every |
901 |
character position in the subject string, so there is no point in retrying the |
character position in the subject string, so there is no point in retrying the |
902 |
overall match at any position after the first. PCRE normally treats such a |
overall match at any position after the first. PCRE normally treats such a |
903 |
pattern as though it were preceded by \\A. |
pattern as though it were preceded by \eA. |
904 |
|
.P |
905 |
In cases where it is known that the subject string contains no newlines, it is |
In cases where it is known that the subject string contains no newlines, it is |
906 |
worth setting PCRE_DOTALL in order to obtain this optimization, or |
worth setting PCRE_DOTALL in order to obtain this optimization, or |
907 |
alternatively using ^ to indicate anchoring explicitly. |
alternatively using ^ to indicate anchoring explicitly. |
908 |
|
.P |
909 |
However, there is one situation where the optimization cannot be used. When .* |
However, there is one situation where the optimization cannot be used. When .* |
910 |
is inside capturing parentheses that are the subject of a backreference |
is inside capturing parentheses that are the subject of a backreference |
911 |
elsewhere in the pattern, a match at the start may fail, and a later one |
elsewhere in the pattern, a match at the start may fail, and a later one |
912 |
succeed. Consider, for example: |
succeed. Consider, for example: |
913 |
|
.sp |
914 |
(.*)abc\\1 |
(.*)abc\e1 |
915 |
|
.sp |
916 |
If the subject is "xyz123abc123" the match point is the fourth character. For |
If the subject is "xyz123abc123" the match point is the fourth character. For |
917 |
this reason, such a pattern is not implicitly anchored. |
this reason, such a pattern is not implicitly anchored. |
918 |
|
.P |
919 |
When a capturing subpattern is repeated, the value captured is the substring |
When a capturing subpattern is repeated, the value captured is the substring |
920 |
that matched the final iteration. For example, after |
that matched the final iteration. For example, after |
921 |
|
.sp |
922 |
(tweedle[dume]{3}\\s*)+ |
(tweedle[dume]{3}\es*)+ |
923 |
|
.sp |
924 |
has matched "tweedledum tweedledee" the value of the captured substring is |
has matched "tweedledum tweedledee" the value of the captured substring is |
925 |
"tweedledee". However, if there are nested capturing subpatterns, the |
"tweedledee". However, if there are nested capturing subpatterns, the |
926 |
corresponding captured values may have been set in previous iterations. For |
corresponding captured values may have been set in previous iterations. For |
927 |
example, after |
example, after |
928 |
|
.sp |
929 |
/(a|(b))+/ |
/(a|(b))+/ |
930 |
|
.sp |
931 |
matches "aba" the value of the second captured substring is "b". |
matches "aba" the value of the second captured substring is "b". |
932 |
|
. |
933 |
.SH ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS |
. |
934 |
|
.\" HTML <a name="atomicgroup"></a> |
935 |
|
.SH "ATOMIC GROUPING AND POSSESSIVE QUANTIFIERS" |
936 |
.rs |
.rs |
937 |
.sp |
.sp |
938 |
With both maximizing and minimizing repetition, failure of what follows |
With both maximizing and minimizing repetition, failure of what follows |
941 |
useful to prevent this, either to change the nature of the match, or to cause |
useful to prevent this, either to change the nature of the match, or to cause |
942 |
it fail earlier than it otherwise might, when the author of the pattern knows |
it fail earlier than it otherwise might, when the author of the pattern knows |
943 |
there is no point in carrying on. |
there is no point in carrying on. |
944 |
|
.P |
945 |
Consider, for example, the pattern \\d+foo when applied to the subject line |
Consider, for example, the pattern \ed+foo when applied to the subject line |
946 |
|
.sp |
947 |
123456bar |
123456bar |
948 |
|
.sp |
949 |
After matching all 6 digits and then failing to match "foo", the normal |
After matching all 6 digits and then failing to match "foo", the normal |
950 |
action of the matcher is to try again with only 5 digits matching the \\d+ |
action of the matcher is to try again with only 5 digits matching the \ed+ |
951 |
item, and then with 4, and so on, before ultimately failing. "Atomic grouping" |
item, and then with 4, and so on, before ultimately failing. "Atomic grouping" |
952 |
(a term taken from Jeffrey Friedl's book) provides the means for specifying |
(a term taken from Jeffrey Friedl's book) provides the means for specifying |
953 |
that once a subpattern has matched, it is not to be re-evaluated in this way. |
that once a subpattern has matched, it is not to be re-evaluated in this way. |
954 |
|
.P |
955 |
If we use atomic grouping for the previous example, the matcher would give up |
If we use atomic grouping for the previous example, the matcher would give up |
956 |
immediately on failing to match "foo" the first time. The notation is a kind of |
immediately on failing to match "foo" the first time. The notation is a kind of |
957 |
special parenthesis, starting with (?> as in this example: |
special parenthesis, starting with (?> as in this example: |
958 |
|
.sp |
959 |
(?>\\d+)foo |
(?>\ed+)foo |
960 |
|
.sp |
961 |
This kind of parenthesis "locks up" the part of the pattern it contains once |
This kind of parenthesis "locks up" the part of the pattern it contains once |
962 |
it has matched, and a failure further into the pattern is prevented from |
it has matched, and a failure further into the pattern is prevented from |
963 |
backtracking into it. Backtracking past it to previous items, however, works as |
backtracking into it. Backtracking past it to previous items, however, works as |
964 |
normal. |
normal. |
965 |
|
.P |
966 |
An alternative description is that a subpattern of this type matches the string |
An alternative description is that a subpattern of this type matches the string |
967 |
of characters that an identical standalone pattern would match, if anchored at |
of characters that an identical standalone pattern would match, if anchored at |
968 |
the current point in the subject string. |
the current point in the subject string. |
969 |
|
.P |
970 |
Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as |
Atomic grouping subpatterns are not capturing subpatterns. Simple cases such as |
971 |
the above example can be thought of as a maximizing repeat that must swallow |
the above example can be thought of as a maximizing repeat that must swallow |
972 |
everything it can. So, while both \\d+ and \\d+? are prepared to adjust the |
everything it can. So, while both \ed+ and \ed+? are prepared to adjust the |
973 |
number of digits they match in order to make the rest of the pattern match, |
number of digits they match in order to make the rest of the pattern match, |
974 |
(?>\\d+) can only match an entire sequence of digits. |
(?>\ed+) can only match an entire sequence of digits. |
975 |
|
.P |
976 |
Atomic groups in general can of course contain arbitrarily complicated |
Atomic groups in general can of course contain arbitrarily complicated |
977 |
subpatterns, and can be nested. However, when the subpattern for an atomic |
subpatterns, and can be nested. However, when the subpattern for an atomic |
978 |
group is just a single repeated item, as in the example above, a simpler |
group is just a single repeated item, as in the example above, a simpler |
979 |
notation, called a "possessive quantifier" can be used. This consists of an |
notation, called a "possessive quantifier" can be used. This consists of an |
980 |
additional + character following a quantifier. Using this notation, the |
additional + character following a quantifier. Using this notation, the |
981 |
previous example can be rewritten as |
previous example can be rewritten as |
982 |
|
.sp |
983 |
\\d++bar |
\ed++foo |
984 |
|
.sp |
985 |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
986 |
option is ignored. They are a convenient notation for the simpler forms of |
option is ignored. They are a convenient notation for the simpler forms of |
987 |
atomic group. However, there is no difference in the meaning or processing of a |
atomic group. However, there is no difference in the meaning or processing of a |
988 |
possessive quantifier and the equivalent atomic group. |
possessive quantifier and the equivalent atomic group. |
989 |
|
.P |
990 |
The possessive quantifier syntax is an extension to the Perl syntax. It |
The possessive quantifier syntax is an extension to the Perl syntax. It |
991 |
originates in Sun's Java package. |
originates in Sun's Java package. |
992 |
|
.P |
993 |
When a pattern contains an unlimited repeat inside a subpattern that can itself |
When a pattern contains an unlimited repeat inside a subpattern that can itself |
994 |
be repeated an unlimited number of times, the use of an atomic group is the |
be repeated an unlimited number of times, the use of an atomic group is the |
995 |
only way to avoid some failing matches taking a very long time indeed. The |
only way to avoid some failing matches taking a very long time indeed. The |
996 |
pattern |
pattern |
997 |
|
.sp |
998 |
(\\D+|<\\d+>)*[!?] |
(\eD+|<\ed+>)*[!?] |
999 |
|
.sp |
1000 |
matches an unlimited number of substrings that either consist of non-digits, or |
matches an unlimited number of substrings that either consist of non-digits, or |
1001 |
digits enclosed in <>, followed by either ! or ?. When it matches, it runs |
digits enclosed in <>, followed by either ! or ?. When it matches, it runs |
1002 |
quickly. However, if it is applied to |
quickly. However, if it is applied to |
1003 |
|
.sp |
1004 |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa |
1005 |
|
.sp |
1006 |
it takes a long time before reporting failure. This is because the string can |
it takes a long time before reporting failure. This is because the string can |
1007 |
be divided between the two repeats in a large number of ways, and all have to |
be divided between the internal \eD+ repeat and the external * repeat in a |
1008 |
be tried. (The example used [!?] rather than a single character at the end, |
large number of ways, and all have to be tried. (The example uses [!?] rather |
1009 |
because both PCRE and Perl have an optimization that allows for fast failure |
than a single character at the end, because both PCRE and Perl have an |
1010 |
when a single character is used. They remember the last single character that |
optimization that allows for fast failure when a single character is used. They |
1011 |
is required for a match, and fail early if it is not present in the string.) |
remember the last single character that is required for a match, and fail early |
1012 |
If the pattern is changed to |
if it is not present in the string.) If the pattern is changed so that it uses |
1013 |
|
an atomic group, like this: |
1014 |
((?>\\D+)|<\\d+>)*[!?] |
.sp |
1015 |
|
((?>\eD+)|<\ed+>)*[!?] |
1016 |
|
.sp |
1017 |
sequences of non-digits cannot be broken, and failure happens quickly. |
sequences of non-digits cannot be broken, and failure happens quickly. |
1018 |
|
. |
1019 |
.SH BACK REFERENCES |
. |
1020 |
|
.\" HTML <a name="backreferences"></a> |
1021 |
|
.SH "BACK REFERENCES" |
1022 |
.rs |
.rs |
1023 |
.sp |
.sp |
1024 |
Outside a character class, a backslash followed by a digit greater than 0 (and |
Outside a character class, a backslash followed by a digit greater than 0 (and |
1025 |
possibly further digits) is a back reference to a capturing subpattern earlier |
possibly further digits) is a back reference to a capturing subpattern earlier |
1026 |
(that is, to its left) in the pattern, provided there have been that many |
(that is, to its left) in the pattern, provided there have been that many |
1027 |
previous capturing left parentheses. |
previous capturing left parentheses. |
1028 |
|
.P |
1029 |
However, if the decimal number following the backslash is less than 10, it is |
However, if the decimal number following the backslash is less than 10, it is |
1030 |
always taken as a back reference, and causes an error only if there are not |
always taken as a back reference, and causes an error only if there are not |
1031 |
that many capturing left parentheses in the entire pattern. In other words, the |
that many capturing left parentheses in the entire pattern. In other words, the |
1032 |
parentheses that are referenced need not be to the left of the reference for |
parentheses that are referenced need not be to the left of the reference for |
1033 |
numbers less than 10. See the section entitled "Backslash" above for further |
numbers less than 10. See the subsection entitled "Non-printing characters" |
1034 |
details of the handling of digits following a backslash. |
.\" HTML <a href="#digitsafterbackslash"> |
1035 |
|
.\" </a> |
1036 |
|
above |
1037 |
|
.\" |
1038 |
|
for further details of the handling of digits following a backslash. |
1039 |
|
.P |
1040 |
A back reference matches whatever actually matched the capturing subpattern in |
A back reference matches whatever actually matched the capturing subpattern in |
1041 |
the current subject string, rather than anything matching the subpattern |
the current subject string, rather than anything matching the subpattern |
1042 |
itself (see |
itself (see |
1045 |
"Subpatterns as subroutines" |
"Subpatterns as subroutines" |
1046 |
.\" |
.\" |
1047 |
below for a way of doing that). So the pattern |
below for a way of doing that). So the pattern |
1048 |
|
.sp |
1049 |
(sens|respons)e and \\1ibility |
(sens|respons)e and \e1ibility |
1050 |
|
.sp |
1051 |
matches "sense and sensibility" and "response and responsibility", but not |
matches "sense and sensibility" and "response and responsibility", but not |
1052 |
"sense and responsibility". If caseful matching is in force at the time of the |
"sense and responsibility". If caseful matching is in force at the time of the |
1053 |
back reference, the case of letters is relevant. For example, |
back reference, the case of letters is relevant. For example, |
1054 |
|
.sp |
1055 |
((?i)rah)\\s+\\1 |
((?i)rah)\es+\e1 |
1056 |
|
.sp |
1057 |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
1058 |
capturing subpattern is matched caselessly. |
capturing subpattern is matched caselessly. |
1059 |
|
.P |
1060 |
Back references to named subpatterns use the Python syntax (?P=name). We could |
Back references to named subpatterns use the Python syntax (?P=name). We could |
1061 |
rewrite the above example as follows: |
rewrite the above example as follows: |
1062 |
|
.sp |
1063 |
(?<p1>(?i)rah)\\s+(?P=p1) |
(?<p1>(?i)rah)\es+(?P=p1) |
1064 |
|
.sp |
1065 |
There may be more than one back reference to the same subpattern. If a |
There may be more than one back reference to the same subpattern. If a |
1066 |
subpattern has not actually been used in a particular match, any back |
subpattern has not actually been used in a particular match, any back |
1067 |
references to it always fail. For example, the pattern |
references to it always fail. For example, the pattern |
1068 |
|
.sp |
1069 |
(a|(bc))\\2 |
(a|(bc))\e2 |
1070 |
|
.sp |
1071 |
always fails if it starts to match "a" rather than "bc". Because there may be |
always fails if it starts to match "a" rather than "bc". Because there may be |
1072 |
many capturing parentheses in a pattern, all digits following the backslash are |
many capturing parentheses in a pattern, all digits following the backslash are |
1073 |
taken as part of a potential back reference number. If the pattern continues |
taken as part of a potential back reference number. If the pattern continues |
1074 |
with a digit character, some delimiter must be used to terminate the back |
with a digit character, some delimiter must be used to terminate the back |
1075 |
reference. If the PCRE_EXTENDED option is set, this can be whitespace. |
reference. If the PCRE_EXTENDED option is set, this can be whitespace. |
1076 |
Otherwise an empty comment can be used. |
Otherwise an empty comment (see |
1077 |
|
.\" HTML <a href="#comments"> |
1078 |
|
.\" </a> |
1079 |
|
"Comments" |
1080 |
|
.\" |
1081 |
|
below) can be used. |
1082 |
|
.P |
1083 |
A back reference that occurs inside the parentheses to which it refers fails |
A back reference that occurs inside the parentheses to which it refers fails |
1084 |
when the subpattern is first used, so, for example, (a\\1) never matches. |
when the subpattern is first used, so, for example, (a\e1) never matches. |
1085 |
However, such references can be useful inside repeated subpatterns. For |
However, such references can be useful inside repeated subpatterns. For |
1086 |
example, the pattern |
example, the pattern |
1087 |
|
.sp |
1088 |
(a|b\\1)+ |
(a|b\e1)+ |
1089 |
|
.sp |
1090 |
matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of |
matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of |
1091 |
the subpattern, the back reference matches the character string corresponding |
the subpattern, the back reference matches the character string corresponding |
1092 |
to the previous iteration. In order for this to work, the pattern must be such |
to the previous iteration. In order for this to work, the pattern must be such |
1093 |
that the first iteration does not need to match the back reference. This can be |
that the first iteration does not need to match the back reference. This can be |
1094 |
done using alternation, as in the example above, or by a quantifier with a |
done using alternation, as in the example above, or by a quantifier with a |
1095 |
minimum of zero. |
minimum of zero. |
1096 |
|
. |
1097 |
|
. |
1098 |
|
.\" HTML <a name="bigassertions"></a> |
1099 |
.SH ASSERTIONS |
.SH ASSERTIONS |
1100 |
.rs |
.rs |
1101 |
.sp |
.sp |
1102 |
An assertion is a test on the characters following or preceding the current |
An assertion is a test on the characters following or preceding the current |
1103 |
matching point that does not actually consume any characters. The simple |
matching point that does not actually consume any characters. The simple |
1104 |
assertions coded as \\b, \\B, \\A, \\G, \\Z, \\z, ^ and $ are described above. |
assertions coded as \eb, \eB, \eA, \eG, \eZ, \ez, ^ and $ are described |
1105 |
|
.\" HTML <a href="#smallassertions"> |
1106 |
|
.\" </a> |
1107 |
|
above. |
1108 |
|
.\" |
1109 |
|
.P |
1110 |
More complicated assertions are coded as subpatterns. There are two kinds: |
More complicated assertions are coded as subpatterns. There are two kinds: |
1111 |
those that look ahead of the current position in the subject string, and those |
those that look ahead of the current position in the subject string, and those |
1112 |
that look behind it. |
that look behind it. An assertion subpattern is matched in the normal way, |
1113 |
|
except that it does not cause the current matching position to be changed. |
1114 |
An assertion subpattern is matched in the normal way, except that it does not |
.P |
1115 |
cause the current matching position to be changed. Lookahead assertions start |
Assertion subpatterns are not capturing subpatterns, and may not be repeated, |
1116 |
|
because it makes no sense to assert the same thing several times. If any kind |
1117 |
|
of assertion contains capturing subpatterns within it, these are counted for |
1118 |
|
the purposes of numbering the capturing subpatterns in the whole pattern. |
1119 |
|
However, substring capturing is carried out only for positive assertions, |
1120 |
|
because it does not make sense for negative assertions. |
1121 |
|
. |
1122 |
|
. |
1123 |
|
.SS "Lookahead assertions" |
1124 |
|
.rs |
1125 |
|
.sp |
1126 |
|
Lookahead assertions start |
1127 |
with (?= for positive assertions and (?! for negative assertions. For example, |
with (?= for positive assertions and (?! for negative assertions. For example, |
1128 |
|
.sp |
1129 |
\\w+(?=;) |
\ew+(?=;) |
1130 |
|
.sp |
1131 |
matches a word followed by a semicolon, but does not include the semicolon in |
matches a word followed by a semicolon, but does not include the semicolon in |
1132 |
the match, and |
the match, and |
1133 |
|
.sp |
1134 |
foo(?!bar) |
foo(?!bar) |
1135 |
|
.sp |
1136 |
matches any occurrence of "foo" that is not followed by "bar". Note that the |
matches any occurrence of "foo" that is not followed by "bar". Note that the |
1137 |
apparently similar pattern |
apparently similar pattern |
1138 |
|
.sp |
1139 |
(?!foo)bar |
(?!foo)bar |
1140 |
|
.sp |
1141 |
does not find an occurrence of "bar" that is preceded by something other than |
does not find an occurrence of "bar" that is preceded by something other than |
1142 |
"foo"; it finds any occurrence of "bar" whatsoever, because the assertion |
"foo"; it finds any occurrence of "bar" whatsoever, because the assertion |
1143 |
(?!foo) is always true when the next three characters are "bar". A |
(?!foo) is always true when the next three characters are "bar". A |
1144 |
lookbehind assertion is needed to achieve this effect. |
lookbehind assertion is needed to achieve the other effect. |
1145 |
|
.P |
1146 |
If you want to force a matching failure at some point in a pattern, the most |
If you want to force a matching failure at some point in a pattern, the most |
1147 |
convenient way to do it is with (?!) because an empty string always matches, so |
convenient way to do it is with (?!) because an empty string always matches, so |
1148 |
an assertion that requires there not to be an empty string must always fail. |
an assertion that requires there not to be an empty string must always fail. |
1149 |
|
. |
1150 |
|
. |
1151 |
|
.\" HTML <a name="lookbehind"></a> |
1152 |
|
.SS "Lookbehind assertions" |
1153 |
|
.rs |
1154 |
|
.sp |
1155 |
Lookbehind assertions start with (?<= for positive assertions and (?<! for |
Lookbehind assertions start with (?<= for positive assertions and (?<! for |
1156 |
negative assertions. For example, |
negative assertions. For example, |
1157 |
|
.sp |
1158 |
(?<!foo)bar |
(?<!foo)bar |
1159 |
|
.sp |
1160 |
does find an occurrence of "bar" that is not preceded by "foo". The contents of |
does find an occurrence of "bar" that is not preceded by "foo". The contents of |
1161 |
a lookbehind assertion are restricted such that all the strings it matches must |
a lookbehind assertion are restricted such that all the strings it matches must |
1162 |
have a fixed length. However, if there are several alternatives, they do not |
have a fixed length. However, if there are several alternatives, they do not |
1163 |
all have to have the same fixed length. Thus |
all have to have the same fixed length. Thus |
1164 |
|
.sp |
1165 |
(?<=bullock|donkey) |
(?<=bullock|donkey) |
1166 |
|
.sp |
1167 |
is permitted, but |
is permitted, but |
1168 |
|
.sp |
1169 |
(?<!dogs?|cats?) |
(?<!dogs?|cats?) |
1170 |
|
.sp |
1171 |
causes an error at compile time. Branches that match different length strings |
causes an error at compile time. Branches that match different length strings |
1172 |
are permitted only at the top level of a lookbehind assertion. This is an |
are permitted only at the top level of a lookbehind assertion. This is an |
1173 |
extension compared with Perl (at least for 5.8), which requires all branches to |
extension compared with Perl (at least for 5.8), which requires all branches to |
1174 |
match the same length of string. An assertion such as |
match the same length of string. An assertion such as |
1175 |
|
.sp |
1176 |
(?<=ab(c|de)) |
(?<=ab(c|de)) |
1177 |
|
.sp |
1178 |
is not permitted, because its single top-level branch can match two different |
is not permitted, because its single top-level branch can match two different |
1179 |
lengths, but it is acceptable if rewritten to use two top-level branches: |
lengths, but it is acceptable if rewritten to use two top-level branches: |
1180 |
|
.sp |
1181 |
(?<=abc|abde) |
(?<=abc|abde) |
1182 |
|
.sp |
1183 |
The implementation of lookbehind assertions is, for each alternative, to |
The implementation of lookbehind assertions is, for each alternative, to |
1184 |
temporarily move the current position back by the fixed width and then try to |
temporarily move the current position back by the fixed width and then try to |
1185 |
match. If there are insufficient characters before the current position, the |
match. If there are insufficient characters before the current position, the |
1186 |
match is deemed to fail. |
match is deemed to fail. |
1187 |
|
.P |
1188 |
PCRE does not allow the \\C escape (which matches a single byte in UTF-8 mode) |
PCRE does not allow the \eC escape (which matches a single byte in UTF-8 mode) |
1189 |
to appear in lookbehind assertions, because it makes it impossible to calculate |
to appear in lookbehind assertions, because it makes it impossible to calculate |
1190 |
the length of the lookbehind. |
the length of the lookbehind. The \eX escape, which can match different numbers |
1191 |
|
of bytes, is also not permitted. |
1192 |
|
.P |
1193 |
Atomic groups can be used in conjunction with lookbehind assertions to specify |
Atomic groups can be used in conjunction with lookbehind assertions to specify |
1194 |
efficient matching at the end of the subject string. Consider a simple pattern |
efficient matching at the end of the subject string. Consider a simple pattern |
1195 |
such as |
such as |
1196 |
|
.sp |
1197 |
abcd$ |
abcd$ |
1198 |
|
.sp |
1199 |
when applied to a long string that does not match. Because matching proceeds |
when applied to a long string that does not match. Because matching proceeds |
1200 |
from left to right, PCRE will look for each "a" in the subject and then see if |
from left to right, PCRE will look for each "a" in the subject and then see if |
1201 |
what follows matches the rest of the pattern. If the pattern is specified as |
what follows matches the rest of the pattern. If the pattern is specified as |
1202 |
|
.sp |
1203 |
^.*abcd$ |
^.*abcd$ |
1204 |
|
.sp |
1205 |
the initial .* matches the entire string at first, but when this fails (because |
the initial .* matches the entire string at first, but when this fails (because |
1206 |
there is no following "a"), it backtracks to match all but the last character, |
there is no following "a"), it backtracks to match all but the last character, |
1207 |
then all but the last two characters, and so on. Once again the search for "a" |
then all but the last two characters, and so on. Once again the search for "a" |
1208 |
covers the entire string, from right to left, so we are no better off. However, |
covers the entire string, from right to left, so we are no better off. However, |
1209 |
if the pattern is written as |
if the pattern is written as |
1210 |
|
.sp |
1211 |
^(?>.*)(?<=abcd) |
^(?>.*)(?<=abcd) |
1212 |
|
.sp |
1213 |
or, equivalently, |
or, equivalently, using the possessive quantifier syntax, |
1214 |
|
.sp |
1215 |
^.*+(?<=abcd) |
^.*+(?<=abcd) |
1216 |
|
.sp |
1217 |
there can be no backtracking for the .* item; it can match only the entire |
there can be no backtracking for the .* item; it can match only the entire |
1218 |
string. The subsequent lookbehind assertion does a single test on the last four |
string. The subsequent lookbehind assertion does a single test on the last four |
1219 |
characters. If it fails, the match fails immediately. For long strings, this |
characters. If it fails, the match fails immediately. For long strings, this |
1220 |
approach makes a significant difference to the processing time. |
approach makes a significant difference to the processing time. |
1221 |
|
. |
1222 |
|
. |
1223 |
|
.SS "Using multiple assertions" |
1224 |
|
.rs |
1225 |
|
.sp |
1226 |
Several assertions (of any sort) may occur in succession. For example, |
Several assertions (of any sort) may occur in succession. For example, |
1227 |
|
.sp |
1228 |
(?<=\\d{3})(?<!999)foo |
(?<=\ed{3})(?<!999)foo |
1229 |
|
.sp |
1230 |
matches "foo" preceded by three digits that are not "999". Notice that each of |
matches "foo" preceded by three digits that are not "999". Notice that each of |
1231 |
the assertions is applied independently at the same point in the subject |
the assertions is applied independently at the same point in the subject |
1232 |
string. First there is a check that the previous three characters are all |
string. First there is a check that the previous three characters are all |
1233 |
digits, and then there is a check that the same three characters are not "999". |
digits, and then there is a check that the same three characters are not "999". |
1234 |
This pattern does \fInot\fR match "foo" preceded by six characters, the first |
This pattern does \fInot\fP match "foo" preceded by six characters, the first |
1235 |
of which are digits and the last three of which are not "999". For example, it |
of which are digits and the last three of which are not "999". For example, it |
1236 |
doesn't match "123abcfoo". A pattern to do that is |
doesn't match "123abcfoo". A pattern to do that is |
1237 |
|
.sp |
1238 |
(?<=\\d{3}...)(?<!999)foo |
(?<=\ed{3}...)(?<!999)foo |
1239 |
|
.sp |
1240 |
This time the first assertion looks at the preceding six characters, checking |
This time the first assertion looks at the preceding six characters, checking |
1241 |
that the first three are digits, and then the second assertion checks that the |
that the first three are digits, and then the second assertion checks that the |
1242 |
preceding three characters are not "999". |
preceding three characters are not "999". |
1243 |
|
.P |
1244 |
Assertions can be nested in any combination. For example, |
Assertions can be nested in any combination. For example, |
1245 |
|
.sp |
1246 |
(?<=(?<!foo)bar)baz |
(?<=(?<!foo)bar)baz |
1247 |
|
.sp |
1248 |
matches an occurrence of "baz" that is preceded by "bar" which in turn is not |
matches an occurrence of "baz" that is preceded by "bar" which in turn is not |
1249 |
preceded by "foo", while |
preceded by "foo", while |
1250 |
|
.sp |
1251 |
(?<=\\d{3}(?!999)...)foo |
(?<=\ed{3}(?!999)...)foo |
1252 |
|
.sp |
1253 |
is another pattern which matches "foo" preceded by three digits and any three |
is another pattern that matches "foo" preceded by three digits and any three |
1254 |
characters that are not "999". |
characters that are not "999". |
1255 |
|
. |
1256 |
Assertion subpatterns are not capturing subpatterns, and may not be repeated, |
. |
1257 |
because it makes no sense to assert the same thing several times. If any kind |
.SH "CONDITIONAL SUBPATTERNS" |
|
of assertion contains capturing subpatterns within it, these are counted for |
|
|
the purposes of numbering the capturing subpatterns in the whole pattern. |
|
|
However, substring capturing is carried out only for positive assertions, |
|
|
because it does not make sense for negative assertions. |
|
|
|
|
|
.SH CONDITIONAL SUBPATTERNS |
|
1258 |
.rs |
.rs |
1259 |
.sp |
.sp |
1260 |
It is possible to cause the matching process to obey a subpattern |
It is possible to cause the matching process to obey a subpattern |
1261 |
conditionally or to choose between two alternative subpatterns, depending on |
conditionally or to choose between two alternative subpatterns, depending on |
1262 |
the result of an assertion, or whether a previous capturing subpattern matched |
the result of an assertion, or whether a previous capturing subpattern matched |
1263 |
or not. The two possible forms of conditional subpattern are |
or not. The two possible forms of conditional subpattern are |
1264 |
|
.sp |
1265 |
(?(condition)yes-pattern) |
(?(condition)yes-pattern) |
1266 |
(?(condition)yes-pattern|no-pattern) |
(?(condition)yes-pattern|no-pattern) |
1267 |
|
.sp |
1268 |
If the condition is satisfied, the yes-pattern is used; otherwise the |
If the condition is satisfied, the yes-pattern is used; otherwise the |
1269 |
no-pattern (if present) is used. If there are more than two alternatives in the |
no-pattern (if present) is used. If there are more than two alternatives in the |
1270 |
subpattern, a compile-time error occurs. |
subpattern, a compile-time error occurs. |
1271 |
|
.P |
1272 |
There are three kinds of condition. If the text between the parentheses |
There are three kinds of condition. If the text between the parentheses |
1273 |
consists of a sequence of digits, the condition is satisfied if the capturing |
consists of a sequence of digits, the condition is satisfied if the capturing |
1274 |
subpattern of that number has previously matched. The number must be greater |
subpattern of that number has previously matched. The number must be greater |
1275 |
than zero. Consider the following pattern, which contains non-significant white |
than zero. Consider the following pattern, which contains non-significant white |
1276 |
space to make it more readable (assume the PCRE_EXTENDED option) and to divide |
space to make it more readable (assume the PCRE_EXTENDED option) and to divide |
1277 |
it into three parts for ease of discussion: |
it into three parts for ease of discussion: |
1278 |
|
.sp |
1279 |
( \\( )? [^()]+ (?(1) \\) ) |
( \e( )? [^()]+ (?(1) \e) ) |
1280 |
|
.sp |
1281 |
The first part matches an optional opening parenthesis, and if that |
The first part matches an optional opening parenthesis, and if that |
1282 |
character is present, sets it as the first captured substring. The second part |
character is present, sets it as the first captured substring. The second part |
1283 |
matches one or more characters that are not parentheses. The third part is a |
matches one or more characters that are not parentheses. The third part is a |
1287 |
parenthesis is required. Otherwise, since no-pattern is not present, the |
parenthesis is required. Otherwise, since no-pattern is not present, the |
1288 |
subpattern matches nothing. In other words, this pattern matches a sequence of |
subpattern matches nothing. In other words, this pattern matches a sequence of |
1289 |
non-parentheses, optionally enclosed in parentheses. |
non-parentheses, optionally enclosed in parentheses. |
1290 |
|
.P |
1291 |
If the condition is the string (R), it is satisfied if a recursive call to the |
If the condition is the string (R), it is satisfied if a recursive call to the |
1292 |
pattern or subpattern has been made. At "top level", the condition is false. |
pattern or subpattern has been made. At "top level", the condition is false. |
1293 |
This is a PCRE extension. Recursive patterns are described in the next section. |
This is a PCRE extension. Recursive patterns are described in the next section. |
1294 |
|
.P |
1295 |
If the condition is not a sequence of digits or (R), it must be an assertion. |
If the condition is not a sequence of digits or (R), it must be an assertion. |
1296 |
This may be a positive or negative lookahead or lookbehind assertion. Consider |
This may be a positive or negative lookahead or lookbehind assertion. Consider |
1297 |
this pattern, again containing non-significant white space, and with the two |
this pattern, again containing non-significant white space, and with the two |
1298 |
alternatives on the second line: |
alternatives on the second line: |
1299 |
|
.sp |
1300 |
(?(?=[^a-z]*[a-z]) |
(?(?=[^a-z]*[a-z]) |
1301 |
\\d{2}-[a-z]{3}-\\d{2} | \\d{2}-\\d{2}-\\d{2} ) |
\ed{2}-[a-z]{3}-\ed{2} | \ed{2}-\ed{2}-\ed{2} ) |
1302 |
|
.sp |
1303 |
The condition is a positive lookahead assertion that matches an optional |
The condition is a positive lookahead assertion that matches an optional |
1304 |
sequence of non-letters followed by a letter. In other words, it tests for the |
sequence of non-letters followed by a letter. In other words, it tests for the |
1305 |
presence of at least one letter in the subject. If a letter is found, the |
presence of at least one letter in the subject. If a letter is found, the |
1306 |
subject is matched against the first alternative; otherwise it is matched |
subject is matched against the first alternative; otherwise it is matched |
1307 |
against the second. This pattern matches strings in one of the two forms |
against the second. This pattern matches strings in one of the two forms |
1308 |
dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. |
dd-aaa-dd or dd-dd-dd, where aaa are letters and dd are digits. |
1309 |
|
. |
1310 |
|
. |
1311 |
|
.\" HTML <a name="comments"></a> |
1312 |
.SH COMMENTS |
.SH COMMENTS |
1313 |
.rs |
.rs |
1314 |
.sp |
.sp |
1315 |
The sequence (?# marks the start of a comment which continues up to the next |
The sequence (?# marks the start of a comment that continues up to the next |
1316 |
closing parenthesis. Nested parentheses are not permitted. The characters |
closing parenthesis. Nested parentheses are not permitted. The characters |
1317 |
that make up a comment play no part in the pattern matching at all. |
that make up a comment play no part in the pattern matching at all. |
1318 |
|
.P |
1319 |
If the PCRE_EXTENDED option is set, an unescaped # character outside a |
If the PCRE_EXTENDED option is set, an unescaped # character outside a |
1320 |
character class introduces a comment that continues up to the next newline |
character class introduces a comment that continues up to the next newline |
1321 |
character in the pattern. |
character in the pattern. |
1322 |
|
. |
1323 |
.SH RECURSIVE PATTERNS |
. |
1324 |
|
.SH "RECURSIVE PATTERNS" |
1325 |
.rs |
.rs |
1326 |
.sp |
.sp |
1327 |
Consider the problem of matching a string in parentheses, allowing for |
Consider the problem of matching a string in parentheses, allowing for |
1328 |
unlimited nested parentheses. Without the use of recursion, the best that can |
unlimited nested parentheses. Without the use of recursion, the best that can |
1329 |
be done is to use a pattern that matches up to some fixed depth of nesting. It |
be done is to use a pattern that matches up to some fixed depth of nesting. It |
1330 |
is not possible to handle an arbitrary nesting depth. Perl has provided an |
is not possible to handle an arbitrary nesting depth. Perl provides a facility |
1331 |
experimental facility that allows regular expressions to recurse (amongst other |
that allows regular expressions to recurse (amongst other things). It does this |
1332 |
things). It does this by interpolating Perl code in the expression at run time, |
by interpolating Perl code in the expression at run time, and the code can |
1333 |
and the code can refer to the expression itself. A Perl pattern to solve the |
refer to the expression itself. A Perl pattern to solve the parentheses problem |
1334 |
parentheses problem can be created like this: |
can be created like this: |
1335 |
|
.sp |
1336 |
$re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x; |
$re = qr{\e( (?: (?>[^()]+) | (?p{$re}) )* \e)}x; |
1337 |
|
.sp |
1338 |
The (?p{...}) item interpolates Perl code at run time, and in this case refers |
The (?p{...}) item interpolates Perl code at run time, and in this case refers |
1339 |
recursively to the pattern in which it appears. Obviously, PCRE cannot support |
recursively to the pattern in which it appears. Obviously, PCRE cannot support |
1340 |
the interpolation of Perl code. Instead, it supports some special syntax for |
the interpolation of Perl code. Instead, it supports some special syntax for |
1341 |
recursion of the entire pattern, and also for individual subpattern recursion. |
recursion of the entire pattern, and also for individual subpattern recursion. |
1342 |
|
.P |
1343 |
The special item that consists of (? followed by a number greater than zero and |
The special item that consists of (? followed by a number greater than zero and |
1344 |
a closing parenthesis is a recursive call of the subpattern of the given |
a closing parenthesis is a recursive call of the subpattern of the given |
1345 |
number, provided that it occurs inside that subpattern. (If not, it is a |
number, provided that it occurs inside that subpattern. (If not, it is a |
1346 |
"subroutine" call, which is described in the next section.) The special item |
"subroutine" call, which is described in the next section.) The special item |
1347 |
(?R) is a recursive call of the entire regular expression. |
(?R) is a recursive call of the entire regular expression. |
1348 |
|
.P |
1349 |
For example, this PCRE pattern solves the nested parentheses problem (assume |
For example, this PCRE pattern solves the nested parentheses problem (assume |
1350 |
the PCRE_EXTENDED option is set so that white space is ignored): |
the PCRE_EXTENDED option is set so that white space is ignored): |
1351 |
|
.sp |
1352 |
\\( ( (?>[^()]+) | (?R) )* \\) |
\e( ( (?>[^()]+) | (?R) )* \e) |
1353 |
|
.sp |
1354 |
First it matches an opening parenthesis. Then it matches any number of |
First it matches an opening parenthesis. Then it matches any number of |
1355 |
substrings which can either be a sequence of non-parentheses, or a recursive |
substrings which can either be a sequence of non-parentheses, or a recursive |
1356 |
match of the pattern itself (that is a correctly parenthesized substring). |
match of the pattern itself (that is a correctly parenthesized substring). |
1357 |
Finally there is a closing parenthesis. |
Finally there is a closing parenthesis. |
1358 |
|
.P |
1359 |
If this were part of a larger pattern, you would not want to recurse the entire |
If this were part of a larger pattern, you would not want to recurse the entire |
1360 |
pattern, so instead you could use this: |
pattern, so instead you could use this: |
1361 |
|
.sp |
1362 |
( \\( ( (?>[^()]+) | (?1) )* \\) ) |
( \e( ( (?>[^()]+) | (?1) )* \e) ) |
1363 |
|
.sp |
1364 |
We have put the pattern into parentheses, and caused the recursion to refer to |
We have put the pattern into parentheses, and caused the recursion to refer to |
1365 |
them instead of the whole pattern. In a larger pattern, keeping track of |
them instead of the whole pattern. In a larger pattern, keeping track of |
1366 |
parenthesis numbers can be tricky. It may be more convenient to use named |
parenthesis numbers can be tricky. It may be more convenient to use named |
1367 |
parentheses instead. For this, PCRE uses (?P>name), which is an extension to |
parentheses instead. For this, PCRE uses (?P>name), which is an extension to |
1368 |
the Python syntax that PCRE uses for named parentheses (Perl does not provide |
the Python syntax that PCRE uses for named parentheses (Perl does not provide |
1369 |
named parentheses). We could rewrite the above example as follows: |
named parentheses). We could rewrite the above example as follows: |
1370 |
|
.sp |
1371 |
(?P<pn> \\( ( (?>[^()]+) | (?P>pn) )* \\) ) |
(?P<pn> \e( ( (?>[^()]+) | (?P>pn) )* \e) ) |
1372 |
|
.sp |
1373 |
This particular example pattern contains nested unlimited repeats, and so the |
This particular example pattern contains nested unlimited repeats, and so the |
1374 |
use of atomic grouping for matching strings of non-parentheses is important |
use of atomic grouping for matching strings of non-parentheses is important |
1375 |
when applying the pattern to strings that do not match. For example, when this |
when applying the pattern to strings that do not match. For example, when this |
1376 |
pattern is applied to |
pattern is applied to |
1377 |
|
.sp |
1378 |
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
(aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa() |
1379 |
|
.sp |
1380 |
it yields "no match" quickly. However, if atomic grouping is not used, |
it yields "no match" quickly. However, if atomic grouping is not used, |
1381 |
the match runs for a very long time indeed because there are so many different |
the match runs for a very long time indeed because there are so many different |
1382 |
ways the + and * repeats can carve up the subject, and all have to be tested |
ways the + and * repeats can carve up the subject, and all have to be tested |
1383 |
before failure can be reported. |
before failure can be reported. |
1384 |
|
.P |
1385 |
At the end of a match, the values set for any capturing subpatterns are those |
At the end of a match, the values set for any capturing subpatterns are those |
1386 |
from the outermost level of the recursion at which the subpattern value is set. |
from the outermost level of the recursion at which the subpattern value is set. |
1387 |
If you want to obtain intermediate values, a callout function can be used (see |
If you want to obtain intermediate values, a callout function can be used (see |
1388 |
below and the |
the next section and the |
1389 |
.\" HREF |
.\" HREF |
1390 |
\fBpcrecallout\fR |
\fBpcrecallout\fP |
1391 |
.\" |
.\" |
1392 |
documentation). If the pattern above is matched against |
documentation). If the pattern above is matched against |
1393 |
|
.sp |
1394 |
(ab(cd)ef) |
(ab(cd)ef) |
1395 |
|
.sp |
1396 |
the value for the capturing parentheses is "ef", which is the last value taken |
the value for the capturing parentheses is "ef", which is the last value taken |
1397 |
on at the top level. If additional parentheses are added, giving |
on at the top level. If additional parentheses are added, giving |
1398 |
|
.sp |
1399 |
\\( ( ( (?>[^()]+) | (?R) )* ) \\) |
\e( ( ( (?>[^()]+) | (?R) )* ) \e) |
1400 |
^ ^ |
^ ^ |
1401 |
^ ^ |
^ ^ |
1402 |
|
.sp |
1403 |
the string they capture is "ab(cd)ef", the contents of the top level |
the string they capture is "ab(cd)ef", the contents of the top level |
1404 |
parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE |
parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE |
1405 |
has to obtain extra memory to store data during a recursion, which it does by |
has to obtain extra memory to store data during a recursion, which it does by |
1406 |
using \fBpcre_malloc\fR, freeing it via \fBpcre_free\fR afterwards. If no |
using \fBpcre_malloc\fP, freeing it via \fBpcre_free\fP afterwards. If no |
1407 |
memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
memory can be obtained, the match fails with the PCRE_ERROR_NOMEMORY error. |
1408 |
|
.P |
1409 |
Do not confuse the (?R) item with the condition (R), which tests for recursion. |
Do not confuse the (?R) item with the condition (R), which tests for recursion. |
1410 |
Consider this pattern, which matches text in angle brackets, allowing for |
Consider this pattern, which matches text in angle brackets, allowing for |
1411 |
arbitrary nesting. Only digits are allowed in nested brackets (that is, when |
arbitrary nesting. Only digits are allowed in nested brackets (that is, when |
1412 |
recursing), whereas any characters are permitted at the outer level. |
recursing), whereas any characters are permitted at the outer level. |
1413 |
|
.sp |
1414 |
< (?: (?(R) \\d++ | [^<>]*+) | (?R)) * > |
< (?: (?(R) \ed++ | [^<>]*+) | (?R)) * > |
1415 |
|
.sp |
1416 |
In this pattern, (?(R) is the start of a conditional subpattern, with two |
In this pattern, (?(R) is the start of a conditional subpattern, with two |
1417 |
different alternatives for the recursive and non-recursive cases. The (?R) item |
different alternatives for the recursive and non-recursive cases. The (?R) item |
1418 |
is the actual recursive call. |
is the actual recursive call. |
1419 |
|
. |
1420 |
|
. |
1421 |
.\" HTML <a name="subpatternsassubroutines"></a> |
.\" HTML <a name="subpatternsassubroutines"></a> |
1422 |
.SH SUBPATTERNS AS SUBROUTINES |
.SH "SUBPATTERNS AS SUBROUTINES" |
1423 |
.rs |
.rs |
1424 |
.sp |
.sp |
1425 |
If the syntax for a recursive subpattern reference (either by number or by |
If the syntax for a recursive subpattern reference (either by number or by |
1426 |
name) is used outside the parentheses to which it refers, it operates like a |
name) is used outside the parentheses to which it refers, it operates like a |
1427 |
subroutine in a programming language. An earlier example pointed out that the |
subroutine in a programming language. An earlier example pointed out that the |
1428 |
pattern |
pattern |
1429 |
|
.sp |
1430 |
(sens|respons)e and \\1ibility |
(sens|respons)e and \e1ibility |
1431 |
|
.sp |
1432 |
matches "sense and sensibility" and "response and responsibility", but not |
matches "sense and sensibility" and "response and responsibility", but not |
1433 |
"sense and responsibility". If instead the pattern |
"sense and responsibility". If instead the pattern |
1434 |
|
.sp |
1435 |
(sens|respons)e and (?1)ibility |
(sens|respons)e and (?1)ibility |
1436 |
|
.sp |
1437 |
is used, it does match "sense and responsibility" as well as the other two |
is used, it does match "sense and responsibility" as well as the other two |
1438 |
strings. Such references must, however, follow the subpattern to which they |
strings. Such references must, however, follow the subpattern to which they |
1439 |
refer. |
refer. |
1440 |
|
. |
1441 |
|
. |
1442 |
.SH CALLOUTS |
.SH CALLOUTS |
1443 |
.rs |
.rs |
1444 |
.sp |
.sp |
1446 |
code to be obeyed in the middle of matching a regular expression. This makes it |
code to be obeyed in the middle of matching a regular expression. This makes it |
1447 |
possible, amongst other things, to extract different substrings that match the |
possible, amongst other things, to extract different substrings that match the |
1448 |
same pair of parentheses when there is a repetition. |
same pair of parentheses when there is a repetition. |
1449 |
|
.P |
1450 |
PCRE provides a similar feature, but of course it cannot obey arbitrary Perl |
PCRE provides a similar feature, but of course it cannot obey arbitrary Perl |
1451 |
code. The feature is called "callout". The caller of PCRE provides an external |
code. The feature is called "callout". The caller of PCRE provides an external |
1452 |
function by putting its entry point in the global variable \fIpcre_callout\fR. |
function by putting its entry point in the global variable \fIpcre_callout\fP. |
1453 |
By default, this variable contains NULL, which disables all calling out. |
By default, this variable contains NULL, which disables all calling out. |
1454 |
|
.P |
1455 |
Within a regular expression, (?C) indicates the points at which the external |
Within a regular expression, (?C) indicates the points at which the external |
1456 |
function is to be called. If you want to identify different callout points, you |
function is to be called. If you want to identify different callout points, you |
1457 |
can put a number less than 256 after the letter C. The default value is zero. |
can put a number less than 256 after the letter C. The default value is zero. |
1458 |
For example, this pattern has two callout points: |
For example, this pattern has two callout points: |
1459 |
|
.sp |
1460 |
(?C1)\dabc(?C2)def |
(?C1)\dabc(?C2)def |
1461 |
|
.sp |
1462 |
During matching, when PCRE reaches a callout point (and \fIpcre_callout\fR is |
If the PCRE_AUTO_CALLOUT flag is passed to \fBpcre_compile()\fP, callouts are |
1463 |
|
automatically installed before each item in the pattern. They are all numbered |
1464 |
|
255. |
1465 |
|
.P |
1466 |
|
During matching, when PCRE reaches a callout point (and \fIpcre_callout\fP is |
1467 |
set), the external function is called. It is provided with the number of the |
set), the external function is called. It is provided with the number of the |
1468 |
callout, and, optionally, one item of data originally supplied by the caller of |
callout, the position in the pattern, and, optionally, one item of data |
1469 |
\fBpcre_exec()\fR. The callout function may cause matching to backtrack, or to |
originally supplied by the caller of \fBpcre_exec()\fP. The callout function |
1470 |
fail altogether. A complete description of the interface to the callout |
may cause matching to proceed, to backtrack, or to fail altogether. A complete |
1471 |
function is given in the |
description of the interface to the callout function is given in the |
1472 |
.\" HREF |
.\" HREF |
1473 |
\fBpcrecallout\fR |
\fBpcrecallout\fP |
1474 |
.\" |
.\" |
1475 |
documentation. |
documentation. |
1476 |
|
.P |
1477 |
.in 0 |
.in 0 |
1478 |
Last updated: 03 February 2003 |
Last updated: 28 February 2005 |
1479 |
.br |
.br |
1480 |
Copyright (c) 1997-2003 University of Cambridge. |
Copyright (c) 1997-2005 University of Cambridge. |