4 |
.SH "PCRE REGULAR EXPRESSION DETAILS" |
.SH "PCRE REGULAR EXPRESSION DETAILS" |
5 |
.rs |
.rs |
6 |
.sp |
.sp |
7 |
The syntax and semantics of the regular expressions supported by PCRE are |
The syntax and semantics of the regular expressions that are supported by PCRE |
8 |
described below. Regular expressions are also described in the Perl |
are described in detail below. There is a quick-reference syntax summary in the |
9 |
documentation and in a number of books, some of which have copious examples. |
.\" HREF |
10 |
Jeffrey Friedl's "Mastering Regular Expressions", published by O'Reilly, covers |
\fBpcresyntax\fP |
11 |
regular expressions in great detail. This description of PCRE's regular |
.\" |
12 |
expressions is intended as reference material. |
page. Perl's regular expressions are described in its own documentation, and |
13 |
|
regular expressions in general are covered in a number of books, some of which |
14 |
|
have copious examples. Jeffrey Friedl's "Mastering Regular Expressions", |
15 |
|
published by O'Reilly, covers regular expressions in great detail. This |
16 |
|
description of PCRE's regular expressions is intended as reference material. |
17 |
.P |
.P |
18 |
The original operation of PCRE was on strings of one-byte characters. However, |
The original operation of PCRE was on strings of one-byte characters. However, |
19 |
there is now also support for UTF-8 character strings. To use this, you must |
there is now also support for UTF-8 character strings. To use this, you must |
34 |
PCRE when its main matching function, \fBpcre_exec()\fP, is used. |
PCRE when its main matching function, \fBpcre_exec()\fP, is used. |
35 |
From release 6.0, PCRE offers a second matching function, |
From release 6.0, PCRE offers a second matching function, |
36 |
\fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not |
\fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not |
37 |
Perl-compatible. The advantages and disadvantages of the alternative function, |
Perl-compatible. Some of the features discussed below are not available when |
38 |
and how it differs from the normal function, are discussed in the |
\fBpcre_dfa_exec()\fP is used. The advantages and disadvantages of the |
39 |
|
alternative function, and how it differs from the normal function, are |
40 |
|
discussed in the |
41 |
.\" HREF |
.\" HREF |
42 |
\fBpcrematching\fP |
\fBpcrematching\fP |
43 |
.\" |
.\" |
44 |
page. |
page. |
45 |
. |
. |
46 |
. |
. |
47 |
|
.SH "NEWLINE CONVENTIONS" |
48 |
|
.rs |
49 |
|
.sp |
50 |
|
PCRE supports five different conventions for indicating line breaks in |
51 |
|
strings: a single CR (carriage return) character, a single LF (linefeed) |
52 |
|
character, the two-character sequence CRLF, any of the three preceding, or any |
53 |
|
Unicode newline sequence. The |
54 |
|
.\" HREF |
55 |
|
\fBpcreapi\fP |
56 |
|
.\" |
57 |
|
page has |
58 |
|
.\" HTML <a href="pcreapi.html#newlines"> |
59 |
|
.\" </a> |
60 |
|
further discussion |
61 |
|
.\" |
62 |
|
about newlines, and shows how to set the newline convention in the |
63 |
|
\fIoptions\fP arguments for the compiling and matching functions. |
64 |
|
.P |
65 |
|
It is also possible to specify a newline convention by starting a pattern |
66 |
|
string with one of the following five sequences: |
67 |
|
.sp |
68 |
|
(*CR) carriage return |
69 |
|
(*LF) linefeed |
70 |
|
(*CRLF) carriage return, followed by linefeed |
71 |
|
(*ANYCRLF) any of the three above |
72 |
|
(*ANY) all Unicode newline sequences |
73 |
|
.sp |
74 |
|
These override the default and the options given to \fBpcre_compile()\fP. For |
75 |
|
example, on a Unix system where LF is the default newline sequence, the pattern |
76 |
|
.sp |
77 |
|
(*CR)a.b |
78 |
|
.sp |
79 |
|
changes the convention to CR. That pattern matches "a\enb" because LF is no |
80 |
|
longer a newline. Note that these special settings, which are not |
81 |
|
Perl-compatible, are recognized only at the very start of a pattern, and that |
82 |
|
they must be in upper case. If more than one of them is present, the last one |
83 |
|
is used. |
84 |
|
.P |
85 |
|
The newline convention does not affect what the \eR escape sequence matches. By |
86 |
|
default, this is any Unicode newline sequence, for Perl compatibility. However, |
87 |
|
this can be changed; see the description of \eR in the section entitled |
88 |
|
.\" HTML <a href="#newlineseq"> |
89 |
|
.\" </a> |
90 |
|
"Newline sequences" |
91 |
|
.\" |
92 |
|
below. |
93 |
|
. |
94 |
|
. |
95 |
.SH "CHARACTERS AND METACHARACTERS" |
.SH "CHARACTERS AND METACHARACTERS" |
96 |
.rs |
.rs |
97 |
.sp |
.sp |
201 |
\ecx "control-x", where x is any character |
\ecx "control-x", where x is any character |
202 |
\ee escape (hex 1B) |
\ee escape (hex 1B) |
203 |
\ef formfeed (hex 0C) |
\ef formfeed (hex 0C) |
204 |
\en newline (hex 0A) |
\en linefeed (hex 0A) |
205 |
\er carriage return (hex 0D) |
\er carriage return (hex 0D) |
206 |
\et tab (hex 09) |
\et tab (hex 09) |
207 |
\eddd character with octal code ddd, or backreference |
\eddd character with octal code ddd, or backreference |
216 |
After \ex, from zero to two hexadecimal digits are read (letters can be in |
After \ex, from zero to two hexadecimal digits are read (letters can be in |
217 |
upper or lower case). Any number of hexadecimal digits may appear between \ex{ |
upper or lower case). Any number of hexadecimal digits may appear between \ex{ |
218 |
and }, but the value of the character code must be less than 256 in non-UTF-8 |
and }, but the value of the character code must be less than 256 in non-UTF-8 |
219 |
mode, and less than 2**31 in UTF-8 mode (that is, the maximum hexadecimal value |
mode, and less than 2**31 in UTF-8 mode. That is, the maximum value in |
220 |
is 7FFFFFFF). If characters other than hexadecimal digits appear between \ex{ |
hexadecimal is 7FFFFFFF. Note that this is bigger than the largest Unicode code |
221 |
and }, or if there is no terminating }, this form of escape is not recognized. |
point, which is 10FFFF. |
222 |
Instead, the initial \ex will be interpreted as a basic hexadecimal escape, |
.P |
223 |
with no following digits, giving a character whose value is zero. |
If characters other than hexadecimal digits appear between \ex{ and }, or if |
224 |
|
there is no terminating }, this form of escape is not recognized. Instead, the |
225 |
|
initial \ex will be interpreted as a basic hexadecimal escape, with no |
226 |
|
following digits, giving a character whose value is zero. |
227 |
.P |
.P |
228 |
Characters whose value is less than 256 can be defined by either of the two |
Characters whose value is less than 256 can be defined by either of the two |
229 |
syntaxes for \ex. There is no difference in the way they are handled. For |
syntaxes for \ex. There is no difference in the way they are handled. For |
295 |
.SS "Absolute and relative back references" |
.SS "Absolute and relative back references" |
296 |
.rs |
.rs |
297 |
.sp |
.sp |
298 |
The sequence \eg followed by a positive or negative number, optionally enclosed |
The sequence \eg followed by an unsigned or a negative number, optionally |
299 |
in braces, is an absolute or relative back reference. Back references are |
enclosed in braces, is an absolute or relative back reference. A named back |
300 |
discussed |
reference can be coded as \eg{name}. Back references are discussed |
301 |
.\" HTML <a href="#backreferences"> |
.\" HTML <a href="#backreferences"> |
302 |
.\" </a> |
.\" </a> |
303 |
later, |
later, |
317 |
.sp |
.sp |
318 |
\ed any decimal digit |
\ed any decimal digit |
319 |
\eD any character that is not a decimal digit |
\eD any character that is not a decimal digit |
320 |
|
\eh any horizontal whitespace character |
321 |
|
\eH any character that is not a horizontal whitespace character |
322 |
\es any whitespace character |
\es any whitespace character |
323 |
\eS any character that is not a whitespace character |
\eS any character that is not a whitespace character |
324 |
|
\ev any vertical whitespace character |
325 |
|
\eV any character that is not a vertical whitespace character |
326 |
\ew any "word" character |
\ew any "word" character |
327 |
\eW any "non-word" character |
\eW any "non-word" character |
328 |
.sp |
.sp |
336 |
.P |
.P |
337 |
For compatibility with Perl, \es does not match the VT character (code 11). |
For compatibility with Perl, \es does not match the VT character (code 11). |
338 |
This makes it different from the the POSIX "space" class. The \es characters |
This makes it different from the the POSIX "space" class. The \es characters |
339 |
are HT (9), LF (10), FF (12), CR (13), and space (32). (If "use locale;" is |
are HT (9), LF (10), FF (12), CR (13), and space (32). If "use locale;" is |
340 |
included in a Perl script, \es may match the VT character. In PCRE, it never |
included in a Perl script, \es may match the VT character. In PCRE, it never |
341 |
does.) |
does. |
342 |
|
.P |
343 |
|
In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or |
344 |
|
\ew, and always match \eD, \eS, and \eW. This is true even when Unicode |
345 |
|
character property support is available. These sequences retain their original |
346 |
|
meanings from before UTF-8 support was available, mainly for efficiency |
347 |
|
reasons. |
348 |
|
.P |
349 |
|
The sequences \eh, \eH, \ev, and \eV are Perl 5.10 features. In contrast to the |
350 |
|
other sequences, these do match certain high-valued codepoints in UTF-8 mode. |
351 |
|
The horizontal space characters are: |
352 |
|
.sp |
353 |
|
U+0009 Horizontal tab |
354 |
|
U+0020 Space |
355 |
|
U+00A0 Non-break space |
356 |
|
U+1680 Ogham space mark |
357 |
|
U+180E Mongolian vowel separator |
358 |
|
U+2000 En quad |
359 |
|
U+2001 Em quad |
360 |
|
U+2002 En space |
361 |
|
U+2003 Em space |
362 |
|
U+2004 Three-per-em space |
363 |
|
U+2005 Four-per-em space |
364 |
|
U+2006 Six-per-em space |
365 |
|
U+2007 Figure space |
366 |
|
U+2008 Punctuation space |
367 |
|
U+2009 Thin space |
368 |
|
U+200A Hair space |
369 |
|
U+202F Narrow no-break space |
370 |
|
U+205F Medium mathematical space |
371 |
|
U+3000 Ideographic space |
372 |
|
.sp |
373 |
|
The vertical space characters are: |
374 |
|
.sp |
375 |
|
U+000A Linefeed |
376 |
|
U+000B Vertical tab |
377 |
|
U+000C Formfeed |
378 |
|
U+000D Carriage return |
379 |
|
U+0085 Next line |
380 |
|
U+2028 Line separator |
381 |
|
U+2029 Paragraph separator |
382 |
.P |
.P |
383 |
A "word" character is an underscore or any character less than 256 that is a |
A "word" character is an underscore or any character less than 256 that is a |
384 |
letter or digit. The definition of letters and digits is controlled by PCRE's |
letter or digit. The definition of letters and digits is controlled by PCRE's |
394 |
.\" |
.\" |
395 |
page). For example, in a French locale such as "fr_FR" in Unix-like systems, |
page). For example, in a French locale such as "fr_FR" in Unix-like systems, |
396 |
or "french" in Windows, some character codes greater than 128 are used for |
or "french" in Windows, some character codes greater than 128 are used for |
397 |
accented letters, and these are matched by \ew. |
accented letters, and these are matched by \ew. The use of locales with Unicode |
398 |
.P |
is discouraged. |
|
In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or |
|
|
\ew, and always match \eD, \eS, and \eW. This is true even when Unicode |
|
|
character property support is available. The use of locales with Unicode is |
|
|
discouraged. |
|
399 |
. |
. |
400 |
. |
. |
401 |
|
.\" HTML <a name="newlineseq"></a> |
402 |
.SS "Newline sequences" |
.SS "Newline sequences" |
403 |
.rs |
.rs |
404 |
.sp |
.sp |
405 |
Outside a character class, the escape sequence \eR matches any Unicode newline |
Outside a character class, by default, the escape sequence \eR matches any |
406 |
sequence. This is an extension to Perl. In non-UTF-8 mode \eR is equivalent to |
Unicode newline sequence. This is a Perl 5.10 feature. In non-UTF-8 mode \eR is |
407 |
the following: |
equivalent to the following: |
408 |
.sp |
.sp |
409 |
(?>\er\en|\en|\ex0b|\ef|\er|\ex85) |
(?>\er\en|\en|\ex0b|\ef|\er|\ex85) |
410 |
.sp |
.sp |
424 |
Unicode character property support is not needed for these characters to be |
Unicode character property support is not needed for these characters to be |
425 |
recognized. |
recognized. |
426 |
.P |
.P |
427 |
|
It is possible to restrict \eR to match only CR, LF, or CRLF (instead of the |
428 |
|
complete set of Unicode line endings) by setting the option PCRE_BSR_ANYCRLF |
429 |
|
either at compile time or when the pattern is matched. This can be made the |
430 |
|
default when PCRE is built; if this is the case, the other behaviour can be |
431 |
|
requested via the PCRE_BSR_UNICODE option. It is also possible to specify these |
432 |
|
settings by starting a pattern string with one of the following sequences: |
433 |
|
.sp |
434 |
|
(*BSR_ANYCRLF) CR, LF, or CRLF only |
435 |
|
(*BSR_UNICODE) any Unicode newline sequence |
436 |
|
.sp |
437 |
|
These override the default and the options given to \fBpcre_compile()\fP, but |
438 |
|
they can be overridden by options given to \fBpcre_exec()\fP. Note that these |
439 |
|
special settings, which are not Perl-compatible, are recognized only at the |
440 |
|
very start of a pattern, and that they must be in upper case. If more than one |
441 |
|
of them is present, the last one is used. |
442 |
|
.P |
443 |
Inside a character class, \eR matches the letter "R". |
Inside a character class, \eR matches the letter "R". |
444 |
. |
. |
445 |
. |
. |
448 |
.rs |
.rs |
449 |
.sp |
.sp |
450 |
When PCRE is built with Unicode character property support, three additional |
When PCRE is built with Unicode character property support, three additional |
451 |
escape sequences to match character properties are available when UTF-8 mode |
escape sequences that match characters with specific properties are available. |
452 |
is selected. They are: |
When not in UTF-8 mode, these sequences are of course limited to testing |
453 |
|
characters whose codepoints are less than 256, but they do work in this mode. |
454 |
|
The extra escape sequences are: |
455 |
.sp |
.sp |
456 |
\ep{\fIxx\fP} a character with the \fIxx\fP property |
\ep{\fIxx\fP} a character with the \fIxx\fP property |
457 |
\eP{\fIxx\fP} a character without the \fIxx\fP property |
\eP{\fIxx\fP} a character without the \fIxx\fP property |
603 |
the Lu, Ll, or Lt property, in other words, a letter that is not classified as |
the Lu, Ll, or Lt property, in other words, a letter that is not classified as |
604 |
a modifier or "other". |
a modifier or "other". |
605 |
.P |
.P |
606 |
|
The Cs (Surrogate) property applies only to characters in the range U+D800 to |
607 |
|
U+DFFF. Such characters are not valid in UTF-8 strings (see RFC 3629) and so |
608 |
|
cannot be tested by PCRE, unless UTF-8 validity checking has been turned off |
609 |
|
(see the discussion of PCRE_NO_UTF8_CHECK in the |
610 |
|
.\" HREF |
611 |
|
\fBpcreapi\fP |
612 |
|
.\" |
613 |
|
page). |
614 |
|
.P |
615 |
The long synonyms for these properties that Perl supports (such as \ep{Letter}) |
The long synonyms for these properties that Perl supports (such as \ep{Letter}) |
616 |
are not supported by PCRE, nor is it permitted to prefix any of these |
are not supported by PCRE, nor is it permitted to prefix any of these |
617 |
properties with "Is". |
properties with "Is". |
636 |
(see below). |
(see below). |
637 |
.\" |
.\" |
638 |
Characters with the "mark" property are typically accents that affect the |
Characters with the "mark" property are typically accents that affect the |
639 |
preceding character. |
preceding character. None of them have codepoints less than 256, so in |
640 |
|
non-UTF-8 mode \eX matches any one character. |
641 |
.P |
.P |
642 |
Matching characters by Unicode property is not fast, because PCRE has to search |
Matching characters by Unicode property is not fast, because PCRE has to search |
643 |
a structure that contains data for over fifteen thousand characters. That is |
a structure that contains data for over fifteen thousand characters. That is |
645 |
properties in PCRE. |
properties in PCRE. |
646 |
. |
. |
647 |
. |
. |
648 |
|
.\" HTML <a name="resetmatchstart"></a> |
649 |
|
.SS "Resetting the match start" |
650 |
|
.rs |
651 |
|
.sp |
652 |
|
The escape sequence \eK, which is a Perl 5.10 feature, causes any previously |
653 |
|
matched characters not to be included in the final matched sequence. For |
654 |
|
example, the pattern: |
655 |
|
.sp |
656 |
|
foo\eKbar |
657 |
|
.sp |
658 |
|
matches "foobar", but reports that it has matched "bar". This feature is |
659 |
|
similar to a lookbehind assertion |
660 |
|
.\" HTML <a href="#lookbehind"> |
661 |
|
.\" </a> |
662 |
|
(described below). |
663 |
|
.\" |
664 |
|
However, in this case, the part of the subject before the real match does not |
665 |
|
have to be of fixed length, as lookbehind assertions do. The use of \eK does |
666 |
|
not interfere with the setting of |
667 |
|
.\" HTML <a href="#subpattern"> |
668 |
|
.\" </a> |
669 |
|
captured substrings. |
670 |
|
.\" |
671 |
|
For example, when the pattern |
672 |
|
.sp |
673 |
|
(foo)\eKbar |
674 |
|
.sp |
675 |
|
matches "foobar", the first substring is still set to "foo". |
676 |
|
. |
677 |
|
. |
678 |
.\" HTML <a name="smallassertions"></a> |
.\" HTML <a name="smallassertions"></a> |
679 |
.SS "Simple assertions" |
.SS "Simple assertions" |
680 |
.rs |
.rs |
987 |
.rs |
.rs |
988 |
.sp |
.sp |
989 |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
990 |
PCRE_EXTENDED options can be changed from within the pattern by a sequence of |
PCRE_EXTENDED options (which are Perl-compatible) can be changed from within |
991 |
Perl option letters enclosed between "(?" and ")". The option letters are |
the pattern by a sequence of Perl option letters enclosed between "(?" and ")". |
992 |
|
The option letters are |
993 |
.sp |
.sp |
994 |
i for PCRE_CASELESS |
i for PCRE_CASELESS |
995 |
m for PCRE_MULTILINE |
m for PCRE_MULTILINE |
1003 |
permitted. If a letter appears both before and after the hyphen, the option is |
permitted. If a letter appears both before and after the hyphen, the option is |
1004 |
unset. |
unset. |
1005 |
.P |
.P |
1006 |
|
The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be |
1007 |
|
changed in the same way as the Perl-compatible options by using the characters |
1008 |
|
J, U and X respectively. |
1009 |
|
.P |
1010 |
When an option change occurs at top level (that is, not inside subpattern |
When an option change occurs at top level (that is, not inside subpattern |
1011 |
parentheses), the change applies to the remainder of the pattern that follows. |
parentheses), the change applies to the remainder of the pattern that follows. |
1012 |
If the change is placed right at the start of a pattern, PCRE extracts it into |
If the change is placed right at the start of a pattern, PCRE extracts it into |
1029 |
branch is abandoned before the option setting. This is because the effects of |
branch is abandoned before the option setting. This is because the effects of |
1030 |
option settings happen at compile time. There would be some very weird |
option settings happen at compile time. There would be some very weird |
1031 |
behaviour otherwise. |
behaviour otherwise. |
|
.P |
|
|
The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA can be |
|
|
changed in the same way as the Perl-compatible options by using the characters |
|
|
J, U and X respectively. |
|
1032 |
. |
. |
1033 |
. |
. |
1034 |
.\" HTML <a name="subpattern"></a> |
.\" HTML <a name="subpattern"></a> |
1083 |
the above patterns match "SUNDAY" as well as "Saturday". |
the above patterns match "SUNDAY" as well as "Saturday". |
1084 |
. |
. |
1085 |
. |
. |
1086 |
|
.SH "DUPLICATE SUBPATTERN NUMBERS" |
1087 |
|
.rs |
1088 |
|
.sp |
1089 |
|
Perl 5.10 introduced a feature whereby each alternative in a subpattern uses |
1090 |
|
the same numbers for its capturing parentheses. Such a subpattern starts with |
1091 |
|
(?| and is itself a non-capturing subpattern. For example, consider this |
1092 |
|
pattern: |
1093 |
|
.sp |
1094 |
|
(?|(Sat)ur|(Sun))day |
1095 |
|
.sp |
1096 |
|
Because the two alternatives are inside a (?| group, both sets of capturing |
1097 |
|
parentheses are numbered one. Thus, when the pattern matches, you can look |
1098 |
|
at captured substring number one, whichever alternative matched. This construct |
1099 |
|
is useful when you want to capture part, but not all, of one of a number of |
1100 |
|
alternatives. Inside a (?| group, parentheses are numbered as usual, but the |
1101 |
|
number is reset at the start of each branch. The numbers of any capturing |
1102 |
|
buffers that follow the subpattern start after the highest number used in any |
1103 |
|
branch. The following example is taken from the Perl documentation. |
1104 |
|
The numbers underneath show in which buffer the captured content will be |
1105 |
|
stored. |
1106 |
|
.sp |
1107 |
|
# before ---------------branch-reset----------- after |
1108 |
|
/ ( a ) (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x |
1109 |
|
# 1 2 2 3 2 3 4 |
1110 |
|
.sp |
1111 |
|
A backreference or a recursive call to a numbered subpattern always refers to |
1112 |
|
the first one in the pattern with the given number. |
1113 |
|
.P |
1114 |
|
An alternative approach to using this "branch reset" feature is to use |
1115 |
|
duplicate named subpatterns, as described in the next section. |
1116 |
|
. |
1117 |
|
. |
1118 |
.SH "NAMED SUBPATTERNS" |
.SH "NAMED SUBPATTERNS" |
1119 |
.rs |
.rs |
1120 |
.sp |
.sp |
1164 |
(?<DN>Sat)(?:urday)? |
(?<DN>Sat)(?:urday)? |
1165 |
.sp |
.sp |
1166 |
There are five capturing substrings, but only one is ever set after a match. |
There are five capturing substrings, but only one is ever set after a match. |
1167 |
|
(An alternative way of solving this problem is to use a "branch reset" |
1168 |
|
subpattern, as described in the previous section.) |
1169 |
|
.P |
1170 |
The convenience function for extracting the data by name returns the substring |
The convenience function for extracting the data by name returns the substring |
1171 |
for the first (and in this example, the only) subpattern of that name that |
for the first (and in this example, the only) subpattern of that name that |
1172 |
matched. This saves searching to find which numbered subpattern it was. If you |
matched. This saves searching to find which numbered subpattern it was. If you |
1372 |
.sp |
.sp |
1373 |
\ed++foo |
\ed++foo |
1374 |
.sp |
.sp |
1375 |
|
Note that a possessive quantifier can be used with an entire group, for |
1376 |
|
example: |
1377 |
|
.sp |
1378 |
|
(abc|xyz){2,3}+ |
1379 |
|
.sp |
1380 |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
Possessive quantifiers are always greedy; the setting of the PCRE_UNGREEDY |
1381 |
option is ignored. They are a convenient notation for the simpler forms of |
option is ignored. They are a convenient notation for the simpler forms of |
1382 |
atomic group. However, there is no difference in the meaning of a possessive |
atomic group. However, there is no difference in the meaning of a possessive |
1451 |
.P |
.P |
1452 |
Another way of avoiding the ambiguity inherent in the use of digits following a |
Another way of avoiding the ambiguity inherent in the use of digits following a |
1453 |
backslash is to use the \eg escape sequence, which is a feature introduced in |
backslash is to use the \eg escape sequence, which is a feature introduced in |
1454 |
Perl 5.10. This escape must be followed by a positive or a negative number, |
Perl 5.10. This escape must be followed by an unsigned number or a negative |
1455 |
optionally enclosed in braces. These examples are all identical: |
number, optionally enclosed in braces. These examples are all identical: |
1456 |
.sp |
.sp |
1457 |
(ring), \e1 |
(ring), \e1 |
1458 |
(ring), \eg1 |
(ring), \eg1 |
1459 |
(ring), \eg{1} |
(ring), \eg{1} |
1460 |
.sp |
.sp |
1461 |
A positive number specifies an absolute reference without the ambiguity that is |
An unsigned number specifies an absolute reference without the ambiguity that |
1462 |
present in the older syntax. It is also useful when literal digits follow the |
is present in the older syntax. It is also useful when literal digits follow |
1463 |
reference. A negative number is a relative reference. Consider this example: |
the reference. A negative number is a relative reference. Consider this |
1464 |
|
example: |
1465 |
.sp |
.sp |
1466 |
(abc(def)ghi)\eg{-1} |
(abc(def)ghi)\eg{-1} |
1467 |
.sp |
.sp |
1491 |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original |
1492 |
capturing subpattern is matched caselessly. |
capturing subpattern is matched caselessly. |
1493 |
.P |
.P |
1494 |
Back references to named subpatterns use the Perl syntax \ek<name> or \ek'name' |
There are several different ways of writing back references to named |
1495 |
or the Python syntax (?P=name). We could rewrite the above example in either of |
subpatterns. The .NET syntax \ek{name} and the Perl syntax \ek<name> or |
1496 |
|
\ek'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified |
1497 |
|
back reference syntax, in which \eg can be used for both numeric and named |
1498 |
|
references, is also supported. We could rewrite the above example in any of |
1499 |
the following ways: |
the following ways: |
1500 |
.sp |
.sp |
1501 |
(?<p1>(?i)rah)\es+\ek<p1> |
(?<p1>(?i)rah)\es+\ek<p1> |
1502 |
|
(?'p1'(?i)rah)\es+\ek{p1} |
1503 |
(?P<p1>(?i)rah)\es+(?P=p1) |
(?P<p1>(?i)rah)\es+(?P=p1) |
1504 |
|
(?<p1>(?i)rah)\es+\eg{p1} |
1505 |
.sp |
.sp |
1506 |
A subpattern that is referenced by name may appear in the pattern before or |
A subpattern that is referenced by name may appear in the pattern before or |
1507 |
after the reference. |
after the reference. |
1624 |
.sp |
.sp |
1625 |
(?<=abc|abde) |
(?<=abc|abde) |
1626 |
.sp |
.sp |
1627 |
|
In some cases, the Perl 5.10 escape sequence \eK |
1628 |
|
.\" HTML <a href="#resetmatchstart"> |
1629 |
|
.\" </a> |
1630 |
|
(see above) |
1631 |
|
.\" |
1632 |
|
can be used instead of a lookbehind assertion; this is not restricted to a |
1633 |
|
fixed-length. |
1634 |
|
.P |
1635 |
The implementation of lookbehind assertions is, for each alternative, to |
The implementation of lookbehind assertions is, for each alternative, to |
1636 |
temporarily move the current position back by the fixed length and then try to |
temporarily move the current position back by the fixed length and then try to |
1637 |
match. If there are insufficient characters before the current position, the |
match. If there are insufficient characters before the current position, the |
1726 |
.sp |
.sp |
1727 |
If the text between the parentheses consists of a sequence of digits, the |
If the text between the parentheses consists of a sequence of digits, the |
1728 |
condition is true if the capturing subpattern of that number has previously |
condition is true if the capturing subpattern of that number has previously |
1729 |
matched. An alternative notation is to precede the digits with a plus or minus |
matched. An alternative notation is to precede the digits with a plus or minus |
1730 |
sign. In this case, the subpattern number is relative rather than absolute. |
sign. In this case, the subpattern number is relative rather than absolute. |
1731 |
The most recently opened parentheses can be referenced by (?(-1), the next most |
The most recently opened parentheses can be referenced by (?(-1), the next most |
1732 |
recent by (?(-2), and so on. In looping constructs it can also make sense to |
recent by (?(-2), and so on. In looping constructs it can also make sense to |
1733 |
refer to subsequent groups with constructs such as (?(+2). |
refer to subsequent groups with constructs such as (?(+2). |
1734 |
.P |
.P |
1748 |
subpattern matches nothing. In other words, this pattern matches a sequence of |
subpattern matches nothing. In other words, this pattern matches a sequence of |
1749 |
non-parentheses, optionally enclosed in parentheses. |
non-parentheses, optionally enclosed in parentheses. |
1750 |
.P |
.P |
1751 |
If you were embedding this pattern in a larger one, you could use a relative |
If you were embedding this pattern in a larger one, you could use a relative |
1752 |
reference: |
reference: |
1753 |
.sp |
.sp |
1754 |
...other stuff... ( \e( )? [^()]+ (?(-1) \e) ) ... |
...other stuff... ( \e( )? [^()]+ (?(-1) \e) ) ... |
1896 |
( \e( ( (?>[^()]+) | (?1) )* \e) ) |
( \e( ( (?>[^()]+) | (?1) )* \e) ) |
1897 |
.sp |
.sp |
1898 |
We have put the pattern into parentheses, and caused the recursion to refer to |
We have put the pattern into parentheses, and caused the recursion to refer to |
1899 |
them instead of the whole pattern. |
them instead of the whole pattern. |
1900 |
.P |
.P |
1901 |
In a larger pattern, keeping track of parenthesis numbers can be tricky. This |
In a larger pattern, keeping track of parenthesis numbers can be tricky. This |
1902 |
is made easier by the use of relative references. (A Perl 5.10 feature.) |
is made easier by the use of relative references. (A Perl 5.10 feature.) |
1917 |
(?<pn> \e( ( (?>[^()]+) | (?&pn) )* \e) ) |
(?<pn> \e( ( (?>[^()]+) | (?&pn) )* \e) ) |
1918 |
.sp |
.sp |
1919 |
If there is more than one subpattern with the same name, the earliest one is |
If there is more than one subpattern with the same name, the earliest one is |
1920 |
used. |
used. |
1921 |
.P |
.P |
1922 |
This particular example pattern that we have been looking at contains nested |
This particular example pattern that we have been looking at contains nested |
1923 |
unlimited repeats, and so the use of atomic grouping for matching strings of |
unlimited repeats, and so the use of atomic grouping for matching strings of |
1979 |
.sp |
.sp |
1980 |
(...(absolute)...)...(?2)... |
(...(absolute)...)...(?2)... |
1981 |
(...(relative)...)...(?-1)... |
(...(relative)...)...(?-1)... |
1982 |
(...(?+1)...(relative)... |
(...(?+1)...(relative)... |
1983 |
.sp |
.sp |
1984 |
An earlier example pointed out that the pattern |
An earlier example pointed out that the pattern |
1985 |
.sp |
.sp |
2044 |
documentation. |
documentation. |
2045 |
. |
. |
2046 |
. |
. |
2047 |
|
.SH "BACTRACKING CONTROL" |
2048 |
|
.rs |
2049 |
|
.sp |
2050 |
|
Perl 5.10 introduced a number of "Special Backtracking Control Verbs", which |
2051 |
|
are described in the Perl documentation as "experimental and subject to change |
2052 |
|
or removal in a future version of Perl". It goes on to say: "Their usage in |
2053 |
|
production code should be noted to avoid problems during upgrades." The same |
2054 |
|
remarks apply to the PCRE features described in this section. |
2055 |
|
.P |
2056 |
|
Since these verbs are specifically related to backtracking, they can be used |
2057 |
|
only when the pattern is to be matched using \fBpcre_exec()\fP, which uses a |
2058 |
|
backtracking algorithm. They cause an error if encountered by |
2059 |
|
\fBpcre_dfa_exec()\fP. |
2060 |
|
.P |
2061 |
|
The new verbs make use of what was previously invalid syntax: an opening |
2062 |
|
parenthesis followed by an asterisk. In Perl, they are generally of the form |
2063 |
|
(*VERB:ARG) but PCRE does not support the use of arguments, so its general |
2064 |
|
form is just (*VERB). Any number of these verbs may occur in a pattern. There |
2065 |
|
are two kinds: |
2066 |
|
. |
2067 |
|
.SS "Verbs that act immediately" |
2068 |
|
.rs |
2069 |
|
.sp |
2070 |
|
The following verbs act as soon as they are encountered: |
2071 |
|
.sp |
2072 |
|
(*ACCEPT) |
2073 |
|
.sp |
2074 |
|
This verb causes the match to end successfully, skipping the remainder of the |
2075 |
|
pattern. When inside a recursion, only the innermost pattern is ended |
2076 |
|
immediately. PCRE differs from Perl in what happens if the (*ACCEPT) is inside |
2077 |
|
capturing parentheses. In Perl, the data so far is captured: in PCRE no data is |
2078 |
|
captured. For example: |
2079 |
|
.sp |
2080 |
|
A(A|B(*ACCEPT)|C)D |
2081 |
|
.sp |
2082 |
|
This matches "AB", "AAD", or "ACD", but when it matches "AB", no data is |
2083 |
|
captured. |
2084 |
|
.sp |
2085 |
|
(*FAIL) or (*F) |
2086 |
|
.sp |
2087 |
|
This verb causes the match to fail, forcing backtracking to occur. It is |
2088 |
|
equivalent to (?!) but easier to read. The Perl documentation notes that it is |
2089 |
|
probably useful only when combined with (?{}) or (??{}). Those are, of course, |
2090 |
|
Perl features that are not present in PCRE. The nearest equivalent is the |
2091 |
|
callout feature, as for example in this pattern: |
2092 |
|
.sp |
2093 |
|
a+(?C)(*FAIL) |
2094 |
|
.sp |
2095 |
|
A match with the string "aaaa" always fails, but the callout is taken before |
2096 |
|
each backtrack happens (in this example, 10 times). |
2097 |
|
. |
2098 |
|
.SS "Verbs that act after backtracking" |
2099 |
|
.rs |
2100 |
|
.sp |
2101 |
|
The following verbs do nothing when they are encountered. Matching continues |
2102 |
|
with what follows, but if there is no subsequent match, a failure is forced. |
2103 |
|
The verbs differ in exactly what kind of failure occurs. |
2104 |
|
.sp |
2105 |
|
(*COMMIT) |
2106 |
|
.sp |
2107 |
|
This verb causes the whole match to fail outright if the rest of the pattern |
2108 |
|
does not match. Even if the pattern is unanchored, no further attempts to find |
2109 |
|
a match by advancing the start point take place. Once (*COMMIT) has been |
2110 |
|
passed, \fBpcre_exec()\fP is committed to finding a match at the current |
2111 |
|
starting point, or not at all. For example: |
2112 |
|
.sp |
2113 |
|
a+(*COMMIT)b |
2114 |
|
.sp |
2115 |
|
This matches "xxaab" but not "aacaab". It can be thought of as a kind of |
2116 |
|
dynamic anchor, or "I've started, so I must finish." |
2117 |
|
.sp |
2118 |
|
(*PRUNE) |
2119 |
|
.sp |
2120 |
|
This verb causes the match to fail at the current position if the rest of the |
2121 |
|
pattern does not match. If the pattern is unanchored, the normal "bumpalong" |
2122 |
|
advance to the next starting character then happens. Backtracking can occur as |
2123 |
|
usual to the left of (*PRUNE), or when matching to the right of (*PRUNE), but |
2124 |
|
if there is no match to the right, backtracking cannot cross (*PRUNE). |
2125 |
|
In simple cases, the use of (*PRUNE) is just an alternative to an atomic |
2126 |
|
group or possessive quantifier, but there are some uses of (*PRUNE) that cannot |
2127 |
|
be expressed in any other way. |
2128 |
|
.sp |
2129 |
|
(*SKIP) |
2130 |
|
.sp |
2131 |
|
This verb is like (*PRUNE), except that if the pattern is unanchored, the |
2132 |
|
"bumpalong" advance is not to the next character, but to the position in the |
2133 |
|
subject where (*SKIP) was encountered. (*SKIP) signifies that whatever text |
2134 |
|
was matched leading up to it cannot be part of a successful match. Consider: |
2135 |
|
.sp |
2136 |
|
a+(*SKIP)b |
2137 |
|
.sp |
2138 |
|
If the subject is "aaaac...", after the first match attempt fails (starting at |
2139 |
|
the first character in the string), the starting point skips on to start the |
2140 |
|
next attempt at "c". Note that a possessive quantifer does not have the same |
2141 |
|
effect in this example; although it would suppress backtracking during the |
2142 |
|
first match attempt, the second attempt would start at the second character |
2143 |
|
instead of skipping on to "c". |
2144 |
|
.sp |
2145 |
|
(*THEN) |
2146 |
|
.sp |
2147 |
|
This verb causes a skip to the next alternation if the rest of the pattern does |
2148 |
|
not match. That is, it cancels pending backtracking, but only within the |
2149 |
|
current alternation. Its name comes from the observation that it can be used |
2150 |
|
for a pattern-based if-then-else block: |
2151 |
|
.sp |
2152 |
|
( COND1 (*THEN) FOO | COND2 (*THEN) BAR | COND3 (*THEN) BAZ ) ... |
2153 |
|
.sp |
2154 |
|
If the COND1 pattern matches, FOO is tried (and possibly further items after |
2155 |
|
the end of the group if FOO succeeds); on failure the matcher skips to the |
2156 |
|
second alternative and tries COND2, without backtracking into COND1. If (*THEN) |
2157 |
|
is used outside of any alternation, it acts exactly like (*PRUNE). |
2158 |
|
. |
2159 |
|
. |
2160 |
.SH "SEE ALSO" |
.SH "SEE ALSO" |
2161 |
.rs |
.rs |
2162 |
.sp |
.sp |
2177 |
.rs |
.rs |
2178 |
.sp |
.sp |
2179 |
.nf |
.nf |
2180 |
Last updated: 09 May 2007 |
Last updated: 11 September 2007 |
2181 |
Copyright (c) 1997-2007 University of Cambridge. |
Copyright (c) 1997-2007 University of Cambridge. |
2182 |
.fi |
.fi |