/[pcre]/code/trunk/doc/pcrepattern.3
ViewVC logotype

Diff of /code/trunk/doc/pcrepattern.3

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 155 by ph10, Tue Apr 24 13:36:11 2007 UTC revision 184 by ph10, Tue Jun 19 13:26:46 2007 UTC
# Line 30  The remainder of this document discusses Line 30  The remainder of this document discusses
30  PCRE when its main matching function, \fBpcre_exec()\fP, is used.  PCRE when its main matching function, \fBpcre_exec()\fP, is used.
31  From release 6.0, PCRE offers a second matching function,  From release 6.0, PCRE offers a second matching function,
32  \fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not  \fBpcre_dfa_exec()\fP, which matches using a different algorithm that is not
33  Perl-compatible. The advantages and disadvantages of the alternative function,  Perl-compatible. Some of the features discussed below are not available when
34  and how it differs from the normal function, are discussed in the  \fBpcre_dfa_exec()\fP is used. The advantages and disadvantages of the
35    alternative function, and how it differs from the normal function, are
36    discussed in the
37  .\" HREF  .\" HREF
38  \fBpcrematching\fP  \fBpcrematching\fP
39  .\"  .\"
# Line 239  meanings Line 241  meanings
241  .rs  .rs
242  .sp  .sp
243  The sequence \eg followed by a positive or negative number, optionally enclosed  The sequence \eg followed by a positive or negative number, optionally enclosed
244  in braces, is an absolute or relative back reference. Back references are  in braces, is an absolute or relative back reference. A named back reference
245  discussed  can be coded as \eg{name}. Back references are discussed
246  .\" HTML <a href="#backreferences">  .\" HTML <a href="#backreferences">
247  .\" </a>  .\" </a>
248  later,  later,
# Line 260  following are always recognized: Line 262  following are always recognized:
262  .sp  .sp
263    \ed     any decimal digit    \ed     any decimal digit
264    \eD     any character that is not a decimal digit    \eD     any character that is not a decimal digit
265      \eh     any horizontal whitespace character
266      \eH     any character that is not a horizontal whitespace character
267    \es     any whitespace character    \es     any whitespace character
268    \eS     any character that is not a whitespace character    \eS     any character that is not a whitespace character
269      \ev     any vertical whitespace character
270      \eV     any character that is not a vertical whitespace character
271    \ew     any "word" character    \ew     any "word" character
272    \eW     any "non-word" character    \eW     any "non-word" character
273  .sp  .sp
# Line 275  there is no character to match. Line 281  there is no character to match.
281  .P  .P
282  For compatibility with Perl, \es does not match the VT character (code 11).  For compatibility with Perl, \es does not match the VT character (code 11).
283  This makes it different from the the POSIX "space" class. The \es characters  This makes it different from the the POSIX "space" class. The \es characters
284  are HT (9), LF (10), FF (12), CR (13), and space (32). (If "use locale;" is  are HT (9), LF (10), FF (12), CR (13), and space (32). If "use locale;" is
285  included in a Perl script, \es may match the VT character. In PCRE, it never  included in a Perl script, \es may match the VT character. In PCRE, it never
286  does.)  does.
287    .P
288    In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or
289    \ew, and always match \eD, \eS, and \eW. This is true even when Unicode
290    character property support is available. These sequences retain their original
291    meanings from before UTF-8 support was available, mainly for efficiency
292    reasons.
293    .P
294    The sequences \eh, \eH, \ev, and \eV are Perl 5.10 features. In contrast to the
295    other sequences, these do match certain high-valued codepoints in UTF-8 mode.
296    The horizontal space characters are:
297    .sp
298      U+0009     Horizontal tab
299      U+0020     Space
300      U+00A0     Non-break space
301      U+1680     Ogham space mark
302      U+180E     Mongolian vowel separator
303      U+2000     En quad
304      U+2001     Em quad
305      U+2002     En space
306      U+2003     Em space
307      U+2004     Three-per-em space
308      U+2005     Four-per-em space
309      U+2006     Six-per-em space
310      U+2007     Figure space
311      U+2008     Punctuation space
312      U+2009     Thin space
313      U+200A     Hair space
314      U+202F     Narrow no-break space
315      U+205F     Medium mathematical space
316      U+3000     Ideographic space
317    .sp
318    The vertical space characters are:
319    .sp
320      U+000A     Linefeed
321      U+000B     Vertical tab
322      U+000C     Formfeed
323      U+000D     Carriage return
324      U+0085     Next line
325      U+2028     Line separator
326      U+2029     Paragraph separator
327  .P  .P
328  A "word" character is an underscore or any character less than 256 that is a  A "word" character is an underscore or any character less than 256 that is a
329  letter or digit. The definition of letters and digits is controlled by PCRE's  letter or digit. The definition of letters and digits is controlled by PCRE's
# Line 293  in the Line 339  in the
339  .\"  .\"
340  page). For example, in a French locale such as "fr_FR" in Unix-like systems,  page). For example, in a French locale such as "fr_FR" in Unix-like systems,
341  or "french" in Windows, some character codes greater than 128 are used for  or "french" in Windows, some character codes greater than 128 are used for
342  accented letters, and these are matched by \ew.  accented letters, and these are matched by \ew. The use of locales with Unicode
343  .P  is discouraged.
 In UTF-8 mode, characters with values greater than 128 never match \ed, \es, or  
 \ew, and always match \eD, \eS, and \eW. This is true even when Unicode  
 character property support is available. The use of locales with Unicode is  
 discouraged.  
344  .  .
345  .  .
346  .SS "Newline sequences"  .SS "Newline sequences"
347  .rs  .rs
348  .sp  .sp
349  Outside a character class, the escape sequence \eR matches any Unicode newline  Outside a character class, the escape sequence \eR matches any Unicode newline
350  sequence. This is an extension to Perl. In non-UTF-8 mode \eR is equivalent to  sequence. This is a Perl 5.10 feature. In non-UTF-8 mode \eR is equivalent to
351  the following:  the following:
352  .sp  .sp
353    (?>\er\en|\en|\ex0b|\ef|\er|\ex85)    (?>\er\en|\en|\ex0b|\ef|\er|\ex85)
# Line 334  Inside a character class, \eR matches th Line 376  Inside a character class, \eR matches th
376  .rs  .rs
377  .sp  .sp
378  When PCRE is built with Unicode character property support, three additional  When PCRE is built with Unicode character property support, three additional
379  escape sequences to match character properties are available when UTF-8 mode  escape sequences that match characters with specific properties are available.
380  is selected. They are:  When not in UTF-8 mode, these sequences are of course limited to testing
381    characters whose codepoints are less than 256, but they do work in this mode.
382    The extra escape sequences are:
383  .sp  .sp
384    \ep{\fIxx\fP}   a character with the \fIxx\fP property    \ep{\fIxx\fP}   a character with the \fIxx\fP property
385    \eP{\fIxx\fP}   a character without the \fIxx\fP property    \eP{\fIxx\fP}   a character without the \fIxx\fP property
# Line 511  atomic group Line 555  atomic group
555  (see below).  (see below).
556  .\"  .\"
557  Characters with the "mark" property are typically accents that affect the  Characters with the "mark" property are typically accents that affect the
558  preceding character.  preceding character. None of them have codepoints less than 256, so in
559    non-UTF-8 mode \eX matches any one character.
560  .P  .P
561  Matching characters by Unicode property is not fast, because PCRE has to search  Matching characters by Unicode property is not fast, because PCRE has to search
562  a structure that contains data for over fifteen thousand characters. That is  a structure that contains data for over fifteen thousand characters. That is
# Line 519  why the traditional escape sequences suc Line 564  why the traditional escape sequences suc
564  properties in PCRE.  properties in PCRE.
565  .  .
566  .  .
567    .\" HTML <a name="resetmatchstart"></a>
568    .SS "Resetting the match start"
569    .rs
570    .sp
571    The escape sequence \eK, which is a Perl 5.10 feature, causes any previously
572    matched characters not to be included in the final matched sequence. For
573    example, the pattern:
574    .sp
575      foo\eKbar
576    .sp
577    matches "foobar", but reports that it has matched "bar". This feature is
578    similar to a lookbehind assertion
579    .\" HTML <a href="#lookbehind">
580    .\" </a>
581    (described below).
582    .\"
583    However, in this case, the part of the subject before the real match does not
584    have to be of fixed length, as lookbehind assertions do. The use of \eK does
585    not interfere with the setting of
586    .\" HTML <a href="#subpattern">
587    .\" </a>
588    captured substrings.
589    .\"
590    For example, when the pattern
591    .sp
592      (foo)\eKbar
593    .sp
594    matches "foobar", the first substring is still set to "foo".
595    .
596    .
597  .\" HTML <a name="smallassertions"></a>  .\" HTML <a name="smallassertions"></a>
598  .SS "Simple assertions"  .SS "Simple assertions"
599  .rs  .rs
# Line 926  is reached, an option setting in one bra Line 1001  is reached, an option setting in one bra
1001  the above patterns match "SUNDAY" as well as "Saturday".  the above patterns match "SUNDAY" as well as "Saturday".
1002  .  .
1003  .  .
1004    .SH "DUPLICATE SUBPATTERN NUMBERS"
1005    .rs
1006    .sp
1007    Perl 5.10 introduced a feature whereby each alternative in a subpattern uses
1008    the same numbers for its capturing parentheses. Such a subpattern starts with
1009    (?| and is itself a non-capturing subpattern. For example, consider this
1010    pattern:
1011    .sp
1012      (?|(Sat)ur|(Sun))day
1013    .sp
1014    Because the two alternatives are inside a (?| group, both sets of capturing
1015    parentheses are numbered one. Thus, when the pattern matches, you can look
1016    at captured substring number one, whichever alternative matched. This construct
1017    is useful when you want to capture part, but not all, of one of a number of
1018    alternatives. Inside a (?| group, parentheses are numbered as usual, but the
1019    number is reset at the start of each branch. The numbers of any capturing
1020    buffers that follow the subpattern start after the highest number used in any
1021    branch. The following example is taken from the Perl documentation.
1022    The numbers underneath show in which buffer the captured content will be
1023    stored.
1024    .sp
1025      # before  ---------------branch-reset----------- after
1026      / ( a )  (?| x ( y ) z | (p (q) r) | (t) u (v) ) ( z ) /x
1027      # 1            2         2  3        2     3     4
1028    .sp
1029    A backreference or a recursive call to a numbered subpattern always refers to
1030    the first one in the pattern with the given number.
1031    .P
1032    An alternative approach to using this "branch reset" feature is to use
1033    duplicate named subpatterns, as described in the next section.
1034    .
1035    .
1036  .SH "NAMED SUBPATTERNS"  .SH "NAMED SUBPATTERNS"
1037  .rs  .rs
1038  .sp  .sp
# Line 975  abbreviation. This pattern (ignoring the Line 1082  abbreviation. This pattern (ignoring the
1082    (?<DN>Sat)(?:urday)?    (?<DN>Sat)(?:urday)?
1083  .sp  .sp
1084  There are five capturing substrings, but only one is ever set after a match.  There are five capturing substrings, but only one is ever set after a match.
1085    (An alternative way of solving this problem is to use a "branch reset"
1086    subpattern, as described in the previous section.)
1087    .P
1088  The convenience function for extracting the data by name returns the substring  The convenience function for extracting the data by name returns the substring
1089  for the first (and in this example, the only) subpattern of that name that  for the first (and in this example, the only) subpattern of that name that
1090  matched. This saves searching to find which numbered subpattern it was. If you  matched. This saves searching to find which numbered subpattern it was. If you
# Line 1293  back reference, the case of letters is r Line 1403  back reference, the case of letters is r
1403  matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original  matches "rah rah" and "RAH RAH", but not "RAH rah", even though the original
1404  capturing subpattern is matched caselessly.  capturing subpattern is matched caselessly.
1405  .P  .P
1406  Back references to named subpatterns use the Perl syntax \ek<name> or \ek'name'  There are several different ways of writing back references to named
1407  or the Python syntax (?P=name). We could rewrite the above example in either of  subpatterns. The .NET syntax \ek{name} and the Perl syntax \ek<name> or
1408    \ek'name' are supported, as is the Python syntax (?P=name). Perl 5.10's unified
1409    back reference syntax, in which \eg can be used for both numeric and named
1410    references, is also supported. We could rewrite the above example in any of
1411  the following ways:  the following ways:
1412  .sp  .sp
1413    (?<p1>(?i)rah)\es+\ek<p1>    (?<p1>(?i)rah)\es+\ek<p1>
1414      (?'p1'(?i)rah)\es+\ek{p1}
1415    (?P<p1>(?i)rah)\es+(?P=p1)    (?P<p1>(?i)rah)\es+(?P=p1)
1416      (?<p1>(?i)rah)\es+\eg{p1}
1417  .sp  .sp
1418  A subpattern that is referenced by name may appear in the pattern before or  A subpattern that is referenced by name may appear in the pattern before or
1419  after the reference.  after the reference.
# Line 1421  lengths, but it is acceptable if rewritt Line 1536  lengths, but it is acceptable if rewritt
1536  .sp  .sp
1537    (?<=abc|abde)    (?<=abc|abde)
1538  .sp  .sp
1539    In some cases, the Perl 5.10 escape sequence \eK
1540    .\" HTML <a href="#resetmatchstart">
1541    .\" </a>
1542    (see above)
1543    .\"
1544    can be used instead of a lookbehind assertion; this is not restricted to a
1545    fixed-length.
1546    .P
1547  The implementation of lookbehind assertions is, for each alternative, to  The implementation of lookbehind assertions is, for each alternative, to
1548  temporarily move the current position back by the fixed length and then try to  temporarily move the current position back by the fixed length and then try to
1549  match. If there are insufficient characters before the current position, the  match. If there are insufficient characters before the current position, the
# Line 1515  recursion, a pseudo-condition called DEF Line 1638  recursion, a pseudo-condition called DEF
1638  .sp  .sp
1639  If the text between the parentheses consists of a sequence of digits, the  If the text between the parentheses consists of a sequence of digits, the
1640  condition is true if the capturing subpattern of that number has previously  condition is true if the capturing subpattern of that number has previously
1641  matched.  matched. An alternative notation is to precede the digits with a plus or minus
1642    sign. In this case, the subpattern number is relative rather than absolute.
1643    The most recently opened parentheses can be referenced by (?(-1), the next most
1644    recent by (?(-2), and so on. In looping constructs it can also make sense to
1645    refer to subsequent groups with constructs such as (?(+2).
1646  .P  .P
1647  Consider the following pattern, which contains non-significant white space to  Consider the following pattern, which contains non-significant white space to
1648  make it more readable (assume the PCRE_EXTENDED option) and to divide it into  make it more readable (assume the PCRE_EXTENDED option) and to divide it into
# Line 1532  the condition is true, and so the yes-pa Line 1659  the condition is true, and so the yes-pa
1659  parenthesis is required. Otherwise, since no-pattern is not present, the  parenthesis is required. Otherwise, since no-pattern is not present, the
1660  subpattern matches nothing. In other words, this pattern matches a sequence of  subpattern matches nothing. In other words, this pattern matches a sequence of
1661  non-parentheses, optionally enclosed in parentheses.  non-parentheses, optionally enclosed in parentheses.
1662    .P
1663    If you were embedding this pattern in a larger one, you could use a relative
1664    reference:
1665    .sp
1666      ...other stuff... ( \e( )?    [^()]+    (?(-1) \e) ) ...
1667    .sp
1668    This makes the fragment independent of the parentheses in the larger pattern.
1669  .  .
1670  .SS "Checking for a used subpattern by name"  .SS "Checking for a used subpattern by name"
1671  .rs  .rs
# Line 1674  pattern, so instead you could use this: Line 1808  pattern, so instead you could use this:
1808    ( \e( ( (?>[^()]+) | (?1) )* \e) )    ( \e( ( (?>[^()]+) | (?1) )* \e) )
1809  .sp  .sp
1810  We have put the pattern into parentheses, and caused the recursion to refer to  We have put the pattern into parentheses, and caused the recursion to refer to
1811  them instead of the whole pattern. In a larger pattern, keeping track of  them instead of the whole pattern.
1812  parenthesis numbers can be tricky. It may be more convenient to use named  .P
1813  parentheses instead. The Perl syntax for this is (?&name); PCRE's earlier  In a larger pattern, keeping track of parenthesis numbers can be tricky. This
1814  syntax (?P>name) is also supported. We could rewrite the above example as  is made easier by the use of relative references. (A Perl 5.10 feature.)
1815  follows:  Instead of (?1) in the pattern above you can write (?-2) to refer to the second
1816    most recently opened parentheses preceding the recursion. In other words, a
1817    negative number counts capturing parentheses leftwards from the point at which
1818    it is encountered.
1819    .P
1820    It is also possible to refer to subsequently opened parentheses, by writing
1821    references such as (?+2). However, these cannot be recursive because the
1822    reference is not inside the parentheses that are referenced. They are always
1823    "subroutine" calls, as described in the next section.
1824    .P
1825    An alternative approach is to use named parentheses instead. The Perl syntax
1826    for this is (?&name); PCRE's earlier syntax (?P>name) is also supported. We
1827    could rewrite the above example as follows:
1828  .sp  .sp
1829    (?<pn> \e( ( (?>[^()]+) | (?&pn) )* \e) )    (?<pn> \e( ( (?>[^()]+) | (?&pn) )* \e) )
1830  .sp  .sp
1831  If there is more than one subpattern with the same name, the earliest one is  If there is more than one subpattern with the same name, the earliest one is
1832  used. This particular example pattern contains nested unlimited repeats, and so  used.
1833  the use of atomic grouping for matching strings of non-parentheses is important  .P
1834  when applying the pattern to strings that do not match. For example, when this  This particular example pattern that we have been looking at contains nested
1835  pattern is applied to  unlimited repeats, and so the use of atomic grouping for matching strings of
1836    non-parentheses is important when applying the pattern to strings that do not
1837    match. For example, when this pattern is applied to
1838  .sp  .sp
1839    (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()    (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()
1840  .sp  .sp
# Line 1738  is the actual recursive call. Line 1886  is the actual recursive call.
1886  If the syntax for a recursive subpattern reference (either by number or by  If the syntax for a recursive subpattern reference (either by number or by
1887  name) is used outside the parentheses to which it refers, it operates like a  name) is used outside the parentheses to which it refers, it operates like a
1888  subroutine in a programming language. The "called" subpattern may be defined  subroutine in a programming language. The "called" subpattern may be defined
1889  before or after the reference. An earlier example pointed out that the pattern  before or after the reference. A numbered reference can be absolute or
1890    relative, as in these examples:
1891    .sp
1892      (...(absolute)...)...(?2)...
1893      (...(relative)...)...(?-1)...
1894      (...(?+1)...(relative)...
1895    .sp
1896    An earlier example pointed out that the pattern
1897  .sp  .sp
1898    (sens|respons)e and \e1ibility    (sens|respons)e and \e1ibility
1899  .sp  .sp
# Line 1759  When a subpattern is used as a subroutin Line 1914  When a subpattern is used as a subroutin
1914  case-independence are fixed when the subpattern is defined. They cannot be  case-independence are fixed when the subpattern is defined. They cannot be
1915  changed for different calls. For example, consider this pattern:  changed for different calls. For example, consider this pattern:
1916  .sp  .sp
1917    (abc)(?i:(?1))    (abc)(?i:(?-1))
1918  .sp  .sp
1919  It matches "abcabc". It does not match "abcABC" because the change of  It matches "abcabc". It does not match "abcABC" because the change of
1920  processing option does not affect the called subpattern.  processing option does not affect the called subpattern.
# Line 1821  Cambridge CB2 3QH, England. Line 1976  Cambridge CB2 3QH, England.
1976  .rs  .rs
1977  .sp  .sp
1978  .nf  .nf
1979  Last updated: 06 March 2007  Last updated: 19 June 2007
1980  Copyright (c) 1997-2007 University of Cambridge.  Copyright (c) 1997-2007 University of Cambridge.
1981  .fi  .fi

Legend:
Removed from v.155  
changed lines
  Added in v.184

  ViewVC Help
Powered by ViewVC 1.1.5