/[pcre]/code/trunk/doc/pcrepattern.3
ViewVC logotype

Diff of /code/trunk/doc/pcrepattern.3

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 491 by ph10, Mon Mar 1 17:45:08 2010 UTC revision 517 by ph10, Wed May 5 10:44:20 2010 UTC
# Line 95  Perl-compatible, are recognized only at Line 95  Perl-compatible, are recognized only at
95  they must be in upper case. If more than one of them is present, the last one  they must be in upper case. If more than one of them is present, the last one
96  is used.  is used.
97  .P  .P
98  The newline convention does not affect what the \eR escape sequence matches. By  The newline convention affects the interpretation of the dot metacharacter when
99  default, this is any Unicode newline sequence, for Perl compatibility. However,  PCRE_DOTALL is not set, and also the behaviour of \eN. However, it does not
100  this can be changed; see the description of \eR in the section entitled  affect what the \eR escape sequence matches. By default, this is any Unicode
101    newline sequence, for Perl compatibility. However, this can be changed; see the
102    description of \eR in the section entitled
103  .\" HTML <a href="#newlineseq">  .\" HTML <a href="#newlineseq">
104  .\" </a>  .\" </a>
105  "Newline sequences"  "Newline sequences"
# Line 295  zero, because no more than three octal d Line 297  zero, because no more than three octal d
297  .P  .P
298  All the sequences that define a single character value can be used both inside  All the sequences that define a single character value can be used both inside
299  and outside character classes. In addition, inside a character class, the  and outside character classes. In addition, inside a character class, the
300  sequence \eb is interpreted as the backspace character (hex 08), and the  sequence \eb is interpreted as the backspace character (hex 08). The sequences
301  sequences \eR and \eX are interpreted as the characters "R" and "X",  \eB, \eN, \eR, and \eX are not special inside a character class. Like any other
302  respectively. Outside a character class, these sequences have different  unrecognized escape sequences, they are treated as the literal characters "B",
303  meanings  "N", "R", and "X" by default, but cause an error if the PCRE_EXTRA option is
304  .\" HTML <a href="#uniextseq">  set. Outside a character class, these sequences have different meanings.
 .\" </a>  
 (see below).  
 .\"  
305  .  .
306  .  .
307  .SS "Absolute and relative back references"  .SS "Absolute and relative back references"
# Line 344  call. Line 343  call.
343  .SS "Generic character types"  .SS "Generic character types"
344  .rs  .rs
345  .sp  .sp
346  Another use of backslash is for specifying generic character types. The  Another use of backslash is for specifying generic character types:
 following are always recognized:  
347  .sp  .sp
348    \ed     any decimal digit    \ed     any decimal digit
349    \eD     any character that is not a decimal digit    \eD     any character that is not a decimal digit
# Line 358  following are always recognized: Line 356  following are always recognized:
356    \ew     any "word" character    \ew     any "word" character
357    \eW     any "non-word" character    \eW     any "non-word" character
358  .sp  .sp
359  Each pair of escape sequences partitions the complete set of characters into  There is also the single sequence \eN, which matches a non-newline character.
360  two disjoint sets. Any given character matches one, and only one, of each pair.  This is the same as
361    .\" HTML <a href="#fullstopdot">
362    .\" </a>
363    the "." metacharacter
364    .\"
365    when PCRE_DOTALL is not set.
366    .P
367    Each pair of lower and upper case escape sequences partitions the complete set
368    of characters into two disjoint sets. Any given character matches one, and only
369    one, of each pair.
370  .P  .P
371  These character type sequences can appear both inside and outside character  These character type sequences can appear both inside and outside character
372  classes. They each match one character of the appropriate type. If the current  classes. They each match one character of the appropriate type. If the current
# Line 478  convention, for example, a pattern can s Line 485  convention, for example, a pattern can s
485  .sp  .sp
486    (*ANY)(*BSR_ANYCRLF)    (*ANY)(*BSR_ANYCRLF)
487  .sp  .sp
488  Inside a character class, \eR matches the letter "R".  Inside a character class, \eR is treated as an unrecognized escape sequence,
489    and so matches the letter "R" by default, but causes an error if PCRE_EXTRA is
490    set.
491  .  .
492  .  .
493  .\" HTML <a name="uniextseq"></a>  .\" HTML <a name="uniextseq"></a>
# Line 496  The extra escape sequences are: Line 505  The extra escape sequences are:
505    \eX       an extended Unicode sequence    \eX       an extended Unicode sequence
506  .sp  .sp
507  The property names represented by \fIxx\fP above are limited to the Unicode  The property names represented by \fIxx\fP above are limited to the Unicode
508  script names, the general category properties, and "Any", which matches any  script names, the general category properties, "Any", which matches any
509  character (including newline). Other properties such as "InMusicalSymbols" are  character (including newline), and some special PCRE properties (described
510  not currently supported by PCRE. Note that \eP{Any} does not match any  in the
511  characters, so always causes a match failure.  .\" HTML <a href="#extraprops">
512    .\" </a>
513    next section).
514    .\"
515    Other Perl properties such as "InMusicalSymbols" are not currently supported by
516    PCRE. Note that \eP{Any} does not match any characters, so always causes a
517    match failure.
518  .P  .P
519  Sets of Unicode characters are defined as belonging to certain scripts. A  Sets of Unicode characters are defined as belonging to certain scripts. A
520  character from one of these sets can be matched using a script name. For  character from one of these sets can be matched using a script name. For
# Line 604  Ugaritic, Line 619  Ugaritic,
619  Vai,  Vai,
620  Yi.  Yi.
621  .P  .P
622  Each character has exactly one general category property, specified by a  Each character has exactly one Unicode general category property, specified by
623  two-letter abbreviation. For compatibility with Perl, negation can be specified  a two-letter abbreviation. For compatibility with Perl, negation can be
624  by including a circumflex between the opening brace and the property name. For  specified by including a circumflex between the opening brace and the property
625  example, \ep{^Lu} is the same as \eP{Lu}.  name. For example, \ep{^Lu} is the same as \eP{Lu}.
626  .P  .P
627  If only one letter is specified with \ep or \eP, it includes all the general  If only one letter is specified with \ep or \eP, it includes all the general
628  category properties that start with that letter. In this case, in the absence  category properties that start with that letter. In this case, in the absence
# Line 709  why the traditional escape sequences suc Line 724  why the traditional escape sequences suc
724  properties in PCRE.  properties in PCRE.
725  .  .
726  .  .
727    .\" HTML <a name="extraprops"></a>
728    .SS PCRE's additional properties
729    .rs
730    .sp
731    As well as the standard Unicode properties described in the previous
732    section, PCRE supports four more that make it possible to convert traditional
733    escape sequences such as \ew and \es and POSIX character classes to use Unicode
734    properties. These are:
735    .sp
736      Xan   Any alphanumeric character
737      Xps   Any POSIX space character
738      Xsp   Any Perl space character
739      Xwd   Any Perl "word" character
740    .sp
741    Xan matches characters that have either the L (letter) or the N (number)
742    property. Xps matches the characters tab, linefeed, vertical tab, formfeed, or
743    carriage return, and any other character that has the Z (separator) property.
744    Xsp is the same as Xps, except that vertical tab is excluded. Xwd matches the
745    same characters as Xan, plus underscore.
746    .
747    .
748  .\" HTML <a name="resetmatchstart"></a>  .\" HTML <a name="resetmatchstart"></a>
749  .SS "Resetting the match start"  .SS "Resetting the match start"
750  .rs  .rs
# Line 737  For example, when the pattern Line 773  For example, when the pattern
773    (foo)\eKbar    (foo)\eKbar
774  .sp  .sp
775  matches "foobar", the first substring is still set to "foo".  matches "foobar", the first substring is still set to "foo".
776    .P
777    Perl documents that the use of \eK within assertions is "not well defined". In
778    PCRE, \eK is acted upon when it occurs inside positive assertions, but is
779    ignored in negative assertions.
780  .  .
781  .  .
782  .\" HTML <a name="smallassertions"></a>  .\" HTML <a name="smallassertions"></a>
# Line 761  The backslashed assertions are: Line 801  The backslashed assertions are:
801    \ez     matches only at the end of the subject    \ez     matches only at the end of the subject
802    \eG     matches at the first matching position in the subject    \eG     matches at the first matching position in the subject
803  .sp  .sp
804  These assertions may not appear in character classes (but note that \eb has a  Inside a character class, \eb has a different meaning; it matches the backspace
805  different meaning, namely the backspace character, inside a character class).  character. If any other of these assertions appears in a character class, by
806    default it matches the corresponding literal character (for example, \eB
807    matches the letter B). However, if the PCRE_EXTRA option is set, an "invalid
808    escape sequence" error is generated instead.
809  .P  .P
810  A word boundary is a position in the subject string where the current character  A word boundary is a position in the subject string where the current character
811  and the previous character do not both match \ew or \eW (i.e. one matches  and the previous character do not both match \ew or \eW (i.e. one matches
# Line 854  end of the subject in both modes, and if Line 897  end of the subject in both modes, and if
897  \eA it is always anchored, whether or not PCRE_MULTILINE is set.  \eA it is always anchored, whether or not PCRE_MULTILINE is set.
898  .  .
899  .  .
900  .SH "FULL STOP (PERIOD, DOT)"  .\" HTML <a name="fullstopdot"></a>
901    .SH "FULL STOP (PERIOD, DOT) AND \eN"
902  .rs  .rs
903  .sp  .sp
904  Outside a character class, a dot in the pattern matches any one character in  Outside a character class, a dot in the pattern matches any one character in
# Line 876  to match it. Line 920  to match it.
920  The handling of dot is entirely independent of the handling of circumflex and  The handling of dot is entirely independent of the handling of circumflex and
921  dollar, the only relationship being that they both involve newlines. Dot has no  dollar, the only relationship being that they both involve newlines. Dot has no
922  special meaning in a character class.  special meaning in a character class.
923    .P
924    The escape sequence \eN always behaves as a dot does when PCRE_DOTALL is not
925    set. In other words, it matches any one character except one that signifies the
926    end of a line.
927  .  .
928  .  .
929  .SH "MATCHING A SINGLE BYTE"  .SH "MATCHING A SINGLE BYTE"
# Line 2314  description of the interface to the call Line 2362  description of the interface to the call
2362  documentation.  documentation.
2363  .  .
2364  .  .
2365    .\" HTML <a name="backtrackcontrol"></a>
2366  .SH "BACKTRACKING CONTROL"  .SH "BACKTRACKING CONTROL"
2367  .rs  .rs
2368  .sp  .sp
# Line 2335  it does not extend to the surrounding pa Line 2384  it does not extend to the surrounding pa
2384  processed as anchored at the point where they are tested.  processed as anchored at the point where they are tested.
2385  .P  .P
2386  The new verbs make use of what was previously invalid syntax: an opening  The new verbs make use of what was previously invalid syntax: an opening
2387  parenthesis followed by an asterisk. In Perl, they are generally of the form  parenthesis followed by an asterisk. They are generally of the form
2388  (*VERB:ARG) but PCRE does not support the use of arguments, so its general  (*VERB) or (*VERB:NAME). Some may take either form, with differing behaviour,
2389  form is just (*VERB). Any number of these verbs may occur in a pattern. There  depending on whether or not an argument is present. An name is a sequence of
2390  are two kinds:  letters, digits, and underscores. If the name is empty, that is, if the closing
2391    parenthesis immediately follows the colon, the effect is as if the colon were
2392    not there. Any number of these verbs may occur in a pattern.
2393    .P
2394    PCRE contains some optimizations that are used to speed up matching by running
2395    some checks at the start of each match attempt. For example, it may know the
2396    minimum length of matching subject, or that a particular character must be
2397    present. When one of these optimizations suppresses the running of a match, any
2398    included backtracking verbs will not, of course, be processed. You can suppress
2399    the start-of-match optimizations by setting the PCRE_NO_START_OPTIMIZE option
2400    when calling \fBpcre_exec()\fP.
2401    .
2402  .  .
2403  .SS "Verbs that act immediately"  .SS "Verbs that act immediately"
2404  .rs  .rs
2405  .sp  .sp
2406  The following verbs act as soon as they are encountered:  The following verbs act as soon as they are encountered. They may not be
2407    followed by a name.
2408  .sp  .sp
2409     (*ACCEPT)     (*ACCEPT)
2410  .sp  .sp
# Line 2370  callout feature, as for example in this Line 2431  callout feature, as for example in this
2431  A match with the string "aaaa" always fails, but the callout is taken before  A match with the string "aaaa" always fails, but the callout is taken before
2432  each backtrack happens (in this example, 10 times).  each backtrack happens (in this example, 10 times).
2433  .  .
2434    .
2435    .SS "Recording which path was taken"
2436    .rs
2437    .sp
2438    There is one verb whose main purpose is to track how a match was arrived at,
2439    though it also has a secondary use in conjunction with advancing the match
2440    starting point (see (*SKIP) below).
2441    .sp
2442      (*MARK:NAME) or (*:NAME)
2443    .sp
2444    A name is always required with this verb. There may be as many instances of
2445    (*MARK) as you like in a pattern, and their names do not have to be unique.
2446    .P
2447    When a match succeeds, the name of the last-encountered (*MARK) is passed back
2448    to the caller via the \fIpcre_extra\fP data structure, as described in the
2449    .\" HTML <a href="pcreapi.html#extradata">
2450    .\" </a>
2451    section on \fIpcre_extra\fP
2452    .\"
2453    in the
2454    .\" HREF
2455    \fBpcreapi\fP
2456    .\"
2457    documentation. No data is returned for a partial match. Here is an example of
2458    \fBpcretest\fP output, where the /K modifier requests the retrieval and
2459    outputting of (*MARK) data:
2460    .sp
2461      /X(*MARK:A)Y|X(*MARK:B)Z/K
2462      XY
2463       0: XY
2464      MK: A
2465      XZ
2466       0: XZ
2467      MK: B
2468    .sp
2469    The (*MARK) name is tagged with "MK:" in this output, and in this example it
2470    indicates which of the two alternatives matched. This is a more efficient way
2471    of obtaining this information than putting each alternative in its own
2472    capturing parentheses.
2473    .P
2474    A name may also be returned after a failed match if the final path through the
2475    pattern involves (*MARK). However, unless (*MARK) used in conjunction with
2476    (*COMMIT), this is unlikely to happen for an unanchored pattern because, as the
2477    starting point for matching is advanced, the final check is often with an empty
2478    string, causing a failure before (*MARK) is reached. For example:
2479    .sp
2480      /X(*MARK:A)Y|X(*MARK:B)Z/K
2481      XP
2482      No match
2483    .sp
2484    There are three potential starting points for this match (starting with X,
2485    starting with P, and with an empty string). If the pattern is anchored, the
2486    result is different:
2487    .sp
2488      /^X(*MARK:A)Y|^X(*MARK:B)Z/K
2489      XP
2490      No match, mark = B
2491    .sp
2492    PCRE's start-of-match optimizations can also interfere with this. For example,
2493    if, as a result of a call to \fBpcre_study()\fP, it knows the minimum
2494    subject length for a match, a shorter subject will not be scanned at all.
2495    .P
2496    Note that similar anomalies (though different in detail) exist in Perl, no
2497    doubt for the same reasons. The use of (*MARK) data after a failed match of an
2498    unanchored pattern is not recommended, unless (*COMMIT) is involved.
2499    .
2500    .
2501  .SS "Verbs that act after backtracking"  .SS "Verbs that act after backtracking"
2502  .rs  .rs
2503  .sp  .sp
2504  The following verbs do nothing when they are encountered. Matching continues  The following verbs do nothing when they are encountered. Matching continues
2505  with what follows, but if there is no subsequent match, a failure is forced.  with what follows, but if there is no subsequent match, causing a backtrack to
2506  The verbs differ in exactly what kind of failure occurs.  the verb, a failure is forced. That is, backtracking cannot pass to the left of
2507    the verb. However, when one of these verbs appears inside an atomic group, its
2508    effect is confined to that group, because once the group has been matched,
2509    there is never any backtracking into it. In this situation, backtracking can
2510    "jump back" to the left of the entire atomic group. (Remember also, as stated
2511    above, that this localization also applies in subroutine calls and assertions.)
2512    .P
2513    These verbs differ in exactly what kind of failure occurs when backtracking
2514    reaches them.
2515  .sp  .sp
2516    (*COMMIT)    (*COMMIT)
2517  .sp  .sp
2518  This verb causes the whole match to fail outright if the rest of the pattern  This verb, which may not be followed by a name, causes the whole match to fail
2519  does not match. Even if the pattern is unanchored, no further attempts to find  outright if the rest of the pattern does not match. Even if the pattern is
2520  a match by advancing the starting point take place. Once (*COMMIT) has been  unanchored, no further attempts to find a match by advancing the starting point
2521  passed, \fBpcre_exec()\fP is committed to finding a match at the current  take place. Once (*COMMIT) has been passed, \fBpcre_exec()\fP is committed to
2522  starting point, or not at all. For example:  finding a match at the current starting point, or not at all. For example:
2523  .sp  .sp
2524    a+(*COMMIT)b    a+(*COMMIT)b
2525  .sp  .sp
2526  This matches "xxaab" but not "aacaab". It can be thought of as a kind of  This matches "xxaab" but not "aacaab". It can be thought of as a kind of
2527  dynamic anchor, or "I've started, so I must finish."  dynamic anchor, or "I've started, so I must finish." The name of the most
2528  .sp  recently passed (*MARK) in the path is passed back when (*COMMIT) forces a
2529    (*PRUNE)  match failure.
2530  .sp  .P
2531  This verb causes the match to fail at the current position if the rest of the  Note that (*COMMIT) at the start of a pattern is not the same as an anchor,
2532  pattern does not match. If the pattern is unanchored, the normal "bumpalong"  unless PCRE's start-of-match optimizations are turned off, as shown in this
2533  advance to the next starting character then happens. Backtracking can occur as  \fBpcretest\fP example:
2534  usual to the left of (*PRUNE), or when matching to the right of (*PRUNE), but  .sp
2535  if there is no match to the right, backtracking cannot cross (*PRUNE).    /(*COMMIT)abc/
2536  In simple cases, the use of (*PRUNE) is just an alternative to an atomic    xyzabc
2537  group or possessive quantifier, but there are some uses of (*PRUNE) that cannot     0: abc
2538  be expressed in any other way.    xyzabc\eY
2539      No match
2540    .sp
2541    PCRE knows that any match must start with "a", so the optimization skips along
2542    the subject to "a" before running the first match attempt, which succeeds. When
2543    the optimization is disabled by the \eY escape in the second subject, the match
2544    starts at "x" and so the (*COMMIT) causes it to fail without trying any other
2545    starting points.
2546    .sp
2547      (*PRUNE) or (*PRUNE:NAME)
2548    .sp
2549    This verb causes the match to fail at the current starting position in the
2550    subject if the rest of the pattern does not match. If the pattern is
2551    unanchored, the normal "bumpalong" advance to the next starting character then
2552    happens. Backtracking can occur as usual to the left of (*PRUNE), before it is
2553    reached, or when matching to the right of (*PRUNE), but if there is no match to
2554    the right, backtracking cannot cross (*PRUNE). In simple cases, the use of
2555    (*PRUNE) is just an alternative to an atomic group or possessive quantifier,
2556    but there are some uses of (*PRUNE) that cannot be expressed in any other way.
2557    The behaviour of (*PRUNE:NAME) is the same as (*MARK:NAME)(*PRUNE) when the
2558    match fails completely; the name is passed back if this is the final attempt.
2559    (*PRUNE:NAME) does not pass back a name if the match succeeds. In an anchored
2560    pattern (*PRUNE) has the same effect as (*COMMIT).
2561  .sp  .sp
2562    (*SKIP)    (*SKIP)
2563  .sp  .sp
2564  This verb is like (*PRUNE), except that if the pattern is unanchored, the  This verb, when given without a name, is like (*PRUNE), except that if the
2565  "bumpalong" advance is not to the next character, but to the position in the  pattern is unanchored, the "bumpalong" advance is not to the next character,
2566  subject where (*SKIP) was encountered. (*SKIP) signifies that whatever text  but to the position in the subject where (*SKIP) was encountered. (*SKIP)
2567  was matched leading up to it cannot be part of a successful match. Consider:  signifies that whatever text was matched leading up to it cannot be part of a
2568    successful match. Consider:
2569  .sp  .sp
2570    a+(*SKIP)b    a+(*SKIP)b
2571  .sp  .sp
# Line 2417  effect as this example; although it woul Line 2576  effect as this example; although it woul
2576  first match attempt, the second attempt would start at the second character  first match attempt, the second attempt would start at the second character
2577  instead of skipping on to "c".  instead of skipping on to "c".
2578  .sp  .sp
2579    (*THEN)    (*SKIP:NAME)
2580    .sp
2581    When (*SKIP) has an associated name, its behaviour is modified. If the
2582    following pattern fails to match, the previous path through the pattern is
2583    searched for the most recent (*MARK) that has the same name. If one is found,
2584    the "bumpalong" advance is to the subject position that corresponds to that
2585    (*MARK) instead of to where (*SKIP) was encountered. If no (*MARK) with a
2586    matching name is found, normal "bumpalong" of one character happens (the
2587    (*SKIP) is ignored).
2588    .sp
2589      (*THEN) or (*THEN:NAME)
2590  .sp  .sp
2591  This verb causes a skip to the next alternation if the rest of the pattern does  This verb causes a skip to the next alternation if the rest of the pattern does
2592  not match. That is, it cancels pending backtracking, but only within the  not match. That is, it cancels pending backtracking, but only within the
# Line 2428  for a pattern-based if-then-else block: Line 2597  for a pattern-based if-then-else block:
2597  .sp  .sp
2598  If the COND1 pattern matches, FOO is tried (and possibly further items after  If the COND1 pattern matches, FOO is tried (and possibly further items after
2599  the end of the group if FOO succeeds); on failure the matcher skips to the  the end of the group if FOO succeeds); on failure the matcher skips to the
2600  second alternative and tries COND2, without backtracking into COND1. If (*THEN)  second alternative and tries COND2, without backtracking into COND1. The
2601  is used outside of any alternation, it acts exactly like (*PRUNE).  behaviour of (*THEN:NAME) is exactly the same as (*MARK:NAME)(*THEN) if the
2602    overall match fails. If (*THEN) is not directly inside an alternation, it acts
2603    like (*PRUNE).
2604  .  .
2605  .  .
2606  .SH "SEE ALSO"  .SH "SEE ALSO"
# Line 2453  Cambridge CB2 3QH, England. Line 2624  Cambridge CB2 3QH, England.
2624  .rs  .rs
2625  .sp  .sp
2626  .nf  .nf
2627  Last updated: 01 March 2010  Last updated: 05 May 2010
2628  Copyright (c) 1997-2010 University of Cambridge.  Copyright (c) 1997-2010 University of Cambridge.
2629  .fi  .fi

Legend:
Removed from v.491  
changed lines
  Added in v.517

  ViewVC Help
Powered by ViewVC 1.1.5