/[pcre]/code/trunk/doc/pcre.3

Diff of /code/trunk/doc/pcre.3

Parent Directory | Revision Log | View Patch Patch

-revision 41 by nigel,
Sat Feb 24 21:39:17 2007 UTC
+revision 51 by nigel,
Sat Feb 24 21:39:37 2007 UTC
 Line 44 
 pcre - Perl-compatible regular expressio
  .B int *\fIovector\fR, int \fIstringcount\fR, "const char ***\fIlistptr\fR);"
  .PP
  .br
+ .B void pcre_free_substring(const char *\fIstringptr\fR);
+ .PP
+ .br
+ .B void pcre_free_substring_list(const char **\fIstringptr\fR);
+ .PP
+ .br
  .B const unsigned char *pcre_maketables(void);
  .PP
  .br
+ .B int pcre_fullinfo(const pcre *\fIcode\fR, "const pcre_extra *\fIextra\fR,"
+ .ti +5n
+ .B int \fIwhat\fR, void *\fIwhere\fR);
+ .PP
+ .br
  .B int pcre_info(const pcre *\fIcode\fR, int *\fIoptptr\fR, int
  .B *\fIfirstcharptr\fR);
  .PP
-Line 64 
 pcre - Perl-compatible regular expressio
+Line 75 
 pcre - Perl-compatible regular expressio
  .SH DESCRIPTION
  The PCRE library is a set of functions that implement regular expression
  pattern matching using the same syntax and semantics as Perl 5, with just a few
- differences (see below). The current implementation corresponds to Perl 5.005.
+ differences (see below). The current implementation corresponds to Perl 5.005,
+ with some additional features from later versions. This includes some
+ experimental, incomplete support for UTF-8 encoded strings. Details of exactly
+ what is and what is not supported are given below.
  PCRE has its own native API, which is described in this document. There is also
- a set of wrapper functions that correspond to the POSIX API. These are
+ a set of wrapper functions that correspond to the POSIX regular expression API.
- described in the \fBpcreposix\fR documentation.
+ These are described in the \fBpcreposix\fR documentation.
  The native API function prototypes are defined in the header file \fBpcre.h\fR,
  and on Unix systems the library itself is called \fBlibpcre.a\fR, so can be
  accessed by adding \fB-lpcre\fR to the command for linking an application which
- calls it.
+ calls it. The header file defines the macros PCRE_MAJOR and PCRE_MINOR to
+ contain the major and minor release numbers for the library. Applications can
+ use these to include support for different releases.
  The functions \fBpcre_compile()\fR, \fBpcre_study()\fR, and \fBpcre_exec()\fR
- are used for compiling and matching regular expressions, while
+ are used for compiling and matching regular expressions.
- \fBpcre_copy_substring()\fR, \fBpcre_get_substring()\fR, and
+ The functions \fBpcre_copy_substring()\fR, \fBpcre_get_substring()\fR, and
  \fBpcre_get_substring_list()\fR are convenience functions for extracting
- captured substrings from a matched subject string. The function
+ captured substrings from a matched subject string; \fBpcre_free_substring()\fR
- \fBpcre_maketables()\fR is used (optionally) to build a set of character tables
+ and \fBpcre_free_substring_list()\fR are also provided, to free the memory used
- in the current locale for passing to \fBpcre_compile()\fR.
+ for extracted strings.
- The function \fBpcre_info()\fR is used to find out information about a compiled
+ The function \fBpcre_maketables()\fR is used (optionally) to build a set of
- pattern, while the function \fBpcre_version()\fR returns a pointer to a string
+ character tables in the current locale for passing to \fBpcre_compile()\fR.
- containing the version of PCRE and its date of release.
+ The function \fBpcre_fullinfo()\fR is used to find out information about a
+ compiled pattern; \fBpcre_info()\fR is an obsolete version which returns only
+ some of the available information, but is retained for backwards compatibility.
+ The function \fBpcre_version()\fR returns a pointer to a string containing the
+ version of PCRE and its date of release.
  The global variables \fBpcre_malloc\fR and \fBpcre_free\fR initially contain
  the entry points of the standard \fBmalloc()\fR and \fBfree()\fR functions
-Line 182 
 sequence (?( which introduces a conditio
+Line 204 
 sequence (?( which introduces a conditio
    PCRE_EXTRA
- This option turns on additional functionality of PCRE that is incompatible with
+ This option was invented in order to turn on additional functionality of PCRE
- Perl. Any backslash in a pattern that is followed by a letter that has no
+ that is incompatible with Perl, but it is currently of very little use. When
+ set, any backslash in a pattern that is followed by a letter that has no
  special meaning causes an error, thus reserving these combinations for future
  expansion. By default, as in Perl, a backslash followed by a letter with no
  special meaning is treated as a literal. There are at present no other features
- controlled by this option.
+ controlled by this option. It can also be set by a (?X) option setting within a
+ pattern.
    PCRE_MULTILINE
-Line 211 
 This option inverts the "greediness" of
+Line 235 
 This option inverts the "greediness" of
  greedy by default, but become greedy if followed by "?". It is not compatible
  with Perl. It can also be set by a (?U) option setting within the pattern.
+   PCRE_UTF8
+ This option causes PCRE to regard both the pattern and the subject as strings
+ of UTF-8 characters instead of just byte strings. However, it is available only
+ if PCRE has been built to include UTF-8 support. If not, the use of this option
+ provokes an error. Support for UTF-8 is new, experimental, and incomplete.
+ Details of exactly what it entails are given below.
  .SH STUDYING A PATTERN
  When a pattern is going to be used several times, it is worth spending more
-Line 261 
 memory containing the tables remains ava
+Line 293 
 memory containing the tables remains ava
  .SH INFORMATION ABOUT A PATTERN
- The \fBpcre_info()\fR function returns information about a compiled pattern.
+ The \fBpcre_fullinfo()\fR function returns information about a compiled
- Its yield is the number of capturing subpatterns, or one of the following
+ pattern. It replaces the obsolete \fBpcre_info()\fR function, which is
- negative numbers:
+ nevertheless retained for backwards compability (and is documented below).
+ The first argument for \fBpcre_fullinfo()\fR is a pointer to the compiled
+ pattern. The second argument is the result of \fBpcre_study()\fR, or NULL if
+ the pattern was not studied. The third argument specifies which piece of
+ information is required, while the fourth argument is a pointer to a variable
+ to receive the data. The yield of the function is zero for success, or one of
+ the following negative numbers:
    PCRE_ERROR_NULL       the argument \fIcode\fR was NULL
+                         the argument \fIwhere\fR was NULL
    PCRE_ERROR_BADMAGIC   the "magic number" was not found
+   PCRE_ERROR_BADOPTION  the value of \fIwhat\fR was invalid
- If the \fIoptptr\fR argument is not NULL, a copy of the options with which the
+ The possible values for the third argument are defined in \fBpcre.h\fR, and are
- pattern was compiled is placed in the integer it points to. These option bits
+ as follows:
+   PCRE_INFO_OPTIONS
+ Return a copy of the options with which the pattern was compiled. The fourth
+ argument should point to au \fBunsigned long int\fR variable. These option bits
  are those specified in the call to \fBpcre_compile()\fR, modified by any
  top-level option settings within the pattern itself, and with the PCRE_ANCHORED
- bit set if the form of the pattern implies that it can match only at the start
+ bit forcibly set if the form of the pattern implies that it can match only at
- of a subject string.
+ the start of a subject string.
- If the pattern is not anchored and the \fIfirstcharptr\fR argument is not NULL,
+   PCRE_INFO_SIZE
- it is used to pass back information about the first character of any matched
- string. If there is a fixed first character, e.g. from a pattern such as
+ Return the size of the compiled pattern, that is, the value that was passed as
- (cat|cow|coyote), then it is returned in the integer pointed to by
+ the argument to \fBpcre_malloc()\fR when PCRE was getting memory in which to
- \fIfirstcharptr\fR. Otherwise, if either
+ place the compiled data. The fourth argument should point to a \fBsize_t\fR
+ variable.
+   PCRE_INFO_CAPTURECOUNT
+ Return the number of capturing subpatterns in the pattern. The fourth argument
+ should point to an \fbint\fR variable.
+   PCRE_INFO_BACKREFMAX
+ Return the number of the highest back reference in the pattern. The fourth
+ argument should point to an \fBint\fR variable. Zero is returned if there are
+ no back references.
+   PCRE_INFO_FIRSTCHAR
+ Return information about the first character of any matched string, for a
+ non-anchored pattern. If there is a fixed first character, e.g. from a pattern
+ such as (cat|cow|coyote), it is returned in the integer pointed to by
+ \fIwhere\fR. Otherwise, if either
  (a) the pattern was compiled with the PCRE_MULTILINE option, and every branch
  starts with "^", or
-Line 287 
 starts with "^", or
+Line 352 
 starts with "^", or
  (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not set
  (if it were set, the pattern would be anchored),
- then -1 is returned, indicating that the pattern matches only at the
+ -1 is returned, indicating that the pattern matches only at the start of a
- start of a subject string or after any "\\n" within the string. Otherwise -2 is
+ subject string or after any "\\n" within the string. Otherwise -2 is returned.
- returned.
+ For anchored patterns, -2 is returned.
+   PCRE_INFO_FIRSTTABLE
+ If the pattern was studied, and this resulted in the construction of a 256-bit
+ table indicating a fixed set of characters for the first character in any
+ matching string, a pointer to the table is returned. Otherwise NULL is
+ returned. The fourth argument should point to an \fBunsigned char *\fR
+ variable.
+   PCRE_INFO_LASTLITERAL
+ For a non-anchored pattern, return the value of the rightmost literal character
+ which must exist in any matched string, other than at its start. The fourth
+ argument should point to an \fBint\fR variable. If there is no such character,
+ or if the pattern is anchored, -1 is returned. For example, for the pattern
+ /a\\d+z\\d+/ the returned value is 'z'.
+ The \fBpcre_info()\fR function is now obsolete because its interface is too
+ restrictive to return all the available data about a compiled pattern. New
+ programs should use \fBpcre_fullinfo()\fR instead. The yield of
+ \fBpcre_info()\fR is the number of capturing subpatterns, or one of the
+ following negative numbers:
+   PCRE_ERROR_NULL       the argument \fIcode\fR was NULL
+   PCRE_ERROR_BADMAGIC   the "magic number" was not found
+ If the \fIoptptr\fR argument is not NULL, a copy of the options with which the
+ pattern was compiled is placed in the integer it points to (see
+ PCRE_INFO_OPTIONS above).
+ If the pattern is not anchored and the \fIfirstcharptr\fR argument is not NULL,
+ it is used to pass back information about the first character of any matched
+ string (see PCRE_INFO_FIRSTCHAR above).
  .SH MATCHING A PATTERN
-Line 472 
 is a pointer to the vector of integer of
+Line 570 
 is a pointer to the vector of integer of
  were captured by the match, including the substring that matched the entire
  regular expression. This is the value returned by \fBpcre_exec\fR if it
  is greater than zero. If \fBpcre_exec()\fR returned zero, indicating that it
- ran out of space in \fIovector\fR, then the value passed as
+ ran out of space in \fIovector\fR, the value passed as \fIstringcount\fR should
- \fIstringcount\fR should be the size of the vector divided by three.
+ be the size of the vector divided by three.
  The functions \fBpcre_copy_substring()\fR and \fBpcre_get_substring()\fR
  extract a single substring, whose number is given as \fIstringnumber\fR. A
  value of zero extracts the substring that matched the entire pattern, while
  higher values extract the captured substrings. For \fBpcre_copy_substring()\fR,
  the string is placed in \fIbuffer\fR, whose length is given by
- \fIbuffersize\fR, while for \fBpcre_get_substring()\fR a new block of store is
+ \fIbuffersize\fR, while for \fBpcre_get_substring()\fR a new block of memory is
  obtained via \fBpcre_malloc\fR, and its address is returned via
  \fIstringptr\fR. The yield of the function is the length of the string, not
  including the terminating zero, or one of
-Line 512 
 string. This can be distinguished from a
+Line 610 
 string. This can be distinguished from a
  inspecting the appropriate offset in \fIovector\fR, which is negative for unset
  substrings.
+ The two convenience functions \fBpcre_free_substring()\fR and
+ \fBpcre_free_substring_list()\fR can be used to free the memory returned by
+ a previous call of \fBpcre_get_substring()\fR or
+ \fBpcre_get_substring_list()\fR, respectively. They do nothing more than call
+ the function pointed to by \fBpcre_free\fR, which of course could be called
+ directly from a C program. However, PCRE is used in some situations where it is
+ linked via a special interface to another programming language which cannot use
+ \fBpcre_free\fR directly; it is for these cases that the functions are
+ provided.
  .SH LIMITATIONS
-Line 564 
 are not part of its pattern matching eng
+Line 671 
 are not part of its pattern matching eng
 . The Perl \\G assertion is not supported as it is not relevant to single
  pattern matches.
-. Fairly obviously, PCRE does not support the (?{code}) construction.
+. Fairly obviously, PCRE does not support the (?{code}) and (?p{code})
+ constructions. However, there is some experimental support for recursive
+ patterns using the non-Perl item (?R).
 . There are at the time of writing some oddities in Perl 5.005_02 concerned
  with the settings of captured strings when part of a pattern is repeated. For
  example, matching "aba" against the pattern /^(a(b)?)+$/ sets $2 to the value
  "b", but matching "aabbaa" against /^(aa(bb)?)+$/ leaves $2 unset. However, if
- the pattern is changed to /^(aa(b(b))?)+$/ then $2 (and $3) get set.
+ the pattern is changed to /^(aa(b(b))?)+$/ then $2 (and $3) are set.
  In Perl 5.004 $2 is set in both cases, and that is also true of PCRE. If in the
  future Perl changes to a consistent state that is different, PCRE may change to
-Line 602 
 of the subject.
+Line 711 
 of the subject.
  (f) The PCRE_NOTBOL, PCRE_NOTEOL, and PCRE_NOTEMPTY options for
  \fBpcre_exec()\fR have no Perl equivalents.
+ (g) The (?R) construct allows for recursive pattern matching (Perl 5.6 can do
+ this using the (?p{code}) construct, which PCRE cannot of course support.)
  .SH REGULAR EXPRESSION DETAILS
  The syntax and semantics of the regular expressions supported by PCRE are
  described below. Regular expressions are also described in the Perl
  documentation and in a number of other books, some of which have copious
  examples. Jeffrey Friedl's "Mastering Regular Expressions", published by
- O'Reilly (ISBN 1-56592-257-3), covers them in great detail. The description
+ O'Reilly (ISBN 1-56592-257), covers them in great detail.
- here is intended as reference documentation.
+ The description here is intended as reference documentation. The basic
+ operation of PCRE is on strings of bytes. However, there is the beginnings of
+ some support for UTF-8 character strings. To use this support you must
+ configure PCRE to include it, and then call \fBpcre_compile()\fR with the
+ PCRE_UTF8 option. How this affects the pattern matching is described in the
+ final section of this document.
  A regular expression is a pattern that is matched against a subject string from
  left to right. Most characters stand for themselves in a pattern, and match the
-Line 837 
 end of the subject in both modes, and if
+Line 955 
 end of the subject in both modes, and if
  .SH FULL STOP (PERIOD, DOT)
  Outside a character class, a dot in the pattern matches any one character in
  the subject, including a non-printing character, but not (by default) newline.
- If the PCRE_DOTALL option is set, then dots match newlines as well. The
+ If the PCRE_DOTALL option is set, dots match newlines as well. The handling of
- handling of dot is entirely independent of the handling of circumflex and
+ dot is entirely independent of the handling of circumflex and dollar, the only
- dollar, the only relationship being that they both involve newline characters.
+ relationship being that they both involve newline characters. Dot has no
- Dot has no special meaning in a character class.
+ special meaning in a character class.
  .SH SQUARE BRACKETS
-Line 906 
 terminating ] are non-special in charact
+Line 1024 
 terminating ] are non-special in charact
  are escaped.
+ .SH POSIX CHARACTER CLASSES
+ Perl 5.6 (not yet released at the time of writing) is going to support the
+ POSIX notation for character classes, which uses names enclosed by [: and :]
+ within the enclosing square brackets. PCRE supports this notation. For example,
+   [01[:alpha:]%]
+ matches "0", "1", any alphabetic character, or "%". The supported class names
+ are
+   alnum    letters and digits
+   alpha    letters
+   ascii    character codes 0 - 127
+   cntrl    control characters
+   digit    decimal digits (same as \\d)
+   graph    printing characters, excluding space
+   lower    lower case letters
+   print    printing characters, including space
+   punct    printing characters, excluding letters and digits
+   space    white space (same as \\s)
+   upper    upper case letters
+   word     "word" characters (same as \\w)
+   xdigit   hexadecimal digits
+ The names "ascii" and "word" are Perl extensions. Another Perl extension is
+ negation, which is indicated by a ^ character after the colon. For example,
+   [12[:^digit:]]
+ matches "1", "2", or any non-digit. PCRE (and Perl) also recogize the POSIX
+ syntax [.ch.] and [=ch=] where "ch" is a "collating element", but these are not
+ supported, and an error is given if they are encountered.
  .SH VERTICAL BAR
  Vertical bar characters are used to separate alternative patterns. For example,
  the pattern
-Line 1093 
 to the string
+Line 1245 
 to the string
    /* first command */  not comment  /* second comment */
- fails, because it matches the entire string due to the greediness of the .*
+ fails, because it matches the entire string owing to the greediness of the .*
  item.
- However, if a quantifier is followed by a question mark, then it ceases to be
+ However, if a quantifier is followed by a question mark, it ceases to be
  greedy, and instead matches the minimum number of times possible, so the
  pattern
-Line 1112 
 own right. Because it has two uses, it c
+Line 1264 
 own right. Because it has two uses, it c
  which matches one digit by preference, but can match two if that is the only
  way the rest of the pattern matches.
- If the PCRE_UNGREEDY option is set (an option which is not available in Perl)
+ If the PCRE_UNGREEDY option is set (an option which is not available in Perl),
- then the quantifiers are not greedy by default, but individual ones can be made
+ the quantifiers are not greedy by default, but individual ones can be made
  greedy by following them with a question mark. In other words, it inverts the
  default behaviour.
-Line 1122 
 is greater than 1 or with a limited maxi
+Line 1274 
 is greater than 1 or with a limited maxi
  compiled pattern, in proportion to the size of the minimum or maximum.
  If a pattern starts with .* or .{0,} and the PCRE_DOTALL option (equivalent
- to Perl's /s) is set, thus allowing the . to match newlines, then the pattern
+ to Perl's /s) is set, thus allowing the . to match newlines, the pattern is
- is implicitly anchored, because whatever follows will be tried against every
+ implicitly anchored, because whatever follows will be tried against every
  character position in the subject string, so there is no point in retrying the
  overall match at any position after the first. PCRE treats such a pattern as
  though it were preceded by \\A. In cases where it is known that the subject
-Line 1167 
 itself. So the pattern
+Line 1319 
 itself. So the pattern
  matches "sense and sensibility" and "response and responsibility", but not
  "sense and responsibility". If caseful matching is in force at the time of the
- back reference, then the case of letters is relevant. For example,
+ back reference, the case of letters is relevant. For example,
    ((?i)rah)\\s+\\1
-Line 1175 
 matches "rah rah" and "RAH RAH", but not
+Line 1327 
 matches "rah rah" and "RAH RAH", but not
  capturing subpattern is matched caselessly.
  There may be more than one back reference to the same subpattern. If a
- subpattern has not actually been used in a particular match, then any back
+ subpattern has not actually been used in a particular match, any back
  references to it always fail. For example, the pattern
    (a|(bc))\\2
-Line 1183 
 references to it always fail. For exampl
+Line 1335 
 references to it always fail. For exampl
  always fails if it starts to match "a" rather than "bc". Because there may be
  up to 99 back references, all digits following the backslash are taken
  as part of a potential back reference number. If the pattern continues with a
- digit character, then some delimiter must be used to terminate the back
+ digit character, some delimiter must be used to terminate the back reference.
- reference. If the PCRE_EXTENDED option is set, this can be whitespace.
+ If the PCRE_EXTENDED option is set, this can be whitespace. Otherwise an empty
- Otherwise an empty comment can be used.
+ comment can be used.
  A back reference that occurs inside the parentheses to which it refers fails
  when the subpattern is first used, so, for example, (a\\1) never matches.
-Line 1194 
 example, the pattern
+Line 1346 
 example, the pattern
    (a|b\\1)+
- matches any number of "a"s and also "aba", "ababaa" etc. At each iteration of
+ matches any number of "a"s and also "aba", "ababbaa" etc. At each iteration of
  the subpattern, the back reference matches the character string corresponding
  to the previous iteration. In order for this to work, the pattern must be such
  that the first iteration does not need to match the back reference. This can be
-Line 1273 
 Several assertions (of any sort) may occ
+Line 1425 
 Several assertions (of any sort) may occ
  matches "foo" preceded by three digits that are not "999". Notice that each of
  the assertions is applied independently at the same point in the subject
  string. First there is a check that the previous three characters are all
- digits, then there is a check that the same three characters are not "999".
+ digits, and then there is a check that the same three characters are not "999".
  This pattern does \fInot\fR match "foo" preceded by six characters, the first
  of which are digits and the last three of which are not "999". For example, it
  doesn't match "123abcfoo". A pattern to do that is
-Line 1352 
 pattern such as
+Line 1504 
 pattern such as
    abcd$
- when applied to a long string which does not match it. Because matching
+ when applied to a long string which does not match. Because matching proceeds
- proceeds from left to right, PCRE will look for each "a" in the subject and
+ from left to right, PCRE will look for each "a" in the subject and then see if
- then see if what follows matches the rest of the pattern. If the pattern is
+ what follows matches the rest of the pattern. If the pattern is specified as
- specified as
    ^.*abcd$
- then the initial .* matches the entire string at first, but when this fails, it
+ the initial .* matches the entire string at first, but when this fails (because
- backtracks to match all but the last character, then all but the last two
+ there is no following "a"), it backtracks to match all but the last character,
- characters, and so on. Once again the search for "a" covers the entire string,
+ then all but the last two characters, and so on. Once again the search for "a"
- from right to left, so we are no better off. However, if the pattern is written
+ covers the entire string, from right to left, so we are no better off. However,
- as
+ if the pattern is written as
    ^(?>.*)(?<=abcd)
- then there can be no backtracking for the .* item; it can match only the entire
+ there can be no backtracking for the .* item; it can match only the entire
  string. The subsequent lookbehind assertion does a single test on the last four
  characters. If it fails, the match fails immediately. For long strings, this
  approach makes a significant difference to the processing time.
+ When a pattern contains an unlimited repeat inside a subpattern that can itself
+ be repeated an unlimited number of times, the use of a once-only subpattern is
+ the only way to avoid some failing matches taking a very long time indeed.
+ The pattern
+   (\\D+|<\\d+>)*[!?]
+ matches an unlimited number of substrings that either consist of non-digits, or
+ digits enclosed in <>, followed by either ! or ?. When it matches, it runs
+ quickly. However, if it is applied to
+   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
+ it takes a long time before reporting failure. This is because the string can
+ be divided between the two repeats in a large number of ways, and all have to
+ be tried. (The example used [!?] rather than a single character at the end,
+ because both PCRE and Perl have an optimization that allows for fast failure
+ when a single character is used. They remember the last single character that
+ is required for a match, and fail early if it is not present in the string.)
+ If the pattern is changed to
+   ((?>\\D+)|<\\d+>)*[!?]
+ sequences of non-digits cannot be broken, and failure happens quickly.
  .SH CONDITIONAL SUBPATTERNS
  It is possible to cause the matching process to obey a subpattern
-Line 1387 
 no-pattern (if present) is used. If ther
+Line 1563 
 no-pattern (if present) is used. If ther
  subpattern, a compile-time error occurs.
  There are two kinds of condition. If the text between the parentheses consists
- of a sequence of digits, then the condition is satisfied if the capturing
+ of a sequence of digits, the condition is satisfied if the capturing subpattern
- subpattern of that number has previously matched. Consider the following
+ of that number has previously matched. The number must be greater than zero.
- pattern, which contains non-significant white space to make it more readable
+ Consider the following pattern, which contains non-significant white space to
- (assume the PCRE_EXTENDED option) and to divide it into three parts for ease
+ make it more readable (assume the PCRE_EXTENDED option) and to divide it into
- of discussion:
+ three parts for ease of discussion:
    ( \\( )?    [^()]+    (?(1) \\) )
-Line 1431 
 character class introduces a comment tha
+Line 1607 
 character class introduces a comment tha
  character in the pattern.
+ .SH RECURSIVE PATTERNS
+ Consider the problem of matching a string in parentheses, allowing for
+ unlimited nested parentheses. Without the use of recursion, the best that can
+ be done is to use a pattern that matches up to some fixed depth of nesting. It
+ is not possible to handle an arbitrary nesting depth. Perl 5.6 has provided an
+ experimental facility that allows regular expressions to recurse (amongst other
+ things). It does this by interpolating Perl code in the expression at run time,
+ and the code can refer to the expression itself. A Perl pattern to solve the
+ parentheses problem can be created like this:
+   $re = qr{\\( (?: (?>[^()]+) | (?p{$re}) )* \\)}x;
+ The (?p{...}) item interpolates Perl code at run time, and in this case refers
+ recursively to the pattern in which it appears. Obviously, PCRE cannot support
+ the interpolation of Perl code. Instead, the special item (?R) is provided for
+ the specific case of recursion. This PCRE pattern solves the parentheses
+ problem (assume the PCRE_EXTENDED option is set so that white space is
+ ignored):
+   \\( ( (?>[^()]+) | (?R) )* \\)
+ First it matches an opening parenthesis. Then it matches any number of
+ substrings which can either be a sequence of non-parentheses, or a recursive
+ match of the pattern itself (i.e. a correctly parenthesized substring). Finally
+ there is a closing parenthesis.
+ This particular example pattern contains nested unlimited repeats, and so the
+ use of a once-only subpattern for matching strings of non-parentheses is
+ important when applying the pattern to strings that do not match. For example,
+ when it is applied to
+   (aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa()
+ it yields "no match" quickly. However, if a once-only subpattern is not used,
+ the match runs for a very long time indeed because there are so many different
+ ways the + and * repeats can carve up the subject, and all have to be tested
+ before failure can be reported.
+ The values set for any capturing subpatterns are those from the outermost level
+ of the recursion at which the subpattern value is set. If the pattern above is
+ matched against
+   (ab(cd)ef)
+ the value for the capturing parentheses is "ef", which is the last value taken
+ on at the top level. If additional parentheses are added, giving
+   \\( ( ( (?>[^()]+) | (?R) )* ) \\)
+      ^                        ^
+      ^                        ^
+ the string they capture is "ab(cd)ef", the contents of the top level
+ parentheses. If there are more than 15 capturing parentheses in a pattern, PCRE
+ has to obtain extra memory to store data during a recursion, which it does by
+ using \fBpcre_malloc\fR, freeing it via \fBpcre_free\fR afterwards. If no
+ memory can be obtained, it saves data for the first 15 capturing parentheses
+ only, as there is no way to give an out-of-memory error from within a
+ recursion.
  .SH PERFORMANCE
  Certain items that may appear in patterns are more efficient than others. It is
  more efficient to use a character class like [aeiou] than a set of alternatives
-Line 1486 
 with the pattern above. The former gives
+Line 1721 
 with the pattern above. The former gives
  applied to a whole line of "a" characters, whereas the latter takes an
  appreciable time with strings longer than about 20 characters.
+ .SH UTF-8 SUPPORT
+ Starting at release 3.3, PCRE has some support for character strings encoded
+ in the UTF-8 format. This is incomplete, and is regarded as experimental. In
+ order to use it, you must configure PCRE to include UTF-8 support in the code,
+ and, in addition, you must call \fBpcre_compile()\fR with the PCRE_UTF8 option
+ flag. When you do this, both the pattern and any subject strings that are
+ matched against it are treated as UTF-8 strings instead of just strings of
+ bytes, but only in the cases that are mentioned below.
+ If you compile PCRE with UTF-8 support, but do not use it at run time, the
+ library will be a bit bigger, but the additional run time overhead is limited
+ to testing the PCRE_UTF8 flag in several places, so should not be very large.
+ PCRE assumes that the strings it is given contain valid UTF-8 codes. It does
+ not diagnose invalid UTF-8 strings. If you pass invalid UTF-8 strings to PCRE,
+ the results are undefined.
+ Running with PCRE_UTF8 set causes these changes in the way PCRE works:
+. In a pattern, the escape sequence \\x{...}, where the contents of the braces
+ is a string of hexadecimal digits, is interpreted as a UTF-8 character whose
+ code number is the given hexadecimal number, for example: \\x{1234}. This
+ inserts from one to six literal bytes into the pattern, using the UTF-8
+ encoding. If a non-hexadecimal digit appears between the braces, the item is
+ not recognized.
+. The original hexadecimal escape sequence, \\xhh, generates a two-byte UTF-8
+ character if its value is greater than 127.
+. Repeat quantifiers are NOT correctly handled if they follow a multibyte
+ character. For example, \\x{100}* and \\xc3+ do not work. If you want to
+ repeat such characters, you must enclose them in non-capturing parentheses,
+ for example (?:\\x{100}), at present.
+. The dot metacharacter matches one UTF-8 character instead of a single byte.
+. Unlike literal UTF-8 characters, the dot metacharacter followed by a
+ repeat quantifier does operate correctly on UTF-8 characters instead of
+ single bytes.
+. Although the \\x{...} escape is permitted in a character class, characters
+ whose values are greater than 255 cannot be included in a class.
+. A class is matched against a UTF-8 character instead of just a single byte,
+ but it can match only characters whose values are less than 256. Characters
+ with greater values always fail to match a class.
+. Repeated classes work correctly on multiple characters.
+. Classes containing just a single character whose value is greater than 127
+ (but less than 256), for example, [\\x80] or [^\\x{93}], do not work because
+ these are optimized into single byte matches. In the first case, of course,
+ the class brackets are just redundant.
+. Lookbehind assertions move backwards in the subject by a fixed number of
+ characters instead of a fixed number of bytes. Simple cases have been tested
+ to work correctly, but there may be hidden gotchas herein.
+. The character types such as \\d and \\w do not work correctly with UTF-8
+ characters. They continue to test a single byte.
+. Anything not explicitly mentioned here continues to work in bytes rather
+ than in characters.
+ The following UTF-8 features of Perl 5.6 are not implemented:
+. The escape sequence \\C to match a single byte.
+. The use of Unicode tables and properties and escapes \\p, \\P, and \\X.
  .SH AUTHOR
  Philip Hazel <ph10@cam.ac.uk>
  .br
-Line 1497 
 Cambridge CB2 3QG, England.
+Line 1803 
 Cambridge CB2 3QG, England.
  .br
  Phone: +44 1223 334714
- Last updated: 29 July 1999
+ Last updated: 28 August 2000,
+ .br
+   the 250th anniversary of the death of J.S. Bach.
  .br
- Copyright (c) 1997-1999 University of Cambridge.
+ Copyright (c) 1997-2000 University of Cambridge.

 Legend:



Removed from v.41
 


changed lines


 
Added in v.51
 Legend:



Removed from v.41
 


changed lines


 
Added in v.51
-Removed from v.41
+Added in v.51

	ViewVC Help
Powered by ViewVC 1.1.5