/[pcre]/code/trunk/doc/pcretest.1

Diff of /code/trunk/doc/pcretest.1

Parent Directory | Revision Log | View Patch Patch

-revision 289 by ph10,
Sun Dec 23 12:17:20 2007 UTC
+revision 606 by ph10,
Mon Jun  6 17:46:22 2011 UTC
 Line 25 
 documentation.
  .rs
  .TP 10
  \fB-b\fP
- Behave as if each regex has the \fB/B\fP (show bytecode) modifier; the internal
+ Behave as if each regex has the \fB/B\fP (show byte code) modifier; the
- form is output after compilation.
+ internal form is output after compilation.
  .TP 10
  \fB-C\fP
  Output the version number of the PCRE library, and all available information
 Line 49 
 Output a brief summary these options and
  Behave as if each regex has the \fB/I\fP modifier; information about the
  compiled pattern is given after compilation.
  .TP 10
+ \fB-M\fP
+ Behave as if each data line contains the \eM escape sequence; this causes
+ PCRE to discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings by
+ calling \fBpcre_exec()\fP repeatedly with different limits.
+ .TP 10
  \fB-m\fP
  Output the size of each compiled pattern after it has been compiled. This is
- equivalent to adding \fB/M\fP to each regular expression. For compatibility
+ equivalent to adding \fB/M\fP to each regular expression.
- with earlier versions of pcretest, \fB-s\fP is a synonym for \fB-m\fP.
  .TP 10
  \fB-o\fP \fIosize\fP
  Set the number of elements in the output vector that is used when calling
-Line 71 
 set.
+Line 75 
 set.
  Do not output the version number of \fBpcretest\fP at the start of execution.
  .TP 10
  \fB-S\fP \fIsize\fP
- On Unix-like systems, set the size of the runtime stack to \fIsize\fP
+ On Unix-like systems, set the size of the run-time stack to \fIsize\fP
  megabytes.
  .TP 10
+ \fB-s\fP
+ Behave as if each regex has the \fB/S\fP modifier; in other words, force each
+ regex to be studied.
+ .TP 10
  \fB-t\fP
  Run each compile, study, and match many times with a timer, and output
  resulting time per compile or match (in milliseconds). Do not set \fB-m\fP with
-Line 149 
 pcretest to read the next line as a cont
+Line 157 
 pcretest to read the next line as a cont
  A pattern may be followed by any number of modifiers, which are mostly single
  characters. Following Perl usage, these are referred to below as, for example,
  "the \fB/i\fP modifier", even though the delimiter of the pattern need not
- always be a slash, and no slash is used when writing modifiers. Whitespace may
+ always be a slash, and no slash is used when writing modifiers. White space may
  appear between the final pattern delimiter and the first modifier, and between
  the modifiers themselves.
  .P
-Line 160 
 effect as they do in Perl. For example:
+Line 168 
 effect as they do in Perl. For example:
  .sp
    /caseless/i
  .sp
- The following table shows additional modifiers for setting PCRE options that do
+ The following table shows additional modifiers for setting PCRE compile-time
- not correspond to anything in Perl:
+ options that do not correspond to anything in Perl:
  .sp
+   \fB/8\fP              PCRE_UTF8
+   \fB/?\fP              PCRE_NO_UTF8_CHECK
    \fB/A\fP              PCRE_ANCHORED
    \fB/C\fP              PCRE_AUTO_CALLOUT
    \fB/E\fP              PCRE_DOLLAR_ENDONLY
-Line 170 
 not correspond to anything in Perl:
+Line 180 
 not correspond to anything in Perl:
    \fB/J\fP              PCRE_DUPNAMES
    \fB/N\fP              PCRE_NO_AUTO_CAPTURE
    \fB/U\fP              PCRE_UNGREEDY
+   \fB/W\fP              PCRE_UCP
    \fB/X\fP              PCRE_EXTRA
+   \fB/Y\fP              PCRE_NO_START_OPTIMIZE
+   \fB/<JS>\fP           PCRE_JAVASCRIPT_COMPAT
    \fB/<cr>\fP           PCRE_NEWLINE_CR
    \fB/<lf>\fP           PCRE_NEWLINE_LF
    \fB/<crlf>\fP         PCRE_NEWLINE_CRLF
-Line 179 
 not correspond to anything in Perl:
+Line 192 
 not correspond to anything in Perl:
    \fB/<bsr_anycrlf>\fP  PCRE_BSR_ANYCRLF
    \fB/<bsr_unicode>\fP  PCRE_BSR_UNICODE
  .sp
- Those specifying line ending sequences are literal strings as shown, but the
+ The modifiers that are enclosed in angle brackets are literal strings as shown,
- letters can be in either case. This example sets multiline matching with CRLF
+ including the angle brackets, but the letters can be in either case. This
- as the line ending sequence:
+ example sets multiline matching with CRLF as the line ending sequence:
  .sp
    /^abc/m<crlf>
  .sp
- Details of the meanings of these PCRE options are given in the
+ As well as turning on the PCRE_UTF8 option, the \fB/8\fP modifier also causes
+ any non-printing characters in output strings to be printed using the
+ \ex{hh...} notation if they are valid UTF-8 sequences. Full details of the PCRE
+ options are given in the
  .\" HREF
  \fBpcreapi\fP
  .\"
-Line 205 
 substring. This makes a difference to th
+Line 221 
 substring. This makes a difference to th
  begins with a lookbehind assertion (including \eb or \eB).
  .P
  If any call to \fBpcre_exec()\fP in a \fB/g\fP or \fB/G\fP sequence matches an
- empty string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED
+ empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and
- flags set in order to search for another, non-empty, match at the same point.
+ PCRE_ANCHORED flags set in order to search for another, non-empty, match at the
- If this second match fails, the start offset is advanced by one, and the normal
+ same point. If this second match fails, the start offset is advanced, and the
- match is retried. This imitates the way Perl handles such cases when using the
+ normal match is retried. This imitates the way Perl handles such cases when
- \fB/g\fP modifier or the \fBsplit()\fP function.
+ using the \fB/g\fP modifier or the \fBsplit()\fP function. Normally, the start
+ offset is advanced by one character, but if the newline convention recognizes
+ CRLF as a newline, and the current character is CR followed by LF, an advance
+ of two is used.
  .
  .
  .SS "Other modifiers"
-Line 230 
 also present, this data is replaced by s
+Line 249 
 also present, this data is replaced by s
  use in the automatic test scripts; it ensures that the same output is generated
  for different internal link sizes.
  .P
- The \fB/L\fP modifier must be followed directly by the name of a locale, for
- example,
- .sp
-   /pattern/Lfr_FR
- .sp
- For this reason, it must be the last modifier. The given locale is set,
- \fBpcre_maketables()\fP is called to build a set of character tables for the
- locale, and this is then passed to \fBpcre_compile()\fP when compiling the
- regular expression. Without an \fB/L\fP modifier, NULL is passed as the tables
- pointer; that is, \fB/L\fP applies only to the expression on which it appears.
- .P
- The \fB/I\fP modifier requests that \fBpcretest\fP output information about the
- compiled pattern (whether it is anchored, has a fixed first character, and
- so on). It does this by calling \fBpcre_fullinfo()\fP after compiling a
- pattern. If the pattern is studied, the results of that are also output.
- .P
  The \fB/D\fP modifier is a PCRE debugging feature, and is equivalent to
  \fB/BI\fP, that is, both the \fB/B\fP and the \fB/I\fP modifiers.
  .P
-Line 257 
 available when the POSIX interface to PC
+Line 260 
 available when the POSIX interface to PC
  \fB/P\fP pattern modifier is specified. See also the section about saving and
  reloading compiled patterns below.
  .P
- The \fB/S\fP modifier causes \fBpcre_study()\fP to be called after the
+ The \fB/I\fP modifier requests that \fBpcretest\fP output information about the
- expression has been compiled, and the results used when the expression is
+ compiled pattern (whether it is anchored, has a fixed first character, and
- matched.
+ so on). It does this by calling \fBpcre_fullinfo()\fP after compiling a
+ pattern. If the pattern is studied, the results of that are also output.
+ .P
+ The \fB/K\fP modifier requests \fBpcretest\fP to show names from backtracking
+ control verbs that are returned from calls to \fBpcre_exec()\fP. It causes
+ \fBpcretest\fP to create a \fBpcre_extra\fP block if one has not already been
+ created by a call to \fBpcre_study()\fP, and to set the PCRE_EXTRA_MARK flag
+ and the \fBmark\fP field within it, every time that \fBpcre_exec()\fP is
+ called. If the variable that the \fBmark\fP field points to is non-NULL for a
+ match, non-match, or partial match, \fBpcretest\fP prints the string to which
+ it points. For a match, this is shown on a line by itself, tagged with "MK:".
+ For a non-match it is added to the message.
+ .P
+ The \fB/L\fP modifier must be followed directly by the name of a locale, for
+ example,
+ .sp
+   /pattern/Lfr_FR
+ .sp
+ For this reason, it must be the last modifier. The given locale is set,
+ \fBpcre_maketables()\fP is called to build a set of character tables for the
+ locale, and this is then passed to \fBpcre_compile()\fP when compiling the
+ regular expression. Without an \fB/L\fP (or \fB/T\fP) modifier, NULL is passed
+ as the tables pointer; that is, \fB/L\fP applies only to the expression on
+ which it appears.
  .P
  The \fB/M\fP modifier causes the size of memory block used to hold the compiled
  pattern to be output.
  .P
+ The \fB/S\fP modifier causes \fBpcre_study()\fP to be called after the
+ expression has been compiled, and the results used when the expression is
+ matched.
+ .P
+ The \fB/T\fP modifier must be followed by a single digit. It causes a specific
+ set of built-in character tables to be passed to \fBpcre_compile()\fP. It is
+ used in the standard PCRE tests to check behaviour with different character
+ tables. The digit specifies the tables as follows:
+ .sp
+  the default ASCII tables, as distributed in
+         pcre_chartables.c.dist
+  a set of tables defining ISO 8859 characters
+ .sp
+ In table 1, some characters whose codes are greater than 128 are identified as
+ letters, digits, spaces, etc.
+ .
+ .
+ .SS "Using the POSIX wrapper API"
+ .rs
+ .sp
  The \fB/P\fP modifier causes \fBpcretest\fP to call PCRE via the POSIX wrapper
- API rather than its native API. When this is done, all other modifiers except
+ API rather than its native API. When \fB/P\fP is set, the following modifiers
- \fB/i\fP, \fB/m\fP, and \fB/+\fP are ignored. REG_ICASE is set if \fB/i\fP is
+ set options for the \fBregcomp()\fP function:
- present, and REG_NEWLINE is set if \fB/m\fP is present. The wrapper functions
+ .sp
- force PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.
+   /i    REG_ICASE
- .P
+   /m    REG_NEWLINE
- The \fB/8\fP modifier causes \fBpcretest\fP to call PCRE with the PCRE_UTF8
+   /N    REG_NOSUB
- option set. This turns on support for UTF-8 character handling in PCRE,
+   /s    REG_DOTALL     )
- provided that it was compiled with this support enabled. This modifier also
+   /U    REG_UNGREEDY   ) These options are not part of
- causes any non-printing characters in output strings to be printed using the
+   /W    REG_UCP        )   the POSIX standard
- \ex{hh...} notation if they are valid UTF-8 sequences.
+   /8    REG_UTF8       )
- .P
+ .sp
- If the \fB/?\fP modifier is used with \fB/8\fP, it causes \fBpcretest\fP to
+ The \fB/+\fP modifier works as described above. All other modifiers are
- call \fBpcre_compile()\fP with the PCRE_NO_UTF8_CHECK option, to suppress the
+ ignored.
- checking of the string for UTF-8 validity.
  .
  .
  .SH "DATA LINES"
  .rs
  .sp
  Before each data line is passed to \fBpcre_exec()\fP, leading and trailing
- whitespace is removed, and it is then scanned for \e escapes. Some of these are
+ white space is removed, and it is then scanned for \e escapes. Some of these
- pretty esoteric features, intended for checking out some of the more
+ are pretty esoteric features, intended for checking out some of the more
  complicated features of PCRE. If you are just testing "ordinary" regular
  expressions, you probably don't need any of these. The following escapes are
  recognized:
-Line 294 
 recognized:
+Line 339 
 recognized:
    \ea         alarm (BEL, \ex07)
    \eb         backspace (\ex08)
    \ee         escape (\ex27)
-   \ef         formfeed (\ex0c)
+   \ef         form feed (\ex0c)
    \en         newline (\ex0a)
  .\" JOIN
    \eqdd       set the PCRE_MATCH_LIMIT limit to dd
-Line 303 
 recognized:
+Line 348 
 recognized:
    \et         tab (\ex09)
    \ev         vertical tab (\ex0b)
    \ennn       octal character (up to 3 octal digits)
-   \exhh       hexadecimal character (up to 2 hex digits)
+                always a byte unless > 255 in UTF-8 mode
+   \exhh       hexadecimal byte (up to 2 hex digits)
  .\" JOIN
    \ex{hh...}  hexadecimal character, any number of digits
                 in UTF-8 mode
-Line 350 
 recognized:
+Line 396 
 recognized:
                 MATCH_LIMIT_RECURSION settings
  .\" JOIN
    \eN         pass the PCRE_NOTEMPTY option to \fBpcre_exec()\fP
-                or \fBpcre_dfa_exec()\fP
+                or \fBpcre_dfa_exec()\fP; if used twice, pass the
+                PCRE_NOTEMPTY_ATSTART option
  .\" JOIN
    \eOdd       set the size of the output vector passed to
                 \fBpcre_exec()\fP to dd (any number of digits)
  .\" JOIN
-   \eP         pass the PCRE_PARTIAL option to \fBpcre_exec()\fP
+   \eP         pass the PCRE_PARTIAL_SOFT option to \fBpcre_exec()\fP
-                or \fBpcre_dfa_exec()\fP
+                or \fBpcre_dfa_exec()\fP; if used twice, pass the
+                PCRE_PARTIAL_HARD option
  .\" JOIN
    \eQdd       set the PCRE_MATCH_LIMIT_RECURSION limit to dd
                 (any number of digits)
    \eR         pass the PCRE_DFA_RESTART option to \fBpcre_dfa_exec()\fP
    \eS         output details of memory get/free calls during matching
  .\" JOIN
+   \eY         pass the PCRE_NO_START_OPTIMIZE option to \fBpcre_exec()\fP
+                or \fBpcre_dfa_exec()\fP
+ .\" JOIN
    \eZ         pass the PCRE_NOTEOL option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
  .\" JOIN
    \e?         pass the PCRE_NO_UTF8_CHECK option to
                 \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP
-   \e>dd       start the match at offset dd (any number of digits);
  .\" JOIN
-                this sets the \fIstartoffset\fP argument for \fBpcre_exec()\fP
+   \e>dd       start the match at offset dd (optional "-"; then
-                or \fBpcre_dfa_exec()\fP
+                any number of digits); this sets the \fIstartoffset\fP
+                argument for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP
  .\" JOIN
    \e<cr>      pass the PCRE_NEWLINE_CR option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
-Line 388 
 recognized:
+Line 439 
 recognized:
    \e<any>     pass the PCRE_NEWLINE_ANY option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
  .sp
+ Note that \exhh always specifies one byte, even in UTF-8 mode; this makes it
+ possible to construct invalid UTF-8 sequences for testing purposes. On the
+ other hand, \ex{hh} is interpreted as a UTF-8 character in UTF-8 mode,
+ generating more than one byte if the value is greater than 127. When not in
+ UTF-8 mode, it generates one byte for values less than 256, and causes an error
+ for greater values.
+ .P
  The escapes that specify line ending sequences are literal strings, exactly as
  shown. No more than one newline setting should be present in any data line.
  .P
-Line 413 
 by the \fB-O\fP command line option (or
+Line 471 
 by the \fB-O\fP command line option (or
  the call of \fBpcre_exec()\fP for the line in which it appears.
  .P
  If the \fB/P\fP modifier was present on the pattern, causing the POSIX wrapper
- API to be used, the only option-setting sequences that have any effect are \eB
+ API to be used, the only option-setting sequences that have any effect are \eB,
- and \eZ, causing REG_NOTBOL and REG_NOTEOL, respectively, to be passed to
+ \eN, and \eZ, causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively,
- \fBregexec()\fP.
+ to be passed to \fBregexec()\fP.
  .P
  The use of \ex{hh...} to represent UTF-8 characters is not dependent on the use
  of the \fB/8\fP modifier on the pattern. It is recognized always. There may be
-Line 452 
 found. This is always the shortest possi
+Line 510 
 found. This is always the shortest possi
  This section describes the output when the normal matching function,
  \fBpcre_exec()\fP, is being used.
  .P
- When a match succeeds, pcretest outputs the list of captured substrings that
+ When a match succeeds, \fBpcretest\fP outputs the list of captured substrings
- \fBpcre_exec()\fP returns, starting with number 0 for the string that matched
+ that \fBpcre_exec()\fP returns, starting with number 0 for the string that
- the whole pattern. Otherwise, it outputs "No match" or "Partial match"
+ matched the whole pattern. Otherwise, it outputs "No match" when the return is
- when \fBpcre_exec()\fP returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL,
+ PCRE_ERROR_NOMATCH, and "Partial match:" followed by the partially matching
- respectively, and otherwise the PCRE negative error number. Here is an example
+ substring when \fBpcre_exec()\fP returns PCRE_ERROR_PARTIAL. (Note that this is
- of an interactive \fBpcretest\fP run.
+ the entire substring that was inspected during the partial match; it may
+ include characters before the actual match start if a lookbehind assertion,
+ \eK, \eb, or \eB was involved.) For any other return, \fBpcretest\fP outputs
+ the PCRE negative error number and a short descriptive phrase. If the error is
+ a failed UTF-8 string check, the byte offset of the start of the failing
+ character and the reason code are also output, provided that the size of the
+ output vector is at least two. Here is an example of an interactive
+ \fBpcretest\fP run.
  .sp
    $ pcretest
-   PCRE version 7.0 30-Nov-2006
+   PCRE version 8.13 2011-04-30
  .sp
      re> /^abc(\ed+)/
    data> abc123
-Line 469 
 of an interactive \fBpcretest\fP run.
+Line 534 
 of an interactive \fBpcretest\fP run.
    data> xyz
    No match
  .sp
- Note that unset capturing substrings that are not followed by one that is set
+ Unset capturing substrings that are not followed by one that is set are not
- are not returned by \fBpcre_exec()\fP, and are not shown by \fBpcretest\fP. In
+ returned by \fBpcre_exec()\fP, and are not shown by \fBpcretest\fP. In the
- the following example, there are two capturing substrings, but when the first
+ following example, there are two capturing substrings, but when the first data
- data line is matched, the second, unset substring is not shown. An "internal"
+ line is matched, the second, unset substring is not shown. An "internal" unset
- unset substring is shown as "<unset>", as for the second data line.
+ substring is shown as "<unset>", as for the second data line.
  .sp
      re> /(a)|(b)/
    data> a
-Line 507 
 matching attempts are output in sequence
+Line 572 
 matching attempts are output in sequence
 : ipp
 : pp
  .sp
- "No match" is output only if the first match attempt fails.
+ "No match" is output only if the first match attempt fails. Here is an example
+ of a failure message (the offset 4 that is specified by \e>4 is past the end of
+ the subject string):
+ .sp
+     re> /xyz/
+   data> xyz\>4
+   Error -24 (bad offset value)
  .P
  If any of the sequences \fB\eC\fP, \fB\eG\fP, or \fB\eL\fP are present in a
  data line that is successfully matched, the substrings extracted by the
-Line 538 
 the subject where there is at least one
+Line 609 
 the subject where there is at least one
 : tan
  .sp
  (Using the normal matching function on this data finds only "tang".) The
- longest matching string is always given first (and numbered zero).
+ longest matching string is always given first (and numbered zero). After a
+ PCRE_ERROR_PARTIAL return, the output is "Partial match:", followed by the
+ partially matching substring. (Note that this is the entire substring that was
+ inspected during the partial match; it may include characters before the actual
+ match start if a lookbehind assertion, \eK, \eb, or \eB was involved.)
  .P
  If \fB/g\fP is present on the pattern, the search for further matches resumes
  at the end of the longest match. For example:
-Line 640 
 function to distinguish printing and non
+Line 715 
 function to distinguish printing and non
  .rs
  .sp
  The facilities described in this section are not available when the POSIX
- inteface to PCRE is being used, that is, when the \fB/P\fP pattern modifier is
+ interface to PCRE is being used, that is, when the \fB/P\fP pattern modifier is
  specified.
  .P
  When the POSIX interface is not in use, you can cause \fBpcretest\fP to write a
-Line 664 
 exact copy of the compiled pattern. If t
+Line 739 
 exact copy of the compiled pattern. If t
  follows immediately after the compiled pattern. After writing the file,
  \fBpcretest\fP expects to read a new pattern.
  .P
- A saved pattern can be reloaded into \fBpcretest\fP by specifing < and a file
+ A saved pattern can be reloaded into \fBpcretest\fP by specifying < and a file
  name instead of a pattern. The name of the file must not contain a < character,
  as otherwise \fBpcretest\fP will interpret the line as a pattern delimited by <
  characters.
-Line 717 
 Cambridge CB2 3QH, England.
+Line 792 
 Cambridge CB2 3QH, England.
  .rs
  .sp
  .nf
- Last updated: 18 December 2007
+ Last updated: 06 June 2011
- Copyright (c) 1997-2007 University of Cambridge.
+ Copyright (c) 1997-2011 University of Cambridge.
  .fi

 Legend:



Removed from v.289
 


changed lines


 
Added in v.606
 Legend:



Removed from v.289
 


changed lines


 
Added in v.606
-Removed from v.289
+Added in v.606

	ViewVC Help
Powered by ViewVC 1.1.5