/[pcre]/code/trunk/doc/pcretest.1

Diff of /code/trunk/doc/pcretest.1

Parent Directory | Revision Log | View Patch Patch

-revision 96 by nigel,
Fri Mar  2 13:10:43 2007 UTC
+revision 567 by ph10,
Sat Nov  6 17:10:00 2010 UTC
 Line 49 
 Output a brief summary these options and
  Behave as if each regex has the \fB/I\fP modifier; information about the
  compiled pattern is given after compilation.
  .TP 10
+ \fB-M\fP
+ Behave as if each data line contains the \eM escape sequence; this causes
+ PCRE to discover the minimum MATCH_LIMIT and MATCH_LIMIT_RECURSION settings by
+ calling \fBpcre_exec()\fP repeatedly with different limits.
+ .TP 10
  \fB-m\fP
  Output the size of each compiled pattern after it has been compiled. This is
  equivalent to adding \fB/M\fP to each regular expression. For compatibility
-Line 97 
 that file and writes to stdout. Otherwis
+Line 102 
 that file and writes to stdout. Otherwis
  stdout, and prompts for each line of input, using "re>" to prompt for regular
  expressions, and "data>" to prompt for data lines.
  .P
+ When \fBpcretest\fP is built, a configuration option can specify that it should
+ be linked with the \fBlibreadline\fP library. When this is done, if the input
+ is from a terminal, it is read using the \fBreadline()\fP function. This
+ provides line-editing and history facilities. The output from the \fB-help\fP
+ option states whether or not \fBreadline()\fP will be used.
+ .P
  The program handles any number of sets of input on a single input file. Each
  set starts with a regular expression, and continues with any number of data
  lines to be matched against the pattern.
-Line 154 
 effect as they do in Perl. For example:
+Line 165 
 effect as they do in Perl. For example:
  .sp
    /caseless/i
  .sp
- The following table shows additional modifiers for setting PCRE options that do
+ The following table shows additional modifiers for setting PCRE compile-time
- not correspond to anything in Perl:
+ options that do not correspond to anything in Perl:
  .sp
-   \fB/A\fP       PCRE_ANCHORED
+   \fB/8\fP              PCRE_UTF8
-   \fB/C\fP       PCRE_AUTO_CALLOUT
+   \fB/?\fP              PCRE_NO_UTF8_CHECK
-   \fB/E\fP       PCRE_DOLLAR_ENDONLY
+   \fB/A\fP              PCRE_ANCHORED
-   \fB/f\fP       PCRE_FIRSTLINE
+   \fB/C\fP              PCRE_AUTO_CALLOUT
-   \fB/J\fP       PCRE_DUPNAMES
+   \fB/E\fP              PCRE_DOLLAR_ENDONLY
-   \fB/N\fP       PCRE_NO_AUTO_CAPTURE
+   \fB/f\fP              PCRE_FIRSTLINE
-   \fB/U\fP       PCRE_UNGREEDY
+   \fB/J\fP              PCRE_DUPNAMES
-   \fB/X\fP       PCRE_EXTRA
+   \fB/N\fP              PCRE_NO_AUTO_CAPTURE
-   \fB/<cr>\fP    PCRE_NEWLINE_CR
+   \fB/U\fP              PCRE_UNGREEDY
-   \fB/<lf>\fP    PCRE_NEWLINE_LF
+   \fB/W\fP              PCRE_UCP
-   \fB/<crlf>\fP  PCRE_NEWLINE_CRLF
+   \fB/X\fP              PCRE_EXTRA
-   \fB/<any>\fP   PCRE_NEWLINE_ANY
+   \fB/<JS>\fP           PCRE_JAVASCRIPT_COMPAT
+   \fB/<cr>\fP           PCRE_NEWLINE_CR
+   \fB/<lf>\fP           PCRE_NEWLINE_LF
+   \fB/<crlf>\fP         PCRE_NEWLINE_CRLF
+   \fB/<anycrlf>\fP      PCRE_NEWLINE_ANYCRLF
+   \fB/<any>\fP          PCRE_NEWLINE_ANY
+   \fB/<bsr_anycrlf>\fP  PCRE_BSR_ANYCRLF
+   \fB/<bsr_unicode>\fP  PCRE_BSR_UNICODE
  .sp
- Those specifying line ending sequencess are literal strings as shown. This
+ The modifiers that are enclosed in angle brackets are literal strings as shown,
+ including the angle brackets, but the letters can be in either case. This
  example sets multiline matching with CRLF as the line ending sequence:
  .sp
    /^abc/m<crlf>
  .sp
- Details of the meanings of these PCRE options are given in the
+ As well as turning on the PCRE_UTF8 option, the \fB/8\fP modifier also causes
+ any non-printing characters in output strings to be printed using the
+ \ex{hh...} notation if they are valid UTF-8 sequences. Full details of the PCRE
+ options are given in the
  .\" HREF
  \fBpcreapi\fP
  .\"
-Line 195 
 substring. This makes a difference to th
+Line 217 
 substring. This makes a difference to th
  begins with a lookbehind assertion (including \eb or \eB).
  .P
  If any call to \fBpcre_exec()\fP in a \fB/g\fP or \fB/G\fP sequence matches an
- empty string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED
+ empty string, the next call is done with the PCRE_NOTEMPTY_ATSTART and
- flags set in order to search for another, non-empty, match at the same point.
+ PCRE_ANCHORED flags set in order to search for another, non-empty, match at the
- If this second match fails, the start offset is advanced by one, and the normal
+ same point. If this second match fails, the start offset is advanced, and the
- match is retried. This imitates the way Perl handles such cases when using the
+ normal match is retried. This imitates the way Perl handles such cases when
- \fB/g\fP modifier or the \fBsplit()\fP function.
+ using the \fB/g\fP modifier or the \fBsplit()\fP function. Normally, the start
+ offset is advanced by one character, but if the newline convention recognizes
+ CRLF as a newline, and the current character is CR followed by LF, an advance
+ of two is used.
  .
  .
  .SS "Other modifiers"
-Line 214 
 the subject string. This is useful for t
+Line 239 
 the subject string. This is useful for t
  multiple copies of the same substring.
  .P
  The \fB/B\fP modifier is a debugging feature. It requests that \fBpcretest\fP
- output a representation of the compiled byte code after compilation.
+ output a representation of the compiled byte code after compilation. Normally
- .P
+ this information contains length and offset values; however, if \fB/Z\fP is
- The \fB/L\fP modifier must be followed directly by the name of a locale, for
+ also present, this data is replaced by spaces. This is a special feature for
- example,
+ use in the automatic test scripts; it ensures that the same output is generated
- .sp
+ for different internal link sizes.
-   /pattern/Lfr_FR
- .sp
- For this reason, it must be the last modifier. The given locale is set,
- \fBpcre_maketables()\fP is called to build a set of character tables for the
- locale, and this is then passed to \fBpcre_compile()\fP when compiling the
- regular expression. Without an \fB/L\fP modifier, NULL is passed as the tables
- pointer; that is, \fB/L\fP applies only to the expression on which it appears.
- .P
- The \fB/I\fP modifier requests that \fBpcretest\fP output information about the
- compiled pattern (whether it is anchored, has a fixed first character, and
- so on). It does this by calling \fBpcre_fullinfo()\fP after compiling a
- pattern. If the pattern is studied, the results of that are also output.
  .P
  The \fB/D\fP modifier is a PCRE debugging feature, and is equivalent to
- \fB/BI\fP, that is, both the \fP/B\fP and the \fB/I\fP modifiers.
+ \fB/BI\fP, that is, both the \fB/B\fP and the \fB/I\fP modifiers.
  .P
  The \fB/F\fP modifier causes \fBpcretest\fP to flip the byte order of the
  fields in the compiled pattern that contain 2-byte and 4-byte numbers. This
-Line 243 
 available when the POSIX interface to PC
+Line 256 
 available when the POSIX interface to PC
  \fB/P\fP pattern modifier is specified. See also the section about saving and
  reloading compiled patterns below.
  .P
- The \fB/S\fP modifier causes \fBpcre_study()\fP to be called after the
+ The \fB/I\fP modifier requests that \fBpcretest\fP output information about the
- expression has been compiled, and the results used when the expression is
+ compiled pattern (whether it is anchored, has a fixed first character, and
- matched.
+ so on). It does this by calling \fBpcre_fullinfo()\fP after compiling a
+ pattern. If the pattern is studied, the results of that are also output.
+ .P
+ The \fB/K\fP modifier requests \fBpcretest\fP to show names from backtracking
+ control verbs that are returned from calls to \fBpcre_exec()\fP. It causes
+ \fBpcretest\fP to create a \fBpcre_extra\fP block if one has not already been
+ created by a call to \fBpcre_study()\fP, and to set the PCRE_EXTRA_MARK flag
+ and the \fBmark\fP field within it, every time that \fBpcre_exec()\fP is
+ called. If the variable that the \fBmark\fP field points to is non-NULL for a
+ match, non-match, or partial match, \fBpcretest\fP prints the string to which
+ it points. For a match, this is shown on a line by itself, tagged with "MK:".
+ For a non-match it is added to the message.
+ .P
+ The \fB/L\fP modifier must be followed directly by the name of a locale, for
+ example,
+ .sp
+   /pattern/Lfr_FR
+ .sp
+ For this reason, it must be the last modifier. The given locale is set,
+ \fBpcre_maketables()\fP is called to build a set of character tables for the
+ locale, and this is then passed to \fBpcre_compile()\fP when compiling the
+ regular expression. Without an \fB/L\fP (or \fB/T\fP) modifier, NULL is passed
+ as the tables pointer; that is, \fB/L\fP applies only to the expression on
+ which it appears.
  .P
  The \fB/M\fP modifier causes the size of memory block used to hold the compiled
  pattern to be output.
  .P
+ The \fB/S\fP modifier causes \fBpcre_study()\fP to be called after the
+ expression has been compiled, and the results used when the expression is
+ matched.
+ .P
+ The \fB/T\fP modifier must be followed by a single digit. It causes a specific
+ set of built-in character tables to be passed to \fBpcre_compile()\fP. It is
+ used in the standard PCRE tests to check behaviour with different character
+ tables. The digit specifies the tables as follows:
+ .sp
+  the default ASCII tables, as distributed in
+         pcre_chartables.c.dist
+  a set of tables defining ISO 8859 characters
+ .sp
+ In table 1, some characters whose codes are greater than 128 are identified as
+ letters, digits, spaces, etc.
+ .
+ .
+ .SS "Using the POSIX wrapper API"
+ .rs
+ .sp
  The \fB/P\fP modifier causes \fBpcretest\fP to call PCRE via the POSIX wrapper
- API rather than its native API. When this is done, all other modifiers except
+ API rather than its native API. When \fB/P\fP is set, the following modifiers
- \fB/i\fP, \fB/m\fP, and \fB/+\fP are ignored. REG_ICASE is set if \fB/i\fP is
+ set options for the \fBregcomp()\fP function:
- present, and REG_NEWLINE is set if \fB/m\fP is present. The wrapper functions
+ .sp
- force PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.
+   /i    REG_ICASE
- .P
+   /m    REG_NEWLINE
- The \fB/8\fP modifier causes \fBpcretest\fP to call PCRE with the PCRE_UTF8
+   /N    REG_NOSUB
- option set. This turns on support for UTF-8 character handling in PCRE,
+   /s    REG_DOTALL     )
- provided that it was compiled with this support enabled. This modifier also
+   /U    REG_UNGREEDY   ) These options are not part of
- causes any non-printing characters in output strings to be printed using the
+   /W    REG_UCP        )   the POSIX standard
- \ex{hh...} notation if they are valid UTF-8 sequences.
+   /8    REG_UTF8       )
- .P
+ .sp
- If the \fB/?\fP modifier is used with \fB/8\fP, it causes \fBpcretest\fP to
+ The \fB/+\fP modifier works as described above. All other modifiers are
- call \fBpcre_compile()\fP with the PCRE_NO_UTF8_CHECK option, to suppress the
+ ignored.
- checking of the string for UTF-8 validity.
  .
  .
  .SH "DATA LINES"
-Line 336 
 recognized:
+Line 391 
 recognized:
                 MATCH_LIMIT_RECURSION settings
  .\" JOIN
    \eN         pass the PCRE_NOTEMPTY option to \fBpcre_exec()\fP
-                or \fBpcre_dfa_exec()\fP
+                or \fBpcre_dfa_exec()\fP; if used twice, pass the
+                PCRE_NOTEMPTY_ATSTART option
  .\" JOIN
    \eOdd       set the size of the output vector passed to
                 \fBpcre_exec()\fP to dd (any number of digits)
  .\" JOIN
-   \eP         pass the PCRE_PARTIAL option to \fBpcre_exec()\fP
+   \eP         pass the PCRE_PARTIAL_SOFT option to \fBpcre_exec()\fP
-                or \fBpcre_dfa_exec()\fP
+                or \fBpcre_dfa_exec()\fP; if used twice, pass the
+                PCRE_PARTIAL_HARD option
  .\" JOIN
    \eQdd       set the PCRE_MATCH_LIMIT_RECURSION limit to dd
                 (any number of digits)
    \eR         pass the PCRE_DFA_RESTART option to \fBpcre_dfa_exec()\fP
    \eS         output details of memory get/free calls during matching
  .\" JOIN
+   \eY         pass the PCRE_NO_START_OPTIMIZE option to \fBpcre_exec()\fP
+                or \fBpcre_dfa_exec()\fP
+ .\" JOIN
    \eZ         pass the PCRE_NOTEOL option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
  .\" JOIN
    \e?         pass the PCRE_NO_UTF8_CHECK option to
                 \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP
-   \e>dd       start the match at offset dd (any number of digits);
  .\" JOIN
-                this sets the \fIstartoffset\fP argument for \fBpcre_exec()\fP
+   \e>dd       start the match at offset dd (optional "-"; then
-                or \fBpcre_dfa_exec()\fP
+                any number of digits); this sets the \fIstartoffset\fP
+                argument for \fBpcre_exec()\fP or \fBpcre_dfa_exec()\fP
  .\" JOIN
    \e<cr>      pass the PCRE_NEWLINE_CR option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
-Line 368 
 recognized:
+Line 428 
 recognized:
    \e<crlf>    pass the PCRE_NEWLINE_CRLF option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
  .\" JOIN
+   \e<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to \fBpcre_exec()\fP
+                or \fBpcre_dfa_exec()\fP
+ .\" JOIN
    \e<any>     pass the PCRE_NEWLINE_ANY option to \fBpcre_exec()\fP
                 or \fBpcre_dfa_exec()\fP
  .sp
-Line 396 
 by the \fB-O\fP command line option (or
+Line 459 
 by the \fB-O\fP command line option (or
  the call of \fBpcre_exec()\fP for the line in which it appears.
  .P
  If the \fB/P\fP modifier was present on the pattern, causing the POSIX wrapper
- API to be used, the only option-setting sequences that have any effect are \eB
+ API to be used, the only option-setting sequences that have any effect are \eB,
- and \eZ, causing REG_NOTBOL and REG_NOTEOL, respectively, to be passed to
+ \eN, and \eZ, causing REG_NOTBOL, REG_NOTEMPTY, and REG_NOTEOL, respectively,
- \fBregexec()\fP.
+ to be passed to \fBregexec()\fP.
  .P
  The use of \ex{hh...} to represent UTF-8 characters is not dependent on the use
  of the \fB/8\fP modifier on the pattern. It is recognized always. There may be
  any number of hexadecimal digits inside the braces. The result is from one to
- six bytes, encoded according to the UTF-8 rules.
+ six bytes, encoded according to the original UTF-8 rules of RFC 2279. This
+ allows for values in the range 0 to 0x7FFFFFFF. Note that not all of those are
+ valid Unicode code points, or indeed valid UTF-8 characters according to the
+ later rules in RFC 3629.
  .
  .
  .SH "THE ALTERNATIVE MATCHING FUNCTION"
-Line 434 
 This section describes the output when t
+Line 500 
 This section describes the output when t
  .P
  When a match succeeds, pcretest outputs the list of captured substrings that
  \fBpcre_exec()\fP returns, starting with number 0 for the string that matched
- the whole pattern. Otherwise, it outputs "No match" or "Partial match"
+ the whole pattern. Otherwise, it outputs "No match" when the return is
- when \fBpcre_exec()\fP returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL,
+ PCRE_ERROR_NOMATCH, and "Partial match:" followed by the partially matching
- respectively, and otherwise the PCRE negative error number. Here is an example
+ substring when \fBpcre_exec()\fP returns PCRE_ERROR_PARTIAL. (Note that this is
- of an interactive \fBpcretest\fP run.
+ the entire substring that was inspected during the partial match; it may
+ include characters before the actual match start if a lookbehind assertion,
+ \eK, \eb, or \eB was involved.) For any other returns, it outputs the PCRE
+ negative error number. Here is an example of an interactive \fBpcretest\fP run.
  .sp
    $ pcretest
    PCRE version 7.0 30-Nov-2006
-Line 449 
 of an interactive \fBpcretest\fP run.
+Line 518 
 of an interactive \fBpcretest\fP run.
    data> xyz
    No match
  .sp
+ Note that unset capturing substrings that are not followed by one that is set
+ are not returned by \fBpcre_exec()\fP, and are not shown by \fBpcretest\fP. In
+ the following example, there are two capturing substrings, but when the first
+ data line is matched, the second, unset substring is not shown. An "internal"
+ unset substring is shown as "<unset>", as for the second data line.
+ .sp
+     re> /(a)|(b)/
+   data> a
+: a
+: a
+   data> b
+: b
+: <unset>
+: b
+ .sp
  If the strings contain any non-printing characters, they are output as \e0x
  escapes, or as \ex{...} escapes if the \fB/8\fP modifier was present on the
  pattern. See below for the definition of non-printing characters. If the
-Line 503 
 the subject where there is at least one
+Line 587 
 the subject where there is at least one
 : tan
  .sp
  (Using the normal matching function on this data finds only "tang".) The
- longest matching string is always given first (and numbered zero).
+ longest matching string is always given first (and numbered zero). After a
+ PCRE_ERROR_PARTIAL return, the output is "Partial match:", followed by the
+ partially matching substring. (Note that this is the entire substring that was
+ inspected during the partial match; it may include characters before the actual
+ match start if a lookbehind assertion, \eK, \eb, or \eB was involved.)
  .P
  If \fB/g\fP is present on the pattern, the search for further matches resumes
  at the end of the longest match. For example:
-Line 529 
 indicating that the subject partially ma
+Line 617 
 indicating that the subject partially ma
  match with additional subject data by means of the \eR escape sequence. For
  example:
  .sp
-     re> /^\d?\d(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\d\d$/
+     re> /^\ed?\ed(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\ed\ed$/
    data> 23ja\eP\eD
    Partial match: 23ja
    data> n05\eR\eD
-Line 665 
 result is undefined.
+Line 753 
 result is undefined.
  .rs
  .sp
  \fBpcre\fP(3), \fBpcreapi\fP(3), \fBpcrecallout\fP(3), \fBpcrematching\fP(3),
- \fBpcrepartial\fP(d), \fPpcrepattern\fP(3), \fBpcreprecompile\fP(3).
+ \fBpcrepartial\fP(d), \fBpcrepattern\fP(3), \fBpcreprecompile\fP(3).
  .
  .
  .SH AUTHOR
  .rs
  .sp
+ .nf
  Philip Hazel
- .br
+ University Computing Service
- University Computing Service,
- .br
  Cambridge CB2 3QH, England.
- .P
+ .fi
- .in 0
+ .
- Last updated: 30 November 2006
+ .
- .br
+ .SH REVISION
- Copyright (c) 1997-2006 University of Cambridge.
+ .rs
+ .sp
+ .nf
+ Last updated: 06 November 2010
+ Copyright (c) 1997-2010 University of Cambridge.
+ .fi

 Legend:



Removed from v.96
 


changed lines


 
Added in v.567
 Legend:



Removed from v.96
 


changed lines


 
Added in v.567
-Removed from v.96
+Added in v.567

	ViewVC Help
Powered by ViewVC 1.1.5