/[pcre]/code/trunk/doc/pcretest.txt
ViewVC logotype

Diff of /code/trunk/doc/pcretest.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 461 by ph10, Mon Oct 5 10:59:35 2009 UTC revision 567 by ph10, Sat Nov 6 17:10:00 2010 UTC
# Line 154  PATTERN MODIFIERS Line 154  PATTERN MODIFIERS
154    
155           /caseless/i           /caseless/i
156    
157         The following table shows additional modifiers for setting PCRE options         The  following  table  shows additional modifiers for setting PCRE com-
158         that do not correspond to anything in Perl:         pile-time options that do not correspond to anything in Perl:
159    
160             /8              PCRE_UTF8
161             /?              PCRE_NO_UTF8_CHECK
162           /A              PCRE_ANCHORED           /A              PCRE_ANCHORED
163           /C              PCRE_AUTO_CALLOUT           /C              PCRE_AUTO_CALLOUT
164           /E              PCRE_DOLLAR_ENDONLY           /E              PCRE_DOLLAR_ENDONLY
# Line 164  PATTERN MODIFIERS Line 166  PATTERN MODIFIERS
166           /J              PCRE_DUPNAMES           /J              PCRE_DUPNAMES
167           /N              PCRE_NO_AUTO_CAPTURE           /N              PCRE_NO_AUTO_CAPTURE
168           /U              PCRE_UNGREEDY           /U              PCRE_UNGREEDY
169             /W              PCRE_UCP
170           /X              PCRE_EXTRA           /X              PCRE_EXTRA
171           /<JS>           PCRE_JAVASCRIPT_COMPAT           /<JS>           PCRE_JAVASCRIPT_COMPAT
172           /<cr>           PCRE_NEWLINE_CR           /<cr>           PCRE_NEWLINE_CR
# Line 174  PATTERN MODIFIERS Line 177  PATTERN MODIFIERS
177           /<bsr_anycrlf>  PCRE_BSR_ANYCRLF           /<bsr_anycrlf>  PCRE_BSR_ANYCRLF
178           /<bsr_unicode>  PCRE_BSR_UNICODE           /<bsr_unicode>  PCRE_BSR_UNICODE
179    
180         Those specifying line ending sequences are literal  strings  as  shown,         The modifiers that are enclosed in angle brackets are  literal  strings
181         but  the  letters  can  be  in either case. This example sets multiline         as  shown,  including  the  angle  brackets,  but the letters can be in
182         matching with CRLF as the line ending sequence:         either case. This example sets multiline matching with CRLF as the line
183           ending sequence:
184    
185           /^abc/m<crlf>           /^abc/m<crlf>
186    
187         Details of the meanings of these PCRE options are given in the  pcreapi         As well as turning on the PCRE_UTF8 option, the /8 modifier also causes
188         documentation.         any non-printing characters in output strings to be printed  using  the
189           \x{hh...}  notation  if they are valid UTF-8 sequences. Full details of
190           the PCRE options are given in the pcreapi documentation.
191    
192     Finding all matches in a string     Finding all matches in a string
193    
194         Searching  for  all  possible matches within each subject string can be         Searching for all possible matches within each subject  string  can  be
195         requested by the /g or /G modifier. After  finding  a  match,  PCRE  is         requested  by  the  /g  or  /G modifier. After finding a match, PCRE is
196         called again to search the remainder of the subject string. The differ-         called again to search the remainder of the subject string. The differ-
197         ence between /g and /G is that the former uses the startoffset argument         ence between /g and /G is that the former uses the startoffset argument
198         to  pcre_exec()  to  start  searching  at a new point within the entire         to pcre_exec() to start searching at a  new  point  within  the  entire
199         string (which is in effect what Perl does), whereas the  latter  passes         string  (which  is in effect what Perl does), whereas the latter passes
200         over  a  shortened  substring.  This makes a difference to the matching         over a shortened substring. This makes a  difference  to  the  matching
201         process if the pattern begins with a lookbehind assertion (including \b         process if the pattern begins with a lookbehind assertion (including \b
202         or \B).         or \B).
203    
204         If  any  call  to  pcre_exec()  in a /g or /G sequence matches an empty         If any call to pcre_exec() in a /g or  /G  sequence  matches  an  empty
205         string, the next  call  is  done  with  the  PCRE_NOTEMPTY_ATSTART  and         string,  the  next  call  is  done  with  the PCRE_NOTEMPTY_ATSTART and
206         PCRE_ANCHORED  flags  set  in  order  to search for another, non-empty,         PCRE_ANCHORED flags set in order  to  search  for  another,  non-empty,
207         match at the same point. If this second match fails, the  start  offset         match  at  the same point. If this second match fails, the start offset
208         is  advanced  by  one  character, and the normal match is retried. This         is advanced, and the normal match is retried.  This  imitates  the  way
209         imitates the way Perl handles such cases when using the /g modifier  or         Perl handles such cases when using the /g modifier or the split() func-
210         the split() function.         tion. Normally, the start offset is advanced by one character,  but  if
211           the  newline  convention  recognizes CRLF as a newline, and the current
212           character is CR followed by LF, an advance of two is used.
213    
214     Other modifiers     Other modifiers
215    
216         There are yet more modifiers for controlling the way pcretest operates.         There are yet more modifiers for controlling the way pcretest operates.
217    
218         The  /+ modifier requests that as well as outputting the substring that         The /+ modifier requests that as well as outputting the substring  that
219         matched the entire pattern, pcretest  should  in  addition  output  the         matched  the  entire  pattern,  pcretest  should in addition output the
220         remainder  of  the  subject  string. This is useful for tests where the         remainder of the subject string. This is useful  for  tests  where  the
221         subject contains multiple copies of the same substring.         subject contains multiple copies of the same substring.
222    
223         The /B modifier is a debugging feature. It requests that pcretest  out-         The  /B modifier is a debugging feature. It requests that pcretest out-
224         put  a representation of the compiled byte code after compilation. Nor-         put a representation of the compiled byte code after compilation.  Nor-
225         mally this information contains length and offset values;  however,  if         mally  this  information contains length and offset values; however, if
226         /Z  is also present, this data is replaced by spaces. This is a special         /Z is also present, this data is replaced by spaces. This is a  special
227         feature for use in the automatic test scripts; it ensures that the same         feature for use in the automatic test scripts; it ensures that the same
228         output is generated for different internal link sizes.         output is generated for different internal link sizes.
229    
230         The  /L modifier must be followed directly by the name of a locale, for         The /D modifier is a PCRE debugging feature, and is equivalent to  /BI,
231         example,         that is, both the /B and the /I modifiers.
   
          /pattern/Lfr_FR  
232    
233         For this reason, it must be the last modifier. The given locale is set,         The /F modifier causes pcretest to flip the byte order of the fields in
234         pcre_maketables()  is called to build a set of character tables for the         the compiled pattern that  contain  2-byte  and  4-byte  numbers.  This
235         locale, and this is then passed to pcre_compile()  when  compiling  the         facility  is  for testing the feature in PCRE that allows it to execute
236         regular  expression.  Without  an  /L  modifier,  NULL is passed as the         patterns that were compiled on a host with a different endianness. This
237         tables pointer; that is, /L applies only to the expression on which  it         feature  is  not  available  when  the POSIX interface to PCRE is being
238         appears.         used, that is, when the /P pattern modifier is specified. See also  the
239           section about saving and reloading compiled patterns below.
240    
241         The  /I  modifier  requests  that pcretest output information about the         The  /I  modifier  requests  that pcretest output information about the
242         compiled pattern (whether it is anchored, has a fixed first  character,         compiled pattern (whether it is anchored, has a fixed first  character,
# Line 237  PATTERN MODIFIERS Line 244  PATTERN MODIFIERS
244         pattern. If the pattern is studied, the results of that are  also  out-         pattern. If the pattern is studied, the results of that are  also  out-
245         put.         put.
246    
247         The  /D modifier is a PCRE debugging feature, and is equivalent to /BI,         The  /K modifier requests pcretest to show names from backtracking con-
248         that is, both the /B and the /I modifiers.         trol verbs that are returned  from  calls  to  pcre_exec().  It  causes
249           pcretest  to create a pcre_extra block if one has not already been cre-
250           ated by a call to pcre_study(), and to set the PCRE_EXTRA_MARK flag and
251           the mark field within it, every time that pcre_exec() is called. If the
252           variable that the mark field points to is non-NULL for  a  match,  non-
253           match, or partial match, pcretest prints the string to which it points.
254           For a match, this is shown on a line by itself, tagged with "MK:".  For
255           a non-match it is added to the message.
256    
257         The /F modifier causes pcretest to flip the byte order of the fields in         The  /L modifier must be followed directly by the name of a locale, for
258         the  compiled  pattern  that  contain  2-byte  and 4-byte numbers. This         example,
        facility is for testing the feature in PCRE that allows it  to  execute  
        patterns that were compiled on a host with a different endianness. This  
        feature is not available when the POSIX  interface  to  PCRE  is  being  
        used,  that is, when the /P pattern modifier is specified. See also the  
        section about saving and reloading compiled patterns below.  
259    
260         The /S modifier causes pcre_study() to be called after  the  expression           /pattern/Lfr_FR
261         has been compiled, and the results used when the expression is matched.  
262           For this reason, it must be the last modifier. The given locale is set,
263           pcre_maketables()  is called to build a set of character tables for the
264           locale, and this is then passed to pcre_compile()  when  compiling  the
265           regular  expression.  Without an /L (or /T) modifier, NULL is passed as
266           the tables pointer; that is, /L applies only to the expression on which
267           it appears.
268    
269         The  /M  modifier causes the size of memory block used to hold the com-         The  /M  modifier causes the size of memory block used to hold the com-
270         piled pattern to be output.         piled pattern to be output.
271    
272         The /P modifier causes pcretest to call PCRE via the POSIX wrapper  API         The /S modifier causes pcre_study() to be called after  the  expression
273         rather  than  its  native  API.  When this is done, all other modifiers         has been compiled, and the results used when the expression is matched.
274         except /i, /m, and /+ are ignored. REG_ICASE is set if /i  is  present,  
275         and  REG_NEWLINE  is  set if /m is present. The wrapper functions force         The  /T  modifier  must be followed by a single digit. It causes a spe-
276         PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.         cific set of built-in character tables to be passed to  pcre_compile().
277           It is used in the standard PCRE tests to check behaviour with different
278         The /8 modifier causes pcretest to call PCRE with the PCRE_UTF8  option         character tables. The digit specifies the tables as follows:
279         set.  This  turns on support for UTF-8 character handling in PCRE, pro-  
280         vided that it was compiled with this  support  enabled.  This  modifier           0   the default ASCII tables, as distributed in
281         also causes any non-printing characters in output strings to be printed                 pcre_chartables.c.dist
282         using the \x{hh...} notation if they are valid UTF-8 sequences.           1   a set of tables defining ISO 8859 characters
283    
284         If the /? modifier  is  used  with  /8,  it  causes  pcretest  to  call         In table 1, some characters whose codes are greater than 128 are  iden-
285         pcre_compile()  with  the  PCRE_NO_UTF8_CHECK  option,  to suppress the         tified as letters, digits, spaces, etc.
286         checking of the string for UTF-8 validity.  
287       Using the POSIX wrapper API
288    
289           The  /P modifier causes pcretest to call PCRE via the POSIX wrapper API
290           rather than its native API. When /P is set, the following modifiers set
291           options for the regcomp() function:
292    
293             /i    REG_ICASE
294             /m    REG_NEWLINE
295             /N    REG_NOSUB
296             /s    REG_DOTALL     )
297             /U    REG_UNGREEDY   ) These options are not part of
298             /W    REG_UCP        )   the POSIX standard
299             /8    REG_UTF8       )
300    
301           The  /+  modifier  works  as  described  above. All other modifiers are
302           ignored.
303    
304    
305  DATA LINES  DATA LINES
# Line 341  DATA LINES Line 372  DATA LINES
372                        or pcre_dfa_exec()                        or pcre_dfa_exec()
373           \?         pass the PCRE_NO_UTF8_CHECK option to           \?         pass the PCRE_NO_UTF8_CHECK option to
374                        pcre_exec() or pcre_dfa_exec()                        pcre_exec() or pcre_dfa_exec()
375           \>dd       start the match at offset dd (any number of digits);           \>dd       start the match at offset dd (optional "-"; then
376                        this sets the startoffset argument for pcre_exec()                        any number of digits); this sets the startoffset
377                        or pcre_dfa_exec()                        argument for pcre_exec() or pcre_dfa_exec()
378           \<cr>      pass the PCRE_NEWLINE_CR option to pcre_exec()           \<cr>      pass the PCRE_NEWLINE_CR option to pcre_exec()
379                        or pcre_dfa_exec()                        or pcre_dfa_exec()
380           \<lf>      pass the PCRE_NEWLINE_LF option to pcre_exec()           \<lf>      pass the PCRE_NEWLINE_LF option to pcre_exec()
# Line 382  DATA LINES Line 413  DATA LINES
413    
414         If the /P modifier was present on the pattern, causing the POSIX  wrap-         If the /P modifier was present on the pattern, causing the POSIX  wrap-
415         per  API  to  be  used, the only option-setting sequences that have any         per  API  to  be  used, the only option-setting sequences that have any
416         effect are \B and \Z, causing REG_NOTBOL and REG_NOTEOL,  respectively,         effect are \B,  \N,  and  \Z,  causing  REG_NOTBOL,  REG_NOTEMPTY,  and
417         to be passed to regexec().         REG_NOTEOL, respectively, to be passed to regexec().
418    
419         The  use of \x{hh...} to represent UTF-8 characters is not dependent on         The  use of \x{hh...} to represent UTF-8 characters is not dependent on
420         the use of the /8 modifier on the pattern.  It  is  recognized  always.         the use of the /8 modifier on the pattern.  It  is  recognized  always.
# Line 420  DEFAULT OUTPUT FROM PCRETEST Line 451  DEFAULT OUTPUT FROM PCRETEST
451         matched  the  whole  pattern. Otherwise, it outputs "No match" when the         matched  the  whole  pattern. Otherwise, it outputs "No match" when the
452         return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the par-         return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the par-
453         tially  matching substring when pcre_exec() returns PCRE_ERROR_PARTIAL.         tially  matching substring when pcre_exec() returns PCRE_ERROR_PARTIAL.
454         For any other returns, it outputs the PCRE negative error number.  Here         (Note that this is the entire substring that was inspected  during  the
455         is an example of an interactive pcretest run.         partial  match; it may include characters before the actual match start
456           if a lookbehind assertion, \K, \b, or \B was involved.) For  any  other
457           returns,  it outputs the PCRE negative error number. Here is an example
458           of an interactive pcretest run.
459    
460           $ pcretest           $ pcretest
461           PCRE version 7.0 30-Nov-2006           PCRE version 7.0 30-Nov-2006
# Line 433  DEFAULT OUTPUT FROM PCRETEST Line 467  DEFAULT OUTPUT FROM PCRETEST
467           data> xyz           data> xyz
468           No match           No match
469    
470         Note  that unset capturing substrings that are not followed by one that         Note that unset capturing substrings that are not followed by one  that
471         is set are not returned by pcre_exec(), and are not shown by  pcretest.         is  set are not returned by pcre_exec(), and are not shown by pcretest.
472         In  the following example, there are two capturing substrings, but when         In the following example, there are two capturing substrings, but  when
473         the first data line is matched, the  second,  unset  substring  is  not         the  first  data  line  is  matched, the second, unset substring is not
474         shown.  An "internal" unset substring is shown as "<unset>", as for the         shown. An "internal" unset substring is shown as "<unset>", as for  the
475         second data line.         second data line.
476    
477             re> /(a)|(b)/             re> /(a)|(b)/
# Line 449  DEFAULT OUTPUT FROM PCRETEST Line 483  DEFAULT OUTPUT FROM PCRETEST
483            1: <unset>            1: <unset>
484            2: b            2: b
485    
486         If the strings contain any non-printing characters, they are output  as         If  the strings contain any non-printing characters, they are output as
487         \0x  escapes,  or  as \x{...} escapes if the /8 modifier was present on         \0x escapes, or as \x{...} escapes if the /8 modifier  was  present  on
488         the pattern. See below for the definition of  non-printing  characters.         the  pattern.  See below for the definition of non-printing characters.
489         If  the pattern has the /+ modifier, the output for substring 0 is fol-         If the pattern has the /+ modifier, the output for substring 0 is  fol-
490         lowed by the the rest of the subject string, identified  by  "0+"  like         lowed  by  the  the rest of the subject string, identified by "0+" like
491         this:         this:
492    
493             re> /cat/+             re> /cat/+
# Line 461  DEFAULT OUTPUT FROM PCRETEST Line 495  DEFAULT OUTPUT FROM PCRETEST
495            0: cat            0: cat
496            0+ aract            0+ aract
497    
498         If  the  pattern  has  the /g or /G modifier, the results of successive         If the pattern has the /g or /G modifier,  the  results  of  successive
499         matching attempts are output in sequence, like this:         matching attempts are output in sequence, like this:
500    
501             re> /\Bi(\w\w)/g             re> /\Bi(\w\w)/g
# Line 475  DEFAULT OUTPUT FROM PCRETEST Line 509  DEFAULT OUTPUT FROM PCRETEST
509    
510         "No match" is output only if the first match attempt fails.         "No match" is output only if the first match attempt fails.
511    
512         If any of the sequences \C, \G, or \L are present in a data  line  that         If  any  of the sequences \C, \G, or \L are present in a data line that
513         is  successfully  matched,  the substrings extracted by the convenience         is successfully matched, the substrings extracted  by  the  convenience
514         functions are output with C, G, or L after the string number instead of         functions are output with C, G, or L after the string number instead of
515         a colon. This is in addition to the normal full list. The string length         a colon. This is in addition to the normal full list. The string length
516         (that is, the return from the extraction function) is given  in  paren-         (that  is,  the return from the extraction function) is given in paren-
517         theses after each string for \C and \G.         theses after each string for \C and \G.
518    
519         Note that whereas patterns can be continued over several lines (a plain         Note that whereas patterns can be continued over several lines (a plain
520         ">" prompt is used for continuations), data lines may not. However new-         ">" prompt is used for continuations), data lines may not. However new-
521         lines  can  be included in data by means of the \n escape (or \r, \r\n,         lines can be included in data by means of the \n escape (or  \r,  \r\n,
522         etc., depending on the newline sequence setting).         etc., depending on the newline sequence setting).
523    
524    
525  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION
526    
527         When the alternative matching function, pcre_dfa_exec(),  is  used  (by         When  the  alternative  matching function, pcre_dfa_exec(), is used (by
528         means  of  the \D escape sequence or the -dfa command line option), the         means of the \D escape sequence or the -dfa command line  option),  the
529         output consists of a list of all the matches that start  at  the  first         output  consists  of  a list of all the matches that start at the first
530         point in the subject where there is at least one match. For example:         point in the subject where there is at least one match. For example:
531    
532             re> /(tang|tangerine|tan)/             re> /(tang|tangerine|tan)/
# Line 501  OUTPUT FROM THE ALTERNATIVE MATCHING FUN Line 535  OUTPUT FROM THE ALTERNATIVE MATCHING FUN
535            1: tang            1: tang
536            2: tan            2: tan
537    
538         (Using  the  normal  matching function on this data finds only "tang".)         (Using the normal matching function on this data  finds  only  "tang".)
539         The longest matching string is always given first (and numbered  zero).         The  longest matching string is always given first (and numbered zero).
540         After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol-         After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol-
541         lowed by the partially matching substring.         lowed  by  the  partially  matching  substring.  (Note that this is the
542           entire substring that was inspected during the partial  match;  it  may
543           include characters before the actual match start if a lookbehind asser-
544           tion, \K, \b, or \B was involved.)
545    
546         If /g is present on the pattern, the search for further matches resumes         If /g is present on the pattern, the search for further matches resumes
547         at the end of the longest match. For example:         at the end of the longest match. For example:
# Line 663  AUTHOR Line 700  AUTHOR
700    
701  REVISION  REVISION
702    
703         Last updated: 26 September 2009         Last updated: 06 November 2010
704         Copyright (c) 1997-2009 University of Cambridge.         Copyright (c) 1997-2010 University of Cambridge.

Legend:
Removed from v.461  
changed lines
  Added in v.567

  ViewVC Help
Powered by ViewVC 1.1.5