/[pcre]/code/trunk/doc/pcretest.txt
ViewVC logotype

Diff of /code/trunk/doc/pcretest.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 429 by ph10, Tue Sep 1 16:10:16 2009 UTC revision 572 by ph10, Wed Nov 17 17:55:57 2010 UTC
# Line 154  PATTERN MODIFIERS Line 154  PATTERN MODIFIERS
154    
155           /caseless/i           /caseless/i
156    
157         The following table shows additional modifiers for setting PCRE options         The  following  table  shows additional modifiers for setting PCRE com-
158         that do not correspond to anything in Perl:         pile-time options that do not correspond to anything in Perl:
159    
160             /8              PCRE_UTF8
161             /?              PCRE_NO_UTF8_CHECK
162           /A              PCRE_ANCHORED           /A              PCRE_ANCHORED
163           /C              PCRE_AUTO_CALLOUT           /C              PCRE_AUTO_CALLOUT
164           /E              PCRE_DOLLAR_ENDONLY           /E              PCRE_DOLLAR_ENDONLY
# Line 164  PATTERN MODIFIERS Line 166  PATTERN MODIFIERS
166           /J              PCRE_DUPNAMES           /J              PCRE_DUPNAMES
167           /N              PCRE_NO_AUTO_CAPTURE           /N              PCRE_NO_AUTO_CAPTURE
168           /U              PCRE_UNGREEDY           /U              PCRE_UNGREEDY
169             /W              PCRE_UCP
170           /X              PCRE_EXTRA           /X              PCRE_EXTRA
171           /<JS>           PCRE_JAVASCRIPT_COMPAT           /<JS>           PCRE_JAVASCRIPT_COMPAT
172           /<cr>           PCRE_NEWLINE_CR           /<cr>           PCRE_NEWLINE_CR
# Line 174  PATTERN MODIFIERS Line 177  PATTERN MODIFIERS
177           /<bsr_anycrlf>  PCRE_BSR_ANYCRLF           /<bsr_anycrlf>  PCRE_BSR_ANYCRLF
178           /<bsr_unicode>  PCRE_BSR_UNICODE           /<bsr_unicode>  PCRE_BSR_UNICODE
179    
180         Those specifying line ending sequences are literal  strings  as  shown,         The modifiers that are enclosed in angle brackets are  literal  strings
181         but  the  letters  can  be  in either case. This example sets multiline         as  shown,  including  the  angle  brackets,  but the letters can be in
182         matching with CRLF as the line ending sequence:         either case. This example sets multiline matching with CRLF as the line
183           ending sequence:
184    
185           /^abc/m<crlf>           /^abc/m<crlf>
186    
187         Details of the meanings of these PCRE options are given in the  pcreapi         As well as turning on the PCRE_UTF8 option, the /8 modifier also causes
188         documentation.         any non-printing characters in output strings to be printed  using  the
189           \x{hh...}  notation  if they are valid UTF-8 sequences. Full details of
190           the PCRE options are given in the pcreapi documentation.
191    
192     Finding all matches in a string     Finding all matches in a string
193    
194         Searching  for  all  possible matches within each subject string can be         Searching for all possible matches within each subject  string  can  be
195         requested by the /g or /G modifier. After  finding  a  match,  PCRE  is         requested  by  the  /g  or  /G modifier. After finding a match, PCRE is
196         called again to search the remainder of the subject string. The differ-         called again to search the remainder of the subject string. The differ-
197         ence between /g and /G is that the former uses the startoffset argument         ence between /g and /G is that the former uses the startoffset argument
198         to  pcre_exec()  to  start  searching  at a new point within the entire         to pcre_exec() to start searching at a  new  point  within  the  entire
199         string (which is in effect what Perl does), whereas the  latter  passes         string  (which  is in effect what Perl does), whereas the latter passes
200         over  a  shortened  substring.  This makes a difference to the matching         over a shortened substring. This makes a  difference  to  the  matching
201         process if the pattern begins with a lookbehind assertion (including \b         process if the pattern begins with a lookbehind assertion (including \b
202         or \B).         or \B).
203    
204         If  any  call  to  pcre_exec()  in a /g or /G sequence matches an empty         If any call to pcre_exec() in a /g or  /G  sequence  matches  an  empty
205         string, the next call is done with the PCRE_NOTEMPTY and  PCRE_ANCHORED         string,  the  next  call  is  done  with  the PCRE_NOTEMPTY_ATSTART and
206         flags  set in order to search for another, non-empty, match at the same         PCRE_ANCHORED flags set in order  to  search  for  another,  non-empty,
207         point.  If this second match fails, the start  offset  is  advanced  by         match  at  the same point. If this second match fails, the start offset
208         one,  and  the normal match is retried. This imitates the way Perl han-         is advanced, and the normal match is retried.  This  imitates  the  way
209         dles such cases when using the /g modifier or the split() function.         Perl handles such cases when using the /g modifier or the split() func-
210           tion. Normally, the start offset is advanced by one character,  but  if
211           the  newline  convention  recognizes CRLF as a newline, and the current
212           character is CR followed by LF, an advance of two is used.
213    
214     Other modifiers     Other modifiers
215    
# Line 218  PATTERN MODIFIERS Line 227  PATTERN MODIFIERS
227         feature for use in the automatic test scripts; it ensures that the same         feature for use in the automatic test scripts; it ensures that the same
228         output is generated for different internal link sizes.         output is generated for different internal link sizes.
229    
        The /L modifier must be followed directly by the name of a locale,  for  
        example,  
   
          /pattern/Lfr_FR  
   
        For this reason, it must be the last modifier. The given locale is set,  
        pcre_maketables() is called to build a set of character tables for  the  
        locale,  and  this  is then passed to pcre_compile() when compiling the  
        regular expression. Without an /L  modifier,  NULL  is  passed  as  the  
        tables  pointer; that is, /L applies only to the expression on which it  
        appears.  
   
        The /I modifier requests that pcretest  output  information  about  the  
        compiled  pattern (whether it is anchored, has a fixed first character,  
        and so on). It does this by calling pcre_fullinfo() after  compiling  a  
        pattern.  If  the pattern is studied, the results of that are also out-  
        put.  
   
230         The /D modifier is a PCRE debugging feature, and is equivalent to  /BI,         The /D modifier is a PCRE debugging feature, and is equivalent to  /BI,
231         that is, both the /B and the /I modifiers.         that is, both the /B and the /I modifiers.
232    
# Line 247  PATTERN MODIFIERS Line 238  PATTERN MODIFIERS
238         used, that is, when the /P pattern modifier is specified. See also  the         used, that is, when the /P pattern modifier is specified. See also  the
239         section about saving and reloading compiled patterns below.         section about saving and reloading compiled patterns below.
240    
241         The  /S  modifier causes pcre_study() to be called after the expression         The  /I  modifier  requests  that pcretest output information about the
242         has been compiled, and the results used when the expression is matched.         compiled pattern (whether it is anchored, has a fixed first  character,
243           and  so  on). It does this by calling pcre_fullinfo() after compiling a
244           pattern. If the pattern is studied, the results of that are  also  out-
245           put.
246    
247           The  /K modifier requests pcretest to show names from backtracking con-
248           trol verbs that are returned  from  calls  to  pcre_exec().  It  causes
249           pcretest  to create a pcre_extra block if one has not already been cre-
250           ated by a call to pcre_study(), and to set the PCRE_EXTRA_MARK flag and
251           the mark field within it, every time that pcre_exec() is called. If the
252           variable that the mark field points to is non-NULL for  a  match,  non-
253           match, or partial match, pcretest prints the string to which it points.
254           For a match, this is shown on a line by itself, tagged with "MK:".  For
255           a non-match it is added to the message.
256    
257           The  /L modifier must be followed directly by the name of a locale, for
258           example,
259    
260             /pattern/Lfr_FR
261    
262         The /M modifier causes the size of memory block used to hold  the  com-         For this reason, it must be the last modifier. The given locale is set,
263           pcre_maketables()  is called to build a set of character tables for the
264           locale, and this is then passed to pcre_compile()  when  compiling  the
265           regular  expression.  Without an /L (or /T) modifier, NULL is passed as
266           the tables pointer; that is, /L applies only to the expression on which
267           it appears.
268    
269           The  /M  modifier causes the size of memory block used to hold the com-
270         piled pattern to be output.         piled pattern to be output.
271    
272           The /S modifier causes pcre_study() to be called after  the  expression
273           has been compiled, and the results used when the expression is matched.
274    
275           The  /T  modifier  must be followed by a single digit. It causes a spe-
276           cific set of built-in character tables to be passed to  pcre_compile().
277           It is used in the standard PCRE tests to check behaviour with different
278           character tables. The digit specifies the tables as follows:
279    
280             0   the default ASCII tables, as distributed in
281                   pcre_chartables.c.dist
282             1   a set of tables defining ISO 8859 characters
283    
284           In table 1, some characters whose codes are greater than 128 are  iden-
285           tified as letters, digits, spaces, etc.
286    
287       Using the POSIX wrapper API
288    
289         The  /P modifier causes pcretest to call PCRE via the POSIX wrapper API         The  /P modifier causes pcretest to call PCRE via the POSIX wrapper API
290         rather than its native API. When this  is  done,  all  other  modifiers         rather than its native API. When /P is set, the following modifiers set
291         except  /i,  /m, and /+ are ignored. REG_ICASE is set if /i is present,         options for the regcomp() function:
292         and REG_NEWLINE is set if /m is present. The  wrapper  functions  force  
293         PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.           /i    REG_ICASE
294             /m    REG_NEWLINE
295         The  /8 modifier causes pcretest to call PCRE with the PCRE_UTF8 option           /N    REG_NOSUB
296         set. This turns on support for UTF-8 character handling in  PCRE,  pro-           /s    REG_DOTALL     )
297         vided  that  it  was  compiled with this support enabled. This modifier           /U    REG_UNGREEDY   ) These options are not part of
298         also causes any non-printing characters in output strings to be printed           /W    REG_UCP        )   the POSIX standard
299         using the \x{hh...} notation if they are valid UTF-8 sequences.           /8    REG_UTF8       )
300    
301         If  the  /?  modifier  is  used  with  /8,  it  causes pcretest to call         The  /+  modifier  works  as  described  above. All other modifiers are
302         pcre_compile() with the  PCRE_NO_UTF8_CHECK  option,  to  suppress  the         ignored.
        checking of the string for UTF-8 validity.  
303    
304    
305  DATA LINES  DATA LINES
306    
307         Before  each  data  line is passed to pcre_exec(), leading and trailing         Before each data line is passed to pcre_exec(),  leading  and  trailing
308         whitespace is removed, and it is then scanned for \  escapes.  Some  of         whitespace  is  removed,  and it is then scanned for \ escapes. Some of
309         these  are  pretty esoteric features, intended for checking out some of         these are pretty esoteric features, intended for checking out  some  of
310         the more complicated features of PCRE. If you are just  testing  "ordi-         the  more  complicated features of PCRE. If you are just testing "ordi-
311         nary"  regular  expressions,  you probably don't need any of these. The         nary" regular expressions, you probably don't need any  of  these.  The
312         following escapes are recognized:         following escapes are recognized:
313    
314           \a         alarm (BEL, \x07)           \a         alarm (BEL, \x07)
# Line 290  DATA LINES Line 322  DATA LINES
322           \t         tab (\x09)           \t         tab (\x09)
323           \v         vertical tab (\x0b)           \v         vertical tab (\x0b)
324           \nnn       octal character (up to 3 octal digits)           \nnn       octal character (up to 3 octal digits)
325           \xhh       hexadecimal character (up to 2 hex digits)                        always a byte unless > 255 in UTF-8 mode
326             \xhh       hexadecimal byte (up to 2 hex digits)
327           \x{hh...}  hexadecimal character, any number of digits           \x{hh...}  hexadecimal character, any number of digits
328                        in UTF-8 mode                        in UTF-8 mode
329           \A         pass the PCRE_ANCHORED option to pcre_exec()           \A         pass the PCRE_ANCHORED option to pcre_exec()
# Line 323  DATA LINES Line 356  DATA LINES
356           \M         discover the minimum MATCH_LIMIT and           \M         discover the minimum MATCH_LIMIT and
357                        MATCH_LIMIT_RECURSION settings                        MATCH_LIMIT_RECURSION settings
358           \N         pass the PCRE_NOTEMPTY option to pcre_exec()           \N         pass the PCRE_NOTEMPTY option to pcre_exec()
359                        or pcre_dfa_exec()                        or pcre_dfa_exec(); if used twice, pass the
360                          PCRE_NOTEMPTY_ATSTART option
361           \Odd       set the size of the output vector passed to           \Odd       set the size of the output vector passed to
362                        pcre_exec() to dd (any number of digits)                        pcre_exec() to dd (any number of digits)
363           \P         pass the PCRE_PARTIAL_SOFT option to pcre_exec()           \P         pass the PCRE_PARTIAL_SOFT option to pcre_exec()
# Line 333  DATA LINES Line 367  DATA LINES
367                        (any number of digits)                        (any number of digits)
368           \R         pass the PCRE_DFA_RESTART option to pcre_dfa_exec()           \R         pass the PCRE_DFA_RESTART option to pcre_dfa_exec()
369           \S         output details of memory get/free calls during matching           \S         output details of memory get/free calls during matching
370             \Y         pass the PCRE_NO_START_OPTIMIZE option to pcre_exec()
371                          or pcre_dfa_exec()
372           \Z         pass the PCRE_NOTEOL option to pcre_exec()           \Z         pass the PCRE_NOTEOL option to pcre_exec()
373                        or pcre_dfa_exec()                        or pcre_dfa_exec()
374           \?         pass the PCRE_NO_UTF8_CHECK option to           \?         pass the PCRE_NO_UTF8_CHECK option to
375                        pcre_exec() or pcre_dfa_exec()                        pcre_exec() or pcre_dfa_exec()
376           \>dd       start the match at offset dd (any number of digits);           \>dd       start the match at offset dd (optional "-"; then
377                        this sets the startoffset argument for pcre_exec()                        any number of digits); this sets the startoffset
378                        or pcre_dfa_exec()                        argument for pcre_exec() or pcre_dfa_exec()
379           \<cr>      pass the PCRE_NEWLINE_CR option to pcre_exec()           \<cr>      pass the PCRE_NEWLINE_CR option to pcre_exec()
380                        or pcre_dfa_exec()                        or pcre_dfa_exec()
381           \<lf>      pass the PCRE_NEWLINE_LF option to pcre_exec()           \<lf>      pass the PCRE_NEWLINE_LF option to pcre_exec()
# Line 351  DATA LINES Line 387  DATA LINES
387           \<any>     pass the PCRE_NEWLINE_ANY option to pcre_exec()           \<any>     pass the PCRE_NEWLINE_ANY option to pcre_exec()
388                        or pcre_dfa_exec()                        or pcre_dfa_exec()
389    
390           Note  that  \xhh  always  specifies  one byte, even in UTF-8 mode; this
391           makes it possible to construct invalid UTF-8 sequences for testing pur-
392           poses. On the other hand, \x{hh} is interpreted as a UTF-8 character in
393           UTF-8 mode, generating more than one byte if the value is greater  than
394           127. When not in UTF-8 mode, it generates one byte for values less than
395           256, and causes an error for greater values.
396    
397         The escapes that specify line ending  sequences  are  literal  strings,         The escapes that specify line ending  sequences  are  literal  strings,
398         exactly as shown. No more than one newline setting should be present in         exactly as shown. No more than one newline setting should be present in
399         any data line.         any data line.
# Line 378  DATA LINES Line 421  DATA LINES
421    
422         If  the /P modifier was present on the pattern, causing the POSIX wrap-         If  the /P modifier was present on the pattern, causing the POSIX wrap-
423         per API to be used, the only option-setting  sequences  that  have  any         per API to be used, the only option-setting  sequences  that  have  any
424         effect  are \B and \Z, causing REG_NOTBOL and REG_NOTEOL, respectively,         effect  are  \B,  \N,  and  \Z,  causing  REG_NOTBOL, REG_NOTEMPTY, and
425         to be passed to regexec().         REG_NOTEOL, respectively, to be passed to regexec().
426    
427         The use of \x{hh...} to represent UTF-8 characters is not dependent  on         The use of \x{hh...} to represent UTF-8 characters is not dependent  on
428         the  use  of  the  /8 modifier on the pattern. It is recognized always.         the  use  of  the  /8 modifier on the pattern. It is recognized always.
# Line 413  DEFAULT OUTPUT FROM PCRETEST Line 456  DEFAULT OUTPUT FROM PCRETEST
456    
457         When a match succeeds, pcretest outputs the list of captured substrings         When a match succeeds, pcretest outputs the list of captured substrings
458         that  pcre_exec()  returns,  starting with number 0 for the string that         that  pcre_exec()  returns,  starting with number 0 for the string that
459         matched the whole pattern. Otherwise, it outputs "No match" or "Partial         matched the whole pattern. Otherwise, it outputs "No  match"  when  the
460         match:"  followed  by the partially matching substring when pcre_exec()         return is PCRE_ERROR_NOMATCH, and "Partial match:" followed by the par-
461         returns PCRE_ERROR_NOMATCH  or  PCRE_ERROR_PARTIAL,  respectively,  and         tially matching substring when pcre_exec() returns  PCRE_ERROR_PARTIAL.
462         otherwise  the  PCRE  negative  error  number. Here is an example of an         (Note  that  this is the entire substring that was inspected during the
463         interactive pcretest run.         partial match; it may include characters before the actual match  start
464           if  a  lookbehind assertion, \K, \b, or \B was involved.) For any other
465           returns, it outputs the PCRE negative error number. Here is an  example
466           of an interactive pcretest run.
467    
468           $ pcretest           $ pcretest
469           PCRE version 7.0 30-Nov-2006           PCRE version 7.0 30-Nov-2006
# Line 429  DEFAULT OUTPUT FROM PCRETEST Line 475  DEFAULT OUTPUT FROM PCRETEST
475           data> xyz           data> xyz
476           No match           No match
477    
478         Note that unset capturing substrings that are not followed by one  that         Note  that unset capturing substrings that are not followed by one that
479         is  set are not returned by pcre_exec(), and are not shown by pcretest.         is set are not returned by pcre_exec(), and are not shown by  pcretest.
480         In the following example, there are two capturing substrings, but  when         In  the following example, there are two capturing substrings, but when
481         the  first  data  line  is  matched, the second, unset substring is not         the first data line is matched, the  second,  unset  substring  is  not
482         shown. An "internal" unset substring is shown as "<unset>", as for  the         shown.  An "internal" unset substring is shown as "<unset>", as for the
483         second data line.         second data line.
484    
485             re> /(a)|(b)/             re> /(a)|(b)/
# Line 445  DEFAULT OUTPUT FROM PCRETEST Line 491  DEFAULT OUTPUT FROM PCRETEST
491            1: <unset>            1: <unset>
492            2: b            2: b
493    
494         If  the strings contain any non-printing characters, they are output as         If the strings contain any non-printing characters, they are output  as
495         \0x escapes, or as \x{...} escapes if the /8 modifier  was  present  on         \0x  escapes,  or  as \x{...} escapes if the /8 modifier was present on
496         the  pattern.  See below for the definition of non-printing characters.         the pattern. See below for the definition of  non-printing  characters.
497         If the pattern has the /+ modifier, the output for substring 0 is  fol-         If  the pattern has the /+ modifier, the output for substring 0 is fol-
498         lowed  by  the  the rest of the subject string, identified by "0+" like         lowed by the the rest of the subject string, identified  by  "0+"  like
499         this:         this:
500    
501             re> /cat/+             re> /cat/+
# Line 457  DEFAULT OUTPUT FROM PCRETEST Line 503  DEFAULT OUTPUT FROM PCRETEST
503            0: cat            0: cat
504            0+ aract            0+ aract
505    
506         If the pattern has the /g or /G modifier,  the  results  of  successive         If  the  pattern  has  the /g or /G modifier, the results of successive
507         matching attempts are output in sequence, like this:         matching attempts are output in sequence, like this:
508    
509             re> /\Bi(\w\w)/g             re> /\Bi(\w\w)/g
# Line 471  DEFAULT OUTPUT FROM PCRETEST Line 517  DEFAULT OUTPUT FROM PCRETEST
517    
518         "No match" is output only if the first match attempt fails.         "No match" is output only if the first match attempt fails.
519    
520         If  any  of the sequences \C, \G, or \L are present in a data line that         If any of the sequences \C, \G, or \L are present in a data  line  that
521         is successfully matched, the substrings extracted  by  the  convenience         is  successfully  matched,  the substrings extracted by the convenience
522         functions are output with C, G, or L after the string number instead of         functions are output with C, G, or L after the string number instead of
523         a colon. This is in addition to the normal full list. The string length         a colon. This is in addition to the normal full list. The string length
524         (that  is,  the return from the extraction function) is given in paren-         (that is, the return from the extraction function) is given  in  paren-
525         theses after each string for \C and \G.         theses after each string for \C and \G.
526    
527         Note that whereas patterns can be continued over several lines (a plain         Note that whereas patterns can be continued over several lines (a plain
528         ">" prompt is used for continuations), data lines may not. However new-         ">" prompt is used for continuations), data lines may not. However new-
529         lines can be included in data by means of the \n escape (or  \r,  \r\n,         lines  can  be included in data by means of the \n escape (or \r, \r\n,
530         etc., depending on the newline sequence setting).         etc., depending on the newline sequence setting).
531    
532    
533  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION
534    
535         When  the  alternative  matching function, pcre_dfa_exec(), is used (by         When the alternative matching function, pcre_dfa_exec(),  is  used  (by
536         means of the \D escape sequence or the -dfa command line  option),  the         means  of  the \D escape sequence or the -dfa command line option), the
537         output  consists  of  a list of all the matches that start at the first         output consists of a list of all the matches that start  at  the  first
538         point in the subject where there is at least one match. For example:         point in the subject where there is at least one match. For example:
539    
540             re> /(tang|tangerine|tan)/             re> /(tang|tangerine|tan)/
# Line 497  OUTPUT FROM THE ALTERNATIVE MATCHING FUN Line 543  OUTPUT FROM THE ALTERNATIVE MATCHING FUN
543            1: tang            1: tang
544            2: tan            2: tan
545    
546         (Using the normal matching function on this data  finds  only  "tang".)         (Using  the  normal  matching function on this data finds only "tang".)
547         The  longest matching string is always given first (and numbered zero).         The longest matching string is always given first (and numbered  zero).
548         After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol-         After a PCRE_ERROR_PARTIAL return, the output is "Partial match:", fol-
549         lowed by the partially matching substring.         lowed by the partially matching  substring.  (Note  that  this  is  the
550           entire  substring  that  was inspected during the partial match; it may
551           include characters before the actual match start if a lookbehind asser-
552           tion, \K, \b, or \B was involved.)
553    
554         If /g is present on the pattern, the search for further matches resumes         If /g is present on the pattern, the search for further matches resumes
555         at the end of the longest match. For example:         at the end of the longest match. For example:
# Line 659  AUTHOR Line 708  AUTHOR
708    
709  REVISION  REVISION
710    
711         Last updated: 29 August 2009         Last updated: 07 November 2010
712         Copyright (c) 1997-2009 University of Cambridge.         Copyright (c) 1997-2010 University of Cambridge.

Legend:
Removed from v.429  
changed lines
  Added in v.572

  ViewVC Help
Powered by ViewVC 1.1.5