/[pcre]/code/trunk/doc/pcretest.txt
ViewVC logotype

Diff of /code/trunk/doc/pcretest.txt

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 77 by nigel, Sat Feb 24 21:40:45 2007 UTC revision 152 by ph10, Tue Apr 17 15:55:53 2007 UTC
# Line 1  Line 1 
1  PCRETEST(1)                                                        PCRETEST(1)  PCRETEST(1)                                                        PCRETEST(1)
2    
3    
   
4  NAME  NAME
5         pcretest - a program for testing Perl-compatible regular expressions.         pcretest - a program for testing Perl-compatible regular expressions.
6    
7    
8  SYNOPSIS  SYNOPSIS
9    
10         pcretest [-C] [-d] [-dfa] [-i] [-m] [-o osize] [-p] [-t] [source]         pcretest [options] [source] [destination]
             [destination]  
11    
12         pcretest  was written as a test program for the PCRE regular expression         pcretest  was written as a test program for the PCRE regular expression
13         library itself, but it can also be used for experimenting with  regular         library itself, but it can also be used for experimenting with  regular
# Line 20  SYNOPSIS Line 19  SYNOPSIS
19    
20  OPTIONS  OPTIONS
21    
22           -b        Behave as if each regex has the /B (show bytecode)  modifier;
23                     the internal form is output after compilation.
24    
25         -C        Output the version number of the PCRE library, and all avail-         -C        Output the version number of the PCRE library, and all avail-
26                   able   information  about  the  optional  features  that  are                   able  information  about  the  optional  features  that   are
27                   included, and then exit.                   included, and then exit.
28    
29         -d        Behave as if each regex has  the  /D  (debug)  modifier;  the         -d        Behave  as  if  each  regex  has the /D (debug) modifier; the
30                   internal form is output after compilation.                   internal form and information about the compiled  pattern  is
31                     output after compilation; -d is equivalent to -b -i.
32    
33         -dfa      Behave  as if each data line contains the \D escape sequence;         -dfa      Behave  as if each data line contains the \D escape sequence;
34                   this    causes    the    alternative    matching    function,                   this    causes    the    alternative    matching    function,
35                   pcre_dfa_exec(),   to   be   used  instead  of  the  standard                   pcre_dfa_exec(),   to   be   used  instead  of  the  standard
36                   pcre_exec() function (more detail is given below).                   pcre_exec() function (more detail is given below).
37    
38           -help     Output a brief summary these options and then exit.
39    
40         -i        Behave as if each regex  has  the  /I  modifier;  information         -i        Behave as if each regex  has  the  /I  modifier;  information
41                   about the compiled pattern is given after compilation.                   about the compiled pattern is given after compilation.
42    
# Line 41  OPTIONS Line 46  OPTIONS
46                   pcretest, -s is a synonym for -m.                   pcretest, -s is a synonym for -m.
47    
48         -o osize  Set the number of elements in the output vector that is  used         -o osize  Set the number of elements in the output vector that is  used
49                   when  calling  pcre_exec()  to be osize. The default value is                   when  calling pcre_exec() or pcre_dfa_exec() to be osize. The
50                   45, which is enough for 14 capturing subexpressions. The vec-                   default value is 45, which is enough for 14 capturing  subex-
51                   tor  size  can  be  changed  for individual matching calls by                   pressions   for  pcre_exec()  or  22  different  matches  for
52                   including \O in the data line (see below).                   pcre_dfa_exec(). The vector size can be changed for  individ-
53                     ual  matching  calls  by  including  \O in the data line (see
54                     below).
55    
56         -p        Behave as if each regex has the /P modifier; the POSIX  wrap-         -p        Behave as if each regex has the /P modifier; the POSIX  wrap-
57                   per  API  is used to call PCRE. None of the other options has                   per  API  is used to call PCRE. None of the other options has
58                   any effect when -p is set.                   any effect when -p is set.
59    
60           -q        Do not output the version number of pcretest at the start  of
61                     execution.
62    
63           -S size   On  Unix-like  systems,  set the size of the runtime stack to
64                     size megabytes.
65    
66         -t        Run each compile, study, and match many times with  a  timer,         -t        Run each compile, study, and match many times with  a  timer,
67                   and  output resulting time per compile or match (in millisec-                   and  output resulting time per compile or match (in millisec-
68                   onds). Do not set -m with -t, because you will then  get  the                   onds). Do not set -m with -t, because you will then  get  the
69                   size  output  a  zillion  times,  and the timing will be dis-                   size  output  a  zillion  times,  and the timing will be dis-
70                   torted.                   torted. You can control the number  of  iterations  that  are
71                     used  for timing by following -t with a number (as a separate
72                     item on the command line). For example, "-t 1000" would iter-
73                     ate 1000 times. The default is to iterate 500000 times.
74    
75           -tm       This is like -t except that it times only the matching phase,
76                     not the compile or study phases.
77    
78    
79  DESCRIPTION  DESCRIPTION
# Line 71  DESCRIPTION Line 90  DESCRIPTION
90         ber of data lines to be matched against the pattern.         ber of data lines to be matched against the pattern.
91    
92         Each  data line is matched separately and independently. If you want to         Each  data line is matched separately and independently. If you want to
93         do multiple-line matches, you have to use the \n escape sequence  in  a         do multi-line matches, you have to use the \n escape sequence (or \r or
94         single  line  of  input  to  encode the newline characters. The maximum         \r\n, etc., depending on the newline setting) in a single line of input
95         length of data line is 30,000 characters.         to encode the newline sequences. There is no limit  on  the  length  of
96           data  lines;  the  input  buffer is automatically extended if it is too
97           small.
98    
99         An empty line signals the end of the data lines, at which point  a  new         An empty line signals the end of the data lines, at which point  a  new
100         regular  expression is read. The regular expressions are given enclosed         regular  expression is read. The regular expressions are given enclosed
101         in any non-alphanumeric delimiters other than backslash, for example         in any non-alphanumeric delimiters other than backslash, for example:
102    
103           /(a|bc)x+yz/           /(a|bc)x+yz/
104    
# Line 125  PATTERN MODIFIERS Line 146  PATTERN MODIFIERS
146         The following table shows additional modifiers for setting PCRE options         The following table shows additional modifiers for setting PCRE options
147         that do not correspond to anything in Perl:         that do not correspond to anything in Perl:
148    
149           /A    PCRE_ANCHORED           /A          PCRE_ANCHORED
150           /C    PCRE_AUTO_CALLOUT           /C          PCRE_AUTO_CALLOUT
151           /E    PCRE_DOLLAR_ENDONLY           /E          PCRE_DOLLAR_ENDONLY
152           /f    PCRE_FIRSTLINE           /f          PCRE_FIRSTLINE
153           /N    PCRE_NO_AUTO_CAPTURE           /J          PCRE_DUPNAMES
154           /U    PCRE_UNGREEDY           /N          PCRE_NO_AUTO_CAPTURE
155           /X    PCRE_EXTRA           /U          PCRE_UNGREEDY
156             /X          PCRE_EXTRA
157             /<cr>       PCRE_NEWLINE_CR
158             /<lf>       PCRE_NEWLINE_LF
159             /<crlf>     PCRE_NEWLINE_CRLF
160             /<anycrlf>  PCRE_NEWLINE_ANYCRLF
161             /<any>      PCRE_NEWLINE_ANY
162    
163           Those  specifying  line ending sequencess are literal strings as shown.
164           This example sets multiline matching  with  CRLF  as  the  line  ending
165           sequence:
166    
167             /^abc/m<crlf>
168    
169         Searching  for  all  possible matches within each subject string can be         Details  of the meanings of these PCRE options are given in the pcreapi
170         requested by the /g or /G modifier. After  finding  a  match,  PCRE  is         documentation.
171    
172       Finding all matches in a string
173    
174           Searching for all possible matches within each subject  string  can  be
175           requested  by  the  /g  or  /G modifier. After finding a match, PCRE is
176         called again to search the remainder of the subject string. The differ-         called again to search the remainder of the subject string. The differ-
177         ence between /g and /G is that the former uses the startoffset argument         ence between /g and /G is that the former uses the startoffset argument
178         to  pcre_exec()  to  start  searching  at a new point within the entire         to pcre_exec() to start searching at a  new  point  within  the  entire
179         string (which is in effect what Perl does), whereas the  latter  passes         string  (which  is in effect what Perl does), whereas the latter passes
180         over  a  shortened  substring.  This makes a difference to the matching         over a shortened substring. This makes a  difference  to  the  matching
181         process if the pattern begins with a lookbehind assertion (including \b         process if the pattern begins with a lookbehind assertion (including \b
182         or \B).         or \B).
183    
184         If  any  call  to  pcre_exec()  in a /g or /G sequence matches an empty         If any call to pcre_exec() in a /g or  /G  sequence  matches  an  empty
185         string, the next call is done with the PCRE_NOTEMPTY and  PCRE_ANCHORED         string,  the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED
186         flags  set in order to search for another, non-empty, match at the same         flags set in order to search for another, non-empty, match at the  same
187         point.  If this second match fails, the start  offset  is  advanced  by         point.   If  this  second  match fails, the start offset is advanced by
188         one,  and  the normal match is retried. This imitates the way Perl han-         one, and the normal match is retried. This imitates the way  Perl  han-
189         dles such cases when using the /g modifier or the split() function.         dles such cases when using the /g modifier or the split() function.
190    
191       Other modifiers
192    
193         There are yet more modifiers for controlling the way pcretest operates.         There are yet more modifiers for controlling the way pcretest operates.
194    
195         The  /+ modifier requests that as well as outputting the substring that         The /+ modifier requests that as well as outputting the substring  that
196         matched the entire pattern, pcretest  should  in  addition  output  the         matched  the  entire  pattern,  pcretest  should in addition output the
197         remainder  of  the  subject  string. This is useful for tests where the         remainder of the subject string. This is useful  for  tests  where  the
198         subject contains multiple copies of the same substring.         subject contains multiple copies of the same substring.
199    
200           The  /B modifier is a debugging feature. It requests that pcretest out-
201           put a representation of the compiled byte code after compilation.  Nor-
202           mally  this  information contains length and offset values; however, if
203           /Z is also present, this data is replaced by spaces. This is a  special
204           feature for use in the automatic test scripts; it ensures that the same
205           output is generated for different internal link sizes.
206    
207         The /L modifier must be followed directly by the name of a locale,  for         The /L modifier must be followed directly by the name of a locale,  for
208         example,         example,
209    
# Line 175  PATTERN MODIFIERS Line 222  PATTERN MODIFIERS
222         pattern.  If  the pattern is studied, the results of that are also out-         pattern.  If  the pattern is studied, the results of that are also out-
223         put.         put.
224    
225         The /D modifier is a PCRE debugging feature, which also assumes /I.  It         The /D modifier is a PCRE debugging feature, and is equivalent to  /BI,
226         causes  the  internal form of compiled regular expressions to be output         that is, both the /B and the /I modifiers.
        after compilation. If the pattern was studied, the information returned  
        is also output.  
227    
228         The /F modifier causes pcretest to flip the byte order of the fields in         The /F modifier causes pcretest to flip the byte order of the fields in
229         the compiled pattern that  contain  2-byte  and  4-byte  numbers.  This         the compiled pattern that  contain  2-byte  and  4-byte  numbers.  This
# Line 220  DATA LINES Line 265  DATA LINES
265         nary"  regular  expressions,  you probably don't need any of these. The         nary"  regular  expressions,  you probably don't need any of these. The
266         following escapes are recognized:         following escapes are recognized:
267    
268           \a         alarm (= BEL)           \a         alarm (BEL, \x07)
269           \b         backspace           \b         backspace (\x08)
270           \e         escape           \e         escape (\x27)
271           \f         formfeed           \f         formfeed (\x0c)
272           \n         newline           \n         newline (\x0a)
273           \r         carriage return           \qdd       set the PCRE_MATCH_LIMIT limit to dd
274           \t         tab                        (any number of digits)
275           \v         vertical tab           \r         carriage return (\x0d)
276             \t         tab (\x09)
277             \v         vertical tab (\x0b)
278           \nnn       octal character (up to 3 octal digits)           \nnn       octal character (up to 3 octal digits)
279           \xhh       hexadecimal character (up to 2 hex digits)           \xhh       hexadecimal character (up to 2 hex digits)
280           \x{hh...}  hexadecimal character, any number of digits           \x{hh...}  hexadecimal character, any number of digits
281                        in UTF-8 mode                        in UTF-8 mode
282           \A         pass the PCRE_ANCHORED option to pcre_exec()           \A         pass the PCRE_ANCHORED option to pcre_exec()
283                          or pcre_dfa_exec()
284           \B         pass the PCRE_NOTBOL option to pcre_exec()           \B         pass the PCRE_NOTBOL option to pcre_exec()
285                          or pcre_dfa_exec()
286           \Cdd       call pcre_copy_substring() for substring dd           \Cdd       call pcre_copy_substring() for substring dd
287                        after a successful match (number less than 32)                        after a successful match (number less than 32)
288           \Cname     call pcre_copy_named_substring() for substring           \Cname     call pcre_copy_named_substring() for substring
# Line 257  DATA LINES Line 306  DATA LINES
306                        ated by next non-alphanumeric character)                        ated by next non-alphanumeric character)
307           \L         call pcre_get_substringlist() after a           \L         call pcre_get_substringlist() after a
308                        successful match                        successful match
309           \M         discover the minimum MATCH_LIMIT setting           \M         discover the minimum MATCH_LIMIT and
310                          MATCH_LIMIT_RECURSION settings
311           \N         pass the PCRE_NOTEMPTY option to pcre_exec()           \N         pass the PCRE_NOTEMPTY option to pcre_exec()
312                          or pcre_dfa_exec()
313           \Odd       set the size of the output vector passed to           \Odd       set the size of the output vector passed to
314                        pcre_exec() to dd (any number of digits)                        pcre_exec() to dd (any number of digits)
315           \P         pass the PCRE_PARTIAL option to pcre_exec()           \P         pass the PCRE_PARTIAL option to pcre_exec()
316                        or pcre_dfa_exec()                        or pcre_dfa_exec()
317             \Qdd       set the PCRE_MATCH_LIMIT_RECURSION limit to dd
318                          (any number of digits)
319           \R         pass the PCRE_DFA_RESTART option to pcre_dfa_exec()           \R         pass the PCRE_DFA_RESTART option to pcre_dfa_exec()
320           \S         output details of memory get/free calls during matching           \S         output details of memory get/free calls during matching
321           \Z         pass the PCRE_NOTEOL option to pcre_exec()           \Z         pass the PCRE_NOTEOL option to pcre_exec()
322                          or pcre_dfa_exec()
323           \?         pass the PCRE_NO_UTF8_CHECK option to           \?         pass the PCRE_NO_UTF8_CHECK option to
324                        pcre_exec()                        pcre_exec() or pcre_dfa_exec()
325           \>dd       start the match at offset dd (any number of digits);           \>dd       start the match at offset dd (any number of digits);
326                        this sets the startoffset argument for pcre_exec()                        this sets the startoffset argument for pcre_exec()
327                          or pcre_dfa_exec()
328             \<cr>      pass the PCRE_NEWLINE_CR option to pcre_exec()
329                          or pcre_dfa_exec()
330             \<lf>      pass the PCRE_NEWLINE_LF option to pcre_exec()
331                          or pcre_dfa_exec()
332             \<crlf>    pass the PCRE_NEWLINE_CRLF option to pcre_exec()
333                          or pcre_dfa_exec()
334             \<anycrlf> pass the PCRE_NEWLINE_ANYCRLF option to pcre_exec()
335                          or pcre_dfa_exec()
336             \<any>     pass the PCRE_NEWLINE_ANY option to pcre_exec()
337                          or pcre_dfa_exec()
338    
339           The escapes that specify line ending  sequences  are  literal  strings,
340           exactly as shown. No more than one newline setting should be present in
341           any data line.
342    
343         A backslash followed by anything else just escapes the  anything  else.         A backslash followed by anything else just escapes the  anything  else.
344         If  the very last character is a backslash, it is ignored. This gives a         If  the very last character is a backslash, it is ignored. This gives a
# Line 277  DATA LINES Line 346  DATA LINES
346         nates the data input.         nates the data input.
347    
348         If  \M  is present, pcretest calls pcre_exec() several times, with dif-         If  \M  is present, pcretest calls pcre_exec() several times, with dif-
349         ferent values in the match_limit field of the  pcre_extra  data  struc-         ferent values in the match_limit and  match_limit_recursion  fields  of
350         ture,  until it finds the minimum number that is needed for pcre_exec()         the  pcre_extra  data structure, until it finds the minimum numbers for
351         to complete. This number is a measure of the amount  of  recursion  and         each parameter that allow pcre_exec() to complete. The match_limit num-
352         backtracking  that takes place, and checking it out can be instructive.         ber  is  a  measure of the amount of backtracking that takes place, and
353         For most simple matches, the number is quite small,  but  for  patterns         checking it out can be instructive. For most simple matches, the number
354         with  very large numbers of matching possibilities, it can become large         is  quite  small,  but for patterns with very large numbers of matching
355         very quickly with increasing length of subject string.         possibilities, it can become large very quickly with increasing  length
356           of subject string. The match_limit_recursion number is a measure of how
357           much stack (or, if PCRE is compiled with  NO_RECURSE,  how  much  heap)
358           memory is needed to complete the match attempt.
359    
360         When \O is used, the value specified may be higher or  lower  than  the         When  \O  is  used, the value specified may be higher or lower than the
361         size set by the -O command line option (or defaulted to 45); \O applies         size set by the -O command line option (or defaulted to 45); \O applies
362         only to the call of pcre_exec() for the line in which it appears.         only to the call of pcre_exec() for the line in which it appears.
363    
364         If the /P modifier was present on the pattern, causing the POSIX  wrap-         If  the /P modifier was present on the pattern, causing the POSIX wrap-
365         per  API to be used, only \B and \Z have any effect, causing REG_NOTBOL         per API to be used, the only option-setting  sequences  that  have  any
366         and REG_NOTEOL to be passed to regexec() respectively.         effect  are \B and \Z, causing REG_NOTBOL and REG_NOTEOL, respectively,
367           to be passed to regexec().
368    
369         The use of \x{hh...} to represent UTF-8 characters is not dependent  on         The use of \x{hh...} to represent UTF-8 characters is not dependent  on
370         the  use  of  the  /8 modifier on the pattern. It is recognized always.         the  use  of  the  /8 modifier on the pattern. It is recognized always.
# Line 327  DEFAULT OUTPUT FROM PCRETEST Line 400  DEFAULT OUTPUT FROM PCRETEST
400         is an example of an interactive pcretest run.         is an example of an interactive pcretest run.
401    
402           $ pcretest           $ pcretest
403           PCRE version 5.00 07-Sep-2004           PCRE version 7.0 30-Nov-2006
404    
405             re> /^abc(\d+)/             re> /^abc(\d+)/
406           data> abc123           data> abc123
# Line 338  DEFAULT OUTPUT FROM PCRETEST Line 411  DEFAULT OUTPUT FROM PCRETEST
411    
412         If the strings contain any non-printing characters, they are output  as         If the strings contain any non-printing characters, they are output  as
413         \0x  escapes,  or  as \x{...} escapes if the /8 modifier was present on         \0x  escapes,  or  as \x{...} escapes if the /8 modifier was present on
414         the pattern. If the pattern has the /+ modifier, the  output  for  sub-         the pattern. See below for the definition of  non-printing  characters.
415         string  0 is followed by the the rest of the subject string, identified         If  the pattern has the /+ modifier, the output for substring 0 is fol-
416         by "0+" like this:         lowed by the the rest of the subject string, identified  by  "0+"  like
417           this:
418    
419             re> /cat/+             re> /cat/+
420           data> cataract           data> cataract
421            0: cat            0: cat
422            0+ aract            0+ aract
423    
424         If the pattern has the /g or /G modifier,  the  results  of  successive         If  the  pattern  has  the /g or /G modifier, the results of successive
425         matching attempts are output in sequence, like this:         matching attempts are output in sequence, like this:
426    
427             re> /\Bi(\w\w)/g             re> /\Bi(\w\w)/g
# Line 361  DEFAULT OUTPUT FROM PCRETEST Line 435  DEFAULT OUTPUT FROM PCRETEST
435    
436         "No match" is output only if the first match attempt fails.         "No match" is output only if the first match attempt fails.
437    
438         If  any  of the sequences \C, \G, or \L are present in a data line that         If any of the sequences \C, \G, or \L are present in a data  line  that
439         is successfully matched, the substrings extracted  by  the  convenience         is  successfully  matched,  the substrings extracted by the convenience
440         functions are output with C, G, or L after the string number instead of         functions are output with C, G, or L after the string number instead of
441         a colon. This is in addition to the normal full list. The string length         a colon. This is in addition to the normal full list. The string length
442         (that  is,  the return from the extraction function) is given in paren-         (that is, the return from the extraction function) is given  in  paren-
443         theses after each string for \C and \G.         theses after each string for \C and \G.
444    
445         Note that while patterns can be continued over several lines  (a  plain         Note that whereas patterns can be continued over several lines (a plain
446         ">" prompt is used for continuations), data lines may not. However new-         ">" prompt is used for continuations), data lines may not. However new-
447         lines can be included in data by means of the \n escape.         lines  can  be included in data by means of the \n escape (or \r, \r\n,
448           etc., depending on the newline sequence setting).
449    
450    
451  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION  OUTPUT FROM THE ALTERNATIVE MATCHING FUNCTION
# Line 389  OUTPUT FROM THE ALTERNATIVE MATCHING FUN Line 464  OUTPUT FROM THE ALTERNATIVE MATCHING FUN
464         (Using  the  normal  matching function on this data finds only "tang".)         (Using  the  normal  matching function on this data finds only "tang".)
465         The longest matching string is always given first (and numbered  zero).         The longest matching string is always given first (and numbered  zero).
466    
467         If  /gP  is  present  on  the  pattern,  the search for further matches         If /g is present on the pattern, the search for further matches resumes
468         resumes at the end of the longest match. For example:         at the end of the longest match. For example:
469    
470             re> /(tang|tangerine|tan)/g             re> /(tang|tangerine|tan)/g
471           data> yellow tangerine and tangy sultana\D           data> yellow tangerine and tangy sultana\D
# Line 463  CALLOUTS Line 538  CALLOUTS
538         the pcrecallout documentation.         the pcrecallout documentation.
539    
540    
541    NON-PRINTING CHARACTERS
542    
543           When  pcretest is outputting text in the compiled version of a pattern,
544           bytes other than 32-126 are always treated as  non-printing  characters
545           are are therefore shown as hex escapes.
546    
547           When  pcretest  is  outputting text that is a matched part of a subject
548           string, it behaves in the same way, unless a different locale has  been
549           set  for  the  pattern  (using  the  /L  modifier).  In  this case, the
550           isprint() function to distinguish printing and non-printing characters.
551    
552    
553  SAVING AND RELOADING COMPILED PATTERNS  SAVING AND RELOADING COMPILED PATTERNS
554    
555         The  facilities  described  in  this section are not available when the         The  facilities  described  in  this section are not available when the
# Line 519  SAVING AND RELOADING COMPILED PATTERNS Line 606  SAVING AND RELOADING COMPILED PATTERNS
606         a file that is not in the correct format, the result is undefined.         a file that is not in the correct format, the result is undefined.
607    
608    
609    SEE ALSO
610    
611           pcre(3), pcreapi(3), pcrecallout(3),  pcrematching(3),  pcrepartial(d),
612           pcrepattern(3), pcreprecompile(3).
613    
614    
615  AUTHOR  AUTHOR
616    
617         Philip Hazel         Philip Hazel
618         University Computing Service,         University Computing Service
619         Cambridge CB2 3QG, England.         Cambridge CB2 3QH, England.
620    
621    
622    REVISION
623    
624  Last updated: 28 February 2005         Last updated: 16 April 2007
625  Copyright (c) 1997-2005 University of Cambridge.         Copyright (c) 1997-2007 University of Cambridge.

Legend:
Removed from v.77  
changed lines
  Added in v.152

  ViewVC Help
Powered by ViewVC 1.1.5