ViewVC logotype

Contents of /code/trunk/doc/html/pcretest.html

Parent Directory Parent Directory | Revision Log Revision Log

Revision 75 - (show annotations)
Sat Feb 24 21:40:37 2007 UTC (13 years, 9 months ago) by nigel
File MIME type: text/html
File size: 21062 byte(s)
Load pcre-5.0 into code/trunk.
1 <html>
2 <head>
3 <title>pcretest specification</title>
4 </head>
5 <body bgcolor="#FFFFFF" text="#00005A" link="#0066FF" alink="#3399FF" vlink="#2222BB">
6 <h1>pcretest man page</h1>
7 <p>
8 Return to the <a href="index.html">PCRE index page</a>.
9 </p>
10 <p>
11 This page is part of the PCRE HTML documentation. It was generated automatically
12 from the original man page. If there is any nonsense in it, please consult the
13 man page, in case the conversion went wrong.
14 <br>
15 <ul>
16 <li><a name="TOC1" href="#SEC1">SYNOPSIS</a>
17 <li><a name="TOC2" href="#SEC2">OPTIONS</a>
18 <li><a name="TOC3" href="#SEC3">DESCRIPTION</a>
19 <li><a name="TOC4" href="#SEC4">PATTERN MODIFIERS</a>
20 <li><a name="TOC5" href="#SEC5">DATA LINES</a>
21 <li><a name="TOC6" href="#SEC6">OUTPUT FROM PCRETEST</a>
22 <li><a name="TOC7" href="#SEC7">CALLOUTS</a>
24 <li><a name="TOC9" href="#SEC9">AUTHOR</a>
25 </ul>
26 <br><a name="SEC1" href="#TOC1">SYNOPSIS</a><br>
27 <P>
28 <b>pcretest [-C] [-d] [-i] [-m] [-o osize] [-p] [-t] [source]</b>
29 <b>[destination]</b>
30 </P>
31 <P>
32 <b>pcretest</b> was written as a test program for the PCRE regular expression
33 library itself, but it can also be used for experimenting with regular
34 expressions. This document describes the features of the test program; for
35 details of the regular expressions themselves, see the
36 <a href="pcrepattern.html"><b>pcrepattern</b></a>
37 documentation. For details of the PCRE library function calls and their
38 options, see the
39 <a href="pcreapi.html"><b>pcreapi</b></a>
40 documentation.
41 </P>
42 <br><a name="SEC2" href="#TOC1">OPTIONS</a><br>
43 <P>
44 <b>-C</b>
45 Output the version number of the PCRE library, and all available information
46 about the optional features that are included, and then exit.
47 </P>
48 <P>
49 <b>-d</b>
50 Behave as if each regex had the <b>/D</b> (debug) modifier; the internal
51 form is output after compilation.
52 </P>
53 <P>
54 <b>-i</b>
55 Behave as if each regex had the <b>/I</b> modifier; information about the
56 compiled pattern is given after compilation.
57 </P>
58 <P>
59 <b>-m</b>
60 Output the size of each compiled pattern after it has been compiled. This is
61 equivalent to adding <b>/M</b> to each regular expression. For compatibility
62 with earlier versions of pcretest, <b>-s</b> is a synonym for <b>-m</b>.
63 </P>
64 <P>
65 <b>-o</b> <i>osize</i>
66 Set the number of elements in the output vector that is used when calling
67 <b>pcre_exec()</b> to be <i>osize</i>. The default value is 45, which is enough
68 for 14 capturing subexpressions. The vector size can be changed for individual
69 matching calls by including \O in the data line (see below).
70 </P>
71 <P>
72 <b>-p</b>
73 Behave as if each regex has <b>/P</b> modifier; the POSIX wrapper API is used
74 to call PCRE. None of the other options has any effect when <b>-p</b> is set.
75 </P>
76 <P>
77 <b>-t</b>
78 Run each compile, study, and match many times with a timer, and output
79 resulting time per compile or match (in milliseconds). Do not set <b>-m</b> with
80 <b>-t</b>, because you will then get the size output a zillion times, and the
81 timing will be distorted.
82 </P>
83 <br><a name="SEC3" href="#TOC1">DESCRIPTION</a><br>
84 <P>
85 If <b>pcretest</b> is given two filename arguments, it reads from the first and
86 writes to the second. If it is given only one filename argument, it reads from
87 that file and writes to stdout. Otherwise, it reads from stdin and writes to
88 stdout, and prompts for each line of input, using "re&#62;" to prompt for regular
89 expressions, and "data&#62;" to prompt for data lines.
90 </P>
91 <P>
92 The program handles any number of sets of input on a single input file. Each
93 set starts with a regular expression, and continues with any number of data
94 lines to be matched against the pattern.
95 </P>
96 <P>
97 Each data line is matched separately and independently. If you want to do
98 multiple-line matches, you have to use the \n escape sequence in a single line
99 of input to encode the newline characters. The maximum length of data line is
100 30,000 characters.
101 </P>
102 <P>
103 An empty line signals the end of the data lines, at which point a new regular
104 expression is read. The regular expressions are given enclosed in any
105 non-alphanumeric delimiters other than backslash, for example
106 <pre>
107 /(a|bc)x+yz/
108 </pre>
109 White space before the initial delimiter is ignored. A regular expression may
110 be continued over several input lines, in which case the newline characters are
111 included within it. It is possible to include the delimiter within the pattern
112 by escaping it, for example
113 <pre>
114 /abc\/def/
115 </pre>
116 If you do so, the escape and the delimiter form part of the pattern, but since
117 delimiters are always non-alphanumeric, this does not affect its interpretation.
118 If the terminating delimiter is immediately followed by a backslash, for
119 example,
120 <pre>
121 /abc/\
122 </pre>
123 then a backslash is added to the end of the pattern. This is done to provide a
124 way of testing the error condition that arises if a pattern finishes with a
125 backslash, because
126 <pre>
127 /abc\/
128 </pre>
129 is interpreted as the first line of a pattern that starts with "abc/", causing
130 pcretest to read the next line as a continuation of the regular expression.
131 </P>
132 <br><a name="SEC4" href="#TOC1">PATTERN MODIFIERS</a><br>
133 <P>
134 A pattern may be followed by any number of modifiers, which are mostly single
135 characters. Following Perl usage, these are referred to below as, for example,
136 "the <b>/i</b> modifier", even though the delimiter of the pattern need not
137 always be a slash, and no slash is used when writing modifiers. Whitespace may
138 appear between the final pattern delimiter and the first modifier, and between
139 the modifiers themselves.
140 </P>
141 <P>
142 The <b>/i</b>, <b>/m</b>, <b>/s</b>, and <b>/x</b> modifiers set the PCRE_CASELESS,
143 PCRE_MULTILINE, PCRE_DOTALL, or PCRE_EXTENDED options, respectively, when
144 <b>pcre_compile()</b> is called. These four modifier letters have the same
145 effect as they do in Perl. For example:
146 <pre>
147 /caseless/i
148 </pre>
149 The following table shows additional modifiers for setting PCRE options that do
150 not correspond to anything in Perl:
151 <pre>
152 <b>/A</b> PCRE_ANCHORED
156 <b>/U</b> PCRE_UNGREEDY
157 <b>/X</b> PCRE_EXTRA
158 </pre>
159 Searching for all possible matches within each subject string can be requested
160 by the <b>/g</b> or <b>/G</b> modifier. After finding a match, PCRE is called
161 again to search the remainder of the subject string. The difference between
162 <b>/g</b> and <b>/G</b> is that the former uses the <i>startoffset</i> argument to
163 <b>pcre_exec()</b> to start searching at a new point within the entire string
164 (which is in effect what Perl does), whereas the latter passes over a shortened
165 substring. This makes a difference to the matching process if the pattern
166 begins with a lookbehind assertion (including \b or \B).
167 </P>
168 <P>
169 If any call to <b>pcre_exec()</b> in a <b>/g</b> or <b>/G</b> sequence matches an
170 empty string, the next call is done with the PCRE_NOTEMPTY and PCRE_ANCHORED
171 flags set in order to search for another, non-empty, match at the same point.
172 If this second match fails, the start offset is advanced by one, and the normal
173 match is retried. This imitates the way Perl handles such cases when using the
174 <b>/g</b> modifier or the <b>split()</b> function.
175 </P>
176 <P>
177 There are yet more modifiers for controlling the way <b>pcretest</b>
178 operates.
179 </P>
180 <P>
181 The <b>/+</b> modifier requests that as well as outputting the substring that
182 matched the entire pattern, pcretest should in addition output the remainder of
183 the subject string. This is useful for tests where the subject contains
184 multiple copies of the same substring.
185 </P>
186 <P>
187 The <b>/L</b> modifier must be followed directly by the name of a locale, for
188 example,
189 <pre>
190 /pattern/Lfr_FR
191 </pre>
192 For this reason, it must be the last modifier. The given locale is set,
193 <b>pcre_maketables()</b> is called to build a set of character tables for the
194 locale, and this is then passed to <b>pcre_compile()</b> when compiling the
195 regular expression. Without an <b>/L</b> modifier, NULL is passed as the tables
196 pointer; that is, <b>/L</b> applies only to the expression on which it appears.
197 </P>
198 <P>
199 The <b>/I</b> modifier requests that <b>pcretest</b> output information about the
200 compiled pattern (whether it is anchored, has a fixed first character, and
201 so on). It does this by calling <b>pcre_fullinfo()</b> after compiling a
202 pattern. If the pattern is studied, the results of that are also output.
203 </P>
204 <P>
205 The <b>/D</b> modifier is a PCRE debugging feature, which also assumes <b>/I</b>.
206 It causes the internal form of compiled regular expressions to be output after
207 compilation. If the pattern was studied, the information returned is also
208 output.
209 </P>
210 <P>
211 The <b>/F</b> modifier causes <b>pcretest</b> to flip the byte order of the
212 fields in the compiled pattern that contain 2-byte and 4-byte numbers. This
213 facility is for testing the feature in PCRE that allows it to execute patterns
214 that were compiled on a host with a different endianness. This feature is not
215 available when the POSIX interface to PCRE is being used, that is, when the
216 <b>/P</b> pattern modifier is specified. See also the section about saving and
217 reloading compiled patterns below.
218 </P>
219 <P>
220 The <b>/S</b> modifier causes <b>pcre_study()</b> to be called after the
221 expression has been compiled, and the results used when the expression is
222 matched.
223 </P>
224 <P>
225 The <b>/M</b> modifier causes the size of memory block used to hold the compiled
226 pattern to be output.
227 </P>
228 <P>
229 The <b>/P</b> modifier causes <b>pcretest</b> to call PCRE via the POSIX wrapper
230 API rather than its native API. When this is done, all other modifiers except
231 <b>/i</b>, <b>/m</b>, and <b>/+</b> are ignored. REG_ICASE is set if <b>/i</b> is
232 present, and REG_NEWLINE is set if <b>/m</b> is present. The wrapper functions
233 force PCRE_DOLLAR_ENDONLY always, and PCRE_DOTALL unless REG_NEWLINE is set.
234 </P>
235 <P>
236 The <b>/8</b> modifier causes <b>pcretest</b> to call PCRE with the PCRE_UTF8
237 option set. This turns on support for UTF-8 character handling in PCRE,
238 provided that it was compiled with this support enabled. This modifier also
239 causes any non-printing characters in output strings to be printed using the
240 \x{hh...} notation if they are valid UTF-8 sequences.
241 </P>
242 <P>
243 If the <b>/?</b> modifier is used with <b>/8</b>, it causes <b>pcretest</b> to
244 call <b>pcre_compile()</b> with the PCRE_NO_UTF8_CHECK option, to suppress the
245 checking of the string for UTF-8 validity.
246 </P>
247 <br><a name="SEC5" href="#TOC1">DATA LINES</a><br>
248 <P>
249 Before each data line is passed to <b>pcre_exec()</b>, leading and trailing
250 whitespace is removed, and it is then scanned for \ escapes. Some of these are
251 pretty esoteric features, intended for checking out some of the more
252 complicated features of PCRE. If you are just testing "ordinary" regular
253 expressions, you probably don't need any of these. The following escapes are
254 recognized:
255 <pre>
256 \a alarm (= BEL)
257 \b backspace
258 \e escape
259 \f formfeed
260 \n newline
261 \r carriage return
262 \t tab
263 \v vertical tab
264 \nnn octal character (up to 3 octal digits)
265 \xhh hexadecimal character (up to 2 hex digits)
266 \x{hh...} hexadecimal character, any number of digits in UTF-8 mode
267 \A pass the PCRE_ANCHORED option to <b>pcre_exec()</b>
268 \B pass the PCRE_NOTBOL option to <b>pcre_exec()</b>
269 \Cdd call pcre_copy_substring() for substring dd after a successful match (number less than 32)
270 \Cname call pcre_copy_named_substring() for substring "name" after a successful match (name termin-
271 ated by next non alphanumeric character)
272 \C+ show the current captured substrings at callout time
273 \C- do not supply a callout function
274 \C!n return 1 instead of 0 when callout number n is reached
275 \C!n!m return 1 instead of 0 when callout number n is reached for the nth time
276 \C*n pass the number n (may be negative) as callout data; this is used as the callout return value
277 \Gdd call pcre_get_substring() for substring dd after a successful match (number less than 32)
278 \Gname call pcre_get_named_substring() for substring "name" after a successful match (name termin-
279 ated by next non-alphanumeric character)
280 \L call pcre_get_substringlist() after a successful match
281 \M discover the minimum MATCH_LIMIT setting
282 \N pass the PCRE_NOTEMPTY option to <b>pcre_exec()</b>
283 \Odd set the size of the output vector passed to <b>pcre_exec()</b> to dd (any number of digits)
284 \P pass the PCRE_PARTIAL option to <b>pcre_exec()</b>
285 \S output details of memory get/free calls during matching
286 \Z pass the PCRE_NOTEOL option to <b>pcre_exec()</b>
287 \? pass the PCRE_NO_UTF8_CHECK option to <b>pcre_exec()</b>
288 \&#62;dd start the match at offset dd (any number of digits);
289 this sets the <i>startoffset</i> argument for <b>pcre_exec()</b>
290 </pre>
291 A backslash followed by anything else just escapes the anything else. If the
292 very last character is a backslash, it is ignored. This gives a way of passing
293 an empty line as data, since a real empty line terminates the data input.
294 </P>
295 <P>
296 If \M is present, <b>pcretest</b> calls <b>pcre_exec()</b> several times, with
297 different values in the <i>match_limit</i> field of the <b>pcre_extra</b> data
298 structure, until it finds the minimum number that is needed for
299 <b>pcre_exec()</b> to complete. This number is a measure of the amount of
300 recursion and backtracking that takes place, and checking it out can be
301 instructive. For most simple matches, the number is quite small, but for
302 patterns with very large numbers of matching possibilities, it can become large
303 very quickly with increasing length of subject string.
304 </P>
305 <P>
306 When \O is used, the value specified may be higher or lower than the size set
307 by the <b>-O</b> command line option (or defaulted to 45); \O applies only to
308 the call of <b>pcre_exec()</b> for the line in which it appears.
309 </P>
310 <P>
311 If the <b>/P</b> modifier was present on the pattern, causing the POSIX wrapper
312 API to be used, only \B and \Z have any effect, causing REG_NOTBOL and
313 REG_NOTEOL to be passed to <b>regexec()</b> respectively.
314 </P>
315 <P>
316 The use of \x{hh...} to represent UTF-8 characters is not dependent on the use
317 of the <b>/8</b> modifier on the pattern. It is recognized always. There may be
318 any number of hexadecimal digits inside the braces. The result is from one to
319 six bytes, encoded according to the UTF-8 rules.
320 </P>
321 <br><a name="SEC6" href="#TOC1">OUTPUT FROM PCRETEST</a><br>
322 <P>
323 When a match succeeds, pcretest outputs the list of captured substrings that
324 <b>pcre_exec()</b> returns, starting with number 0 for the string that matched
325 the whole pattern. Otherwise, it outputs "No match" or "Partial match"
326 when <b>pcre_exec()</b> returns PCRE_ERROR_NOMATCH or PCRE_ERROR_PARTIAL,
327 respectively, and otherwise the PCRE negative error number. Here is an example
328 of an interactive pcretest run.
329 <pre>
330 $ pcretest
331 PCRE version 5.00 07-Sep-2004
333 re&#62; /^abc(\d+)/
334 data&#62; abc123
335 0: abc123
336 1: 123
337 data&#62; xyz
338 No match
339 </pre>
340 If the strings contain any non-printing characters, they are output as \0x
341 escapes, or as \x{...} escapes if the <b>/8</b> modifier was present on the
342 pattern. If the pattern has the <b>/+</b> modifier, the output for substring 0
343 is followed by the the rest of the subject string, identified by "0+" like
344 this:
345 <pre>
346 re&#62; /cat/+
347 data&#62; cataract
348 0: cat
349 0+ aract
350 </pre>
351 If the pattern has the <b>/g</b> or <b>/G</b> modifier, the results of successive
352 matching attempts are output in sequence, like this:
353 <pre>
354 re&#62; /\Bi(\w\w)/g
355 data&#62; Mississippi
356 0: iss
357 1: ss
358 0: iss
359 1: ss
360 0: ipp
361 1: pp
362 </pre>
363 "No match" is output only if the first match attempt fails.
364 </P>
365 <P>
366 If any of the sequences <b>\C</b>, <b>\G</b>, or <b>\L</b> are present in a
367 data line that is successfully matched, the substrings extracted by the
368 convenience functions are output with C, G, or L after the string number
369 instead of a colon. This is in addition to the normal full list. The string
370 length (that is, the return from the extraction function) is given in
371 parentheses after each string for <b>\C</b> and <b>\G</b>.
372 </P>
373 <P>
374 Note that while patterns can be continued over several lines (a plain "&#62;"
375 prompt is used for continuations), data lines may not. However newlines can be
376 included in data by means of the \n escape.
377 </P>
378 <br><a name="SEC7" href="#TOC1">CALLOUTS</a><br>
379 <P>
380 If the pattern contains any callout requests, <b>pcretest</b>'s callout function
381 is called during matching. By default, it displays the callout number, the
382 start and current positions in the text at the callout time, and the next
383 pattern item to be tested. For example, the output
384 <pre>
385 ---&#62;pqrabcdef
386 0 ^ ^ \d
387 </pre>
388 indicates that callout number 0 occurred for a match attempt starting at the
389 fourth character of the subject string, when the pointer was at the seventh
390 character of the data, and when the next pattern item was \d. Just one
391 circumflex is output if the start and current positions are the same.
392 </P>
393 <P>
394 Callouts numbered 255 are assumed to be automatic callouts, inserted as a
395 result of the <b>/C</b> pattern modifier. In this case, instead of showing the
396 callout number, the offset in the pattern, preceded by a plus, is output. For
397 example:
398 <pre>
399 re&#62; /\d?[A-E]\*/C
400 data&#62; E*
401 ---&#62;E*
402 +0 ^ \d?
403 +3 ^ [A-E]
404 +8 ^^ \*
405 +10 ^ ^
406 0: E*
407 </pre>
408 The callout function in <b>pcretest</b> returns zero (carry on matching) by
409 default, but you can use an \C item in a data line (as described above) to
410 change this.
411 </P>
412 <P>
413 Inserting callouts can be helpful when using <b>pcretest</b> to check
414 complicated regular expressions. For further information about callouts, see
415 the
416 <a href="pcrecallout.html"><b>pcrecallout</b></a>
417 documentation.
418 </P>
419 <br><a name="SEC8" href="#TOC1">SAVING AND RELOADING COMPILED PATTERNS</a><br>
420 <P>
421 The facilities described in this section are not available when the POSIX
422 inteface to PCRE is being used, that is, when the <b>/P</b> pattern modifier is
423 specified.
424 </P>
425 <P>
426 When the POSIX interface is not in use, you can cause <b>pcretest</b> to write a
427 compiled pattern to a file, by following the modifiers with &#62; and a file name.
428 For example:
429 <pre>
430 /pattern/im &#62;/some/file
431 </pre>
432 See the
433 <a href="pcreprecompile.html"><b>pcreprecompile</b></a>
434 documentation for a discussion about saving and re-using compiled patterns.
435 </P>
436 <P>
437 The data that is written is binary. The first eight bytes are the length of the
438 compiled pattern data followed by the length of the optional study data, each
439 written as four bytes in big-endian order (most significant byte first). If
440 there is no study data (either the pattern was not studied, or studying did not
441 return any data), the second length is zero. The lengths are followed by an
442 exact copy of the compiled pattern. If there is additional study data, this
443 follows immediately after the compiled pattern. After writing the file,
444 <b>pcretest</b> expects to read a new pattern.
445 </P>
446 <P>
447 A saved pattern can be reloaded into <b>pcretest</b> by specifing &#60; and a file
448 name instead of a pattern. The name of the file must not contain a &#60; character,
449 as otherwise <b>pcretest</b> will interpret the line as a pattern delimited by &#60;
450 characters.
451 For example:
452 <pre>
453 re&#62; &#60;/some/file
454 Compiled regex loaded from /some/file
455 No study data
456 </pre>
457 When the pattern has been loaded, <b>pcretest</b> proceeds to read data lines in
458 the usual way.
459 </P>
460 <P>
461 You can copy a file written by <b>pcretest</b> to a different host and reload it
462 there, even if the new host has opposite endianness to the one on which the
463 pattern was compiled. For example, you can compile on an i86 machine and run on
464 a SPARC machine.
465 </P>
466 <P>
467 File names for saving and reloading can be absolute or relative, but note that
468 the shell facility of expanding a file name that starts with a tilde (~) is not
469 available.
470 </P>
471 <P>
472 The ability to save and reload files in <b>pcretest</b> is intended for testing
473 and experimentation. It is not intended for production use because only a
474 single pattern can be written to a file. Furthermore, there is no facility for
475 supplying custom character tables for use with a reloaded pattern. If the
476 original pattern was compiled with custom tables, an attempt to match a subject
477 string using a reloaded pattern is likely to cause <b>pcretest</b> to crash.
478 Finally, if you attempt to load a file that is not in the correct format, the
479 result is undefined.
480 </P>
481 <br><a name="SEC9" href="#TOC1">AUTHOR</a><br>
482 <P>
483 Philip Hazel &#60;ph10@cam.ac.uk&#62;
484 <br>
485 University Computing Service,
486 <br>
487 Cambridge CB2 3QG, England.
488 </P>
489 <P>
490 Last updated: 10 September 2004
491 <br>
492 Copyright &copy; 1997-2004 University of Cambridge.
493 <p>
494 Return to the <a href="index.html">PCRE index page</a>.
495 </p>

  ViewVC Help
Powered by ViewVC 1.1.5