Parent Directory
|
Revision Log
Final tidies and documentation updates for 8.44.
1 | ----------------------------------------------------------------------------- |
2 | This file contains a concatenation of the PCRE man pages, converted to plain |
3 | text format for ease of searching with a text editor, or for use on systems |
4 | that do not have a man page processor. The small individual files that give |
5 | synopses of each function in the library have not been included. Neither has |
6 | the pcredemo program. There are separate text files for the pcregrep and |
7 | pcretest commands. |
8 | ----------------------------------------------------------------------------- |
9 | |
10 | |
11 | PCRE(3) Library Functions Manual PCRE(3) |
12 | |
13 | |
14 | |
15 | NAME |
16 | PCRE - Perl-compatible regular expressions (original API) |
17 | |
18 | PLEASE TAKE NOTE |
19 | |
20 | This document relates to PCRE releases that use the original API, with |
21 | library names libpcre, libpcre16, and libpcre32. January 2015 saw the |
22 | first release of a new API, known as PCRE2, with release numbers start- |
23 | ing at 10.00 and library names libpcre2-8, libpcre2-16, and |
24 | libpcre2-32. The old libraries (now called PCRE1) are still being main- |
25 | tained for bug fixes, but there will be no new development. New |
26 | projects are advised to use the new PCRE2 libraries. |
27 | |
28 | |
29 | INTRODUCTION |
30 | |
31 | The PCRE library is a set of functions that implement regular expres- |
32 | sion pattern matching using the same syntax and semantics as Perl, with |
33 | just a few differences. Some features that appeared in Python and PCRE |
34 | before they appeared in Perl are also available using the Python syn- |
35 | tax, there is some support for one or two .NET and Oniguruma syntax |
36 | items, and there is an option for requesting some minor changes that |
37 | give better JavaScript compatibility. |
38 | |
39 | Starting with release 8.30, it is possible to compile two separate PCRE |
40 | libraries: the original, which supports 8-bit character strings (in- |
41 | cluding UTF-8 strings), and a second library that supports 16-bit char- |
42 | acter strings (including UTF-16 strings). The build process allows ei- |
43 | ther one or both to be built. The majority of the work to make this |
44 | possible was done by Zoltan Herczeg. |
45 | |
46 | Starting with release 8.32 it is possible to compile a third separate |
47 | PCRE library that supports 32-bit character strings (including UTF-32 |
48 | strings). The build process allows any combination of the 8-, 16- and |
49 | 32-bit libraries. The work to make this possible was done by Christian |
50 | Persch. |
51 | |
52 | The three libraries contain identical sets of functions, except that |
53 | the names in the 16-bit library start with pcre16_ instead of pcre_, |
54 | and the names in the 32-bit library start with pcre32_ instead of |
55 | pcre_. To avoid over-complication and reduce the documentation mainte- |
56 | nance load, most of the documentation describes the 8-bit library, with |
57 | the differences for the 16-bit and 32-bit libraries described sepa- |
58 | rately in the pcre16 and pcre32 pages. References to functions or |
59 | structures of the form pcre[16|32]_xxx should be read as meaning |
60 | "pcre_xxx when using the 8-bit library, pcre16_xxx when using the |
61 | 16-bit library, or pcre32_xxx when using the 32-bit library". |
62 | |
63 | The current implementation of PCRE corresponds approximately with Perl |
64 | 5.12, including support for UTF-8/16/32 encoded strings and Unicode |
65 | general category properties. However, UTF-8/16/32 and Unicode support |
66 | has to be explicitly enabled; it is not the default. The Unicode tables |
67 | correspond to Unicode release 6.3.0. |
68 | |
69 | In addition to the Perl-compatible matching function, PCRE contains an |
70 | alternative function that matches the same compiled patterns in a dif- |
71 | ferent way. In certain circumstances, the alternative function has some |
72 | advantages. For a discussion of the two matching algorithms, see the |
73 | pcrematching page. |
74 | |
75 | PCRE is written in C and released as a C library. A number of people |
76 | have written wrappers and interfaces of various kinds. In particular, |
77 | Google Inc. have provided a comprehensive C++ wrapper for the 8-bit |
78 | library. This is now included as part of the PCRE distribution. The |
79 | pcrecpp page has details of this interface. Other people's contribu- |
80 | tions can be found in the Contrib directory at the primary FTP site, |
81 | which is: |
82 | |
83 | ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre |
84 | |
85 | Details of exactly which Perl regular expression features are and are |
86 | not supported by PCRE are given in separate documents. See the pcrepat- |
87 | tern and pcrecompat pages. There is a syntax summary in the pcresyntax |
88 | page. |
89 | |
90 | Some features of PCRE can be included, excluded, or changed when the |
91 | library is built. The pcre_config() function makes it possible for a |
92 | client to discover which features are available. The features them- |
93 | selves are described in the pcrebuild page. Documentation about build- |
94 | ing PCRE for various operating systems can be found in the README and |
95 | NON-AUTOTOOLS_BUILD files in the source distribution. |
96 | |
97 | The libraries contains a number of undocumented internal functions and |
98 | data tables that are used by more than one of the exported external |
99 | functions, but which are not intended for use by external callers. |
100 | Their names all begin with "_pcre_" or "_pcre16_" or "_pcre32_", which |
101 | hopefully will not provoke any name clashes. In some environments, it |
102 | is possible to control which external symbols are exported when a |
103 | shared library is built, and in these cases the undocumented symbols |
104 | are not exported. |
105 | |
106 | |
107 | SECURITY CONSIDERATIONS |
108 | |
109 | If you are using PCRE in a non-UTF application that permits users to |
110 | supply arbitrary patterns for compilation, you should be aware of a |
111 | feature that allows users to turn on UTF support from within a pattern, |
112 | provided that PCRE was built with UTF support. For example, an 8-bit |
113 | pattern that begins with "(*UTF8)" or "(*UTF)" turns on UTF-8 mode, |
114 | which interprets patterns and subjects as strings of UTF-8 characters |
115 | instead of individual 8-bit characters. This causes both the pattern |
116 | and any data against which it is matched to be checked for UTF-8 valid- |
117 | ity. If the data string is very long, such a check might use suffi- |
118 | ciently many resources as to cause your application to lose perfor- |
119 | mance. |
120 | |
121 | One way of guarding against this possibility is to use the |
122 | pcre_fullinfo() function to check the compiled pattern's options for |
123 | UTF. Alternatively, from release 8.33, you can set the PCRE_NEVER_UTF |
124 | option at compile time. This causes a compile time error if a pattern |
125 | contains a UTF-setting sequence. |
126 | |
127 | If your application is one that supports UTF, be aware that validity |
128 | checking can take time. If the same data string is to be matched many |
129 | times, you can use the PCRE_NO_UTF[8|16|32]_CHECK option for the second |
130 | and subsequent matches to save redundant checks. |
131 | |
132 | Another way that performance can be hit is by running a pattern that |
133 | has a very large search tree against a string that will never match. |
134 | Nested unlimited repeats in a pattern are a common example. PCRE pro- |
135 | vides some protection against this: see the PCRE_EXTRA_MATCH_LIMIT fea- |
136 | ture in the pcreapi page. |
137 | |
138 | |
139 | USER DOCUMENTATION |
140 | |
141 | The user documentation for PCRE comprises a number of different sec- |
142 | tions. In the "man" format, each of these is a separate "man page". In |
143 | the HTML format, each is a separate page, linked from the index page. |
144 | In the plain text format, the descriptions of the pcregrep and pcretest |
145 | programs are in files called pcregrep.txt and pcretest.txt, respec- |
146 | tively. The remaining sections, except for the pcredemo section (which |
147 | is a program listing), are concatenated in pcre.txt, for ease of |
148 | searching. The sections are as follows: |
149 | |
150 | pcre this document |
151 | pcre-config show PCRE installation configuration information |
152 | pcre16 details of the 16-bit library |
153 | pcre32 details of the 32-bit library |
154 | pcreapi details of PCRE's native C API |
155 | pcrebuild building PCRE |
156 | pcrecallout details of the callout feature |
157 | pcrecompat discussion of Perl compatibility |
158 | pcrecpp details of the C++ wrapper for the 8-bit library |
159 | pcredemo a demonstration C program that uses PCRE |
160 | pcregrep description of the pcregrep command (8-bit only) |
161 | pcrejit discussion of the just-in-time optimization support |
162 | pcrelimits details of size and other limits |
163 | pcrematching discussion of the two matching algorithms |
164 | pcrepartial details of the partial matching facility |
165 | pcrepattern syntax and semantics of supported |
166 | regular expressions |
167 | pcreperform discussion of performance issues |
168 | pcreposix the POSIX-compatible C API for the 8-bit library |
169 | pcreprecompile details of saving and re-using precompiled patterns |
170 | pcresample discussion of the pcredemo program |
171 | pcrestack discussion of stack usage |
172 | pcresyntax quick syntax reference |
173 | pcretest description of the pcretest testing command |
174 | pcreunicode discussion of Unicode and UTF-8/16/32 support |
175 | |
176 | In the "man" and HTML formats, there is also a short page for each C |
177 | library function, listing its arguments and results. |
178 | |
179 | |
180 | AUTHOR |
181 | |
182 | Philip Hazel |
183 | University Computing Service |
184 | Cambridge CB2 3QH, England. |
185 | |
186 | Putting an actual email address here seems to have been a spam magnet, |
187 | so I've taken it away. If you want to email me, use my two initials, |
188 | followed by the two digits 10, at the domain cam.ac.uk. |
189 | |
190 | |
191 | REVISION |
192 | |
193 | Last updated: 10 February 2015 |
194 | Copyright (c) 1997-2015 University of Cambridge. |
195 | ------------------------------------------------------------------------------ |
196 | |
197 | |
198 | PCRE(3) Library Functions Manual PCRE(3) |
199 | |
200 | |
201 | |
202 | NAME |
203 | PCRE - Perl-compatible regular expressions |
204 | |
205 | #include <pcre.h> |
206 | |
207 | |
208 | PCRE 16-BIT API BASIC FUNCTIONS |
209 | |
210 | pcre16 *pcre16_compile(PCRE_SPTR16 pattern, int options, |
211 | const char **errptr, int *erroffset, |
212 | const unsigned char *tableptr); |
213 | |
214 | pcre16 *pcre16_compile2(PCRE_SPTR16 pattern, int options, |
215 | int *errorcodeptr, |
216 | const char **errptr, int *erroffset, |
217 | const unsigned char *tableptr); |
218 | |
219 | pcre16_extra *pcre16_study(const pcre16 *code, int options, |
220 | const char **errptr); |
221 | |
222 | void pcre16_free_study(pcre16_extra *extra); |
223 | |
224 | int pcre16_exec(const pcre16 *code, const pcre16_extra *extra, |
225 | PCRE_SPTR16 subject, int length, int startoffset, |
226 | int options, int *ovector, int ovecsize); |
227 | |
228 | int pcre16_dfa_exec(const pcre16 *code, const pcre16_extra *extra, |
229 | PCRE_SPTR16 subject, int length, int startoffset, |
230 | int options, int *ovector, int ovecsize, |
231 | int *workspace, int wscount); |
232 | |
233 | |
234 | PCRE 16-BIT API STRING EXTRACTION FUNCTIONS |
235 | |
236 | int pcre16_copy_named_substring(const pcre16 *code, |
237 | PCRE_SPTR16 subject, int *ovector, |
238 | int stringcount, PCRE_SPTR16 stringname, |
239 | PCRE_UCHAR16 *buffer, int buffersize); |
240 | |
241 | int pcre16_copy_substring(PCRE_SPTR16 subject, int *ovector, |
242 | int stringcount, int stringnumber, PCRE_UCHAR16 *buffer, |
243 | int buffersize); |
244 | |
245 | int pcre16_get_named_substring(const pcre16 *code, |
246 | PCRE_SPTR16 subject, int *ovector, |
247 | int stringcount, PCRE_SPTR16 stringname, |
248 | PCRE_SPTR16 *stringptr); |
249 | |
250 | int pcre16_get_stringnumber(const pcre16 *code, |
251 | PCRE_SPTR16 name); |
252 | |
253 | int pcre16_get_stringtable_entries(const pcre16 *code, |
254 | PCRE_SPTR16 name, PCRE_UCHAR16 **first, PCRE_UCHAR16 **last); |
255 | |
256 | int pcre16_get_substring(PCRE_SPTR16 subject, int *ovector, |
257 | int stringcount, int stringnumber, |
258 | PCRE_SPTR16 *stringptr); |
259 | |
260 | int pcre16_get_substring_list(PCRE_SPTR16 subject, |
261 | int *ovector, int stringcount, PCRE_SPTR16 **listptr); |
262 | |
263 | void pcre16_free_substring(PCRE_SPTR16 stringptr); |
264 | |
265 | void pcre16_free_substring_list(PCRE_SPTR16 *stringptr); |
266 | |
267 | |
268 | PCRE 16-BIT API AUXILIARY FUNCTIONS |
269 | |
270 | pcre16_jit_stack *pcre16_jit_stack_alloc(int startsize, int maxsize); |
271 | |
272 | void pcre16_jit_stack_free(pcre16_jit_stack *stack); |
273 | |
274 | void pcre16_assign_jit_stack(pcre16_extra *extra, |
275 | pcre16_jit_callback callback, void *data); |
276 | |
277 | const unsigned char *pcre16_maketables(void); |
278 | |
279 | int pcre16_fullinfo(const pcre16 *code, const pcre16_extra *extra, |
280 | int what, void *where); |
281 | |
282 | int pcre16_refcount(pcre16 *code, int adjust); |
283 | |
284 | int pcre16_config(int what, void *where); |
285 | |
286 | const char *pcre16_version(void); |
287 | |
288 | int pcre16_pattern_to_host_byte_order(pcre16 *code, |
289 | pcre16_extra *extra, const unsigned char *tables); |
290 | |
291 | |
292 | PCRE 16-BIT API INDIRECTED FUNCTIONS |
293 | |
294 | void *(*pcre16_malloc)(size_t); |
295 | |
296 | void (*pcre16_free)(void *); |
297 | |
298 | void *(*pcre16_stack_malloc)(size_t); |
299 | |
300 | void (*pcre16_stack_free)(void *); |
301 | |
302 | int (*pcre16_callout)(pcre16_callout_block *); |
303 | |
304 | |
305 | PCRE 16-BIT API 16-BIT-ONLY FUNCTION |
306 | |
307 | int pcre16_utf16_to_host_byte_order(PCRE_UCHAR16 *output, |
308 | PCRE_SPTR16 input, int length, int *byte_order, |
309 | int keep_boms); |
310 | |
311 | |
312 | THE PCRE 16-BIT LIBRARY |
313 | |
314 | Starting with release 8.30, it is possible to compile a PCRE library |
315 | that supports 16-bit character strings, including UTF-16 strings, as |
316 | well as or instead of the original 8-bit library. The majority of the |
317 | work to make this possible was done by Zoltan Herczeg. The two li- |
318 | braries contain identical sets of functions, used in exactly the same |
319 | way. Only the names of the functions and the data types of their argu- |
320 | ments and results are different. To avoid over-complication and reduce |
321 | the documentation maintenance load, most of the PCRE documentation de- |
322 | scribes the 8-bit library, with only occasional references to the |
323 | 16-bit library. This page describes what is different when you use the |
324 | 16-bit library. |
325 | |
326 | WARNING: A single application can be linked with both libraries, but |
327 | you must take care when processing any particular pattern to use func- |
328 | tions from just one library. For example, if you want to study a pat- |
329 | tern that was compiled with pcre16_compile(), you must do so with |
330 | pcre16_study(), not pcre_study(), and you must free the study data with |
331 | pcre16_free_study(). |
332 | |
333 | |
334 | THE HEADER FILE |
335 | |
336 | There is only one header file, pcre.h. It contains prototypes for all |
337 | the functions in all libraries, as well as definitions of flags, struc- |
338 | tures, error codes, etc. |
339 | |
340 | |
341 | THE LIBRARY NAME |
342 | |
343 | In Unix-like systems, the 16-bit library is called libpcre16, and can |
344 | normally be accesss by adding -lpcre16 to the command for linking an |
345 | application that uses PCRE. |
346 | |
347 | |
348 | STRING TYPES |
349 | |
350 | In the 8-bit library, strings are passed to PCRE library functions as |
351 | vectors of bytes with the C type "char *". In the 16-bit library, |
352 | strings are passed as vectors of unsigned 16-bit quantities. The macro |
353 | PCRE_UCHAR16 specifies an appropriate data type, and PCRE_SPTR16 is de- |
354 | fined as "const PCRE_UCHAR16 *". In very many environments, "short int" |
355 | is a 16-bit data type. When PCRE is built, it defines PCRE_UCHAR16 as |
356 | "unsigned short int", but checks that it really is a 16-bit data type. |
357 | If it is not, the build fails with an error message telling the main- |
358 | tainer to modify the definition appropriately. |
359 | |
360 | |
361 | STRUCTURE TYPES |
362 | |
363 | The types of the opaque structures that are used for compiled 16-bit |
364 | patterns and JIT stacks are pcre16 and pcre16_jit_stack respectively. |
365 | The type of the user-accessible structure that is returned by |
366 | pcre16_study() is pcre16_extra, and the type of the structure that is |
367 | used for passing data to a callout function is pcre16_callout_block. |
368 | These structures contain the same fields, with the same names, as their |
369 | 8-bit counterparts. The only difference is that pointers to character |
370 | strings are 16-bit instead of 8-bit types. |
371 | |
372 | |
373 | 16-BIT FUNCTIONS |
374 | |
375 | For every function in the 8-bit library there is a corresponding func- |
376 | tion in the 16-bit library with a name that starts with pcre16_ instead |
377 | of pcre_. The prototypes are listed above. In addition, there is one |
378 | extra function, pcre16_utf16_to_host_byte_order(). This is a utility |
379 | function that converts a UTF-16 character string to host byte order if |
380 | necessary. The other 16-bit functions expect the strings they are |
381 | passed to be in host byte order. |
382 | |
383 | The input and output arguments of pcre16_utf16_to_host_byte_order() may |
384 | point to the same address, that is, conversion in place is supported. |
385 | The output buffer must be at least as long as the input. |
386 | |
387 | The length argument specifies the number of 16-bit data units in the |
388 | input string; a negative value specifies a zero-terminated string. |
389 | |
390 | If byte_order is NULL, it is assumed that the string starts off in host |
391 | byte order. This may be changed by byte-order marks (BOMs) anywhere in |
392 | the string (commonly as the first character). |
393 | |
394 | If byte_order is not NULL, a non-zero value of the integer to which it |
395 | points means that the input starts off in host byte order, otherwise |
396 | the opposite order is assumed. Again, BOMs in the string can change |
397 | this. The final byte order is passed back at the end of processing. |
398 | |
399 | If keep_boms is not zero, byte-order mark characters (0xfeff) are |
400 | copied into the output string. Otherwise they are discarded. |
401 | |
402 | The result of the function is the number of 16-bit units placed into |
403 | the output buffer, including the zero terminator if the string was |
404 | zero-terminated. |
405 | |
406 | |
407 | SUBJECT STRING OFFSETS |
408 | |
409 | The lengths and starting offsets of subject strings must be specified |
410 | in 16-bit data units, and the offsets within subject strings that are |
411 | returned by the matching functions are in also 16-bit units rather than |
412 | bytes. |
413 | |
414 | |
415 | NAMED SUBPATTERNS |
416 | |
417 | The name-to-number translation table that is maintained for named sub- |
418 | patterns uses 16-bit characters. The pcre16_get_stringtable_entries() |
419 | function returns the length of each entry in the table as the number of |
420 | 16-bit data units. |
421 | |
422 | |
423 | OPTION NAMES |
424 | |
425 | There are two new general option names, PCRE_UTF16 and |
426 | PCRE_NO_UTF16_CHECK, which correspond to PCRE_UTF8 and |
427 | PCRE_NO_UTF8_CHECK in the 8-bit library. In fact, these new options de- |
428 | fine the same bits in the options word. There is a discussion about the |
429 | validity of UTF-16 strings in the pcreunicode page. |
430 | |
431 | For the pcre16_config() function there is an option PCRE_CONFIG_UTF16 |
432 | that returns 1 if UTF-16 support is configured, otherwise 0. If this |
433 | option is given to pcre_config() or pcre32_config(), or if the |
434 | PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF32 option is given to pcre16_con- |
435 | fig(), the result is the PCRE_ERROR_BADOPTION error. |
436 | |
437 | |
438 | CHARACTER CODES |
439 | |
440 | In 16-bit mode, when PCRE_UTF16 is not set, character values are |
441 | treated in the same way as in 8-bit, non UTF-8 mode, except, of course, |
442 | that they can range from 0 to 0xffff instead of 0 to 0xff. Character |
443 | types for characters less than 0xff can therefore be influenced by the |
444 | locale in the same way as before. Characters greater than 0xff have |
445 | only one case, and no "type" (such as letter or digit). |
446 | |
447 | In UTF-16 mode, the character code is Unicode, in the range 0 to |
448 | 0x10ffff, with the exception of values in the range 0xd800 to 0xdfff |
449 | because those are "surrogate" values that are used in pairs to encode |
450 | values greater than 0xffff. |
451 | |
452 | A UTF-16 string can indicate its endianness by special code knows as a |
453 | byte-order mark (BOM). The PCRE functions do not handle this, expecting |
454 | strings to be in host byte order. A utility function called |
455 | pcre16_utf16_to_host_byte_order() is provided to help with this (see |
456 | above). |
457 | |
458 | |
459 | ERROR NAMES |
460 | |
461 | The errors PCRE_ERROR_BADUTF16_OFFSET and PCRE_ERROR_SHORTUTF16 corre- |
462 | spond to their 8-bit counterparts. The error PCRE_ERROR_BADMODE is |
463 | given when a compiled pattern is passed to a function that processes |
464 | patterns in the other mode, for example, if a pattern compiled with |
465 | pcre_compile() is passed to pcre16_exec(). |
466 | |
467 | There are new error codes whose names begin with PCRE_UTF16_ERR for in- |
468 | valid UTF-16 strings, corresponding to the PCRE_UTF8_ERR codes for |
469 | UTF-8 strings that are described in the section entitled "Reason codes |
470 | for invalid UTF-8 strings" in the main pcreapi page. The UTF-16 errors |
471 | are: |
472 | |
473 | PCRE_UTF16_ERR1 Missing low surrogate at end of string |
474 | PCRE_UTF16_ERR2 Invalid low surrogate follows high surrogate |
475 | PCRE_UTF16_ERR3 Isolated low surrogate |
476 | PCRE_UTF16_ERR4 Non-character |
477 | |
478 | |
479 | ERROR TEXTS |
480 | |
481 | If there is an error while compiling a pattern, the error text that is |
482 | passed back by pcre16_compile() or pcre16_compile2() is still an 8-bit |
483 | character string, zero-terminated. |
484 | |
485 | |
486 | CALLOUTS |
487 | |
488 | The subject and mark fields in the callout block that is passed to a |
489 | callout function point to 16-bit vectors. |
490 | |
491 | |
492 | TESTING |
493 | |
494 | The pcretest program continues to operate with 8-bit input and output |
495 | files, but it can be used for testing the 16-bit library. If it is run |
496 | with the command line option -16, patterns and subject strings are con- |
497 | verted from 8-bit to 16-bit before being passed to PCRE, and the 16-bit |
498 | library functions are used instead of the 8-bit ones. Returned 16-bit |
499 | strings are converted to 8-bit for output. If both the 8-bit and the |
500 | 32-bit libraries were not compiled, pcretest defaults to 16-bit and the |
501 | -16 option is ignored. |
502 | |
503 | When PCRE is being built, the RunTest script that is called by "make |
504 | check" uses the pcretest -C option to discover which of the 8-bit, |
505 | 16-bit and 32-bit libraries has been built, and runs the tests appro- |
506 | priately. |
507 | |
508 | |
509 | NOT SUPPORTED IN 16-BIT MODE |
510 | |
511 | Not all the features of the 8-bit library are available with the 16-bit |
512 | library. The C++ and POSIX wrapper functions support only the 8-bit li- |
513 | brary, and the pcregrep program is at present 8-bit only. |
514 | |
515 | |
516 | AUTHOR |
517 | |
518 | Philip Hazel |
519 | University Computing Service |
520 | Cambridge CB2 3QH, England. |
521 | |
522 | |
523 | REVISION |
524 | |
525 | Last updated: 12 May 2013 |
526 | Copyright (c) 1997-2013 University of Cambridge. |
527 | ------------------------------------------------------------------------------ |
528 | |
529 | |
530 | PCRE(3) Library Functions Manual PCRE(3) |
531 | |
532 | |
533 | |
534 | NAME |
535 | PCRE - Perl-compatible regular expressions |
536 | |
537 | #include <pcre.h> |
538 | |
539 | |
540 | PCRE 32-BIT API BASIC FUNCTIONS |
541 | |
542 | pcre32 *pcre32_compile(PCRE_SPTR32 pattern, int options, |
543 | const char **errptr, int *erroffset, |
544 | const unsigned char *tableptr); |
545 | |
546 | pcre32 *pcre32_compile2(PCRE_SPTR32 pattern, int options, |
547 | int *errorcodeptr, |
548 | const unsigned char *tableptr); |
549 | |
550 | pcre32_extra *pcre32_study(const pcre32 *code, int options, |
551 | const char **errptr); |
552 | |
553 | void pcre32_free_study(pcre32_extra *extra); |
554 | |
555 | int pcre32_exec(const pcre32 *code, const pcre32_extra *extra, |
556 | PCRE_SPTR32 subject, int length, int startoffset, |
557 | int options, int *ovector, int ovecsize); |
558 | |
559 | int pcre32_dfa_exec(const pcre32 *code, const pcre32_extra *extra, |
560 | PCRE_SPTR32 subject, int length, int startoffset, |
561 | int options, int *ovector, int ovecsize, |
562 | int *workspace, int wscount); |
563 | |
564 | |
565 | PCRE 32-BIT API STRING EXTRACTION FUNCTIONS |
566 | |
567 | int pcre32_copy_named_substring(const pcre32 *code, |
568 | PCRE_SPTR32 subject, int *ovector, |
569 | int stringcount, PCRE_SPTR32 stringname, |
570 | PCRE_UCHAR32 *buffer, int buffersize); |
571 | |
572 | int pcre32_copy_substring(PCRE_SPTR32 subject, int *ovector, |
573 | int stringcount, int stringnumber, PCRE_UCHAR32 *buffer, |
574 | int buffersize); |
575 | |
576 | int pcre32_get_named_substring(const pcre32 *code, |
577 | PCRE_SPTR32 subject, int *ovector, |
578 | int stringcount, PCRE_SPTR32 stringname, |
579 | PCRE_SPTR32 *stringptr); |
580 | |
581 | int pcre32_get_stringnumber(const pcre32 *code, |
582 | PCRE_SPTR32 name); |
583 | |
584 | int pcre32_get_stringtable_entries(const pcre32 *code, |
585 | PCRE_SPTR32 name, PCRE_UCHAR32 **first, PCRE_UCHAR32 **last); |
586 | |
587 | int pcre32_get_substring(PCRE_SPTR32 subject, int *ovector, |
588 | int stringcount, int stringnumber, |
589 | PCRE_SPTR32 *stringptr); |
590 | |
591 | int pcre32_get_substring_list(PCRE_SPTR32 subject, |
592 | int *ovector, int stringcount, PCRE_SPTR32 **listptr); |
593 | |
594 | void pcre32_free_substring(PCRE_SPTR32 stringptr); |
595 | |
596 | void pcre32_free_substring_list(PCRE_SPTR32 *stringptr); |
597 | |
598 | |
599 | PCRE 32-BIT API AUXILIARY FUNCTIONS |
600 | |
601 | pcre32_jit_stack *pcre32_jit_stack_alloc(int startsize, int maxsize); |
602 | |
603 | void pcre32_jit_stack_free(pcre32_jit_stack *stack); |
604 | |
605 | void pcre32_assign_jit_stack(pcre32_extra *extra, |
606 | pcre32_jit_callback callback, void *data); |
607 | |
608 | const unsigned char *pcre32_maketables(void); |
609 | |
610 | int pcre32_fullinfo(const pcre32 *code, const pcre32_extra *extra, |
611 | int what, void *where); |
612 | |
613 | int pcre32_refcount(pcre32 *code, int adjust); |
614 | |
615 | int pcre32_config(int what, void *where); |
616 | |
617 | const char *pcre32_version(void); |
618 | |
619 | int pcre32_pattern_to_host_byte_order(pcre32 *code, |
620 | pcre32_extra *extra, const unsigned char *tables); |
621 | |
622 | |
623 | PCRE 32-BIT API INDIRECTED FUNCTIONS |
624 | |
625 | void *(*pcre32_malloc)(size_t); |
626 | |
627 | void (*pcre32_free)(void *); |
628 | |
629 | void *(*pcre32_stack_malloc)(size_t); |
630 | |
631 | void (*pcre32_stack_free)(void *); |
632 | |
633 | int (*pcre32_callout)(pcre32_callout_block *); |
634 | |
635 | |
636 | PCRE 32-BIT API 32-BIT-ONLY FUNCTION |
637 | |
638 | int pcre32_utf32_to_host_byte_order(PCRE_UCHAR32 *output, |
639 | PCRE_SPTR32 input, int length, int *byte_order, |
640 | int keep_boms); |
641 | |
642 | |
643 | THE PCRE 32-BIT LIBRARY |
644 | |
645 | Starting with release 8.32, it is possible to compile a PCRE library |
646 | that supports 32-bit character strings, including UTF-32 strings, as |
647 | well as or instead of the original 8-bit library. This work was done by |
648 | Christian Persch, based on the work done by Zoltan Herczeg for the |
649 | 16-bit library. All three libraries contain identical sets of func- |
650 | tions, used in exactly the same way. Only the names of the functions |
651 | and the data types of their arguments and results are different. To |
652 | avoid over-complication and reduce the documentation maintenance load, |
653 | most of the PCRE documentation describes the 8-bit library, with only |
654 | occasional references to the 16-bit and 32-bit libraries. This page de- |
655 | scribes what is different when you use the 32-bit library. |
656 | |
657 | WARNING: A single application can be linked with all or any of the |
658 | three libraries, but you must take care when processing any particular |
659 | pattern to use functions from just one library. For example, if you |
660 | want to study a pattern that was compiled with pcre32_compile(), you |
661 | must do so with pcre32_study(), not pcre_study(), and you must free the |
662 | study data with pcre32_free_study(). |
663 | |
664 | |
665 | THE HEADER FILE |
666 | |
667 | There is only one header file, pcre.h. It contains prototypes for all |
668 | the functions in all libraries, as well as definitions of flags, struc- |
669 | tures, error codes, etc. |
670 | |
671 | |
672 | THE LIBRARY NAME |
673 | |
674 | In Unix-like systems, the 32-bit library is called libpcre32, and can |
675 | normally be accesss by adding -lpcre32 to the command for linking an |
676 | application that uses PCRE. |
677 | |
678 | |
679 | STRING TYPES |
680 | |
681 | In the 8-bit library, strings are passed to PCRE library functions as |
682 | vectors of bytes with the C type "char *". In the 32-bit library, |
683 | strings are passed as vectors of unsigned 32-bit quantities. The macro |
684 | PCRE_UCHAR32 specifies an appropriate data type, and PCRE_SPTR32 is de- |
685 | fined as "const PCRE_UCHAR32 *". In very many environments, "unsigned |
686 | int" is a 32-bit data type. When PCRE is built, it defines PCRE_UCHAR32 |
687 | as "unsigned int", but checks that it really is a 32-bit data type. If |
688 | it is not, the build fails with an error message telling the maintainer |
689 | to modify the definition appropriately. |
690 | |
691 | |
692 | STRUCTURE TYPES |
693 | |
694 | The types of the opaque structures that are used for compiled 32-bit |
695 | patterns and JIT stacks are pcre32 and pcre32_jit_stack respectively. |
696 | The type of the user-accessible structure that is returned by |
697 | pcre32_study() is pcre32_extra, and the type of the structure that is |
698 | used for passing data to a callout function is pcre32_callout_block. |
699 | These structures contain the same fields, with the same names, as their |
700 | 8-bit counterparts. The only difference is that pointers to character |
701 | strings are 32-bit instead of 8-bit types. |
702 | |
703 | |
704 | 32-BIT FUNCTIONS |
705 | |
706 | For every function in the 8-bit library there is a corresponding func- |
707 | tion in the 32-bit library with a name that starts with pcre32_ instead |
708 | of pcre_. The prototypes are listed above. In addition, there is one |
709 | extra function, pcre32_utf32_to_host_byte_order(). This is a utility |
710 | function that converts a UTF-32 character string to host byte order if |
711 | necessary. The other 32-bit functions expect the strings they are |
712 | passed to be in host byte order. |
713 | |
714 | The input and output arguments of pcre32_utf32_to_host_byte_order() may |
715 | point to the same address, that is, conversion in place is supported. |
716 | The output buffer must be at least as long as the input. |
717 | |
718 | The length argument specifies the number of 32-bit data units in the |
719 | input string; a negative value specifies a zero-terminated string. |
720 | |
721 | If byte_order is NULL, it is assumed that the string starts off in host |
722 | byte order. This may be changed by byte-order marks (BOMs) anywhere in |
723 | the string (commonly as the first character). |
724 | |
725 | If byte_order is not NULL, a non-zero value of the integer to which it |
726 | points means that the input starts off in host byte order, otherwise |
727 | the opposite order is assumed. Again, BOMs in the string can change |
728 | this. The final byte order is passed back at the end of processing. |
729 | |
730 | If keep_boms is not zero, byte-order mark characters (0xfeff) are |
731 | copied into the output string. Otherwise they are discarded. |
732 | |
733 | The result of the function is the number of 32-bit units placed into |
734 | the output buffer, including the zero terminator if the string was |
735 | zero-terminated. |
736 | |
737 | |
738 | SUBJECT STRING OFFSETS |
739 | |
740 | The lengths and starting offsets of subject strings must be specified |
741 | in 32-bit data units, and the offsets within subject strings that are |
742 | returned by the matching functions are in also 32-bit units rather than |
743 | bytes. |
744 | |
745 | |
746 | NAMED SUBPATTERNS |
747 | |
748 | The name-to-number translation table that is maintained for named sub- |
749 | patterns uses 32-bit characters. The pcre32_get_stringtable_entries() |
750 | function returns the length of each entry in the table as the number of |
751 | 32-bit data units. |
752 | |
753 | |
754 | OPTION NAMES |
755 | |
756 | There are two new general option names, PCRE_UTF32 and |
757 | PCRE_NO_UTF32_CHECK, which correspond to PCRE_UTF8 and |
758 | PCRE_NO_UTF8_CHECK in the 8-bit library. In fact, these new options de- |
759 | fine the same bits in the options word. There is a discussion about the |
760 | validity of UTF-32 strings in the pcreunicode page. |
761 | |
762 | For the pcre32_config() function there is an option PCRE_CONFIG_UTF32 |
763 | that returns 1 if UTF-32 support is configured, otherwise 0. If this |
764 | option is given to pcre_config() or pcre16_config(), or if the |
765 | PCRE_CONFIG_UTF8 or PCRE_CONFIG_UTF16 option is given to pcre32_con- |
766 | fig(), the result is the PCRE_ERROR_BADOPTION error. |
767 | |
768 | |
769 | CHARACTER CODES |
770 | |
771 | In 32-bit mode, when PCRE_UTF32 is not set, character values are |
772 | treated in the same way as in 8-bit, non UTF-8 mode, except, of course, |
773 | that they can range from 0 to 0x7fffffff instead of 0 to 0xff. Charac- |
774 | ter types for characters less than 0xff can therefore be influenced by |
775 | the locale in the same way as before. Characters greater than 0xff |
776 | have only one case, and no "type" (such as letter or digit). |
777 | |
778 | In UTF-32 mode, the character code is Unicode, in the range 0 to |
779 | 0x10ffff, with the exception of values in the range 0xd800 to 0xdfff |
780 | because those are "surrogate" values that are ill-formed in UTF-32. |
781 | |
782 | A UTF-32 string can indicate its endianness by special code knows as a |
783 | byte-order mark (BOM). The PCRE functions do not handle this, expecting |
784 | strings to be in host byte order. A utility function called |
785 | pcre32_utf32_to_host_byte_order() is provided to help with this (see |
786 | above). |
787 | |
788 | |
789 | ERROR NAMES |
790 | |
791 | The error PCRE_ERROR_BADUTF32 corresponds to its 8-bit counterpart. |
792 | The error PCRE_ERROR_BADMODE is given when a compiled pattern is passed |
793 | to a function that processes patterns in the other mode, for example, |
794 | if a pattern compiled with pcre_compile() is passed to pcre32_exec(). |
795 | |
796 | There are new error codes whose names begin with PCRE_UTF32_ERR for in- |
797 | valid UTF-32 strings, corresponding to the PCRE_UTF8_ERR codes for |
798 | UTF-8 strings that are described in the section entitled "Reason codes |
799 | for invalid UTF-8 strings" in the main pcreapi page. The UTF-32 errors |
800 | are: |
801 | |
802 | PCRE_UTF32_ERR1 Surrogate character (range from 0xd800 to 0xdfff) |
803 | PCRE_UTF32_ERR2 Non-character |
804 | PCRE_UTF32_ERR3 Character > 0x10ffff |
805 | |
806 | |
807 | ERROR TEXTS |
808 | |
809 | If there is an error while compiling a pattern, the error text that is |
810 | passed back by pcre32_compile() or pcre32_compile2() is still an 8-bit |
811 | character string, zero-terminated. |
812 | |
813 | |
814 | CALLOUTS |
815 | |
816 | The subject and mark fields in the callout block that is passed to a |
817 | callout function point to 32-bit vectors. |
818 | |
819 | |
820 | TESTING |
821 | |
822 | The pcretest program continues to operate with 8-bit input and output |
823 | files, but it can be used for testing the 32-bit library. If it is run |
824 | with the command line option -32, patterns and subject strings are con- |
825 | verted from 8-bit to 32-bit before being passed to PCRE, and the 32-bit |
826 | library functions are used instead of the 8-bit ones. Returned 32-bit |
827 | strings are converted to 8-bit for output. If both the 8-bit and the |
828 | 16-bit libraries were not compiled, pcretest defaults to 32-bit and the |
829 | -32 option is ignored. |
830 | |
831 | When PCRE is being built, the RunTest script that is called by "make |
832 | check" uses the pcretest -C option to discover which of the 8-bit, |
833 | 16-bit and 32-bit libraries has been built, and runs the tests appro- |
834 | priately. |
835 | |
836 | |
837 | NOT SUPPORTED IN 32-BIT MODE |
838 | |
839 | Not all the features of the 8-bit library are available with the 32-bit |
840 | library. The C++ and POSIX wrapper functions support only the 8-bit li- |
841 | brary, and the pcregrep program is at present 8-bit only. |
842 | |
843 | |
844 | AUTHOR |
845 | |
846 | Philip Hazel |
847 | University Computing Service |
848 | Cambridge CB2 3QH, England. |
849 | |
850 | |
851 | REVISION |
852 | |
853 | Last updated: 12 May 2013 |
854 | Copyright (c) 1997-2013 University of Cambridge. |
855 | ------------------------------------------------------------------------------ |
856 | |
857 | |
858 | PCREBUILD(3) Library Functions Manual PCREBUILD(3) |
859 | |
860 | |
861 | |
862 | NAME |
863 | PCRE - Perl-compatible regular expressions |
864 | |
865 | BUILDING PCRE |
866 | |
867 | PCRE is distributed with a configure script that can be used to build |
868 | the library in Unix-like environments using the applications known as |
869 | Autotools. Also in the distribution are files to support building us- |
870 | ing CMake instead of configure. The text file README contains general |
871 | information about building with Autotools (some of which is repeated |
872 | below), and also has some comments about building on various operating |
873 | systems. There is a lot more information about building PCRE without |
874 | using Autotools (including information about using CMake and building |
875 | "by hand") in the text file called NON-AUTOTOOLS-BUILD. You should |
876 | consult this file as well as the README file if you are building in a |
877 | non-Unix-like environment. |
878 | |
879 | |
880 | PCRE BUILD-TIME OPTIONS |
881 | |
882 | The rest of this document describes the optional features of PCRE that |
883 | can be selected when the library is compiled. It assumes use of the |
884 | configure script, where the optional features are selected or dese- |
885 | lected by providing options to configure before running the make com- |
886 | mand. However, the same options can be selected in both Unix-like and |
887 | non-Unix-like environments using the GUI facility of cmake-gui if you |
888 | are using CMake instead of configure to build PCRE. |
889 | |
890 | If you are not using Autotools or CMake, option selection can be done |
891 | by editing the config.h file, or by passing parameter settings to the |
892 | compiler, as described in NON-AUTOTOOLS-BUILD. |
893 | |
894 | The complete list of options for configure (which includes the standard |
895 | ones such as the selection of the installation directory) can be ob- |
896 | tained by running |
897 | |
898 | ./configure --help |
899 | |
900 | The following sections include descriptions of options whose names be- |
901 | gin with --enable or --disable. These settings specify changes to the |
902 | defaults for the configure command. Because of the way that configure |
903 | works, --enable and --disable always come in pairs, so the complemen- |
904 | tary option always exists as well, but as it specifies the default, it |
905 | is not described. |
906 | |
907 | |
908 | BUILDING 8-BIT, 16-BIT AND 32-BIT LIBRARIES |
909 | |
910 | By default, a library called libpcre is built, containing functions |
911 | that take string arguments contained in vectors of bytes, either as |
912 | single-byte characters, or interpreted as UTF-8 strings. You can also |
913 | build a separate library, called libpcre16, in which strings are con- |
914 | tained in vectors of 16-bit data units and interpreted either as sin- |
915 | gle-unit characters or UTF-16 strings, by adding |
916 | |
917 | --enable-pcre16 |
918 | |
919 | to the configure command. You can also build yet another separate li- |
920 | brary, called libpcre32, in which strings are contained in vectors of |
921 | 32-bit data units and interpreted either as single-unit characters or |
922 | UTF-32 strings, by adding |
923 | |
924 | --enable-pcre32 |
925 | |
926 | to the configure command. If you do not want the 8-bit library, add |
927 | |
928 | --disable-pcre8 |
929 | |
930 | as well. At least one of the three libraries must be built. Note that |
931 | the C++ and POSIX wrappers are for the 8-bit library only, and that |
932 | pcregrep is an 8-bit program. None of these are built if you select |
933 | only the 16-bit or 32-bit libraries. |
934 | |
935 | |
936 | BUILDING SHARED AND STATIC LIBRARIES |
937 | |
938 | The Autotools PCRE building process uses libtool to build both shared |
939 | and static libraries by default. You can suppress one of these by |
940 | adding one of |
941 | |
942 | --disable-shared |
943 | --disable-static |
944 | |
945 | to the configure command, as required. |
946 | |
947 | |
948 | C++ SUPPORT |
949 | |
950 | By default, if the 8-bit library is being built, the configure script |
951 | will search for a C++ compiler and C++ header files. If it finds them, |
952 | it automatically builds the C++ wrapper library (which supports only |
953 | 8-bit strings). You can disable this by adding |
954 | |
955 | --disable-cpp |
956 | |
957 | to the configure command. |
958 | |
959 | |
960 | UTF-8, UTF-16 AND UTF-32 SUPPORT |
961 | |
962 | To build PCRE with support for UTF Unicode character strings, add |
963 | |
964 | --enable-utf |
965 | |
966 | to the configure command. This setting applies to all three libraries, |
967 | adding support for UTF-8 to the 8-bit library, support for UTF-16 to |
968 | the 16-bit library, and support for UTF-32 to the to the 32-bit li- |
969 | brary. There are no separate options for enabling UTF-8, UTF-16 and |
970 | UTF-32 independently because that would allow ridiculous settings such |
971 | as requesting UTF-16 support while building only the 8-bit library. It |
972 | is not possible to build one library with UTF support and another with- |
973 | out in the same configuration. (For backwards compatibility, --enable- |
974 | utf8 is a synonym of --enable-utf.) |
975 | |
976 | Of itself, this setting does not make PCRE treat strings as UTF-8, |
977 | UTF-16 or UTF-32. As well as compiling PCRE with this option, you also |
978 | have have to set the PCRE_UTF8, PCRE_UTF16 or PCRE_UTF32 option (as ap- |
979 | propriate) when you call one of the pattern compiling functions. |
980 | |
981 | If you set --enable-utf when compiling in an EBCDIC environment, PCRE |
982 | expects its input to be either ASCII or UTF-8 (depending on the run- |
983 | time option). It is not possible to support both EBCDIC and UTF-8 codes |
984 | in the same version of the library. Consequently, --enable-utf and |
985 | --enable-ebcdic are mutually exclusive. |
986 | |
987 | |
988 | UNICODE CHARACTER PROPERTY SUPPORT |
989 | |
990 | UTF support allows the libraries to process character codepoints up to |
991 | 0x10ffff in the strings that they handle. On its own, however, it does |
992 | not provide any facilities for accessing the properties of such charac- |
993 | ters. If you want to be able to use the pattern escapes \P, \p, and \X, |
994 | which refer to Unicode character properties, you must add |
995 | |
996 | --enable-unicode-properties |
997 | |
998 | to the configure command. This implies UTF support, even if you have |
999 | not explicitly requested it. |
1000 | |
1001 | Including Unicode property support adds around 30K of tables to the |
1002 | PCRE library. Only the general category properties such as Lu and Nd |
1003 | are supported. Details are given in the pcrepattern documentation. |
1004 | |
1005 | |
1006 | JUST-IN-TIME COMPILER SUPPORT |
1007 | |
1008 | Just-in-time compiler support is included in the build by specifying |
1009 | |
1010 | --enable-jit |
1011 | |
1012 | This support is available only for certain hardware architectures. If |
1013 | this option is set for an unsupported architecture, a compile time er- |
1014 | ror occurs. See the pcrejit documentation for a discussion of JIT us- |
1015 | age. When JIT support is enabled, pcregrep automatically makes use of |
1016 | it, unless you add |
1017 | |
1018 | --disable-pcregrep-jit |
1019 | |
1020 | to the "configure" command. |
1021 | |
1022 | |
1023 | CODE VALUE OF NEWLINE |
1024 | |
1025 | By default, PCRE interprets the linefeed (LF) character as indicating |
1026 | the end of a line. This is the normal newline character on Unix-like |
1027 | systems. You can compile PCRE to use carriage return (CR) instead, by |
1028 | adding |
1029 | |
1030 | --enable-newline-is-cr |
1031 | |
1032 | to the configure command. There is also a --enable-newline-is-lf op- |
1033 | tion, which explicitly specifies linefeed as the newline character. |
1034 | |
1035 | Alternatively, you can specify that line endings are to be indicated by |
1036 | the two character sequence CRLF. If you want this, add |
1037 | |
1038 | --enable-newline-is-crlf |
1039 | |
1040 | to the configure command. There is a fourth option, specified by |
1041 | |
1042 | --enable-newline-is-anycrlf |
1043 | |
1044 | which causes PCRE to recognize any of the three sequences CR, LF, or |
1045 | CRLF as indicating a line ending. Finally, a fifth option, specified by |
1046 | |
1047 | --enable-newline-is-any |
1048 | |
1049 | causes PCRE to recognize any Unicode newline sequence. |
1050 | |
1051 | Whatever line ending convention is selected when PCRE is built can be |
1052 | overridden when the library functions are called. At build time it is |
1053 | conventional to use the standard for your operating system. |
1054 | |
1055 | |
1056 | WHAT \R MATCHES |
1057 | |
1058 | By default, the sequence \R in a pattern matches any Unicode newline |
1059 | sequence, whatever has been selected as the line ending sequence. If |
1060 | you specify |
1061 | |
1062 | --enable-bsr-anycrlf |
1063 | |
1064 | the default is changed so that \R matches only CR, LF, or CRLF. What- |
1065 | ever is selected when PCRE is built can be overridden when the library |
1066 | functions are called. |
1067 | |
1068 | |
1069 | POSIX MALLOC USAGE |
1070 | |
1071 | When the 8-bit library is called through the POSIX interface (see the |
1072 | pcreposix documentation), additional working storage is required for |
1073 | holding the pointers to capturing substrings, because PCRE requires |
1074 | three integers per substring, whereas the POSIX interface provides only |
1075 | two. If the number of expected substrings is small, the wrapper func- |
1076 | tion uses space on the stack, because this is faster than using mal- |
1077 | loc() for each call. The default threshold above which the stack is no |
1078 | longer used is 10; it can be changed by adding a setting such as |
1079 | |
1080 | --with-posix-malloc-threshold=20 |
1081 | |
1082 | to the configure command. |
1083 | |
1084 | |
1085 | HANDLING VERY LARGE PATTERNS |
1086 | |
1087 | Within a compiled pattern, offset values are used to point from one |
1088 | part to another (for example, from an opening parenthesis to an alter- |
1089 | nation metacharacter). By default, in the 8-bit and 16-bit libraries, |
1090 | two-byte values are used for these offsets, leading to a maximum size |
1091 | for a compiled pattern of around 64K. This is sufficient to handle all |
1092 | but the most gigantic patterns. Nevertheless, some people do want to |
1093 | process truly enormous patterns, so it is possible to compile PCRE to |
1094 | use three-byte or four-byte offsets by adding a setting such as |
1095 | |
1096 | --with-link-size=3 |
1097 | |
1098 | to the configure command. The value given must be 2, 3, or 4. For the |
1099 | 16-bit library, a value of 3 is rounded up to 4. In these libraries, |
1100 | using longer offsets slows down the operation of PCRE because it has to |
1101 | load additional data when handling them. For the 32-bit library the |
1102 | value is always 4 and cannot be overridden; the value of --with-link- |
1103 | size is ignored. |
1104 | |
1105 | |
1106 | AVOIDING EXCESSIVE STACK USAGE |
1107 | |
1108 | When matching with the pcre_exec() function, PCRE implements backtrack- |
1109 | ing by making recursive calls to an internal function called match(). |
1110 | In environments where the size of the stack is limited, this can se- |
1111 | verely limit PCRE's operation. (The Unix environment does not usually |
1112 | suffer from this problem, but it may sometimes be necessary to increase |
1113 | the maximum stack size. There is a discussion in the pcrestack docu- |
1114 | mentation.) An alternative approach to recursion that uses memory from |
1115 | the heap to remember data, instead of using recursive function calls, |
1116 | has been implemented to work round the problem of limited stack size. |
1117 | If you want to build a version of PCRE that works this way, add |
1118 | |
1119 | --disable-stack-for-recursion |
1120 | |
1121 | to the configure command. With this configuration, PCRE will use the |
1122 | pcre_stack_malloc and pcre_stack_free variables to call memory manage- |
1123 | ment functions. By default these point to malloc() and free(), but you |
1124 | can replace the pointers so that your own functions are used instead. |
1125 | |
1126 | Separate functions are provided rather than using pcre_malloc and |
1127 | pcre_free because the usage is very predictable: the block sizes re- |
1128 | quested are always the same, and the blocks are always freed in reverse |
1129 | order. A calling program might be able to implement optimized functions |
1130 | that perform better than malloc() and free(). PCRE runs noticeably more |
1131 | slowly when built in this way. This option affects only the pcre_exec() |
1132 | function; it is not relevant for pcre_dfa_exec(). |
1133 | |
1134 | |
1135 | LIMITING PCRE RESOURCE USAGE |
1136 | |
1137 | Internally, PCRE has a function called match(), which it calls repeat- |
1138 | edly (sometimes recursively) when matching a pattern with the |
1139 | pcre_exec() function. By controlling the maximum number of times this |
1140 | function may be called during a single matching operation, a limit can |
1141 | be placed on the resources used by a single call to pcre_exec(). The |
1142 | limit can be changed at run time, as described in the pcreapi documen- |
1143 | tation. The default is 10 million, but this can be changed by adding a |
1144 | setting such as |
1145 | |
1146 | --with-match-limit=500000 |
1147 | |
1148 | to the configure command. This setting has no effect on the |
1149 | pcre_dfa_exec() matching function. |
1150 | |
1151 | In some environments it is desirable to limit the depth of recursive |
1152 | calls of match() more strictly than the total number of calls, in order |
1153 | to restrict the maximum amount of stack (or heap, if --disable-stack- |
1154 | for-recursion is specified) that is used. A second limit controls this; |
1155 | it defaults to the value that is set for --with-match-limit, which im- |
1156 | poses no additional constraints. However, you can set a lower limit by |
1157 | adding, for example, |
1158 | |
1159 | --with-match-limit-recursion=10000 |
1160 | |
1161 | to the configure command. This value can also be overridden at run |
1162 | time. |
1163 | |
1164 | |
1165 | CREATING CHARACTER TABLES AT BUILD TIME |
1166 | |
1167 | PCRE uses fixed tables for processing characters whose code values are |
1168 | less than 256. By default, PCRE is built with a set of tables that are |
1169 | distributed in the file pcre_chartables.c.dist. These tables are for |
1170 | ASCII codes only. If you add |
1171 | |
1172 | --enable-rebuild-chartables |
1173 | |
1174 | to the configure command, the distributed tables are no longer used. |
1175 | Instead, a program called dftables is compiled and run. This outputs |
1176 | the source for new set of tables, created in the default locale of your |
1177 | C run-time system. (This method of replacing the tables does not work |
1178 | if you are cross compiling, because dftables is run on the local host. |
1179 | If you need to create alternative tables when cross compiling, you will |
1180 | have to do so "by hand".) |
1181 | |
1182 | |
1183 | USING EBCDIC CODE |
1184 | |
1185 | PCRE assumes by default that it will run in an environment where the |
1186 | character code is ASCII (or Unicode, which is a superset of ASCII). |
1187 | This is the case for most computer operating systems. PCRE can, how- |
1188 | ever, be compiled to run in an EBCDIC environment by adding |
1189 | |
1190 | --enable-ebcdic |
1191 | |
1192 | to the configure command. This setting implies --enable-rebuild-charta- |
1193 | bles. You should only use it if you know that you are in an EBCDIC en- |
1194 | vironment (for example, an IBM mainframe operating system). The --en- |
1195 | able-ebcdic option is incompatible with --enable-utf. |
1196 | |
1197 | The EBCDIC character that corresponds to an ASCII LF is assumed to have |
1198 | the value 0x15 by default. However, in some EBCDIC environments, 0x25 |
1199 | is used. In such an environment you should use |
1200 | |
1201 | --enable-ebcdic-nl25 |
1202 | |
1203 | as well as, or instead of, --enable-ebcdic. The EBCDIC character for CR |
1204 | has the same value as in ASCII, namely, 0x0d. Whichever of 0x15 and |
1205 | 0x25 is not chosen as LF is made to correspond to the Unicode NEL char- |
1206 | acter (which, in Unicode, is 0x85). |
1207 | |
1208 | The options that select newline behaviour, such as --enable-newline-is- |
1209 | cr, and equivalent run-time options, refer to these character values in |
1210 | an EBCDIC environment. |
1211 | |
1212 | |
1213 | PCREGREP OPTIONS FOR COMPRESSED FILE SUPPORT |
1214 | |
1215 | By default, pcregrep reads all files as plain text. You can build it so |
1216 | that it recognizes files whose names end in .gz or .bz2, and reads them |
1217 | with libz or libbz2, respectively, by adding one or both of |
1218 | |
1219 | --enable-pcregrep-libz |
1220 | --enable-pcregrep-libbz2 |
1221 | |
1222 | to the configure command. These options naturally require that the rel- |
1223 | evant libraries are installed on your system. Configuration will fail |
1224 | if they are not. |
1225 | |
1226 | |
1227 | PCREGREP BUFFER SIZE |
1228 | |
1229 | pcregrep uses an internal buffer to hold a "window" on the file it is |
1230 | scanning, in order to be able to output "before" and "after" lines when |
1231 | it finds a match. The size of the buffer is controlled by a parameter |
1232 | whose default value is 20K. The buffer itself is three times this size, |
1233 | but because of the way it is used for holding "before" lines, the long- |
1234 | est line that is guaranteed to be processable is the parameter size. |
1235 | You can change the default parameter value by adding, for example, |
1236 | |
1237 | --with-pcregrep-bufsize=50K |
1238 | |
1239 | to the configure command. The caller of pcregrep can, however, override |
1240 | this value by specifying a run-time option. |
1241 | |
1242 | |
1243 | PCRETEST OPTION FOR LIBREADLINE SUPPORT |
1244 | |
1245 | If you add |
1246 | |
1247 | --enable-pcretest-libreadline |
1248 | |
1249 | to the configure command, pcretest is linked with the libreadline li- |
1250 | brary, and when its input is from a terminal, it reads it using the |
1251 | readline() function. This provides line-editing and history facilities. |
1252 | Note that libreadline is GPL-licensed, so if you distribute a binary of |
1253 | pcretest linked in this way, there may be licensing issues. |
1254 | |
1255 | Setting this option causes the -lreadline option to be added to the |
1256 | pcretest build. In many operating environments with a sytem-installed |
1257 | libreadline this is sufficient. However, in some environments (e.g. if |
1258 | an unmodified distribution version of readline is in use), some extra |
1259 | configuration may be necessary. The INSTALL file for libreadline says |
1260 | this: |
1261 | |
1262 | "Readline uses the termcap functions, but does not link with the |
1263 | termcap or curses library itself, allowing applications which link |
1264 | with readline the to choose an appropriate library." |
1265 | |
1266 | If your environment has not been set up so that an appropriate library |
1267 | is automatically included, you may need to add something like |
1268 | |
1269 | LIBS="-ncurses" |
1270 | |
1271 | immediately before the configure command. |
1272 | |
1273 | |
1274 | DEBUGGING WITH VALGRIND SUPPORT |
1275 | |
1276 | By adding the |
1277 | |
1278 | --enable-valgrind |
1279 | |
1280 | option to to the configure command, PCRE will use valgrind annotations |
1281 | to mark certain memory regions as unaddressable. This allows it to de- |
1282 | tect invalid memory accesses, and is mostly useful for debugging PCRE |
1283 | itself. |
1284 | |
1285 | |
1286 | CODE COVERAGE REPORTING |
1287 | |
1288 | If your C compiler is gcc, you can build a version of PCRE that can |
1289 | generate a code coverage report for its test suite. To enable this, you |
1290 | must install lcov version 1.6 or above. Then specify |
1291 | |
1292 | --enable-coverage |
1293 | |
1294 | to the configure command and build PCRE in the usual way. |
1295 | |
1296 | Note that using ccache (a caching C compiler) is incompatible with code |
1297 | coverage reporting. If you have configured ccache to run automatically |
1298 | on your system, you must set the environment variable |
1299 | |
1300 | CCACHE_DISABLE=1 |
1301 | |
1302 | before running make to build PCRE, so that ccache is not used. |
1303 | |
1304 | When --enable-coverage is used, the following addition targets are |
1305 | added to the Makefile: |
1306 | |
1307 | make coverage |
1308 | |
1309 | This creates a fresh coverage report for the PCRE test suite. It is |
1310 | equivalent to running "make coverage-reset", "make coverage-baseline", |
1311 | "make check", and then "make coverage-report". |
1312 | |
1313 | make coverage-reset |
1314 | |
1315 | This zeroes the coverage counters, but does nothing else. |
1316 | |
1317 | make coverage-baseline |
1318 | |
1319 | This captures baseline coverage information. |
1320 | |
1321 | make coverage-report |
1322 | |
1323 | This creates the coverage report. |
1324 | |
1325 | make coverage-clean-report |
1326 | |
1327 | This removes the generated coverage report without cleaning the cover- |
1328 | age data itself. |
1329 | |
1330 | make coverage-clean-data |
1331 | |
1332 | This removes the captured coverage data without removing the coverage |
1333 | files created at compile time (*.gcno). |
1334 | |
1335 | make coverage-clean |
1336 | |
1337 | This cleans all coverage data including the generated coverage report. |
1338 | For more information about code coverage, see the gcov and lcov docu- |
1339 | mentation. |
1340 | |
1341 | |
1342 | SEE ALSO |
1343 | |
1344 | pcreapi(3), pcre16, pcre32, pcre_config(3). |
1345 | |
1346 | |
1347 | AUTHOR |
1348 | |
1349 | Philip Hazel |
1350 | University Computing Service |
1351 | Cambridge CB2 3QH, England. |
1352 | |
1353 | |
1354 | REVISION |
1355 | |
1356 | Last updated: 12 May 2013 |
1357 | Copyright (c) 1997-2013 University of Cambridge. |
1358 | ------------------------------------------------------------------------------ |
1359 | |
1360 | |
1361 | PCREMATCHING(3) Library Functions Manual PCREMATCHING(3) |
1362 | |
1363 | |
1364 | |
1365 | NAME |
1366 | PCRE - Perl-compatible regular expressions |
1367 | |
1368 | PCRE MATCHING ALGORITHMS |
1369 | |
1370 | This document describes the two different algorithms that are available |
1371 | in PCRE for matching a compiled regular expression against a given sub- |
1372 | ject string. The "standard" algorithm is the one provided by the |
1373 | pcre_exec(), pcre16_exec() and pcre32_exec() functions. These work in |
1374 | the same as as Perl's matching function, and provide a Perl-compatible |
1375 | matching operation. The just-in-time (JIT) optimization that is de- |
1376 | scribed in the pcrejit documentation is compatible with these func- |
1377 | tions. |
1378 | |
1379 | An alternative algorithm is provided by the pcre_dfa_exec(), |
1380 | pcre16_dfa_exec() and pcre32_dfa_exec() functions; they operate in a |
1381 | different way, and are not Perl-compatible. This alternative has advan- |
1382 | tages and disadvantages compared with the standard algorithm, and these |
1383 | are described below. |
1384 | |
1385 | When there is only one possible way in which a given subject string can |
1386 | match a pattern, the two algorithms give the same answer. A difference |
1387 | arises, however, when there are multiple possibilities. For example, if |
1388 | the pattern |
1389 | |
1390 | ^<.*> |
1391 | |
1392 | is matched against the string |
1393 | |
1394 | <something> <something else> <something further> |
1395 | |
1396 | there are three possible answers. The standard algorithm finds only one |
1397 | of them, whereas the alternative algorithm finds all three. |
1398 | |
1399 | |
1400 | REGULAR EXPRESSIONS AS TREES |
1401 | |
1402 | The set of strings that are matched by a regular expression can be rep- |
1403 | resented as a tree structure. An unlimited repetition in the pattern |
1404 | makes the tree of infinite size, but it is still a tree. Matching the |
1405 | pattern to a given subject string (from a given starting point) can be |
1406 | thought of as a search of the tree. There are two ways to search a |
1407 | tree: depth-first and breadth-first, and these correspond to the two |
1408 | matching algorithms provided by PCRE. |
1409 | |
1410 | |
1411 | THE STANDARD MATCHING ALGORITHM |
1412 | |
1413 | In the terminology of Jeffrey Friedl's book "Mastering Regular Expres- |
1414 | sions", the standard algorithm is an "NFA algorithm". It conducts a |
1415 | depth-first search of the pattern tree. That is, it proceeds along a |
1416 | single path through the tree, checking that the subject matches what is |
1417 | required. When there is a mismatch, the algorithm tries any alterna- |
1418 | tives at the current point, and if they all fail, it backs up to the |
1419 | previous branch point in the tree, and tries the next alternative |
1420 | branch at that level. This often involves backing up (moving to the |
1421 | left) in the subject string as well. The order in which repetition |
1422 | branches are tried is controlled by the greedy or ungreedy nature of |
1423 | the quantifier. |
1424 | |
1425 | If a leaf node is reached, a matching string has been found, and at |
1426 | that point the algorithm stops. Thus, if there is more than one possi- |
1427 | ble match, this algorithm returns the first one that it finds. Whether |
1428 | this is the shortest, the longest, or some intermediate length depends |
1429 | on the way the greedy and ungreedy repetition quantifiers are specified |
1430 | in the pattern. |
1431 | |
1432 | Because it ends up with a single path through the tree, it is rela- |
1433 | tively straightforward for this algorithm to keep track of the sub- |
1434 | strings that are matched by portions of the pattern in parentheses. |
1435 | This provides support for capturing parentheses and back references. |
1436 | |
1437 | |
1438 | THE ALTERNATIVE MATCHING ALGORITHM |
1439 | |
1440 | This algorithm conducts a breadth-first search of the tree. Starting |
1441 | from the first matching point in the subject, it scans the subject |
1442 | string from left to right, once, character by character, and as it does |
1443 | this, it remembers all the paths through the tree that represent valid |
1444 | matches. In Friedl's terminology, this is a kind of "DFA algorithm", |
1445 | though it is not implemented as a traditional finite state machine (it |
1446 | keeps multiple states active simultaneously). |
1447 | |
1448 | Although the general principle of this matching algorithm is that it |
1449 | scans the subject string only once, without backtracking, there is one |
1450 | exception: when a lookaround assertion is encountered, the characters |
1451 | following or preceding the current point have to be independently in- |
1452 | spected. |
1453 | |
1454 | The scan continues until either the end of the subject is reached, or |
1455 | there are no more unterminated paths. At this point, terminated paths |
1456 | represent the different matching possibilities (if there are none, the |
1457 | match has failed). Thus, if there is more than one possible match, |
1458 | this algorithm finds all of them, and in particular, it finds the long- |
1459 | est. The matches are returned in decreasing order of length. There is |
1460 | an option to stop the algorithm after the first match (which is neces- |
1461 | sarily the shortest) is found. |
1462 | |
1463 | Note that all the matches that are found start at the same point in the |
1464 | subject. If the pattern |
1465 | |
1466 | cat(er(pillar)?)? |
1467 | |
1468 | is matched against the string "the caterpillar catchment", the result |
1469 | will be the three strings "caterpillar", "cater", and "cat" that start |
1470 | at the fifth character of the subject. The algorithm does not automati- |
1471 | cally move on to find matches that start at later positions. |
1472 | |
1473 | PCRE's "auto-possessification" optimization usually applies to charac- |
1474 | ter repeats at the end of a pattern (as well as internally). For exam- |
1475 | ple, the pattern "a\d+" is compiled as if it were "a\d++" because there |
1476 | is no point even considering the possibility of backtracking into the |
1477 | repeated digits. For DFA matching, this means that only one possible |
1478 | match is found. If you really do want multiple matches in such cases, |
1479 | either use an ungreedy repeat ("a\d+?") or set the PCRE_NO_AUTO_POSSESS |
1480 | option when compiling. |
1481 | |
1482 | There are a number of features of PCRE regular expressions that are not |
1483 | supported by the alternative matching algorithm. They are as follows: |
1484 | |
1485 | 1. Because the algorithm finds all possible matches, the greedy or un- |
1486 | greedy nature of repetition quantifiers is not relevant. Greedy and un- |
1487 | greedy quantifiers are treated in exactly the same way. However, pos- |
1488 | sessive quantifiers can make a difference when what follows could also |
1489 | match what is quantified, for example in a pattern like this: |
1490 | |
1491 | ^a++\w! |
1492 | |
1493 | This pattern matches "aaab!" but not "aaa!", which would be matched by |
1494 | a non-possessive quantifier. Similarly, if an atomic group is present, |
1495 | it is matched as if it were a standalone pattern at the current point, |
1496 | and the longest match is then "locked in" for the rest of the overall |
1497 | pattern. |
1498 | |
1499 | 2. When dealing with multiple paths through the tree simultaneously, it |
1500 | is not straightforward to keep track of captured substrings for the |
1501 | different matching possibilities, and PCRE's implementation of this al- |
1502 | gorithm does not attempt to do this. This means that no captured sub- |
1503 | strings are available. |
1504 | |
1505 | 3. Because no substrings are captured, back references within the pat- |
1506 | tern are not supported, and cause errors if encountered. |
1507 | |
1508 | 4. For the same reason, conditional expressions that use a backrefer- |
1509 | ence as the condition or test for a specific group recursion are not |
1510 | supported. |
1511 | |
1512 | 5. Because many paths through the tree may be active, the \K escape se- |
1513 | quence, which resets the start of the match when encountered (but may |
1514 | be on some paths and not on others), is not supported. It causes an er- |
1515 | ror if encountered. |
1516 | |
1517 | 6. Callouts are supported, but the value of the capture_top field is |
1518 | always 1, and the value of the capture_last field is always -1. |
1519 | |
1520 | 7. The \C escape sequence, which (in the standard algorithm) always |
1521 | matches a single data unit, even in UTF-8, UTF-16 or UTF-32 modes, is |
1522 | not supported in these modes, because the alternative algorithm moves |
1523 | through the subject string one character (not data unit) at a time, for |
1524 | all active paths through the tree. |
1525 | |
1526 | 8. Except for (*FAIL), the backtracking control verbs such as (*PRUNE) |
1527 | are not supported. (*FAIL) is supported, and behaves like a failing |
1528 | negative assertion. |
1529 | |
1530 | |
1531 | ADVANTAGES OF THE ALTERNATIVE ALGORITHM |
1532 | |
1533 | Using the alternative matching algorithm provides the following advan- |
1534 | tages: |
1535 | |
1536 | 1. All possible matches (at a single point in the subject) are automat- |
1537 | ically found, and in particular, the longest match is found. To find |
1538 | more than one match using the standard algorithm, you have to do kludgy |
1539 | things with callouts. |
1540 | |
1541 | 2. Because the alternative algorithm scans the subject string just |
1542 | once, and never needs to backtrack (except for lookbehinds), it is pos- |
1543 | sible to pass very long subject strings to the matching function in |
1544 | several pieces, checking for partial matching each time. Although it is |
1545 | possible to do multi-segment matching using the standard algorithm by |
1546 | retaining partially matched substrings, it is more complicated. The |
1547 | pcrepartial documentation gives details of partial matching and dis- |
1548 | cusses multi-segment matching. |
1549 | |
1550 | |
1551 | DISADVANTAGES OF THE ALTERNATIVE ALGORITHM |
1552 | |
1553 | The alternative algorithm suffers from a number of disadvantages: |
1554 | |
1555 | 1. It is substantially slower than the standard algorithm. This is |
1556 | partly because it has to search for all possible matches, but is also |
1557 | because it is less susceptible to optimization. |
1558 | |
1559 | 2. Capturing parentheses and back references are not supported. |
1560 | |
1561 | 3. Although atomic groups are supported, their use does not provide the |
1562 | performance advantage that it does for the standard algorithm. |
1563 | |
1564 | |
1565 | AUTHOR |
1566 | |
1567 | Philip Hazel |
1568 | University Computing Service |
1569 | Cambridge CB2 3QH, England. |
1570 | |
1571 | |
1572 | REVISION |
1573 | |
1574 | Last updated: 12 November 2013 |
1575 | Copyright (c) 1997-2012 University of Cambridge. |
1576 | ------------------------------------------------------------------------------ |
1577 | |
1578 | |
1579 | PCREAPI(3) Library Functions Manual PCREAPI(3) |
1580 | |
1581 | |
1582 | |
1583 | NAME |
1584 | PCRE - Perl-compatible regular expressions |
1585 | |
1586 | #include <pcre.h> |
1587 | |
1588 | |
1589 | PCRE NATIVE API BASIC FUNCTIONS |
1590 | |
1591 | pcre *pcre_compile(const char *pattern, int options, |
1592 | const char **errptr, int *erroffset, |
1593 | const unsigned char *tableptr); |
1594 | |
1595 | pcre *pcre_compile2(const char *pattern, int options, |
1596 | int *errorcodeptr, |
1597 | const char **errptr, int *erroffset, |
1598 | const unsigned char *tableptr); |
1599 | |
1600 | pcre_extra *pcre_study(const pcre *code, int options, |
1601 | const char **errptr); |
1602 | |
1603 | void pcre_free_study(pcre_extra *extra); |
1604 | |
1605 | int pcre_exec(const pcre *code, const pcre_extra *extra, |
1606 | const char *subject, int length, int startoffset, |
1607 | int options, int *ovector, int ovecsize); |
1608 | |
1609 | int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, |
1610 | const char *subject, int length, int startoffset, |
1611 | int options, int *ovector, int ovecsize, |
1612 | int *workspace, int wscount); |
1613 | |
1614 | |
1615 | PCRE NATIVE API STRING EXTRACTION FUNCTIONS |
1616 | |
1617 | int pcre_copy_named_substring(const pcre *code, |
1618 | const char *subject, int *ovector, |
1619 | int stringcount, const char *stringname, |
1620 | char *buffer, int buffersize); |
1621 | |
1622 | int pcre_copy_substring(const char *subject, int *ovector, |
1623 | int stringcount, int stringnumber, char *buffer, |
1624 | int buffersize); |
1625 | |
1626 | int pcre_get_named_substring(const pcre *code, |
1627 | const char *subject, int *ovector, |
1628 | int stringcount, const char *stringname, |
1629 | const char **stringptr); |
1630 | |
1631 | int pcre_get_stringnumber(const pcre *code, |
1632 | const char *name); |
1633 | |
1634 | int pcre_get_stringtable_entries(const pcre *code, |
1635 | const char *name, char **first, char **last); |
1636 | |
1637 | int pcre_get_substring(const char *subject, int *ovector, |
1638 | int stringcount, int stringnumber, |
1639 | const char **stringptr); |
1640 | |
1641 | int pcre_get_substring_list(const char *subject, |
1642 | int *ovector, int stringcount, const char ***listptr); |
1643 | |
1644 | void pcre_free_substring(const char *stringptr); |
1645 | |
1646 | void pcre_free_substring_list(const char **stringptr); |
1647 | |
1648 | |
1649 | PCRE NATIVE API AUXILIARY FUNCTIONS |
1650 | |
1651 | int pcre_jit_exec(const pcre *code, const pcre_extra *extra, |
1652 | const char *subject, int length, int startoffset, |
1653 | int options, int *ovector, int ovecsize, |
1654 | pcre_jit_stack *jstack); |
1655 | |
1656 | pcre_jit_stack *pcre_jit_stack_alloc(int startsize, int maxsize); |
1657 | |
1658 | void pcre_jit_stack_free(pcre_jit_stack *stack); |
1659 | |
1660 | void pcre_assign_jit_stack(pcre_extra *extra, |
1661 | pcre_jit_callback callback, void *data); |
1662 | |
1663 | const unsigned char *pcre_maketables(void); |
1664 | |
1665 | int pcre_fullinfo(const pcre *code, const pcre_extra *extra, |
1666 | int what, void *where); |
1667 | |
1668 | int pcre_refcount(pcre *code, int adjust); |
1669 | |
1670 | int pcre_config(int what, void *where); |
1671 | |
1672 | const char *pcre_version(void); |
1673 | |
1674 | int pcre_pattern_to_host_byte_order(pcre *code, |
1675 | pcre_extra *extra, const unsigned char *tables); |
1676 | |
1677 | |
1678 | PCRE NATIVE API INDIRECTED FUNCTIONS |
1679 | |
1680 | void *(*pcre_malloc)(size_t); |
1681 | |
1682 | void (*pcre_free)(void *); |
1683 | |
1684 | void *(*pcre_stack_malloc)(size_t); |
1685 | |
1686 | void (*pcre_stack_free)(void *); |
1687 | |
1688 | int (*pcre_callout)(pcre_callout_block *); |
1689 | |
1690 | int (*pcre_stack_guard)(void); |
1691 | |
1692 | |
1693 | PCRE 8-BIT, 16-BIT, AND 32-BIT LIBRARIES |
1694 | |
1695 | As well as support for 8-bit character strings, PCRE also supports |
1696 | 16-bit strings (from release 8.30) and 32-bit strings (from release |
1697 | 8.32), by means of two additional libraries. They can be built as well |
1698 | as, or instead of, the 8-bit library. To avoid too much complication, |
1699 | this document describes the 8-bit versions of the functions, with only |
1700 | occasional references to the 16-bit and 32-bit libraries. |
1701 | |
1702 | The 16-bit and 32-bit functions operate in the same way as their 8-bit |
1703 | counterparts; they just use different data types for their arguments |
1704 | and results, and their names start with pcre16_ or pcre32_ instead of |
1705 | pcre_. For every option that has UTF8 in its name (for example, |
1706 | PCRE_UTF8), there are corresponding 16-bit and 32-bit names with UTF8 |
1707 | replaced by UTF16 or UTF32, respectively. This facility is in fact just |
1708 | cosmetic; the 16-bit and 32-bit option names define the same bit val- |
1709 | ues. |
1710 | |
1711 | References to bytes and UTF-8 in this document should be read as refer- |
1712 | ences to 16-bit data units and UTF-16 when using the 16-bit library, or |
1713 | 32-bit data units and UTF-32 when using the 32-bit library, unless |
1714 | specified otherwise. More details of the specific differences for the |
1715 | 16-bit and 32-bit libraries are given in the pcre16 and pcre32 pages. |
1716 | |
1717 | |
1718 | PCRE API OVERVIEW |
1719 | |
1720 | PCRE has its own native API, which is described in this document. There |
1721 | are also some wrapper functions (for the 8-bit library only) that cor- |
1722 | respond to the POSIX regular expression API, but they do not give ac- |
1723 | cess to all the functionality. They are described in the pcreposix doc- |
1724 | umentation. Both of these APIs define a set of C function calls. A C++ |
1725 | wrapper (again for the 8-bit library only) is also distributed with |
1726 | PCRE. It is documented in the pcrecpp page. |
1727 | |
1728 | The native API C function prototypes are defined in the header file |
1729 | pcre.h, and on Unix-like systems the (8-bit) library itself is called |
1730 | libpcre. It can normally be accessed by adding -lpcre to the command |
1731 | for linking an application that uses PCRE. The header file defines the |
1732 | macros PCRE_MAJOR and PCRE_MINOR to contain the major and minor release |
1733 | numbers for the library. Applications can use these to include support |
1734 | for different releases of PCRE. |
1735 | |
1736 | In a Windows environment, if you want to statically link an application |
1737 | program against a non-dll pcre.a file, you must define PCRE_STATIC be- |
1738 | fore including pcre.h or pcrecpp.h, because otherwise the pcre_malloc() |
1739 | and pcre_free() exported functions will be declared __declspec(dl- |
1740 | limport), with unwanted results. |
1741 | |
1742 | The functions pcre_compile(), pcre_compile2(), pcre_study(), and |
1743 | pcre_exec() are used for compiling and matching regular expressions in |
1744 | a Perl-compatible manner. A sample program that demonstrates the sim- |
1745 | plest way of using them is provided in the file called pcredemo.c in |
1746 | the PCRE source distribution. A listing of this program is given in the |
1747 | pcredemo documentation, and the pcresample documentation describes how |
1748 | to compile and run it. |
1749 | |
1750 | Just-in-time compiler support is an optional feature of PCRE that can |
1751 | be built in appropriate hardware environments. It greatly speeds up the |
1752 | matching performance of many patterns. Simple programs can easily re- |
1753 | quest that it be used if available, by setting an option that is ig- |
1754 | nored when it is not relevant. More complicated programs might need to |
1755 | make use of the functions pcre_jit_stack_alloc(), |
1756 | pcre_jit_stack_free(), and pcre_assign_jit_stack() in order to control |
1757 | the JIT code's memory usage. |
1758 | |
1759 | From release 8.32 there is also a direct interface for JIT execution, |
1760 | which gives improved performance. The JIT-specific functions are dis- |
1761 | cussed in the pcrejit documentation. |
1762 | |
1763 | A second matching function, pcre_dfa_exec(), which is not Perl-compati- |
1764 | ble, is also provided. This uses a different algorithm for the match- |
1765 | ing. The alternative algorithm finds all possible matches (at a given |
1766 | point in the subject), and scans the subject just once (unless there |
1767 | are lookbehind assertions). However, this algorithm does not return |
1768 | captured substrings. A description of the two matching algorithms and |
1769 | their advantages and disadvantages is given in the pcrematching docu- |
1770 | mentation. |
1771 | |
1772 | In addition to the main compiling and matching functions, there are |
1773 | convenience functions for extracting captured substrings from a subject |
1774 | string that is matched by pcre_exec(). They are: |
1775 | |
1776 | pcre_copy_substring() |
1777 | pcre_copy_named_substring() |
1778 | pcre_get_substring() |
1779 | pcre_get_named_substring() |
1780 | pcre_get_substring_list() |
1781 | pcre_get_stringnumber() |
1782 | pcre_get_stringtable_entries() |
1783 | |
1784 | pcre_free_substring() and pcre_free_substring_list() are also provided, |
1785 | to free the memory used for extracted strings. |
1786 | |
1787 | The function pcre_maketables() is used to build a set of character ta- |
1788 | bles in the current locale for passing to pcre_compile(), pcre_exec(), |
1789 | or pcre_dfa_exec(). This is an optional facility that is provided for |
1790 | specialist use. Most commonly, no special tables are passed, in which |
1791 | case internal tables that are generated when PCRE is built are used. |
1792 | |
1793 | The function pcre_fullinfo() is used to find out information about a |
1794 | compiled pattern. The function pcre_version() returns a pointer to a |
1795 | string containing the version of PCRE and its date of release. |
1796 | |
1797 | The function pcre_refcount() maintains a reference count in a data |
1798 | block containing a compiled pattern. This is provided for the benefit |
1799 | of object-oriented applications. |
1800 | |
1801 | The global variables pcre_malloc and pcre_free initially contain the |
1802 | entry points of the standard malloc() and free() functions, respec- |
1803 | tively. PCRE calls the memory management functions via these variables, |
1804 | so a calling program can replace them if it wishes to intercept the |
1805 | calls. This should be done before calling any PCRE functions. |
1806 | |
1807 | The global variables pcre_stack_malloc and pcre_stack_free are also in- |
1808 | directions to memory management functions. These special functions are |
1809 | used only when PCRE is compiled to use the heap for remembering data, |
1810 | instead of recursive function calls, when running the pcre_exec() func- |
1811 | tion. See the pcrebuild documentation for details of how to do this. It |
1812 | is a non-standard way of building PCRE, for use in environments that |
1813 | have limited stacks. Because of the greater use of memory management, |
1814 | it runs more slowly. Separate functions are provided so that special- |
1815 | purpose external code can be used for this case. When used, these func- |
1816 | tions always allocate memory blocks of the same size. There is a dis- |
1817 | cussion about PCRE's stack usage in the pcrestack documentation. |
1818 | |
1819 | The global variable pcre_callout initially contains NULL. It can be set |
1820 | by the caller to a "callout" function, which PCRE will then call at |
1821 | specified points during a matching operation. Details are given in the |
1822 | pcrecallout documentation. |
1823 | |
1824 | The global variable pcre_stack_guard initially contains NULL. It can be |
1825 | set by the caller to a function that is called by PCRE whenever it |
1826 | starts to compile a parenthesized part of a pattern. When parentheses |
1827 | are nested, PCRE uses recursive function calls, which use up the system |
1828 | stack. This function is provided so that applications with restricted |
1829 | stacks can force a compilation error if the stack runs out. The func- |
1830 | tion should return zero if all is well, or non-zero to force an error. |
1831 | |
1832 | |
1833 | NEWLINES |
1834 | |
1835 | PCRE supports five different conventions for indicating line breaks in |
1836 | strings: a single CR (carriage return) character, a single LF (line- |
1837 | feed) character, the two-character sequence CRLF, any of the three pre- |
1838 | ceding, or any Unicode newline sequence. The Unicode newline sequences |
1839 | are the three just mentioned, plus the single characters VT (vertical |
1840 | tab, U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line |
1841 | separator, U+2028), and PS (paragraph separator, U+2029). |
1842 | |
1843 | Each of the first three conventions is used by at least one operating |
1844 | system as its standard newline sequence. When PCRE is built, a default |
1845 | can be specified. The default default is LF, which is the Unix stan- |
1846 | dard. When PCRE is run, the default can be overridden, either when a |
1847 | pattern is compiled, or when it is matched. |
1848 | |
1849 | At compile time, the newline convention can be specified by the options |
1850 | argument of pcre_compile(), or it can be specified by special text at |
1851 | the start of the pattern itself; this overrides any other settings. See |
1852 | the pcrepattern page for details of the special character sequences. |
1853 | |
1854 | In the PCRE documentation the word "newline" is used to mean "the char- |
1855 | acter or pair of characters that indicate a line break". The choice of |
1856 | newline convention affects the handling of the dot, circumflex, and |
1857 | dollar metacharacters, the handling of #-comments in /x mode, and, when |
1858 | CRLF is a recognized line ending sequence, the match position advance- |
1859 | ment for a non-anchored pattern. There is more detail about this in the |
1860 | section on pcre_exec() options below. |
1861 | |
1862 | The choice of newline convention does not affect the interpretation of |
1863 | the \n or \r escape sequences, nor does it affect what \R matches, |
1864 | which is controlled in a similar way, but by separate options. |
1865 | |
1866 | |
1867 | MULTITHREADING |
1868 | |
1869 | The PCRE functions can be used in multi-threading applications, with |
1870 | the proviso that the memory management functions pointed to by |
1871 | pcre_malloc, pcre_free, pcre_stack_malloc, and pcre_stack_free, and the |
1872 | callout and stack-checking functions pointed to by pcre_callout and |
1873 | pcre_stack_guard, are shared by all threads. |
1874 | |
1875 | The compiled form of a regular expression is not altered during match- |
1876 | ing, so the same compiled pattern can safely be used by several threads |
1877 | at once. |
1878 | |
1879 | If the just-in-time optimization feature is being used, it needs sepa- |
1880 | rate memory stack areas for each thread. See the pcrejit documentation |
1881 | for more details. |
1882 | |
1883 | |
1884 | SAVING PRECOMPILED PATTERNS FOR LATER USE |
1885 | |
1886 | The compiled form of a regular expression can be saved and re-used at a |
1887 | later time, possibly by a different program, and even on a host other |
1888 | than the one on which it was compiled. Details are given in the |
1889 | pcreprecompile documentation, which includes a description of the |
1890 | pcre_pattern_to_host_byte_order() function. However, compiling a regu- |
1891 | lar expression with one version of PCRE for use with a different ver- |
1892 | sion is not guaranteed to work and may cause crashes. |
1893 | |
1894 | |
1895 | CHECKING BUILD-TIME OPTIONS |
1896 | |
1897 | int pcre_config(int what, void *where); |
1898 | |
1899 | The function pcre_config() makes it possible for a PCRE client to dis- |
1900 | cover which optional features have been compiled into the PCRE library. |
1901 | The pcrebuild documentation has more details about these optional fea- |
1902 | tures. |
1903 | |
1904 | The first argument for pcre_config() is an integer, specifying which |
1905 | information is required; the second argument is a pointer to a variable |
1906 | into which the information is placed. The returned value is zero on |
1907 | success, or the negative error code PCRE_ERROR_BADOPTION if the value |
1908 | in the first argument is not recognized. The following information is |
1909 | available: |
1910 | |
1911 | PCRE_CONFIG_UTF8 |
1912 | |
1913 | The output is an integer that is set to one if UTF-8 support is avail- |
1914 | able; otherwise it is set to zero. This value should normally be given |
1915 | to the 8-bit version of this function, pcre_config(). If it is given to |
1916 | the 16-bit or 32-bit version of this function, the result is PCRE_ER- |
1917 | ROR_BADOPTION. |
1918 | |
1919 | PCRE_CONFIG_UTF16 |
1920 | |
1921 | The output is an integer that is set to one if UTF-16 support is avail- |
1922 | able; otherwise it is set to zero. This value should normally be given |
1923 | to the 16-bit version of this function, pcre16_config(). If it is given |
1924 | to the 8-bit or 32-bit version of this function, the result is PCRE_ER- |
1925 | ROR_BADOPTION. |
1926 | |
1927 | PCRE_CONFIG_UTF32 |
1928 | |
1929 | The output is an integer that is set to one if UTF-32 support is avail- |
1930 | able; otherwise it is set to zero. This value should normally be given |
1931 | to the 32-bit version of this function, pcre32_config(). If it is given |
1932 | to the 8-bit or 16-bit version of this function, the result is PCRE_ER- |
1933 | ROR_BADOPTION. |
1934 | |
1935 | PCRE_CONFIG_UNICODE_PROPERTIES |
1936 | |
1937 | The output is an integer that is set to one if support for Unicode |
1938 | character properties is available; otherwise it is set to zero. |
1939 | |
1940 | PCRE_CONFIG_JIT |
1941 | |
1942 | The output is an integer that is set to one if support for just-in-time |
1943 | compiling is available; otherwise it is set to zero. |
1944 | |
1945 | PCRE_CONFIG_JITTARGET |
1946 | |
1947 | The output is a pointer to a zero-terminated "const char *" string. If |
1948 | JIT support is available, the string contains the name of the architec- |
1949 | ture for which the JIT compiler is configured, for example "x86 32bit |
1950 | (little endian + unaligned)". If JIT support is not available, the re- |
1951 | sult is NULL. |
1952 | |
1953 | PCRE_CONFIG_NEWLINE |
1954 | |
1955 | The output is an integer whose value specifies the default character |
1956 | sequence that is recognized as meaning "newline". The values that are |
1957 | supported in ASCII/Unicode environments are: 10 for LF, 13 for CR, 3338 |
1958 | for CRLF, -2 for ANYCRLF, and -1 for ANY. In EBCDIC environments, CR, |
1959 | ANYCRLF, and ANY yield the same values. However, the value for LF is |
1960 | normally 21, though some EBCDIC environments use 37. The corresponding |
1961 | values for CRLF are 3349 and 3365. The default should normally corre- |
1962 | spond to the standard sequence for your operating system. |
1963 | |
1964 | PCRE_CONFIG_BSR |
1965 | |
1966 | The output is an integer whose value indicates what character sequences |
1967 | the \R escape sequence matches by default. A value of 0 means that \R |
1968 | matches any Unicode line ending sequence; a value of 1 means that \R |
1969 | matches only CR, LF, or CRLF. The default can be overridden when a pat- |
1970 | tern is compiled or matched. |
1971 | |
1972 | PCRE_CONFIG_LINK_SIZE |
1973 | |
1974 | The output is an integer that contains the number of bytes used for in- |
1975 | ternal linkage in compiled regular expressions. For the 8-bit library, |
1976 | the value can be 2, 3, or 4. For the 16-bit library, the value is ei- |
1977 | ther 2 or 4 and is still a number of bytes. For the 32-bit library, the |
1978 | value is either 2 or 4 and is still a number of bytes. The default |
1979 | value of 2 is sufficient for all but the most massive patterns, since |
1980 | it allows the compiled pattern to be up to 64K in size. Larger values |
1981 | allow larger regular expressions to be compiled, at the expense of |
1982 | slower matching. |
1983 | |
1984 | PCRE_CONFIG_POSIX_MALLOC_THRESHOLD |
1985 | |
1986 | The output is an integer that contains the threshold above which the |
1987 | POSIX interface uses malloc() for output vectors. Further details are |
1988 | given in the pcreposix documentation. |
1989 | |
1990 | PCRE_CONFIG_PARENS_LIMIT |
1991 | |
1992 | The output is a long integer that gives the maximum depth of nesting of |
1993 | parentheses (of any kind) in a pattern. This limit is imposed to cap |
1994 | the amount of system stack used when a pattern is compiled. It is spec- |
1995 | ified when PCRE is built; the default is 250. This limit does not take |
1996 | into account the stack that may already be used by the calling applica- |
1997 | tion. For finer control over compilation stack usage, you can set a |
1998 | pointer to an external checking function in pcre_stack_guard. |
1999 | |
2000 | PCRE_CONFIG_MATCH_LIMIT |
2001 | |
2002 | The output is a long integer that gives the default limit for the num- |
2003 | ber of internal matching function calls in a pcre_exec() execution. |
2004 | Further details are given with pcre_exec() below. |
2005 | |
2006 | PCRE_CONFIG_MATCH_LIMIT_RECURSION |
2007 | |
2008 | The output is a long integer that gives the default limit for the depth |
2009 | of recursion when calling the internal matching function in a |
2010 | pcre_exec() execution. Further details are given with pcre_exec() be- |
2011 | low. |
2012 | |
2013 | PCRE_CONFIG_STACKRECURSE |
2014 | |
2015 | The output is an integer that is set to one if internal recursion when |
2016 | running pcre_exec() is implemented by recursive function calls that use |
2017 | the stack to remember their state. This is the usual way that PCRE is |
2018 | compiled. The output is zero if PCRE was compiled to use blocks of data |
2019 | on the heap instead of recursive function calls. In this case, |
2020 | pcre_stack_malloc and pcre_stack_free are called to manage memory |
2021 | blocks on the heap, thus avoiding the use of the stack. |
2022 | |
2023 | |
2024 | COMPILING A PATTERN |
2025 | |
2026 | pcre *pcre_compile(const char *pattern, int options, |
2027 | const char **errptr, int *erroffset, |
2028 | const unsigned char *tableptr); |
2029 | |
2030 | pcre *pcre_compile2(const char *pattern, int options, |
2031 | int *errorcodeptr, |
2032 | const char **errptr, int *erroffset, |
2033 | const unsigned char *tableptr); |
2034 | |
2035 | Either of the functions pcre_compile() or pcre_compile2() can be called |
2036 | to compile a pattern into an internal form. The only difference between |
2037 | the two interfaces is that pcre_compile2() has an additional argument, |
2038 | errorcodeptr, via which a numerical error code can be returned. To |
2039 | avoid too much repetition, we refer just to pcre_compile() below, but |
2040 | the information applies equally to pcre_compile2(). |
2041 | |
2042 | The pattern is a C string terminated by a binary zero, and is passed in |
2043 | the pattern argument. A pointer to a single block of memory that is ob- |
2044 | tained via pcre_malloc is returned. This contains the compiled code and |
2045 | related data. The pcre type is defined for the returned block; this is |
2046 | a typedef for a structure whose contents are not externally defined. It |
2047 | is up to the caller to free the memory (via pcre_free) when it is no |
2048 | longer required. |
2049 | |
2050 | Although the compiled code of a PCRE regex is relocatable, that is, it |
2051 | does not depend on memory location, the complete pcre data block is not |
2052 | fully relocatable, because it may contain a copy of the tableptr argu- |
2053 | ment, which is an address (see below). |
2054 | |
2055 | The options argument contains various bit settings that affect the com- |
2056 | pilation. It should be zero if no options are required. The available |
2057 | options are described below. Some of them (in particular, those that |
2058 | are compatible with Perl, but some others as well) can also be set and |
2059 | unset from within the pattern (see the detailed description in the |
2060 | pcrepattern documentation). For those options that can be different in |
2061 | different parts of the pattern, the contents of the options argument |
2062 | specifies their settings at the start of compilation and execution. The |
2063 | PCRE_ANCHORED, PCRE_BSR_xxx, PCRE_NEWLINE_xxx, PCRE_NO_UTF8_CHECK, and |
2064 | PCRE_NO_START_OPTIMIZE options can be set at the time of matching as |
2065 | well as at compile time. |
2066 | |
2067 | If errptr is NULL, pcre_compile() returns NULL immediately. Otherwise, |
2068 | if compilation of a pattern fails, pcre_compile() returns NULL, and |
2069 | sets the variable pointed to by errptr to point to a textual error mes- |
2070 | sage. This is a static string that is part of the library. You must not |
2071 | try to free it. Normally, the offset from the start of the pattern to |
2072 | the data unit that was being processed when the error was discovered is |
2073 | placed in the variable pointed to by erroffset, which must not be NULL |
2074 | (if it is, an immediate error is given). However, for an invalid UTF-8 |
2075 | or UTF-16 string, the offset is that of the first data unit of the |
2076 | failing character. |
2077 | |
2078 | Some errors are not detected until the whole pattern has been scanned; |
2079 | in these cases, the offset passed back is the length of the pattern. |
2080 | Note that the offset is in data units, not characters, even in a UTF |
2081 | mode. It may sometimes point into the middle of a UTF-8 or UTF-16 char- |
2082 | acter. |
2083 | |
2084 | If pcre_compile2() is used instead of pcre_compile(), and the error- |
2085 | codeptr argument is not NULL, a non-zero error code number is returned |
2086 | via this argument in the event of an error. This is in addition to the |
2087 | textual error message. Error codes and messages are listed below. |
2088 | |
2089 | If the final argument, tableptr, is NULL, PCRE uses a default set of |
2090 | character tables that are built when PCRE is compiled, using the de- |
2091 | fault C locale. Otherwise, tableptr must be an address that is the re- |
2092 | sult of a call to pcre_maketables(). This value is stored with the com- |
2093 | piled pattern, and used again by pcre_exec() and pcre_dfa_exec() when |
2094 | the pattern is matched. For more discussion, see the section on locale |
2095 | support below. |
2096 | |
2097 | This code fragment shows a typical straightforward call to pcre_com- |
2098 | pile(): |
2099 | |
2100 | pcre *re; |
2101 | const char *error; |
2102 | int erroffset; |
2103 | re = pcre_compile( |
2104 | "^A.*Z", /* the pattern */ |
2105 | 0, /* default options */ |
2106 | &error, /* for error message */ |
2107 | &erroffset, /* for error offset */ |
2108 | NULL); /* use default character tables */ |
2109 | |
2110 | The following names for option bits are defined in the pcre.h header |
2111 | file: |
2112 | |
2113 | PCRE_ANCHORED |
2114 | |
2115 | If this bit is set, the pattern is forced to be "anchored", that is, it |
2116 | is constrained to match only at the first matching point in the string |
2117 | that is being searched (the "subject string"). This effect can also be |
2118 | achieved by appropriate constructs in the pattern itself, which is the |
2119 | only way to do it in Perl. |
2120 | |
2121 | PCRE_AUTO_CALLOUT |
2122 | |
2123 | If this bit is set, pcre_compile() automatically inserts callout items, |
2124 | all with number 255, before each pattern item. For discussion of the |
2125 | callout facility, see the pcrecallout documentation. |
2126 | |
2127 | PCRE_BSR_ANYCRLF |
2128 | PCRE_BSR_UNICODE |
2129 | |
2130 | These options (which are mutually exclusive) control what the \R escape |
2131 | sequence matches. The choice is either to match only CR, LF, or CRLF, |
2132 | or to match any Unicode newline sequence. The default is specified when |
2133 | PCRE is built. It can be overridden from within the pattern, or by set- |
2134 | ting an option when a compiled pattern is matched. |
2135 | |
2136 | PCRE_CASELESS |
2137 | |
2138 | If this bit is set, letters in the pattern match both upper and lower |
2139 | case letters. It is equivalent to Perl's /i option, and it can be |
2140 | changed within a pattern by a (?i) option setting. In UTF-8 mode, PCRE |
2141 | always understands the concept of case for characters whose values are |
2142 | less than 128, so caseless matching is always possible. For characters |
2143 | with higher values, the concept of case is supported if PCRE is com- |
2144 | piled with Unicode property support, but not otherwise. If you want to |
2145 | use caseless matching for characters 128 and above, you must ensure |
2146 | that PCRE is compiled with Unicode property support as well as with |
2147 | UTF-8 support. |
2148 | |
2149 | PCRE_DOLLAR_ENDONLY |
2150 | |
2151 | If this bit is set, a dollar metacharacter in the pattern matches only |
2152 | at the end of the subject string. Without this option, a dollar also |
2153 | matches immediately before a newline at the end of the string (but not |
2154 | before any other newlines). The PCRE_DOLLAR_ENDONLY option is ignored |
2155 | if PCRE_MULTILINE is set. There is no equivalent to this option in |
2156 | Perl, and no way to set it within a pattern. |
2157 | |
2158 | PCRE_DOTALL |
2159 | |
2160 | If this bit is set, a dot metacharacter in the pattern matches a char- |
2161 | acter of any value, including one that indicates a newline. However, it |
2162 | only ever matches one character, even if newlines are coded as CRLF. |
2163 | Without this option, a dot does not match when the current position is |
2164 | at a newline. This option is equivalent to Perl's /s option, and it can |
2165 | be changed within a pattern by a (?s) option setting. A negative class |
2166 | such as [^a] always matches newline characters, independent of the set- |
2167 | ting of this option. |
2168 | |
2169 | PCRE_DUPNAMES |
2170 | |
2171 | If this bit is set, names used to identify capturing subpatterns need |
2172 | not be unique. This can be helpful for certain types of pattern when it |
2173 | is known that only one instance of the named subpattern can ever be |
2174 | matched. There are more details of named subpatterns below; see also |
2175 | the pcrepattern documentation. |
2176 | |
2177 | PCRE_EXTENDED |
2178 | |
2179 | If this bit is set, most white space characters in the pattern are to- |
2180 | tally ignored except when escaped or inside a character class. However, |
2181 | white space is not allowed within sequences such as (?> that introduce |
2182 | various parenthesized subpatterns, nor within a numerical quantifier |
2183 | such as {1,3}. However, ignorable white space is permitted between an |
2184 | item and a following quantifier and between a quantifier and a follow- |
2185 | ing + that indicates possessiveness. |
2186 | |
2187 | White space did not used to include the VT character (code 11), because |
2188 | Perl did not treat this character as white space. However, Perl changed |
2189 | at release 5.18, so PCRE followed at release 8.34, and VT is now |
2190 | treated as white space. |
2191 | |
2192 | PCRE_EXTENDED also causes characters between an unescaped # outside a |
2193 | character class and the next newline, inclusive, to be ignored. |
2194 | PCRE_EXTENDED is equivalent to Perl's /x option, and it can be changed |
2195 | within a pattern by a (?x) option setting. |
2196 | |
2197 | Which characters are interpreted as newlines is controlled by the op- |
2198 | tions passed to pcre_compile() or by a special sequence at the start of |
2199 | the pattern, as described in the section entitled "Newline conventions" |
2200 | in the pcrepattern documentation. Note that the end of this type of |
2201 | comment is a literal newline sequence in the pattern; escape sequences |
2202 | that happen to represent a newline do not count. |
2203 | |
2204 | This option makes it possible to include comments inside complicated |
2205 | patterns. Note, however, that this applies only to data characters. |
2206 | White space characters may never appear within special character se- |
2207 | quences in a pattern, for example within the sequence (?( that intro- |
2208 | duces a conditional subpattern. |
2209 | |
2210 | PCRE_EXTRA |
2211 | |
2212 | This option was invented in order to turn on additional functionality |
2213 | of PCRE that is incompatible with Perl, but it is currently of very |
2214 | little use. When set, any backslash in a pattern that is followed by a |
2215 | letter that has no special meaning causes an error, thus reserving |
2216 | these combinations for future expansion. By default, as in Perl, a |
2217 | backslash followed by a letter with no special meaning is treated as a |
2218 | literal. (Perl can, however, be persuaded to give an error for this, by |
2219 | running it with the -w option.) There are at present no other features |
2220 | controlled by this option. It can also be set by a (?X) option setting |
2221 | within a pattern. |
2222 | |
2223 | PCRE_FIRSTLINE |
2224 | |
2225 | If this option is set, an unanchored pattern is required to match be- |
2226 | fore or at the first newline in the subject string, though the matched |
2227 | text may continue over the newline. |
2228 | |
2229 | PCRE_JAVASCRIPT_COMPAT |
2230 | |
2231 | If this option is set, PCRE's behaviour is changed in some ways so that |
2232 | it is compatible with JavaScript rather than Perl. The changes are as |
2233 | follows: |
2234 | |
2235 | (1) A lone closing square bracket in a pattern causes a compile-time |
2236 | error, because this is illegal in JavaScript (by default it is treated |
2237 | as a data character). Thus, the pattern AB]CD becomes illegal when this |
2238 | option is set. |
2239 | |
2240 | (2) At run time, a back reference to an unset subpattern group matches |
2241 | an empty string (by default this causes the current matching alterna- |
2242 | tive to fail). A pattern such as (\1)(a) succeeds when this option is |
2243 | set (assuming it can find an "a" in the subject), whereas it fails by |
2244 | default, for Perl compatibility. |
2245 | |
2246 | (3) \U matches an upper case "U" character; by default \U causes a com- |
2247 | pile time error (Perl uses \U to upper case subsequent characters). |
2248 | |
2249 | (4) \u matches a lower case "u" character unless it is followed by four |
2250 | hexadecimal digits, in which case the hexadecimal number defines the |
2251 | code point to match. By default, \u causes a compile time error (Perl |
2252 | uses it to upper case the following character). |
2253 | |
2254 | (5) \x matches a lower case "x" character unless it is followed by two |
2255 | hexadecimal digits, in which case the hexadecimal number defines the |
2256 | code point to match. By default, as in Perl, a hexadecimal number is |
2257 | always expected after \x, but it may have zero, one, or two digits (so, |
2258 | for example, \xz matches a binary zero character followed by z). |
2259 | |
2260 | PCRE_MULTILINE |
2261 | |
2262 | By default, for the purposes of matching "start of line" and "end of |
2263 | line", PCRE treats the subject string as consisting of a single line of |
2264 | characters, even if it actually contains newlines. The "start of line" |
2265 | metacharacter (^) matches only at the start of the string, and the "end |
2266 | of line" metacharacter ($) matches only at the end of the string, or |
2267 | before a terminating newline (except when PCRE_DOLLAR_ENDONLY is set). |
2268 | Note, however, that unless PCRE_DOTALL is set, the "any character" |
2269 | metacharacter (.) does not match at a newline. This behaviour (for ^, |
2270 | $, and dot) is the same as Perl. |
2271 | |
2272 | When PCRE_MULTILINE it is set, the "start of line" and "end of line" |
2273 | constructs match immediately following or immediately before internal |
2274 | newlines in the subject string, respectively, as well as at the very |
2275 | start and end. This is equivalent to Perl's /m option, and it can be |
2276 | changed within a pattern by a (?m) option setting. If there are no new- |
2277 | lines in a subject string, or no occurrences of ^ or $ in a pattern, |
2278 | setting PCRE_MULTILINE has no effect. |
2279 | |
2280 | PCRE_NEVER_UTF |
2281 | |
2282 | This option locks out interpretation of the pattern as UTF-8 (or UTF-16 |
2283 | or UTF-32 in the 16-bit and 32-bit libraries). In particular, it pre- |
2284 | vents the creator of the pattern from switching to UTF interpretation |
2285 | by starting the pattern with (*UTF). This may be useful in applications |
2286 | that process patterns from external sources. The combination of |
2287 | PCRE_UTF8 and PCRE_NEVER_UTF also causes an error. |
2288 | |
2289 | PCRE_NEWLINE_CR |
2290 | PCRE_NEWLINE_LF |
2291 | PCRE_NEWLINE_CRLF |
2292 | PCRE_NEWLINE_ANYCRLF |
2293 | PCRE_NEWLINE_ANY |
2294 | |
2295 | These options override the default newline definition that was chosen |
2296 | when PCRE was built. Setting the first or the second specifies that a |
2297 | newline is indicated by a single character (CR or LF, respectively). |
2298 | Setting PCRE_NEWLINE_CRLF specifies that a newline is indicated by the |
2299 | two-character CRLF sequence. Setting PCRE_NEWLINE_ANYCRLF specifies |
2300 | that any of the three preceding sequences should be recognized. Setting |
2301 | PCRE_NEWLINE_ANY specifies that any Unicode newline sequence should be |
2302 | recognized. |
2303 | |
2304 | In an ASCII/Unicode environment, the Unicode newline sequences are the |
2305 | three just mentioned, plus the single characters VT (vertical tab, |
2306 | U+000B), FF (form feed, U+000C), NEL (next line, U+0085), LS (line sep- |
2307 | arator, U+2028), and PS (paragraph separator, U+2029). For the 8-bit |
2308 | library, the last two are recognized only in UTF-8 mode. |
2309 | |
2310 | When PCRE is compiled to run in an EBCDIC (mainframe) environment, the |
2311 | code for CR is 0x0d, the same as ASCII. However, the character code for |
2312 | LF is normally 0x15, though in some EBCDIC environments 0x25 is used. |
2313 | Whichever of these is not LF is made to correspond to Unicode's NEL |
2314 | character. EBCDIC codes are all less than 256. For more details, see |
2315 | the pcrebuild documentation. |
2316 | |
2317 | The newline setting in the options word uses three bits that are |
2318 | treated as a number, giving eight possibilities. Currently only six are |
2319 | used (default plus the five values above). This means that if you set |
2320 | more than one newline option, the combination may or may not be sensi- |
2321 | ble. For example, PCRE_NEWLINE_CR with PCRE_NEWLINE_LF is equivalent to |
2322 | PCRE_NEWLINE_CRLF, but other combinations may yield unused numbers and |
2323 | cause an error. |
2324 | |
2325 | The only time that a line break in a pattern is specially recognized |
2326 | when compiling is when PCRE_EXTENDED is set. CR and LF are white space |
2327 | characters, and so are ignored in this mode. Also, an unescaped # out- |
2328 | side a character class indicates a comment that lasts until after the |
2329 | next line break sequence. In other circumstances, line break sequences |
2330 | in patterns are treated as literal data. |
2331 | |
2332 | The newline option that is set at compile time becomes the default that |
2333 | is used for pcre_exec() and pcre_dfa_exec(), but it can be overridden. |
2334 | |
2335 | PCRE_NO_AUTO_CAPTURE |
2336 | |
2337 | If this option is set, it disables the use of numbered capturing paren- |
2338 | theses in the pattern. Any opening parenthesis that is not followed by |
2339 | ? behaves as if it were followed by ?: but named parentheses can still |
2340 | be used for capturing (and they acquire numbers in the usual way). |
2341 | There is no equivalent of this option in Perl. |
2342 | |
2343 | PCRE_NO_AUTO_POSSESS |
2344 | |
2345 | If this option is set, it disables "auto-possessification". This is an |
2346 | optimization that, for example, turns a+b into a++b in order to avoid |
2347 | backtracks into a+ that can never be successful. However, if callouts |
2348 | are in use, auto-possessification means that some of them are never |
2349 | taken. You can set this option if you want the matching functions to do |
2350 | a full unoptimized search and run all the callouts, but it is mainly |
2351 | provided for testing purposes. |
2352 | |
2353 | PCRE_NO_START_OPTIMIZE |
2354 | |
2355 | This is an option that acts at matching time; that is, it is really an |
2356 | option for pcre_exec() or pcre_dfa_exec(). If it is set at compile |
2357 | time, it is remembered with the compiled pattern and assumed at match- |
2358 | ing time. This is necessary if you want to use JIT execution, because |
2359 | the JIT compiler needs to know whether or not this option is set. For |
2360 | details see the discussion of PCRE_NO_START_OPTIMIZE below. |
2361 | |
2362 | PCRE_UCP |
2363 | |
2364 | This option changes the way PCRE processes \B, \b, \D, \d, \S, \s, \W, |
2365 | \w, and some of the POSIX character classes. By default, only ASCII |
2366 | characters are recognized, but if PCRE_UCP is set, Unicode properties |
2367 | are used instead to classify characters. More details are given in the |
2368 | section on generic character types in the pcrepattern page. If you set |
2369 | PCRE_UCP, matching one of the items it affects takes much longer. The |
2370 | option is available only if PCRE has been compiled with Unicode prop- |
2371 | erty support. |
2372 | |
2373 | PCRE_UNGREEDY |
2374 | |
2375 | This option inverts the "greediness" of the quantifiers so that they |
2376 | are not greedy by default, but become greedy if followed by "?". It is |
2377 | not compatible with Perl. It can also be set by a (?U) option setting |
2378 | within the pattern. |
2379 | |
2380 | PCRE_UTF8 |
2381 | |
2382 | This option causes PCRE to regard both the pattern and the subject as |
2383 | strings of UTF-8 characters instead of single-byte strings. However, it |
2384 | is available only when PCRE is built to include UTF support. If not, |
2385 | the use of this option provokes an error. Details of how this option |
2386 | changes the behaviour of PCRE are given in the pcreunicode page. |
2387 | |
2388 | PCRE_NO_UTF8_CHECK |
2389 | |
2390 | When PCRE_UTF8 is set, the validity of the pattern as a UTF-8 string is |
2391 | automatically checked. There is a discussion about the validity of |
2392 | UTF-8 strings in the pcreunicode page. If an invalid UTF-8 sequence is |
2393 | found, pcre_compile() returns an error. If you already know that your |
2394 | pattern is valid, and you want to skip this check for performance rea- |
2395 | sons, you can set the PCRE_NO_UTF8_CHECK option. When it is set, the |
2396 | effect of passing an invalid UTF-8 string as a pattern is undefined. It |
2397 | may cause your program to crash or loop. Note that this option can also |
2398 | be passed to pcre_exec() and pcre_dfa_exec(), to suppress the validity |
2399 | checking of subject strings only. If the same string is being matched |
2400 | many times, the option can be safely set for the second and subsequent |
2401 | matchings to improve performance. |
2402 | |
2403 | |
2404 | COMPILATION ERROR CODES |
2405 | |
2406 | The following table lists the error codes than may be returned by |
2407 | pcre_compile2(), along with the error messages that may be returned by |
2408 | both compiling functions. Note that error messages are always 8-bit |
2409 | ASCII strings, even in 16-bit or 32-bit mode. As PCRE has developed, |
2410 | some error codes have fallen out of use. To avoid confusion, they have |
2411 | not been re-used. |
2412 | |
2413 | 0 no error |
2414 | 1 \ at end of pattern |
2415 | 2 \c at end of pattern |
2416 | 3 unrecognized character follows \ |
2417 | 4 numbers out of order in {} quantifier |
2418 | 5 number too big in {} quantifier |
2419 | 6 missing terminating ] for character class |
2420 | 7 invalid escape sequence in character class |
2421 | 8 range out of order in character class |
2422 | 9 nothing to repeat |
2423 | 10 [this code is not in use] |
2424 | 11 internal error: unexpected repeat |
2425 | 12 unrecognized character after (? or (?- |
2426 | 13 POSIX named classes are supported only within a class |
2427 | 14 missing ) |
2428 | 15 reference to non-existent subpattern |
2429 | 16 erroffset passed as NULL |
2430 | 17 unknown option bit(s) set |
2431 | 18 missing ) after comment |
2432 | 19 [this code is not in use] |
2433 | 20 regular expression is too large |
2434 | 21 failed to get memory |
2435 | 22 unmatched parentheses |
2436 | 23 internal error: code overflow |
2437 | 24 unrecognized character after (?< |
2438 | 25 lookbehind assertion is not fixed length |
2439 | 26 malformed number or name after (?( |
2440 | 27 conditional group contains more than two branches |
2441 | 28 assertion expected after (?( |
2442 | 29 (?R or (?[+-]digits must be followed by ) |
2443 | 30 unknown POSIX class name |
2444 | 31 POSIX collating elements are not supported |
2445 | 32 this version of PCRE is compiled without UTF support |
2446 | 33 [this code is not in use] |
2447 | 34 character value in \x{} or \o{} is too large |
2448 | 35 invalid condition (?(0) |
2449 | 36 \C not allowed in lookbehind assertion |
2450 | 37 PCRE does not support \L, \l, \N{name}, \U, or \u |
2451 | 38 number after (?C is > 255 |
2452 | 39 closing ) for (?C expected |
2453 | 40 recursive call could loop indefinitely |
2454 | 41 unrecognized character after (?P |
2455 | 42 syntax error in subpattern name (missing terminator) |
2456 | 43 two named subpatterns have the same name |
2457 | 44 invalid UTF-8 string (specifically UTF-8) |
2458 | 45 support for \P, \p, and \X has not been compiled |
2459 | 46 malformed \P or \p sequence |
2460 | 47 unknown property name after \P or \p |
2461 | 48 subpattern name is too long (maximum 32 characters) |
2462 | 49 too many named subpatterns (maximum 10000) |
2463 | 50 [this code is not in use] |
2464 | 51 octal value is greater than \377 in 8-bit non-UTF-8 mode |
2465 | 52 internal error: overran compiling workspace |
2466 | 53 internal error: previously-checked referenced subpattern |
2467 | not found |
2468 | 54 DEFINE group contains more than one branch |
2469 | 55 repeating a DEFINE group is not allowed |
2470 | 56 inconsistent NEWLINE options |
2471 | 57 \g is not followed by a braced, angle-bracketed, or quoted |
2472 | name/number or by a plain number |
2473 | 58 a numbered reference must not be zero |
2474 | 59 an argument is not allowed for (*ACCEPT), (*FAIL), or (*COMMIT) |
2475 | 60 (*VERB) not recognized or malformed |
2476 | 61 number is too big |
2477 | 62 subpattern name expected |
2478 | 63 digit expected after (?+ |
2479 | 64 ] is an invalid data character in JavaScript compatibility mode |
2480 | 65 different names for subpatterns of the same number are |
2481 | not allowed |
2482 | 66 (*MARK) must have an argument |
2483 | 67 this version of PCRE is not compiled with Unicode property |
2484 | support |
2485 | 68 \c must be followed by an ASCII character |
2486 | 69 \k is not followed by a braced, angle-bracketed, or quoted name |
2487 | 70 internal error: unknown opcode in find_fixedlength() |
2488 | 71 \N is not supported in a class |
2489 | 72 too many forward references |
2490 | 73 disallowed Unicode code point (>= 0xd800 && <= 0xdfff) |
2491 | 74 invalid UTF-16 string (specifically UTF-16) |
2492 | 75 name is too long in (*MARK), (*PRUNE), (*SKIP), or (*THEN) |
2493 | 76 character value in \u.... sequence is too large |
2494 | 77 invalid UTF-32 string (specifically UTF-32) |
2495 | 78 setting UTF is disabled by the application |
2496 | 79 non-hex character in \x{} (closing brace missing?) |
2497 | 80 non-octal character in \o{} (closing brace missing?) |
2498 | 81 missing opening brace after \o |
2499 | 82 parentheses are too deeply nested |
2500 | 83 invalid range in character class |
2501 | 84 group name must start with a non-digit |
2502 | 85 parentheses are too deeply nested (stack check) |
2503 | |
2504 | The numbers 32 and 10000 in errors 48 and 49 are defaults; different |
2505 | values may be used if the limits were changed when PCRE was built. |
2506 | |
2507 | |
2508 | STUDYING A PATTERN |
2509 | |
2510 | pcre_extra *pcre_study(const pcre *code, int options, |
2511 | const char **errptr); |
2512 | |
2513 | If a compiled pattern is going to be used several times, it is worth |
2514 | spending more time analyzing it in order to speed up the time taken for |
2515 | matching. The function pcre_study() takes a pointer to a compiled pat- |
2516 | tern as its first argument. If studying the pattern produces additional |
2517 | information that will help speed up matching, pcre_study() returns a |
2518 | pointer to a pcre_extra block, in which the study_data field points to |
2519 | the results of the study. |
2520 | |
2521 | The returned value from pcre_study() can be passed directly to |
2522 | pcre_exec() or pcre_dfa_exec(). However, a pcre_extra block also con- |
2523 | tains other fields that can be set by the caller before the block is |
2524 | passed; these are described below in the section on matching a pattern. |
2525 | |
2526 | If studying the pattern does not produce any useful information, |
2527 | pcre_study() returns NULL by default. In that circumstance, if the |
2528 | calling program wants to pass any of the other fields to pcre_exec() or |
2529 | pcre_dfa_exec(), it must set up its own pcre_extra block. However, if |
2530 | pcre_study() is called with the PCRE_STUDY_EXTRA_NEEDED option, it re- |
2531 | turns a pcre_extra block even if studying did not find any additional |
2532 | information. It may still return NULL, however, if an error occurs in |
2533 | pcre_study(). |
2534 | |
2535 | The second argument of pcre_study() contains option bits. There are |
2536 | three further options in addition to PCRE_STUDY_EXTRA_NEEDED: |
2537 | |
2538 | PCRE_STUDY_JIT_COMPILE |
2539 | PCRE_STUDY_JIT_PARTIAL_HARD_COMPILE |
2540 | PCRE_STUDY_JIT_PARTIAL_SOFT_COMPILE |
2541 | |
2542 | If any of these are set, and the just-in-time compiler is available, |
2543 | the pattern is further compiled into machine code that executes much |
2544 | faster than the pcre_exec() interpretive matching function. If the |
2545 | just-in-time compiler is not available, these options are ignored. All |
2546 | undefined bits in the options argument must be zero. |
2547 | |
2548 | JIT compilation is a heavyweight optimization. It can take some time |
2549 | for patterns to be analyzed, and for one-off matches and simple pat- |
2550 | terns the benefit of faster execution might be offset by a much slower |
2551 | study time. Not all patterns can be optimized by the JIT compiler. For |
2552 | those that cannot be handled, matching automatically falls back to the |
2553 | pcre_exec() interpreter. For more details, see the pcrejit documenta- |
2554 | tion. |
2555 | |
2556 | The third argument for pcre_study() is a pointer for an error message. |
2557 | If studying succeeds (even if no data is returned), the variable it |
2558 | points to is set to NULL. Otherwise it is set to point to a textual er- |
2559 | ror message. This is a static string that is part of the library. You |
2560 | must not try to free it. You should test the error pointer for NULL af- |
2561 | ter calling pcre_study(), to be sure that it has run successfully. |
2562 | |
2563 | When you are finished with a pattern, you can free the memory used for |
2564 | the study data by calling pcre_free_study(). This function was added to |
2565 | the API for release 8.20. For earlier versions, the memory could be |
2566 | freed with pcre_free(), just like the pattern itself. This will still |
2567 | work in cases where JIT optimization is not used, but it is advisable |
2568 | to change to the new function when convenient. |
2569 | |
2570 | This is a typical way in which pcre_study() is used (except that in a |
2571 | real application there should be tests for errors): |
2572 | |
2573 | int rc; |
2574 | pcre *re; |
2575 | pcre_extra *sd; |
2576 | re = pcre_compile("pattern", 0, &error, &erroroffset, NULL); |
2577 | sd = pcre_study( |
2578 | re, /* result of pcre_compile() */ |
2579 | 0, /* no options */ |
2580 | &error); /* set to NULL or points to a message */ |
2581 | rc = pcre_exec( /* see below for details of pcre_exec() options */ |
2582 | re, sd, "subject", 7, 0, 0, ovector, 30); |
2583 | ... |
2584 | pcre_free_study(sd); |
2585 | pcre_free(re); |
2586 | |
2587 | Studying a pattern does two things: first, a lower bound for the length |
2588 | of subject string that is needed to match the pattern is computed. This |
2589 | does not mean that there are any strings of that length that match, but |
2590 | it does guarantee that no shorter strings match. The value is used to |
2591 | avoid wasting time by trying to match strings that are shorter than the |
2592 | lower bound. You can find out the value in a calling program via the |
2593 | pcre_fullinfo() function. |
2594 | |
2595 | Studying a pattern is also useful for non-anchored patterns that do not |
2596 | have a single fixed starting character. A bitmap of possible starting |
2597 | bytes is created. This speeds up finding a position in the subject at |
2598 | which to start matching. (In 16-bit mode, the bitmap is used for 16-bit |
2599 | values less than 256. In 32-bit mode, the bitmap is used for 32-bit |
2600 | values less than 256.) |
2601 | |
2602 | These two optimizations apply to both pcre_exec() and pcre_dfa_exec(), |
2603 | and the information is also used by the JIT compiler. The optimiza- |
2604 | tions can be disabled by setting the PCRE_NO_START_OPTIMIZE option. |
2605 | You might want to do this if your pattern contains callouts or (*MARK) |
2606 | and you want to make use of these facilities in cases where matching |
2607 | fails. |
2608 | |
2609 | PCRE_NO_START_OPTIMIZE can be specified at either compile time or exe- |
2610 | cution time. However, if PCRE_NO_START_OPTIMIZE is passed to |
2611 | pcre_exec(), (that is, after any JIT compilation has happened) JIT exe- |
2612 | cution is disabled. For JIT execution to work with PCRE_NO_START_OPTI- |
2613 | MIZE, the option must be set at compile time. |
2614 | |
2615 | There is a longer discussion of PCRE_NO_START_OPTIMIZE below. |
2616 | |
2617 | |
2618 | LOCALE SUPPORT |
2619 | |
2620 | PCRE handles caseless matching, and determines whether characters are |
2621 | letters, digits, or whatever, by reference to a set of tables, indexed |
2622 | by character code point. When running in UTF-8 mode, or in the 16- or |
2623 | 32-bit libraries, this applies only to characters with code points less |
2624 | than 256. By default, higher-valued code points never match escapes |
2625 | such as \w or \d. However, if PCRE is built with Unicode property sup- |
2626 | port, all characters can be tested with \p and \P, or, alternatively, |
2627 | the PCRE_UCP option can be set when a pattern is compiled; this causes |
2628 | \w and friends to use Unicode property support instead of the built-in |
2629 | tables. |
2630 | |
2631 | The use of locales with Unicode is discouraged. If you are handling |
2632 | characters with code points greater than 128, you should either use |
2633 | Unicode support, or use locales, but not try to mix the two. |
2634 | |
2635 | PCRE contains an internal set of tables that are used when the final |
2636 | argument of pcre_compile() is NULL. These are sufficient for many ap- |
2637 | plications. Normally, the internal tables recognize only ASCII charac- |
2638 | ters. However, when PCRE is built, it is possible to cause the internal |
2639 | tables to be rebuilt in the default "C" locale of the local system, |
2640 | which may cause them to be different. |
2641 | |
2642 | The internal tables can always be overridden by tables supplied by the |
2643 | application that calls PCRE. These may be created in a different locale |
2644 | from the default. As more and more applications change to using Uni- |
2645 | code, the need for this locale support is expected to die away. |
2646 | |
2647 | External tables are built by calling the pcre_maketables() function, |
2648 | which has no arguments, in the relevant locale. The result can then be |
2649 | passed to pcre_compile() as often as necessary. For example, to build |
2650 | and use tables that are appropriate for the French locale (where ac- |
2651 | cented characters with values greater than 128 are treated as letters), |
2652 | the following code could be used: |
2653 | |
2654 | setlocale(LC_CTYPE, "fr_FR"); |
2655 | tables = pcre_maketables(); |
2656 | re = pcre_compile(..., tables); |
2657 | |
2658 | The locale name "fr_FR" is used on Linux and other Unix-like systems; |
2659 | if you are using Windows, the name for the French locale is "french". |
2660 | |
2661 | When pcre_maketables() runs, the tables are built in memory that is ob- |
2662 | tained via pcre_malloc. It is the caller's responsibility to ensure |
2663 | that the memory containing the tables remains available for as long as |
2664 | it is needed. |
2665 | |
2666 | The pointer that is passed to pcre_compile() is saved with the compiled |
2667 | pattern, and the same tables are used via this pointer by pcre_study() |
2668 | and also by pcre_exec() and pcre_dfa_exec(). Thus, for any single pat- |
2669 | tern, compilation, studying and matching all happen in the same locale, |
2670 | but different patterns can be processed in different locales. |
2671 | |
2672 | It is possible to pass a table pointer or NULL (indicating the use of |
2673 | the internal tables) to pcre_exec() or pcre_dfa_exec() (see the discus- |
2674 | sion below in the section on matching a pattern). This facility is pro- |
2675 | vided for use with pre-compiled patterns that have been saved and |
2676 | reloaded. Character tables are not saved with patterns, so if a non- |
2677 | standard table was used at compile time, it must be provided again when |
2678 | the reloaded pattern is matched. Attempting to use this facility to |
2679 | match a pattern in a different locale from the one in which it was com- |
2680 | piled is likely to lead to anomalous (usually incorrect) results. |
2681 | |
2682 | |
2683 | INFORMATION ABOUT A PATTERN |
2684 | |
2685 | int pcre_fullinfo(const pcre *code, const pcre_extra *extra, |
2686 | int what, void *where); |
2687 | |
2688 | The pcre_fullinfo() function returns information about a compiled pat- |
2689 | tern. It replaces the pcre_info() function, which was removed from the |
2690 | library at version 8.30, after more than 10 years of obsolescence. |
2691 | |
2692 | The first argument for pcre_fullinfo() is a pointer to the compiled |
2693 | pattern. The second argument is the result of pcre_study(), or NULL if |
2694 | the pattern was not studied. The third argument specifies which piece |
2695 | of information is required, and the fourth argument is a pointer to a |
2696 | variable to receive the data. The yield of the function is zero for |
2697 | success, or one of the following negative numbers: |
2698 | |
2699 | PCRE_ERROR_NULL the argument code was NULL |
2700 | the argument where was NULL |
2701 | PCRE_ERROR_BADMAGIC the "magic number" was not found |
2702 | PCRE_ERROR_BADENDIANNESS the pattern was compiled with different |
2703 | endianness |
2704 | PCRE_ERROR_BADOPTION the value of what was invalid |
2705 | PCRE_ERROR_UNSET the requested field is not set |
2706 | |
2707 | The "magic number" is placed at the start of each compiled pattern as a |
2708 | simple check against passing an arbitrary memory pointer. The endian- |
2709 | ness error can occur if a compiled pattern is saved and reloaded on a |
2710 | different host. Here is a typical call of pcre_fullinfo(), to obtain |
2711 | the length of the compiled pattern: |
2712 | |
2713 | int rc; |
2714 | size_t length; |
2715 | rc = pcre_fullinfo( |
2716 | re, /* result of pcre_compile() */ |
2717 | sd, /* result of pcre_study(), or NULL */ |
2718 | PCRE_INFO_SIZE, /* what is required */ |
2719 | &length); /* where to put the data */ |
2720 | |
2721 | The possible values for the third argument are defined in pcre.h, and |
2722 | are as follows: |
2723 | |
2724 | PCRE_INFO_BACKREFMAX |
2725 | |
2726 | Return the number of the highest back reference in the pattern. The |
2727 | fourth argument should point to an int variable. Zero is returned if |
2728 | there are no back references. |
2729 | |
2730 | PCRE_INFO_CAPTURECOUNT |
2731 | |
2732 | Return the number of capturing subpatterns in the pattern. The fourth |
2733 | argument should point to an int variable. |
2734 | |
2735 | PCRE_INFO_DEFAULT_TABLES |
2736 | |
2737 | Return a pointer to the internal default character tables within PCRE. |
2738 | The fourth argument should point to an unsigned char * variable. This |
2739 | information call is provided for internal use by the pcre_study() func- |
2740 | tion. External callers can cause PCRE to use its internal tables by |
2741 | passing a NULL table pointer. |
2742 | |
2743 | PCRE_INFO_FIRSTBYTE (deprecated) |
2744 | |
2745 | Return information about the first data unit of any matched string, for |
2746 | a non-anchored pattern. The name of this option refers to the 8-bit li- |
2747 | brary, where data units are bytes. The fourth argument should point to |
2748 | an int variable. Negative values are used for special cases. However, |
2749 | this means that when the 32-bit library is in non-UTF-32 mode, the full |
2750 | 32-bit range of characters cannot be returned. For this reason, this |
2751 | value is deprecated; use PCRE_INFO_FIRSTCHARACTERFLAGS and |
2752 | PCRE_INFO_FIRSTCHARACTER instead. |
2753 | |
2754 | If there is a fixed first value, for example, the letter "c" from a |
2755 | pattern such as (cat|cow|coyote), its value is returned. In the 8-bit |
2756 | library, the value is always less than 256. In the 16-bit library the |
2757 | value can be up to 0xffff. In the 32-bit library the value can be up to |
2758 | 0x10ffff. |
2759 | |
2760 | If there is no fixed first value, and if either |
2761 | |
2762 | (a) the pattern was compiled with the PCRE_MULTILINE option, and every |
2763 | branch starts with "^", or |
2764 | |
2765 | (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not |
2766 | set (if it were set, the pattern would be anchored), |
2767 | |
2768 | -1 is returned, indicating that the pattern matches only at the start |
2769 | of a subject string or after any newline within the string. Otherwise |
2770 | -2 is returned. For anchored patterns, -2 is returned. |
2771 | |
2772 | PCRE_INFO_FIRSTCHARACTER |
2773 | |
2774 | Return the value of the first data unit (non-UTF character) of any |
2775 | matched string in the situation where PCRE_INFO_FIRSTCHARACTERFLAGS re- |
2776 | turns 1; otherwise return 0. The fourth argument should point to a |
2777 | uint_t variable. |
2778 | |
2779 | In the 8-bit library, the value is always less than 256. In the 16-bit |
2780 | library the value can be up to 0xffff. In the 32-bit library in UTF-32 |
2781 | mode the value can be up to 0x10ffff, and up to 0xffffffff when not us- |
2782 | ing UTF-32 mode. |
2783 | |
2784 | PCRE_INFO_FIRSTCHARACTERFLAGS |
2785 | |
2786 | Return information about the first data unit of any matched string, for |
2787 | a non-anchored pattern. The fourth argument should point to an int |
2788 | variable. |
2789 | |
2790 | If there is a fixed first value, for example, the letter "c" from a |
2791 | pattern such as (cat|cow|coyote), 1 is returned, and the character |
2792 | value can be retrieved using PCRE_INFO_FIRSTCHARACTER. If there is no |
2793 | fixed first value, and if either |
2794 | |
2795 | (a) the pattern was compiled with the PCRE_MULTILINE option, and every |
2796 | branch starts with "^", or |
2797 | |
2798 | (b) every branch of the pattern starts with ".*" and PCRE_DOTALL is not |
2799 | set (if it were set, the pattern would be anchored), |
2800 | |
2801 | 2 is returned, indicating that the pattern matches only at the start of |
2802 | a subject string or after any newline within the string. Otherwise 0 is |
2803 | returned. For anchored patterns, 0 is returned. |
2804 | |
2805 | PCRE_INFO_FIRSTTABLE |
2806 | |
2807 | If the pattern was studied, and this resulted in the construction of a |
2808 | 256-bit table indicating a fixed set of values for the first data unit |
2809 | in any matching string, a pointer to the table is returned. Otherwise |
2810 | NULL is returned. The fourth argument should point to an unsigned char |
2811 | * variable. |
2812 | |
2813 | PCRE_INFO_HASCRORLF |
2814 | |
2815 | Return 1 if the pattern contains any explicit matches for CR or LF |
2816 | characters, otherwise 0. The fourth argument should point to an int |
2817 | variable. An explicit match is either a literal CR or LF character, or |
2818 | \r or \n. |
2819 | |
2820 | PCRE_INFO_JCHANGED |
2821 | |
2822 | Return 1 if the (?J) or (?-J) option setting is used in the pattern, |
2823 | otherwise 0. The fourth argument should point to an int variable. (?J) |
2824 | and (?-J) set and unset the local PCRE_DUPNAMES option, respectively. |
2825 | |
2826 | PCRE_INFO_JIT |
2827 | |
2828 | Return 1 if the pattern was studied with one of the JIT options, and |
2829 | just-in-time compiling was successful. The fourth argument should point |
2830 | to an int variable. A return value of 0 means that JIT support is not |
2831 | available in this version of PCRE, or that the pattern was not studied |
2832 | with a JIT option, or that the JIT compiler could not handle this par- |
2833 | ticular pattern. See the pcrejit documentation for details of what can |
2834 | and cannot be handled. |
2835 | |
2836 | PCRE_INFO_JITSIZE |
2837 | |
2838 | If the pattern was successfully studied with a JIT option, return the |
2839 | size of the JIT compiled code, otherwise return zero. The fourth argu- |
2840 | ment should point to a size_t variable. |
2841 | |
2842 | PCRE_INFO_LASTLITERAL |
2843 | |
2844 | Return the value of the rightmost literal data unit that must exist in |
2845 | any matched string, other than at its start, if such a value has been |
2846 | recorded. The fourth argument should point to an int variable. If there |
2847 | is no such value, -1 is returned. For anchored patterns, a last literal |
2848 | value is recorded only if it follows something of variable length. For |
2849 | example, for the pattern /^a\d+z\d+/ the returned value is "z", but for |
2850 | /^a\dz\d/ the returned value is -1. |
2851 | |
2852 | Since for the 32-bit library using the non-UTF-32 mode, this function |
2853 | is unable to return the full 32-bit range of characters, this value is |
2854 | deprecated; instead the PCRE_INFO_REQUIREDCHARFLAGS and PCRE_INFO_RE- |
2855 | QUIREDCHAR values should be used. |
2856 | |
2857 | PCRE_INFO_MATCH_EMPTY |
2858 | |
2859 | Return 1 if the pattern can match an empty string, otherwise 0. The |
2860 | fourth argument should point to an int variable. |
2861 | |
2862 | PCRE_INFO_MATCHLIMIT |
2863 | |
2864 | If the pattern set a match limit by including an item of the form |
2865 | (*LIMIT_MATCH=nnnn) at the start, the value is returned. The fourth ar- |
2866 | gument should point to an unsigned 32-bit integer. If no such value has |
2867 | been set, the call to pcre_fullinfo() returns the error PCRE_ERROR_UN- |
2868 | SET. |
2869 | |
2870 | PCRE_INFO_MAXLOOKBEHIND |
2871 | |
2872 | Return the number of characters (NB not data units) in the longest |
2873 | lookbehind assertion in the pattern. This information is useful when |
2874 | doing multi-segment matching using the partial matching facilities. |
2875 | Note that the simple assertions \b and \B require a one-character look- |
2876 | behind. \A also registers a one-character lookbehind, though it does |
2877 | not actually inspect the previous character. This is to ensure that at |
2878 | least one character from the old segment is retained when a new segment |
2879 | is processed. Otherwise, if there are no lookbehinds in the pattern, \A |
2880 | might match incorrectly at the start of a new segment. |
2881 | |
2882 | PCRE_INFO_MINLENGTH |
2883 | |
2884 | If the pattern was studied and a minimum length for matching subject |
2885 | strings was computed, its value is returned. Otherwise the returned |
2886 | value is -1. The value is a number of characters, which in UTF mode may |
2887 | be different from the number of data units. The fourth argument should |
2888 | point to an int variable. A non-negative value is a lower bound to the |
2889 | length of any matching string. There may not be any strings of that |
2890 | length that do actually match, but every string that does match is at |
2891 | least that long. |
2892 | |
2893 | PCRE_INFO_NAMECOUNT |
2894 | PCRE_INFO_NAMEENTRYSIZE |
2895 | PCRE_INFO_NAMETABLE |
2896 | |
2897 | PCRE supports the use of named as well as numbered capturing parenthe- |
2898 | ses. The names are just an additional way of identifying the parenthe- |
2899 | ses, which still acquire numbers. Several convenience functions such as |
2900 | pcre_get_named_substring() are provided for extracting captured sub- |
2901 | strings by name. It is also possible to extract the data directly, by |
2902 | first converting the name to a number in order to access the correct |
2903 | pointers in the output vector (described with pcre_exec() below). To do |
2904 | the conversion, you need to use the name-to-number map, which is de- |
2905 | scribed by these three values. |
2906 | |
2907 | The map consists of a number of fixed-size entries. PCRE_INFO_NAMECOUNT |
2908 | gives the number of entries, and PCRE_INFO_NAMEENTRYSIZE gives the size |
2909 | of each entry; both of these return an int value. The entry size de- |
2910 | pends on the length of the longest name. PCRE_INFO_NAMETABLE returns a |
2911 | pointer to the first entry of the table. This is a pointer to char in |
2912 | the 8-bit library, where the first two bytes of each entry are the num- |
2913 | ber of the capturing parenthesis, most significant byte first. In the |
2914 | 16-bit library, the pointer points to 16-bit data units, the first of |
2915 | which contains the parenthesis number. In the 32-bit library, the |
2916 | pointer points to 32-bit data units, the first of which contains the |
2917 | parenthesis number. The rest of the entry is the corresponding name, |
2918 | zero terminated. |
2919 | |
2920 | The names are in alphabetical order. If (?| is used to create multiple |
2921 | groups with the same number, as described in the section on duplicate |
2922 | subpattern numbers in the pcrepattern page, the groups may be given the |
2923 | same name, but there is only one entry in the table. Different names |
2924 | for groups of the same number are not permitted. Duplicate names for |
2925 | subpatterns with different numbers are permitted, but only if PCRE_DUP- |
2926 | NAMES is set. They appear in the table in the order in which they were |
2927 | found in the pattern. In the absence of (?| this is the order of in- |
2928 | creasing number; when (?| is used this is not necessarily the case be- |
2929 | cause later subpatterns may have lower numbers. |
2930 | |
2931 | As a simple example of the name/number table, consider the following |
2932 | pattern after compilation by the 8-bit library (assume PCRE_EXTENDED is |
2933 | set, so white space - including newlines - is ignored): |
2934 | |
2935 | (?<date> (?<year>(\d\d)?\d\d) - |
2936 | (?<month>\d\d) - (?<day>\d\d) ) |
2937 | |
2938 | There are four named subpatterns, so the table has four entries, and |
2939 | each entry in the table is eight bytes long. The table is as follows, |
2940 | with non-printing bytes shows in hexadecimal, and undefined bytes shown |
2941 | as ??: |
2942 | |
2943 | 00 01 d a t e 00 ?? |
2944 | 00 05 d a y 00 ?? ?? |
2945 | 00 04 m o n t h 00 |
2946 | 00 02 y e a r 00 ?? |
2947 | |
2948 | When writing code to extract data from named subpatterns using the |
2949 | name-to-number map, remember that the length of the entries is likely |
2950 | to be different for each compiled pattern. |
2951 | |
2952 | PCRE_INFO_OKPARTIAL |
2953 | |
2954 | Return 1 if the pattern can be used for partial matching with |
2955 | pcre_exec(), otherwise 0. The fourth argument should point to an int |
2956 | variable. From release 8.00, this always returns 1, because the re- |
2957 | strictions that previously applied to partial matching have been |
2958 | lifted. The pcrepartial documentation gives details of partial match- |
2959 | ing. |
2960 | |
2961 | PCRE_INFO_OPTIONS |
2962 | |
2963 | Return a copy of the options with which the pattern was compiled. The |
2964 | fourth argument should point to an unsigned long int variable. These |
2965 | option bits are those specified in the call to pcre_compile(), modified |
2966 | by any top-level option settings at the start of the pattern itself. In |
2967 | other words, they are the options that will be in force when matching |
2968 | starts. For example, if the pattern /(?im)abc(?-i)d/ is compiled with |
2969 | the PCRE_EXTENDED option, the result is PCRE_CASELESS, PCRE_MULTILINE, |
2970 | and PCRE_EXTENDED. |
2971 | |
2972 | A pattern is automatically anchored by PCRE if all of its top-level al- |
2973 | ternatives begin with one of the following: |
2974 | |
2975 | ^ unless PCRE_MULTILINE is set |
2976 | \A always |
2977 | \G always |
2978 | .* if PCRE_DOTALL is set and there are no back |
2979 | references to the subpattern in which .* appears |
2980 | |
2981 | For such patterns, the PCRE_ANCHORED bit is set in the options returned |
2982 | by pcre_fullinfo(). |
2983 | |
2984 | PCRE_INFO_RECURSIONLIMIT |
2985 | |
2986 | If the pattern set a recursion limit by including an item of the form |
2987 | (*LIMIT_RECURSION=nnnn) at the start, the value is returned. The fourth |
2988 | argument should point to an unsigned 32-bit integer. If no such value |
2989 | has been set, the call to pcre_fullinfo() returns the error PCRE_ER- |
2990 | ROR_UNSET. |
2991 | |
2992 | PCRE_INFO_SIZE |
2993 | |
2994 | Return the size of the compiled pattern in bytes (for all three li- |
2995 | braries). The fourth argument should point to a size_t variable. This |
2996 | value does not include the size of the pcre structure that is returned |
2997 | by pcre_compile(). The value that is passed as the argument to |
2998 | pcre_malloc() when pcre_compile() is getting memory in which to place |
2999 | the compiled data is the value returned by this option plus the size of |
3000 | the pcre structure. Studying a compiled pattern, with or without JIT, |
3001 | does not alter the value returned by this option. |
3002 | |
3003 | PCRE_INFO_STUDYSIZE |
3004 | |
3005 | Return the size in bytes (for all three libraries) of the data block |
3006 | pointed to by the study_data field in a pcre_extra block. If pcre_extra |
3007 | is NULL, or there is no study data, zero is returned. The fourth argu- |
3008 | ment should point to a size_t variable. The study_data field is set by |
3009 | pcre_study() to record information that will speed up matching (see the |
3010 | section entitled "Studying a pattern" above). The format of the |
3011 | study_data block is private, but its length is made available via this |
3012 | option so that it can be saved and restored (see the pcreprecompile |
3013 | documentation for details). |
3014 | |
3015 | PCRE_INFO_REQUIREDCHARFLAGS |
3016 | |
3017 | Returns 1 if there is a rightmost literal data unit that must exist in |
3018 | any matched string, other than at its start. The fourth argument should |
3019 | point to an int variable. If there is no such value, 0 is returned. If |
3020 | returning 1, the character value itself can be retrieved using |
3021 | PCRE_INFO_REQUIREDCHAR. |
3022 | |
3023 | For anchored patterns, a last literal value is recorded only if it fol- |
3024 | lows something of variable length. For example, for the pattern |
3025 | /^a\d+z\d+/ the returned value 1 (with "z" returned from PCRE_INFO_RE- |
3026 | QUIREDCHAR), but for /^a\dz\d/ the returned value is 0. |
3027 | |
3028 | PCRE_INFO_REQUIREDCHAR |
3029 | |
3030 | Return the value of the rightmost literal data unit that must exist in |
3031 | any matched string, other than at its start, if such a value has been |
3032 | recorded. The fourth argument should point to a uint32_t variable. If |
3033 | there is no such value, 0 is returned. |
3034 | |
3035 | |
3036 | REFERENCE COUNTS |
3037 | |
3038 | int pcre_refcount(pcre *code, int adjust); |
3039 | |
3040 | The pcre_refcount() function is used to maintain a reference count in |
3041 | the data block that contains a compiled pattern. It is provided for the |
3042 | benefit of applications that operate in an object-oriented manner, |
3043 | where different parts of the application may be using the same compiled |
3044 | pattern, but you want to free the block when they are all done. |
3045 | |
3046 | When a pattern is compiled, the reference count field is initialized to |
3047 | zero. It is changed only by calling this function, whose action is to |
3048 | add the adjust value (which may be positive or negative) to it. The |
3049 | yield of the function is the new value. However, the value of the count |
3050 | is constrained to lie between 0 and 65535, inclusive. If the new value |
3051 | is outside these limits, it is forced to the appropriate limit value. |
3052 | |
3053 | Except when it is zero, the reference count is not correctly preserved |
3054 | if a pattern is compiled on one host and then transferred to a host |
3055 | whose byte-order is different. (This seems a highly unlikely scenario.) |
3056 | |
3057 | |
3058 | MATCHING A PATTERN: THE TRADITIONAL FUNCTION |
3059 | |
3060 | int pcre_exec(const pcre *code, const pcre_extra *extra, |
3061 | const char *subject, int length, int startoffset, |
3062 | int options, int *ovector, int ovecsize); |
3063 | |
3064 | The function pcre_exec() is called to match a subject string against a |
3065 | compiled pattern, which is passed in the code argument. If the pattern |
3066 | was studied, the result of the study should be passed in the extra ar- |
3067 | gument. You can call pcre_exec() with the same code and extra arguments |
3068 | as many times as you like, in order to match different subject strings |
3069 | with the same pattern. |
3070 | |
3071 | This function is the main matching facility of the library, and it op- |
3072 | erates in a Perl-like manner. For specialist use there is also an al- |
3073 | ternative matching function, which is described below in the section |
3074 | about the pcre_dfa_exec() function. |
3075 | |
3076 | In most applications, the pattern will have been compiled (and option- |
3077 | ally studied) in the same process that calls pcre_exec(). However, it |
3078 | is possible to save compiled patterns and study data, and then use them |
3079 | later in different processes, possibly even on different hosts. For a |
3080 | discussion about this, see the pcreprecompile documentation. |
3081 | |
3082 | Here is an example of a simple call to pcre_exec(): |
3083 | |
3084 | int rc; |
3085 | int ovector[30]; |
3086 | rc = pcre_exec( |
3087 | re, /* result of pcre_compile() */ |
3088 | NULL, /* we didn't study the pattern */ |
3089 | "some string", /* the subject string */ |
3090 | 11, /* the length of the subject string */ |
3091 | 0, /* start at offset 0 in the subject */ |
3092 | 0, /* default options */ |
3093 | ovector, /* vector of integers for substring information */ |
3094 | 30); /* number of elements (NOT size in bytes) */ |
3095 | |
3096 | Extra data for pcre_exec() |
3097 | |
3098 | If the extra argument is not NULL, it must point to a pcre_extra data |
3099 | block. The pcre_study() function returns such a block (when it doesn't |
3100 | return NULL), but you can also create one for yourself, and pass addi- |
3101 | tional information in it. The pcre_extra block contains the following |
3102 | fields (not necessarily in this order): |
3103 | |
3104 | unsigned long int flags; |
3105 | void *study_data; |
3106 | void *executable_jit; |
3107 | unsigned long int match_limit; |
3108 | unsigned long int match_limit_recursion; |
3109 | void *callout_data; |
3110 | const unsigned char *tables; |
3111 | unsigned char **mark; |
3112 | |
3113 | In the 16-bit version of this structure, the mark field has type |
3114 | "PCRE_UCHAR16 **". |
3115 | |
3116 | In the 32-bit version of this structure, the mark field has type |
3117 | "PCRE_UCHAR32 **". |
3118 | |
3119 | The flags field is used to specify which of the other fields are set. |
3120 | The flag bits are: |
3121 | |
3122 | PCRE_EXTRA_CALLOUT_DATA |
3123 | PCRE_EXTRA_EXECUTABLE_JIT |
3124 | PCRE_EXTRA_MARK |
3125 | PCRE_EXTRA_MATCH_LIMIT |
3126 | PCRE_EXTRA_MATCH_LIMIT_RECURSION |
3127 | PCRE_EXTRA_STUDY_DATA |
3128 | PCRE_EXTRA_TABLES |
3129 | |
3130 | Other flag bits should be set to zero. The study_data field and some- |
3131 | times the executable_jit field are set in the pcre_extra block that is |
3132 | returned by pcre_study(), together with the appropriate flag bits. You |
3133 | should not set these yourself, but you may add to the block by setting |
3134 | other fields and their corresponding flag bits. |
3135 | |
3136 | The match_limit field provides a means of preventing PCRE from using up |
3137 | a vast amount of resources when running patterns that are not going to |
3138 | match, but which have a very large number of possibilities in their |
3139 | search trees. The classic example is a pattern that uses nested unlim- |
3140 | ited repeats. |
3141 | |
3142 | Internally, pcre_exec() uses a function called match(), which it calls |
3143 | repeatedly (sometimes recursively). The limit set by match_limit is im- |
3144 | posed on the number of times this function is called during a match, |
3145 | which has the effect of limiting the amount of backtracking that can |
3146 | take place. For patterns that are not anchored, the count restarts from |
3147 | zero for each position in the subject string. |
3148 | |
3149 | When pcre_exec() is called with a pattern that was successfully studied |
3150 | with a JIT option, the way that the matching is executed is entirely |
3151 | different. However, there is still the possibility of runaway matching |
3152 | that goes on for a very long time, and so the match_limit value is also |
3153 | used in this case (but in a different way) to limit how long the match- |
3154 | ing can continue. |
3155 | |
3156 | The default value for the limit can be set when PCRE is built; the de- |
3157 | fault default is 10 million, which handles all but the most extreme |
3158 | cases. You can override the default by suppling pcre_exec() with a |
3159 | pcre_extra block in which match_limit is set, and PCRE_EX- |
3160 | TRA_MATCH_LIMIT is set in the flags field. If the limit is exceeded, |
3161 | pcre_exec() returns PCRE_ERROR_MATCHLIMIT. |
3162 | |
3163 | A value for the match limit may also be supplied by an item at the |
3164 | start of a pattern of the form |
3165 | |
3166 | (*LIMIT_MATCH=d) |
3167 | |
3168 | where d is a decimal number. However, such a setting is ignored unless |
3169 | d is less than the limit set by the caller of pcre_exec() or, if no |
3170 | such limit is set, less than the default. |
3171 | |
3172 | The match_limit_recursion field is similar to match_limit, but instead |
3173 | of limiting the total number of times that match() is called, it limits |
3174 | the depth of recursion. The recursion depth is a smaller number than |
3175 | the total number of calls, because not all calls to match() are recur- |
3176 | sive. This limit is of use only if it is set smaller than match_limit. |
3177 | |
3178 | Limiting the recursion depth limits the amount of machine stack that |
3179 | can be used, or, when PCRE has been compiled to use memory on the heap |
3180 | instead of the stack, the amount of heap memory that can be used. This |
3181 | limit is not relevant, and is ignored, when matching is done using JIT |
3182 | compiled code. |
3183 | |
3184 | The default value for match_limit_recursion can be set when PCRE is |
3185 | built; the default default is the same value as the default for |
3186 | match_limit. You can override the default by suppling pcre_exec() with |
3187 | a pcre_extra block in which match_limit_recursion is set, and PCRE_EX- |
3188 | TRA_MATCH_LIMIT_RECURSION is set in the flags field. If the limit is |
3189 | exceeded, pcre_exec() returns PCRE_ERROR_RECURSIONLIMIT. |
3190 | |
3191 | A value for the recursion limit may also be supplied by an item at the |
3192 | start of a pattern of the form |
3193 | |
3194 | (*LIMIT_RECURSION=d) |
3195 | |
3196 | where d is a decimal number. However, such a setting is ignored unless |
3197 | d is less than the limit set by the caller of pcre_exec() or, if no |
3198 | such limit is set, less than the default. |
3199 | |
3200 | The callout_data field is used in conjunction with the "callout" fea- |
3201 | ture, and is described in the pcrecallout documentation. |
3202 | |
3203 | The tables field is provided for use with patterns that have been pre- |
3204 | compiled using custom character tables, saved to disc or elsewhere, and |
3205 | then reloaded, because the tables that were used to compile a pattern |
3206 | are not saved with it. See the pcreprecompile documentation for a dis- |
3207 | cussion of saving compiled patterns for later use. If NULL is passed |
3208 | using this mechanism, it forces PCRE's internal tables to be used. |
3209 | |
3210 | Warning: The tables that pcre_exec() uses must be the same as those |
3211 | that were used when the pattern was compiled. If this is not the case, |
3212 | the behaviour of pcre_exec() is undefined. Therefore, when a pattern is |
3213 | compiled and matched in the same process, this field should never be |
3214 | set. In this (the most common) case, the correct table pointer is auto- |
3215 | matically passed with the compiled pattern from pcre_compile() to |
3216 | pcre_exec(). |
3217 | |
3218 | If PCRE_EXTRA_MARK is set in the flags field, the mark field must be |
3219 | set to point to a suitable variable. If the pattern contains any back- |
3220 | tracking control verbs such as (*MARK:NAME), and the execution ends up |
3221 | with a name to pass back, a pointer to the name string (zero termi- |
3222 | nated) is placed in the variable pointed to by the mark field. The |
3223 | names are within the compiled pattern; if you wish to retain such a |
3224 | name you must copy it before freeing the memory of a compiled pattern. |
3225 | If there is no name to pass back, the variable pointed to by the mark |
3226 | field is set to NULL. For details of the backtracking control verbs, |
3227 | see the section entitled "Backtracking control" in the pcrepattern doc- |
3228 | umentation. |
3229 | |
3230 | Option bits for pcre_exec() |
3231 | |
3232 | The unused bits of the options argument for pcre_exec() must be zero. |
3233 | The only bits that may be set are PCRE_ANCHORED, PCRE_NEWLINE_xxx, |
3234 | PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, |
3235 | PCRE_NO_START_OPTIMIZE, PCRE_NO_UTF8_CHECK, PCRE_PARTIAL_HARD, and |
3236 | PCRE_PARTIAL_SOFT. |
3237 | |
3238 | If the pattern was successfully studied with one of the just-in-time |
3239 | (JIT) compile options, the only supported options for JIT execution are |
3240 | PCRE_NO_UTF8_CHECK, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, |
3241 | PCRE_NOTEMPTY_ATSTART, PCRE_PARTIAL_HARD, and PCRE_PARTIAL_SOFT. If an |
3242 | unsupported option is used, JIT execution is disabled and the normal |
3243 | interpretive code in pcre_exec() is run. |
3244 | |
3245 | PCRE_ANCHORED |
3246 | |
3247 | The PCRE_ANCHORED option limits pcre_exec() to matching at the first |
3248 | matching position. If a pattern was compiled with PCRE_ANCHORED, or |
3249 | turned out to be anchored by virtue of its contents, it cannot be made |
3250 | unachored at matching time. |
3251 | |
3252 | PCRE_BSR_ANYCRLF |
3253 | PCRE_BSR_UNICODE |
3254 | |
3255 | These options (which are mutually exclusive) control what the \R escape |
3256 | sequence matches. The choice is either to match only CR, LF, or CRLF, |
3257 | or to match any Unicode newline sequence. These options override the |
3258 | choice that was made or defaulted when the pattern was compiled. |
3259 | |
3260 | PCRE_NEWLINE_CR |
3261 | PCRE_NEWLINE_LF |
3262 | PCRE_NEWLINE_CRLF |
3263 | PCRE_NEWLINE_ANYCRLF |
3264 | PCRE_NEWLINE_ANY |
3265 | |
3266 | These options override the newline definition that was chosen or de- |
3267 | faulted when the pattern was compiled. For details, see the description |
3268 | of pcre_compile() above. During matching, the newline choice affects |
3269 | the behaviour of the dot, circumflex, and dollar metacharacters. It may |
3270 | also alter the way the match position is advanced after a match failure |
3271 | for an unanchored pattern. |
3272 | |
3273 | When PCRE_NEWLINE_CRLF, PCRE_NEWLINE_ANYCRLF, or PCRE_NEWLINE_ANY is |
3274 | set, and a match attempt for an unanchored pattern fails when the cur- |
3275 | rent position is at a CRLF sequence, and the pattern contains no ex- |
3276 | plicit matches for CR or LF characters, the match position is advanced |
3277 | by two characters instead of one, in other words, to after the CRLF. |
3278 | |
3279 | The above rule is a compromise that makes the most common cases work as |
3280 | expected. For example, if the pattern is .+A (and the PCRE_DOTALL op- |
3281 | tion is not set), it does not match the string "\r\nA" because, after |
3282 | failing at the start, it skips both the CR and the LF before retrying. |
3283 | However, the pattern [\r\n]A does match that string, because it con- |
3284 | tains an explicit CR or LF reference, and so advances only by one char- |
3285 | acter after the first failure. |
3286 | |
3287 | An explicit match for CR of LF is either a literal appearance of one of |
3288 | those characters, or one of the \r or \n escape sequences. Implicit |
3289 | matches such as [^X] do not count, nor does \s (which includes CR and |
3290 | LF in the characters that it matches). |
3291 | |
3292 | Notwithstanding the above, anomalous effects may still occur when CRLF |
3293 | is a valid newline sequence and explicit \r or \n escapes appear in the |
3294 | pattern. |
3295 | |
3296 | PCRE_NOTBOL |
3297 | |
3298 | This option specifies that first character of the subject string is not |
3299 | the beginning of a line, so the circumflex metacharacter should not |
3300 | match before it. Setting this without PCRE_MULTILINE (at compile time) |
3301 | causes circumflex never to match. This option affects only the behav- |
3302 | iour of the circumflex metacharacter. It does not affect \A. |
3303 | |
3304 | PCRE_NOTEOL |
3305 | |
3306 | This option specifies that the end of the subject string is not the end |
3307 | of a line, so the dollar metacharacter should not match it nor (except |
3308 | in multiline mode) a newline immediately before it. Setting this with- |
3309 | out PCRE_MULTILINE (at compile time) causes dollar never to match. This |
3310 | option affects only the behaviour of the dollar metacharacter. It does |
3311 | not affect \Z or \z. |
3312 | |
3313 | PCRE_NOTEMPTY |
3314 | |
3315 | An empty string is not considered to be a valid match if this option is |
3316 | set. If there are alternatives in the pattern, they are tried. If all |
3317 | the alternatives match the empty string, the entire match fails. For |
3318 | example, if the pattern |
3319 | |
3320 | a?b? |
3321 | |
3322 | is applied to a string not beginning with "a" or "b", it matches an |
3323 | empty string at the start of the subject. With PCRE_NOTEMPTY set, this |
3324 | match is not valid, so PCRE searches further into the string for occur- |
3325 | rences of "a" or "b". |
3326 | |
3327 | PCRE_NOTEMPTY_ATSTART |
3328 | |
3329 | This is like PCRE_NOTEMPTY, except that an empty string match that is |
3330 | not at the start of the subject is permitted. If the pattern is an- |
3331 | chored, such a match can occur only if the pattern contains \K. |
3332 | |
3333 | Perl has no direct equivalent of PCRE_NOTEMPTY or PCRE_NOTEMPTY_AT- |
3334 | START, but it does make a special case of a pattern match of the empty |
3335 | string within its split() function, and when using the /g modifier. It |
3336 | is possible to emulate Perl's behaviour after matching a null string by |
3337 | first trying the match again at the same offset with PCRE_NOTEMPTY_AT- |
3338 | START and PCRE_ANCHORED, and then if that fails, by advancing the |
3339 | starting offset (see below) and trying an ordinary match again. There |
3340 | is some code that demonstrates how to do this in the pcredemo sample |
3341 | program. In the most general case, you have to check to see if the new- |
3342 | line convention recognizes CRLF as a newline, and if so, and the cur- |
3343 | rent character is CR followed by LF, advance the starting offset by two |
3344 | characters instead of one. |
3345 | |
3346 | PCRE_NO_START_OPTIMIZE |
3347 | |
3348 | There are a number of optimizations that pcre_exec() uses at the start |
3349 | of a match, in order to speed up the process. For example, if it is |
3350 | known that an unanchored match must start with a specific character, it |
3351 | searches the subject for that character, and fails immediately if it |
3352 | cannot find it, without actually running the main matching function. |
3353 | This means that a special item such as (*COMMIT) at the start of a pat- |
3354 | tern is not considered until after a suitable starting point for the |
3355 | match has been found. Also, when callouts or (*MARK) items are in use, |
3356 | these "start-up" optimizations can cause them to be skipped if the pat- |
3357 | tern is never actually used. The start-up optimizations are in effect a |
3358 | pre-scan of the subject that takes place before the pattern is run. |
3359 | |
3360 | The PCRE_NO_START_OPTIMIZE option disables the start-up optimizations, |
3361 | possibly causing performance to suffer, but ensuring that in cases |
3362 | where the result is "no match", the callouts do occur, and that items |
3363 | such as (*COMMIT) and (*MARK) are considered at every possible starting |
3364 | position in the subject string. If PCRE_NO_START_OPTIMIZE is set at |
3365 | compile time, it cannot be unset at matching time. The use of |
3366 | PCRE_NO_START_OPTIMIZE at matching time (that is, passing it to |
3367 | pcre_exec()) disables JIT execution; in this situation, matching is al- |
3368 | ways done using interpretively. |
3369 | |
3370 | Setting PCRE_NO_START_OPTIMIZE can change the outcome of a matching op- |
3371 | eration. Consider the pattern |
3372 | |
3373 | (*COMMIT)ABC |
3374 | |
3375 | When this is compiled, PCRE records the fact that a match must start |
3376 | with the character "A". Suppose the subject string is "DEFABC". The |
3377 | start-up optimization scans along the subject, finds "A" and runs the |
3378 | first match attempt from there. The (*COMMIT) item means that the pat- |
3379 | tern must match the current starting position, which in this case, it |
3380 | does. However, if the same match is run with PCRE_NO_START_OPTIMIZE |
3381 | set, the initial scan along the subject string does not happen. The |
3382 | first match attempt is run starting from "D" and when this fails, |
3383 | (*COMMIT) prevents any further matches being tried, so the overall re- |
3384 | sult is "no match". If the pattern is studied, more start-up optimiza- |
3385 | tions may be used. For example, a minimum length for the subject may be |
3386 | recorded. Consider the pattern |
3387 | |
3388 | (*MARK:A)(X|Y) |
3389 | |
3390 | The minimum length for a match is one character. If the subject is |
3391 | "ABC", there will be attempts to match "ABC", "BC", "C", and then fi- |
3392 | nally an empty string. If the pattern is studied, the final attempt |
3393 | does not take place, because PCRE knows that the subject is too short, |
3394 | and so the (*MARK) is never encountered. In this case, studying the |
3395 | pattern does not affect the overall match result, which is still "no |
3396 | match", but it does affect the auxiliary information that is returned. |
3397 | |
3398 | PCRE_NO_UTF8_CHECK |
3399 | |
3400 | When PCRE_UTF8 is set at compile time, the validity of the subject as a |
3401 | UTF-8 string is automatically checked when pcre_exec() is subsequently |
3402 | called. The entire string is checked before any other processing takes |
3403 | place. The value of startoffset is also checked to ensure that it |
3404 | points to the start of a UTF-8 character. There is a discussion about |
3405 | the validity of UTF-8 strings in the pcreunicode page. If an invalid |
3406 | sequence of bytes is found, pcre_exec() returns the error PCRE_ER- |
3407 | ROR_BADUTF8 or, if PCRE_PARTIAL_HARD is set and the problem is a trun- |
3408 | cated character at the end of the subject, PCRE_ERROR_SHORTUTF8. In |
3409 | both cases, information about the precise nature of the error may also |
3410 | be returned (see the descriptions of these errors in the section enti- |
3411 | tled Error return values from pcre_exec() below). If startoffset con- |
3412 | tains a value that does not point to the start of a UTF-8 character (or |
3413 | to the end of the subject), PCRE_ERROR_BADUTF8_OFFSET is returned. |
3414 | |
3415 | If you already know that your subject is valid, and you want to skip |
3416 | these checks for performance reasons, you can set the |
3417 | PCRE_NO_UTF8_CHECK option when calling pcre_exec(). You might want to |
3418 | do this for the second and subsequent calls to pcre_exec() if you are |
3419 | making repeated calls to find all the matches in a single subject |
3420 | string. However, you should be sure that the value of startoffset |
3421 | points to the start of a character (or the end of the subject). When |
3422 | PCRE_NO_UTF8_CHECK is set, the effect of passing an invalid string as a |
3423 | subject or an invalid value of startoffset is undefined. Your program |
3424 | may crash or loop. |
3425 | |
3426 | PCRE_PARTIAL_HARD |
3427 | PCRE_PARTIAL_SOFT |
3428 | |
3429 | These options turn on the partial matching feature. For backwards com- |
3430 | patibility, PCRE_PARTIAL is a synonym for PCRE_PARTIAL_SOFT. A partial |
3431 | match occurs if the end of the subject string is reached successfully, |
3432 | but there are not enough subject characters to complete the match. If |
3433 | this happens when PCRE_PARTIAL_SOFT (but not PCRE_PARTIAL_HARD) is set, |
3434 | matching continues by testing any remaining alternatives. Only if no |
3435 | complete match can be found is PCRE_ERROR_PARTIAL returned instead of |
3436 | PCRE_ERROR_NOMATCH. In other words, PCRE_PARTIAL_SOFT says that the |
3437 | caller is prepared to handle a partial match, but only if no complete |
3438 | match can be found. |
3439 | |
3440 | If PCRE_PARTIAL_HARD is set, it overrides PCRE_PARTIAL_SOFT. In this |
3441 | case, if a partial match is found, pcre_exec() immediately returns |
3442 | PCRE_ERROR_PARTIAL, without considering any other alternatives. In |
3443 | other words, when PCRE_PARTIAL_HARD is set, a partial match is consid- |
3444 | ered to be more important that an alternative complete match. |
3445 | |
3446 | In both cases, the portion of the string that was inspected when the |
3447 | partial match was found is set as the first matching string. There is a |
3448 | more detailed discussion of partial and multi-segment matching, with |
3449 | examples, in the pcrepartial documentation. |
3450 | |
3451 | The string to be matched by pcre_exec() |
3452 | |
3453 | The subject string is passed to pcre_exec() as a pointer in subject, a |
3454 | length in length, and a starting offset in startoffset. The units for |
3455 | length and startoffset are bytes for the 8-bit library, 16-bit data |
3456 | items for the 16-bit library, and 32-bit data items for the 32-bit li- |
3457 | brary. |
3458 | |
3459 | If startoffset is negative or greater than the length of the subject, |
3460 | pcre_exec() returns PCRE_ERROR_BADOFFSET. When the starting offset is |
3461 | zero, the search for a match starts at the beginning of the subject, |
3462 | and this is by far the most common case. In UTF-8 or UTF-16 mode, the |
3463 | offset must point to the start of a character, or the end of the sub- |
3464 | ject (in UTF-32 mode, one data unit equals one character, so all off- |
3465 | sets are valid). Unlike the pattern string, the subject may contain bi- |
3466 | nary zeroes. |
3467 | |
3468 | A non-zero starting offset is useful when searching for another match |
3469 | in the same subject by calling pcre_exec() again after a previous suc- |
3470 | cess. Setting startoffset differs from just passing over a shortened |
3471 | string and setting PCRE_NOTBOL in the case of a pattern that begins |
3472 | with any kind of lookbehind. For example, consider the pattern |
3473 | |
3474 | \Biss\B |
3475 | |
3476 | which finds occurrences of "iss" in the middle of words. (\B matches |
3477 | only if the current position in the subject is not a word boundary.) |
3478 | When applied to the string "Mississipi" the first call to pcre_exec() |
3479 | finds the first occurrence. If pcre_exec() is called again with just |
3480 | the remainder of the subject, namely "issipi", it does not match, be- |
3481 | cause \B is always false at the start of the subject, which is deemed |
3482 | to be a word boundary. However, if pcre_exec() is passed the entire |
3483 | string again, but with startoffset set to 4, it finds the second occur- |
3484 | rence of "iss" because it is able to look behind the starting point to |
3485 | discover that it is preceded by a letter. |
3486 | |
3487 | Finding all the matches in a subject is tricky when the pattern can |
3488 | match an empty string. It is possible to emulate Perl's /g behaviour by |
3489 | first trying the match again at the same offset, with the |
3490 | PCRE_NOTEMPTY_ATSTART and PCRE_ANCHORED options, and then if that |
3491 | fails, advancing the starting offset and trying an ordinary match |
3492 | again. There is some code that demonstrates how to do this in the pcre- |
3493 | demo sample program. In the most general case, you have to check to see |
3494 | if the newline convention recognizes CRLF as a newline, and if so, and |
3495 | the current character is CR followed by LF, advance the starting offset |
3496 | by two characters instead of one. |
3497 | |
3498 | If a non-zero starting offset is passed when the pattern is anchored, |
3499 | one attempt to match at the given offset is made. This can only succeed |
3500 | if the pattern does not require the match to be at the start of the |
3501 | subject. |
3502 | |
3503 | How pcre_exec() returns captured substrings |
3504 | |
3505 | In general, a pattern matches a certain portion of the subject, and in |
3506 | addition, further substrings from the subject may be picked out by |
3507 | parts of the pattern. Following the usage in Jeffrey Friedl's book, |
3508 | this is called "capturing" in what follows, and the phrase "capturing |
3509 | subpattern" is used for a fragment of a pattern that picks out a sub- |
3510 | string. PCRE supports several other kinds of parenthesized subpattern |
3511 | that do not cause substrings to be captured. |
3512 | |
3513 | Captured substrings are returned to the caller via a vector of integers |
3514 | whose address is passed in ovector. The number of elements in the vec- |
3515 | tor is passed in ovecsize, which must be a non-negative number. Note: |
3516 | this argument is NOT the size of ovector in bytes. |
3517 | |
3518 | The first two-thirds of the vector is used to pass back captured sub- |
3519 | strings, each substring using a pair of integers. The remaining third |
3520 | of the vector is used as workspace by pcre_exec() while matching cap- |
3521 | turing subpatterns, and is not available for passing back information. |
3522 | The number passed in ovecsize should always be a multiple of three. If |
3523 | it is not, it is rounded down. |
3524 | |
3525 | When a match is successful, information about captured substrings is |
3526 | returned in pairs of integers, starting at the beginning of ovector, |
3527 | and continuing up to two-thirds of its length at the most. The first |
3528 | element of each pair is set to the offset of the first character in a |
3529 | substring, and the second is set to the offset of the first character |
3530 | after the end of a substring. These values are always data unit off- |
3531 | sets, even in UTF mode. They are byte offsets in the 8-bit library, |
3532 | 16-bit data item offsets in the 16-bit library, and 32-bit data item |
3533 | offsets in the 32-bit library. Note: they are not character counts. |
3534 | |
3535 | The first pair of integers, ovector[0] and ovector[1], identify the |
3536 | portion of the subject string matched by the entire pattern. The next |
3537 | pair is used for the first capturing subpattern, and so on. The value |
3538 | returned by pcre_exec() is one more than the highest numbered pair that |
3539 | has been set. For example, if two substrings have been captured, the |
3540 | returned value is 3. If there are no capturing subpatterns, the return |
3541 | value from a successful match is 1, indicating that just the first pair |
3542 | of offsets has been set. |
3543 | |
3544 | If a capturing subpattern is matched repeatedly, it is the last portion |
3545 | of the string that it matched that is returned. |
3546 | |
3547 | If the vector is too small to hold all the captured substring offsets, |
3548 | it is used as far as possible (up to two-thirds of its length), and the |
3549 | function returns a value of zero. If neither the actual string matched |
3550 | nor any captured substrings are of interest, pcre_exec() may be called |
3551 | with ovector passed as NULL and ovecsize as zero. However, if the pat- |
3552 | tern contains back references and the ovector is not big enough to re- |
3553 | member the related substrings, PCRE has to get additional memory for |
3554 | use during matching. Thus it is usually advisable to supply an ovector |
3555 | of reasonable size. |
3556 | |
3557 | There are some cases where zero is returned (indicating vector over- |
3558 | flow) when in fact the vector is exactly the right size for the final |
3559 | match. For example, consider the pattern |
3560 | |
3561 | (a)(?:(b)c|bd) |
3562 | |
3563 | If a vector of 6 elements (allowing for only 1 captured substring) is |
3564 | given with subject string "abd", pcre_exec() will try to set the second |
3565 | captured string, thereby recording a vector overflow, before failing to |
3566 | match "c" and backing up to try the second alternative. The zero re- |
3567 | turn, however, does correctly indicate that the maximum number of slots |
3568 | (namely 2) have been filled. In similar cases where there is temporary |
3569 | overflow, but the final number of used slots is actually less than the |
3570 | maximum, a non-zero value is returned. |
3571 | |
3572 | The pcre_fullinfo() function can be used to find out how many capturing |
3573 | subpatterns there are in a compiled pattern. The smallest size for |
3574 | ovector that will allow for n captured substrings, in addition to the |
3575 | offsets of the substring matched by the whole pattern, is (n+1)*3. |
3576 | |
3577 | It is possible for capturing subpattern number n+1 to match some part |
3578 | of the subject when subpattern n has not been used at all. For example, |
3579 | if the string "abc" is matched against the pattern (a|(z))(bc) the re- |
3580 | turn from the function is 4, and subpatterns 1 and 3 are matched, but 2 |
3581 | is not. When this happens, both values in the offset pairs correspond- |
3582 | ing to unused subpatterns are set to -1. |
3583 | |
3584 | Offset values that correspond to unused subpatterns at the end of the |
3585 | expression are also set to -1. For example, if the string "abc" is |
3586 | matched against the pattern (abc)(x(yz)?)? subpatterns 2 and 3 are not |
3587 | matched. The return from the function is 2, because the highest used |
3588 | capturing subpattern number is 1, and the offsets for for the second |
3589 | and third capturing subpatterns (assuming the vector is large enough, |
3590 | of course) are set to -1. |
3591 | |
3592 | Note: Elements in the first two-thirds of ovector that do not corre- |
3593 | spond to capturing parentheses in the pattern are never changed. That |
3594 | is, if a pattern contains n capturing parentheses, no more than ovec- |
3595 | tor[0] to ovector[2n+1] are set by pcre_exec(). The other elements (in |
3596 | the first two-thirds) retain whatever values they previously had. |
3597 | |
3598 | Some convenience functions are provided for extracting the captured |
3599 | substrings as separate strings. These are described below. |
3600 | |
3601 | Error return values from pcre_exec() |
3602 | |
3603 | If pcre_exec() fails, it returns a negative number. The following are |
3604 | defined in the header file: |
3605 | |
3606 | PCRE_ERROR_NOMATCH (-1) |
3607 | |
3608 | The subject string did not match the pattern. |
3609 | |
3610 | PCRE_ERROR_NULL (-2) |
3611 | |
3612 | Either code or subject was passed as NULL, or ovector was NULL and |
3613 | ovecsize was not zero. |
3614 | |
3615 | PCRE_ERROR_BADOPTION (-3) |
3616 | |
3617 | An unrecognized bit was set in the options argument. |
3618 | |
3619 | PCRE_ERROR_BADMAGIC (-4) |
3620 | |
3621 | PCRE stores a 4-byte "magic number" at the start of the compiled code, |
3622 | to catch the case when it is passed a junk pointer and to detect when a |
3623 | pattern that was compiled in an environment of one endianness is run in |
3624 | an environment with the other endianness. This is the error that PCRE |
3625 | gives when the magic number is not present. |
3626 | |
3627 | PCRE_ERROR_UNKNOWN_OPCODE (-5) |
3628 | |
3629 | While running the pattern match, an unknown item was encountered in the |
3630 | compiled pattern. This error could be caused by a bug in PCRE or by |
3631 | overwriting of the compiled pattern. |
3632 | |
3633 | PCRE_ERROR_NOMEMORY (-6) |
3634 | |
3635 | If a pattern contains back references, but the ovector that is passed |
3636 | to pcre_exec() is not big enough to remember the referenced substrings, |
3637 | PCRE gets a block of memory at the start of matching to use for this |
3638 | purpose. If the call via pcre_malloc() fails, this error is given. The |
3639 | memory is automatically freed at the end of matching. |
3640 | |
3641 | This error is also given if pcre_stack_malloc() fails in pcre_exec(). |
3642 | This can happen only when PCRE has been compiled with --disable-stack- |
3643 | for-recursion. |
3644 | |
3645 | PCRE_ERROR_NOSUBSTRING (-7) |
3646 | |
3647 | This error is used by the pcre_copy_substring(), pcre_get_substring(), |
3648 | and pcre_get_substring_list() functions (see below). It is never re- |
3649 | turned by pcre_exec(). |
3650 | |
3651 | PCRE_ERROR_MATCHLIMIT (-8) |
3652 | |
3653 | The backtracking limit, as specified by the match_limit field in a |
3654 | pcre_extra structure (or defaulted) was reached. See the description |
3655 | above. |
3656 | |
3657 | PCRE_ERROR_CALLOUT (-9) |
3658 | |
3659 | This error is never generated by pcre_exec() itself. It is provided for |
3660 | use by callout functions that want to yield a distinctive error code. |
3661 | See the pcrecallout documentation for details. |
3662 | |
3663 | PCRE_ERROR_BADUTF8 (-10) |
3664 | |
3665 | A string that contains an invalid UTF-8 byte sequence was passed as a |
3666 | subject, and the PCRE_NO_UTF8_CHECK option was not set. If the size of |
3667 | the output vector (ovecsize) is at least 2, the byte offset to the |
3668 | start of the the invalid UTF-8 character is placed in the first ele- |
3669 | ment, and a reason code is placed in the second element. The reason |
3670 | codes are listed in the following section. For backward compatibility, |
3671 | if PCRE_PARTIAL_HARD is set and the problem is a truncated UTF-8 char- |
3672 | acter at the end of the subject (reason codes 1 to 5), PCRE_ER- |
3673 | ROR_SHORTUTF8 is returned instead of PCRE_ERROR_BADUTF8. |
3674 | |
3675 | PCRE_ERROR_BADUTF8_OFFSET (-11) |
3676 | |
3677 | The UTF-8 byte sequence that was passed as a subject was checked and |
3678 | found to be valid (the PCRE_NO_UTF8_CHECK option was not set), but the |
3679 | value of startoffset did not point to the beginning of a UTF-8 charac- |
3680 | ter or the end of the subject. |
3681 | |
3682 | PCRE_ERROR_PARTIAL (-12) |
3683 | |
3684 | The subject string did not match, but it did match partially. See the |
3685 | pcrepartial documentation for details of partial matching. |
3686 | |
3687 | PCRE_ERROR_BADPARTIAL (-13) |
3688 | |
3689 | This code is no longer in use. It was formerly returned when the |
3690 | PCRE_PARTIAL option was used with a compiled pattern containing items |
3691 | that were not supported for partial matching. From release 8.00 on- |
3692 | wards, there are no restrictions on partial matching. |
3693 | |
3694 | PCRE_ERROR_INTERNAL (-14) |
3695 | |
3696 | An unexpected internal error has occurred. This error could be caused |
3697 | by a bug in PCRE or by overwriting of the compiled pattern. |
3698 | |
3699 | PCRE_ERROR_BADCOUNT (-15) |
3700 | |
3701 | This error is given if the value of the ovecsize argument is negative. |
3702 | |
3703 | PCRE_ERROR_RECURSIONLIMIT (-21) |
3704 | |
3705 | The internal recursion limit, as specified by the match_limit_recursion |
3706 | field in a pcre_extra structure (or defaulted) was reached. See the de- |
3707 | scription above. |
3708 | |
3709 | PCRE_ERROR_BADNEWLINE (-23) |
3710 | |
3711 | An invalid combination of PCRE_NEWLINE_xxx options was given. |
3712 | |
3713 | PCRE_ERROR_BADOFFSET (-24) |
3714 | |
3715 | The value of startoffset was negative or greater than the length of the |
3716 | subject, that is, the value in length. |
3717 | |
3718 | PCRE_ERROR_SHORTUTF8 (-25) |
3719 | |
3720 | This error is returned instead of PCRE_ERROR_BADUTF8 when the subject |
3721 | string ends with a truncated UTF-8 character and the PCRE_PARTIAL_HARD |
3722 | option is set. Information about the failure is returned as for |
3723 | PCRE_ERROR_BADUTF8. It is in fact sufficient to detect this case, but |
3724 | this special error code for PCRE_PARTIAL_HARD precedes the implementa- |
3725 | tion of returned information; it is retained for backwards compatibil- |
3726 | ity. |
3727 | |
3728 | PCRE_ERROR_RECURSELOOP (-26) |
3729 | |
3730 | This error is returned when pcre_exec() detects a recursion loop within |
3731 | the pattern. Specifically, it means that either the whole pattern or a |
3732 | subpattern has been called recursively for the second time at the same |
3733 | position in the subject string. Some simple patterns that might do this |
3734 | are detected and faulted at compile time, but more complicated cases, |
3735 | in particular mutual recursions between two different subpatterns, can- |
3736 | not be detected until run time. |
3737 | |
3738 | PCRE_ERROR_JIT_STACKLIMIT (-27) |
3739 | |
3740 | This error is returned when a pattern that was successfully studied us- |
3741 | ing a JIT compile option is being matched, but the memory available for |
3742 | the just-in-time processing stack is not large enough. See the pcrejit |
3743 | documentation for more details. |
3744 | |
3745 | PCRE_ERROR_BADMODE (-28) |
3746 | |
3747 | This error is given if a pattern that was compiled by the 8-bit library |
3748 | is passed to a 16-bit or 32-bit library function, or vice versa. |
3749 | |
3750 | PCRE_ERROR_BADENDIANNESS (-29) |
3751 | |
3752 | This error is given if a pattern that was compiled and saved is |
3753 | reloaded on a host with different endianness. The utility function |
3754 | pcre_pattern_to_host_byte_order() can be used to convert such a pattern |
3755 | so that it runs on the new host. |
3756 | |
3757 | PCRE_ERROR_JIT_BADOPTION |
3758 | |
3759 | This error is returned when a pattern that was successfully studied us- |
3760 | ing a JIT compile option is being matched, but the matching mode (par- |
3761 | tial or complete match) does not correspond to any JIT compilation |
3762 | mode. When the JIT fast path function is used, this error may be also |
3763 | given for invalid options. See the pcrejit documentation for more de- |
3764 | tails. |
3765 | |
3766 | PCRE_ERROR_BADLENGTH (-32) |
3767 | |
3768 | This error is given if pcre_exec() is called with a negative value for |
3769 | the length argument. |
3770 | |
3771 | Error numbers -16 to -20, -22, and 30 are not used by pcre_exec(). |
3772 | |
3773 | Reason codes for invalid UTF-8 strings |
3774 | |
3775 | This section applies only to the 8-bit library. The corresponding in- |
3776 | formation for the 16-bit and 32-bit libraries is given in the pcre16 |
3777 | and pcre32 pages. |
3778 | |
3779 | When pcre_exec() returns either PCRE_ERROR_BADUTF8 or PCRE_ERROR_SHORT- |
3780 | UTF8, and the size of the output vector (ovecsize) is at least 2, the |
3781 | offset of the start of the invalid UTF-8 character is placed in the |
3782 | first output vector element (ovector[0]) and a reason code is placed in |
3783 | the second element (ovector[1]). The reason codes are given names in |
3784 | the pcre.h header file: |
3785 | |
3786 | PCRE_UTF8_ERR1 |
3787 | PCRE_UTF8_ERR2 |
3788 | PCRE_UTF8_ERR3 |
3789 | PCRE_UTF8_ERR4 |
3790 | PCRE_UTF8_ERR5 |
3791 | |
3792 | The string ends with a truncated UTF-8 character; the code specifies |
3793 | how many bytes are missing (1 to 5). Although RFC 3629 restricts UTF-8 |
3794 | characters to be no longer than 4 bytes, the encoding scheme (origi- |
3795 | nally defined by RFC 2279) allows for up to 6 bytes, and this is |
3796 | checked first; hence the possibility of 4 or 5 missing bytes. |
3797 | |
3798 | PCRE_UTF8_ERR6 |
3799 | PCRE_UTF8_ERR7 |
3800 | PCRE_UTF8_ERR8 |
3801 | PCRE_UTF8_ERR9 |
3802 | PCRE_UTF8_ERR10 |
3803 | |
3804 | The two most significant bits of the 2nd, 3rd, 4th, 5th, or 6th byte of |
3805 | the character do not have the binary value 0b10 (that is, either the |
3806 | most significant bit is 0, or the next bit is 1). |
3807 | |
3808 | PCRE_UTF8_ERR11 |
3809 | PCRE_UTF8_ERR12 |
3810 | |
3811 | A character that is valid by the RFC 2279 rules is either 5 or 6 bytes |
3812 | long; these code points are excluded by RFC 3629. |
3813 | |
3814 | PCRE_UTF8_ERR13 |
3815 | |
3816 | A 4-byte character has a value greater than 0x10fff; these code points |
3817 | are excluded by RFC 3629. |
3818 | |
3819 | PCRE_UTF8_ERR14 |
3820 | |
3821 | A 3-byte character has a value in the range 0xd800 to 0xdfff; this |
3822 | range of code points are reserved by RFC 3629 for use with UTF-16, and |
3823 | so are excluded from UTF-8. |
3824 | |
3825 | PCRE_UTF8_ERR15 |
3826 | PCRE_UTF8_ERR16 |
3827 | PCRE_UTF8_ERR17 |
3828 | PCRE_UTF8_ERR18 |
3829 | PCRE_UTF8_ERR19 |
3830 | |
3831 | A 2-, 3-, 4-, 5-, or 6-byte character is "overlong", that is, it codes |
3832 | for a value that can be represented by fewer bytes, which is invalid. |
3833 | For example, the two bytes 0xc0, 0xae give the value 0x2e, whose cor- |
3834 | rect coding uses just one byte. |
3835 | |
3836 | PCRE_UTF8_ERR20 |
3837 | |
3838 | The two most significant bits of the first byte of a character have the |
3839 | binary value 0b10 (that is, the most significant bit is 1 and the sec- |
3840 | ond is 0). Such a byte can only validly occur as the second or subse- |
3841 | quent byte of a multi-byte character. |
3842 | |
3843 | PCRE_UTF8_ERR21 |
3844 | |
3845 | The first byte of a character has the value 0xfe or 0xff. These values |
3846 | can never occur in a valid UTF-8 string. |
3847 | |
3848 | PCRE_UTF8_ERR22 |
3849 | |
3850 | This error code was formerly used when the presence of a so-called |
3851 | "non-character" caused an error. Unicode corrigendum #9 makes it clear |
3852 | that such characters should not cause a string to be rejected, and so |
3853 | this code is no longer in use and is never returned. |
3854 | |
3855 | |
3856 | EXTRACTING CAPTURED SUBSTRINGS BY NUMBER |
3857 | |
3858 | int pcre_copy_substring(const char *subject, int *ovector, |
3859 | int stringcount, int stringnumber, char *buffer, |
3860 | int buffersize); |
3861 | |
3862 | int pcre_get_substring(const char *subject, int *ovector, |
3863 | int stringcount, int stringnumber, |
3864 | const char **stringptr); |
3865 | |
3866 | int pcre_get_substring_list(const char *subject, |
3867 | int *ovector, int stringcount, const char ***listptr); |
3868 | |
3869 | Captured substrings can be accessed directly by using the offsets re- |
3870 | turned by pcre_exec() in ovector. For convenience, the functions |
3871 | pcre_copy_substring(), pcre_get_substring(), and pcre_get_sub- |
3872 | string_list() are provided for extracting captured substrings as new, |
3873 | separate, zero-terminated strings. These functions identify substrings |
3874 | by number. The next section describes functions for extracting named |
3875 | substrings. |
3876 | |
3877 | A substring that contains a binary zero is correctly extracted and has |
3878 | a further zero added on the end, but the result is not, of course, a C |
3879 | string. However, you can process such a string by referring to the |
3880 | length that is returned by pcre_copy_substring() and pcre_get_sub- |
3881 | string(). Unfortunately, the interface to pcre_get_substring_list() is |
3882 | not adequate for handling strings containing binary zeros, because the |
3883 | end of the final string is not independently indicated. |
3884 | |
3885 | The first three arguments are the same for all three of these func- |
3886 | tions: subject is the subject string that has just been successfully |
3887 | matched, ovector is a pointer to the vector of integer offsets that was |
3888 | passed to pcre_exec(), and stringcount is the number of substrings that |
3889 | were captured by the match, including the substring that matched the |
3890 | entire regular expression. This is the value returned by pcre_exec() if |
3891 | it is greater than zero. If pcre_exec() returned zero, indicating that |
3892 | it ran out of space in ovector, the value passed as stringcount should |
3893 | be the number of elements in the vector divided by three. |
3894 | |
3895 | The functions pcre_copy_substring() and pcre_get_substring() extract a |
3896 | single substring, whose number is given as stringnumber. A value of |
3897 | zero extracts the substring that matched the entire pattern, whereas |
3898 | higher values extract the captured substrings. For pcre_copy_sub- |
3899 | string(), the string is placed in buffer, whose length is given by |
3900 | buffersize, while for pcre_get_substring() a new block of memory is ob- |
3901 | tained via pcre_malloc, and its address is returned via stringptr. The |
3902 | yield of the function is the length of the string, not including the |
3903 | terminating zero, or one of these error codes: |
3904 | |
3905 | PCRE_ERROR_NOMEMORY (-6) |
3906 | |
3907 | The buffer was too small for pcre_copy_substring(), or the attempt to |
3908 | get memory failed for pcre_get_substring(). |
3909 | |
3910 | PCRE_ERROR_NOSUBSTRING (-7) |
3911 | |
3912 | There is no substring whose number is stringnumber. |
3913 | |
3914 | The pcre_get_substring_list() function extracts all available sub- |
3915 | strings and builds a list of pointers to them. All this is done in a |
3916 | single block of memory that is obtained via pcre_malloc. The address of |
3917 | the memory block is returned via listptr, which is also the start of |
3918 | the list of string pointers. The end of the list is marked by a NULL |
3919 | pointer. The yield of the function is zero if all went well, or the er- |
3920 | ror code |
3921 | |
3922 | PCRE_ERROR_NOMEMORY (-6) |
3923 | |
3924 | if the attempt to get the memory block failed. |
3925 | |
3926 | When any of these functions encounter a substring that is unset, which |
3927 | can happen when capturing subpattern number n+1 matches some part of |
3928 | the subject, but subpattern n has not been used at all, they return an |
3929 | empty string. This can be distinguished from a genuine zero-length sub- |
3930 | string by inspecting the appropriate offset in ovector, which is nega- |
3931 | tive for unset substrings. |
3932 | |
3933 | The two convenience functions pcre_free_substring() and pcre_free_sub- |
3934 | string_list() can be used to free the memory returned by a previous |
3935 | call of pcre_get_substring() or pcre_get_substring_list(), respec- |
3936 | tively. They do nothing more than call the function pointed to by |
3937 | pcre_free, which of course could be called directly from a C program. |
3938 | However, PCRE is used in some situations where it is linked via a spe- |
3939 | cial interface to another programming language that cannot use |
3940 | pcre_free directly; it is for these cases that the functions are pro- |
3941 | vided. |
3942 | |
3943 | |
3944 | EXTRACTING CAPTURED SUBSTRINGS BY NAME |
3945 | |
3946 | int pcre_get_stringnumber(const pcre *code, |
3947 | const char *name); |
3948 | |
3949 | int pcre_copy_named_substring(const pcre *code, |
3950 | const char *subject, int *ovector, |
3951 | int stringcount, const char *stringname, |
3952 | char *buffer, int buffersize); |
3953 | |
3954 | int pcre_get_named_substring(const pcre *code, |
3955 | const char *subject, int *ovector, |
3956 | int stringcount, const char *stringname, |
3957 | const char **stringptr); |
3958 | |
3959 | To extract a substring by name, you first have to find associated num- |
3960 | ber. For example, for this pattern |
3961 | |
3962 | (a+)b(?<xxx>\d+)... |
3963 | |
3964 | the number of the subpattern called "xxx" is 2. If the name is known to |
3965 | be unique (PCRE_DUPNAMES was not set), you can find the number from the |
3966 | name by calling pcre_get_stringnumber(). The first argument is the com- |
3967 | piled pattern, and the second is the name. The yield of the function is |
3968 | the subpattern number, or PCRE_ERROR_NOSUBSTRING (-7) if there is no |
3969 | subpattern of that name. |
3970 | |
3971 | Given the number, you can extract the substring directly, or use one of |
3972 | the functions described in the previous section. For convenience, there |
3973 | are also two functions that do the whole job. |
3974 | |
3975 | Most of the arguments of pcre_copy_named_substring() and |
3976 | pcre_get_named_substring() are the same as those for the similarly |
3977 | named functions that extract by number. As these are described in the |
3978 | previous section, they are not re-described here. There are just two |
3979 | differences: |
3980 | |
3981 | First, instead of a substring number, a substring name is given. Sec- |
3982 | ond, there is an extra argument, given at the start, which is a pointer |
3983 | to the compiled pattern. This is needed in order to gain access to the |
3984 | name-to-number translation table. |
3985 | |
3986 | These functions call pcre_get_stringnumber(), and if it succeeds, they |
3987 | then call pcre_copy_substring() or pcre_get_substring(), as appropri- |
3988 | ate. NOTE: If PCRE_DUPNAMES is set and there are duplicate names, the |
3989 | behaviour may not be what you want (see the next section). |
3990 | |
3991 | Warning: If the pattern uses the (?| feature to set up multiple subpat- |
3992 | terns with the same number, as described in the section on duplicate |
3993 | subpattern numbers in the pcrepattern page, you cannot use names to |
3994 | distinguish the different subpatterns, because names are not included |
3995 | in the compiled code. The matching process uses only numbers. For this |
3996 | reason, the use of different names for subpatterns of the same number |
3997 | causes an error at compile time. |
3998 | |
3999 | |
4000 | DUPLICATE SUBPATTERN NAMES |
4001 | |
4002 | int pcre_get_stringtable_entries(const pcre *code, |
4003 | const char *name, char **first, char **last); |
4004 | |
4005 | When a pattern is compiled with the PCRE_DUPNAMES option, names for |
4006 | subpatterns are not required to be unique. (Duplicate names are always |
4007 | allowed for subpatterns with the same number, created by using the (?| |
4008 | feature. Indeed, if such subpatterns are named, they are required to |
4009 | use the same names.) |
4010 | |
4011 | Normally, patterns with duplicate names are such that in any one match, |
4012 | only one of the named subpatterns participates. An example is shown in |
4013 | the pcrepattern documentation. |
4014 | |
4015 | When duplicates are present, pcre_copy_named_substring() and |
4016 | pcre_get_named_substring() return the first substring corresponding to |
4017 | the given name that is set. If none are set, PCRE_ERROR_NOSUBSTRING |
4018 | (-7) is returned; no data is returned. The pcre_get_stringnumber() |
4019 | function returns one of the numbers that are associated with the name, |
4020 | but it is not defined which it is. |
4021 | |
4022 | If you want to get full details of all captured substrings for a given |
4023 | name, you must use the pcre_get_stringtable_entries() function. The |
4024 | first argument is the compiled pattern, and the second is the name. The |
4025 | third and fourth are pointers to variables which are updated by the |
4026 | function. After it has run, they point to the first and last entries in |
4027 | the name-to-number table for the given name. The function itself re- |
4028 | turns the length of each entry, or PCRE_ERROR_NOSUBSTRING (-7) if there |
4029 | are none. The format of the table is described above in the section en- |
4030 | titled Information about a pattern above. Given all the relevant en- |
4031 | tries for the name, you can extract each of their numbers, and hence |
4032 | the captured data, if any. |
4033 | |
4034 | |
4035 | FINDING ALL POSSIBLE MATCHES |
4036 | |
4037 | The traditional matching function uses a similar algorithm to Perl, |
4038 | which stops when it finds the first match, starting at a given point in |
4039 | the subject. If you want to find all possible matches, or the longest |
4040 | possible match, consider using the alternative matching function (see |
4041 | below) instead. If you cannot use the alternative function, but still |
4042 | need to find all possible matches, you can kludge it up by making use |
4043 | of the callout facility, which is described in the pcrecallout documen- |
4044 | tation. |
4045 | |
4046 | What you have to do is to insert a callout right at the end of the pat- |
4047 | tern. When your callout function is called, extract and save the cur- |
4048 | rent matched substring. Then return 1, which forces pcre_exec() to |
4049 | backtrack and try other alternatives. Ultimately, when it runs out of |
4050 | matches, pcre_exec() will yield PCRE_ERROR_NOMATCH. |
4051 | |
4052 | |
4053 | OBTAINING AN ESTIMATE OF STACK USAGE |
4054 | |
4055 | Matching certain patterns using pcre_exec() can use a lot of process |
4056 | stack, which in certain environments can be rather limited in size. |
4057 | Some users find it helpful to have an estimate of the amount of stack |
4058 | that is used by pcre_exec(), to help them set recursion limits, as de- |
4059 | scribed in the pcrestack documentation. The estimate that is output by |
4060 | pcretest when called with the -m and -C options is obtained by calling |
4061 | pcre_exec with the values NULL, NULL, NULL, -999, and -999 for its |
4062 | first five arguments. |
4063 | |
4064 | Normally, if its first argument is NULL, pcre_exec() immediately re- |
4065 | turns the negative error code PCRE_ERROR_NULL, but with this special |
4066 | combination of arguments, it returns instead a negative number whose |
4067 | absolute value is the approximate stack frame size in bytes. (A nega- |
4068 | tive number is used so that it is clear that no match has happened.) |
4069 | The value is approximate because in some cases, recursive calls to |
4070 | pcre_exec() occur when there are one or two additional variables on the |
4071 | stack. |
4072 | |
4073 | If PCRE has been compiled to use the heap instead of the stack for re- |
4074 | cursion, the value returned is the size of each block that is obtained |
4075 | from the heap. |
4076 | |
4077 | |
4078 | MATCHING A PATTERN: THE ALTERNATIVE FUNCTION |
4079 | |
4080 | int pcre_dfa_exec(const pcre *code, const pcre_extra *extra, |
4081 | const char *subject, int length, int startoffset, |
4082 | int options, int *ovector, int ovecsize, |
4083 | int *workspace, int wscount); |
4084 | |
4085 | The function pcre_dfa_exec() is called to match a subject string |
4086 | against a compiled pattern, using a matching algorithm that scans the |
4087 | subject string just once, and does not backtrack. This has different |
4088 | characteristics to the normal algorithm, and is not compatible with |
4089 | Perl. Some of the features of PCRE patterns are not supported. Never- |
4090 | theless, there are times when this kind of matching can be useful. For |
4091 | a discussion of the two matching algorithms, and a list of features |
4092 | that pcre_dfa_exec() does not support, see the pcrematching documenta- |
4093 | tion. |
4094 | |
4095 | The arguments for the pcre_dfa_exec() function are the same as for |
4096 | pcre_exec(), plus two extras. The ovector argument is used in a differ- |
4097 | ent way, and this is described below. The other common arguments are |
4098 | used in the same way as for pcre_exec(), so their description is not |
4099 | repeated here. |
4100 | |
4101 | The two additional arguments provide workspace for the function. The |
4102 | workspace vector should contain at least 20 elements. It is used for |
4103 | keeping track of multiple paths through the pattern tree. More |
4104 | workspace will be needed for patterns and subjects where there are a |
4105 | lot of potential matches. |
4106 | |
4107 | Here is an example of a simple call to pcre_dfa_exec(): |
4108 | |
4109 | int rc; |
4110 | int ovector[10]; |
4111 | int wspace[20]; |
4112 | rc = pcre_dfa_exec( |
4113 | re, /* result of pcre_compile() */ |
4114 | NULL, /* we didn't study the pattern */ |
4115 | "some string", /* the subject string */ |
4116 | 11, /* the length of the subject string */ |
4117 | 0, /* start at offset 0 in the subject */ |
4118 | 0, /* default options */ |
4119 | ovector, /* vector of integers for substring information */ |
4120 | 10, /* number of elements (NOT size in bytes) */ |
4121 | wspace, /* working space vector */ |
4122 | 20); /* number of elements (NOT size in bytes) */ |
4123 | |
4124 | Option bits for pcre_dfa_exec() |
4125 | |
4126 | The unused bits of the options argument for pcre_dfa_exec() must be |
4127 | zero. The only bits that may be set are PCRE_ANCHORED, PCRE_NEW- |
4128 | LINE_xxx, PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_AT- |
4129 | START, PCRE_NO_UTF8_CHECK, PCRE_BSR_ANYCRLF, PCRE_BSR_UNICODE, |
4130 | PCRE_NO_START_OPTIMIZE, PCRE_PARTIAL_HARD, PCRE_PARTIAL_SOFT, |
4131 | PCRE_DFA_SHORTEST, and PCRE_DFA_RESTART. All but the last four of |
4132 | these are exactly the same as for pcre_exec(), so their description is |
4133 | not repeated here. |
4134 | |
4135 | PCRE_PARTIAL_HARD |
4136 | PCRE_PARTIAL_SOFT |
4137 | |
4138 | These have the same general effect as they do for pcre_exec(), but the |
4139 | details are slightly different. When PCRE_PARTIAL_HARD is set for |
4140 | pcre_dfa_exec(), it returns PCRE_ERROR_PARTIAL if the end of the sub- |
4141 | ject is reached and there is still at least one matching possibility |
4142 | that requires additional characters. This happens even if some complete |
4143 | matches have also been found. When PCRE_PARTIAL_SOFT is set, the return |
4144 | code PCRE_ERROR_NOMATCH is converted into PCRE_ERROR_PARTIAL if the end |
4145 | of the subject is reached, there have been no complete matches, but |
4146 | there is still at least one matching possibility. The portion of the |
4147 | string that was inspected when the longest partial match was found is |
4148 | set as the first matching string in both cases. There is a more de- |
4149 | tailed discussion of partial and multi-segment matching, with examples, |
4150 | in the pcrepartial documentation. |
4151 | |
4152 | PCRE_DFA_SHORTEST |
4153 | |
4154 | Setting the PCRE_DFA_SHORTEST option causes the matching algorithm to |
4155 | stop as soon as it has found one match. Because of the way the alterna- |
4156 | tive algorithm works, this is necessarily the shortest possible match |
4157 | at the first possible matching point in the subject string. |
4158 | |
4159 | PCRE_DFA_RESTART |
4160 | |
4161 | When pcre_dfa_exec() returns a partial match, it is possible to call it |
4162 | again, with additional subject characters, and have it continue with |
4163 | the same match. The PCRE_DFA_RESTART option requests this action; when |
4164 | it is set, the workspace and wscount options must reference the same |
4165 | vector as before because data about the match so far is left in them |
4166 | after a partial match. There is more discussion of this facility in the |
4167 | pcrepartial documentation. |
4168 | |
4169 | Successful returns from pcre_dfa_exec() |
4170 | |
4171 | When pcre_dfa_exec() succeeds, it may have matched more than one sub- |
4172 | string in the subject. Note, however, that all the matches from one run |
4173 | of the function start at the same point in the subject. The shorter |
4174 | matches are all initial substrings of the longer matches. For example, |
4175 | if the pattern |
4176 | |
4177 | <.*> |
4178 | |
4179 | is matched against the string |
4180 | |
4181 | This is <something> <something else> <something further> no more |
4182 | |
4183 | the three matched strings are |
4184 | |
4185 | <something> |
4186 | <something> <something else> |
4187 | <something> <something else> <something further> |
4188 | |
4189 | On success, the yield of the function is a number greater than zero, |
4190 | which is the number of matched substrings. The substrings themselves |
4191 | are returned in ovector. Each string uses two elements; the first is |
4192 | the offset to the start, and the second is the offset to the end. In |
4193 | fact, all the strings have the same start offset. (Space could have |
4194 | been saved by giving this only once, but it was decided to retain some |
4195 | compatibility with the way pcre_exec() returns data, even though the |
4196 | meaning of the strings is different.) |
4197 | |
4198 | The strings are returned in reverse order of length; that is, the long- |
4199 | est matching string is given first. If there were too many matches to |
4200 | fit into ovector, the yield of the function is zero, and the vector is |
4201 | filled with the longest matches. Unlike pcre_exec(), pcre_dfa_exec() |
4202 | can use the entire ovector for returning matched strings. |
4203 | |
4204 | NOTE: PCRE's "auto-possessification" optimization usually applies to |
4205 | character repeats at the end of a pattern (as well as internally). For |
4206 | example, the pattern "a\d+" is compiled as if it were "a\d++" because |
4207 | there is no point even considering the possibility of backtracking into |
4208 | the repeated digits. For DFA matching, this means that only one possi- |
4209 | ble match is found. If you really do want multiple matches in such |
4210 | cases, either use an ungreedy repeat ("a\d+?") or set the |
4211 | PCRE_NO_AUTO_POSSESS option when compiling. |
4212 | |
4213 | Error returns from pcre_dfa_exec() |
4214 | |
4215 | The pcre_dfa_exec() function returns a negative number when it fails. |
4216 | Many of the errors are the same as for pcre_exec(), and these are de- |
4217 | scribed above. There are in addition the following errors that are |
4218 | specific to pcre_dfa_exec(): |
4219 | |
4220 | PCRE_ERROR_DFA_UITEM (-16) |
4221 | |
4222 | This return is given if pcre_dfa_exec() encounters an item in the pat- |
4223 | tern that it does not support, for instance, the use of \C or a back |
4224 | reference. |
4225 | |
4226 | PCRE_ERROR_DFA_UCOND (-17) |
4227 | |
4228 | This return is given if pcre_dfa_exec() encounters a condition item |
4229 | that uses a back reference for the condition, or a test for recursion |
4230 | in a specific group. These are not supported. |
4231 | |
4232 | PCRE_ERROR_DFA_UMLIMIT (-18) |
4233 | |
4234 | This return is given if pcre_dfa_exec() is called with an extra block |
4235 | that contains a setting of the match_limit or match_limit_recursion |
4236 | fields. This is not supported (these fields are meaningless for DFA |
4237 | matching). |
4238 | |
4239 | PCRE_ERROR_DFA_WSSIZE (-19) |
4240 | |
4241 | This return is given if pcre_dfa_exec() runs out of space in the |
4242 | workspace vector. |
4243 | |
4244 | PCRE_ERROR_DFA_RECURSE (-20) |
4245 | |
4246 | When a recursive subpattern is processed, the matching function calls |
4247 | itself recursively, using private vectors for ovector and workspace. |
4248 | This error is given if the output vector is not large enough. This |
4249 | should be extremely rare, as a vector of size 1000 is used. |
4250 | |
4251 | PCRE_ERROR_DFA_BADRESTART (-30) |
4252 | |
4253 | When pcre_dfa_exec() is called with the PCRE_DFA_RESTART option, some |
4254 | plausibility checks are made on the contents of the workspace, which |
4255 | should contain data about the previous partial match. If any of these |
4256 | checks fail, this error is given. |
4257 | |
4258 | |
4259 | SEE ALSO |
4260 | |
4261 | pcre16(3), pcre32(3), pcrebuild(3), pcrecallout(3), pcrecpp(3)(3), |
4262 | pcrematching(3), pcrepartial(3), pcreposix(3), pcreprecompile(3), pcre- |
4263 | sample(3), pcrestack(3). |
4264 | |
4265 | |
4266 | AUTHOR |
4267 | |
4268 | Philip Hazel |
4269 | University Computing Service |
4270 | Cambridge CB2 3QH, England. |
4271 | |
4272 | |
4273 | REVISION |
4274 | |
4275 | Last updated: 18 December 2015 |
4276 | Copyright (c) 1997-2015 University of Cambridge. |
4277 | ------------------------------------------------------------------------------ |
4278 | |
4279 | |
4280 | PCRECALLOUT(3) Library Functions Manual PCRECALLOUT(3) |
4281 | |
4282 | |
4283 | |
4284 | NAME |
4285 | PCRE - Perl-compatible regular expressions |
4286 | |
4287 | SYNOPSIS |
4288 | |
4289 | #include <pcre.h> |
4290 | |
4291 | int (*pcre_callout)(pcre_callout_block *); |
4292 | |
4293 | int (*pcre16_callout)(pcre16_callout_block *); |
4294 | |
4295 | int (*pcre32_callout)(pcre32_callout_block *); |
4296 | |
4297 | |
4298 | DESCRIPTION |
4299 | |
4300 | PCRE provides a feature called "callout", which is a means of temporar- |
4301 | ily passing control to the caller of PCRE in the middle of pattern |
4302 | matching. The caller of PCRE provides an external function by putting |
4303 | its entry point in the global variable pcre_callout (pcre16_callout for |
4304 | the 16-bit library, pcre32_callout for the 32-bit library). By default, |
4305 | this variable contains NULL, which disables all calling out. |
4306 | |
4307 | Within a regular expression, (?C) indicates the points at which the ex- |
4308 | ternal function is to be called. Different callout points can be iden- |
4309 | tified by putting a number less than 256 after the letter C. The de- |
4310 | fault value is zero. For example, this pattern has two callout points: |
4311 | |
4312 | (?C1)abc(?C2)def |
4313 | |
4314 | If the PCRE_AUTO_CALLOUT option bit is set when a pattern is compiled, |
4315 | PCRE automatically inserts callouts, all with number 255, before each |
4316 | item in the pattern. For example, if PCRE_AUTO_CALLOUT is used with the |
4317 | pattern |
4318 | |
4319 | A(\d{2}|--) |
4320 | |
4321 | it is processed as if it were |
4322 | |
4323 | (?C255)A(?C255)((?C255)\d{2}(?C255)|(?C255)-(?C255)-(?C255))(?C255) |
4324 | |
4325 | Notice that there is a callout before and after each parenthesis and |
4326 | alternation bar. If the pattern contains a conditional group whose con- |
4327 | dition is an assertion, an automatic callout is inserted immediately |
4328 | before the condition. Such a callout may also be inserted explicitly, |
4329 | for example: |
4330 | |
4331 | (?(?C9)(?=a)ab|de) |
4332 | |
4333 | This applies only to assertion conditions (because they are themselves |
4334 | independent groups). |
4335 | |
4336 | Automatic callouts can be used for tracking the progress of pattern |
4337 | matching. The pcretest program has a pattern qualifier (/C) that sets |
4338 | automatic callouts; when it is used, the output indicates how the pat- |
4339 | tern is being matched. This is useful information when you are trying |
4340 | to optimize the performance of a particular pattern. |
4341 | |
4342 | |
4343 | MISSING CALLOUTS |
4344 | |
4345 | You should be aware that, because of optimizations in the way PCRE com- |
4346 | piles and matches patterns, callouts sometimes do not happen exactly as |
4347 | you might expect. |
4348 | |
4349 | At compile time, PCRE "auto-possessifies" repeated items when it knows |
4350 | that what follows cannot be part of the repeat. For example, a+[bc] is |
4351 | compiled as if it were a++[bc]. The pcretest output when this pattern |
4352 | is anchored and then applied with automatic callouts to the string |
4353 | "aaaa" is: |
4354 | |
4355 | --->aaaa |
4356 | +0 ^ ^ |
4357 | +1 ^ a+ |
4358 | +3 ^ ^ [bc] |
4359 | No match |
4360 | |
4361 | This indicates that when matching [bc] fails, there is no backtracking |
4362 | into a+ and therefore the callouts that would be taken for the back- |
4363 | tracks do not occur. You can disable the auto-possessify feature by |
4364 | passing PCRE_NO_AUTO_POSSESS to pcre_compile(), or starting the pattern |
4365 | with (*NO_AUTO_POSSESS). If this is done in pcretest (using the /O |
4366 | qualifier), the output changes to this: |
4367 | |
4368 | --->aaaa |
4369 | +0 ^ ^ |
4370 | +1 ^ a+ |
4371 | +3 ^ ^ [bc] |
4372 | +3 ^ ^ [bc] |
4373 | +3 ^ ^ [bc] |
4374 | +3 ^^ [bc] |
4375 | No match |
4376 | |
4377 | This time, when matching [bc] fails, the matcher backtracks into a+ and |
4378 | tries again, repeatedly, until a+ itself fails. |
4379 | |
4380 | Other optimizations that provide fast "no match" results also affect |
4381 | callouts. For example, if the pattern is |
4382 | |
4383 | ab(?C4)cd |
4384 | |
4385 | PCRE knows that any matching string must contain the letter "d". If the |
4386 | subject string is "abyz", the lack of "d" means that matching doesn't |
4387 | ever start, and the callout is never reached. However, with "abyd", |
4388 | though the result is still no match, the callout is obeyed. |
4389 | |
4390 | If the pattern is studied, PCRE knows the minimum length of a matching |
4391 | string, and will immediately give a "no match" return without actually |
4392 | running a match if the subject is not long enough, or, for unanchored |
4393 | patterns, if it has been scanned far enough. |
4394 | |
4395 | You can disable these optimizations by passing the PCRE_NO_START_OPTI- |
4396 | MIZE option to the matching function, or by starting the pattern with |
4397 | (*NO_START_OPT). This slows down the matching process, but does ensure |
4398 | that callouts such as the example above are obeyed. |
4399 | |
4400 | |
4401 | THE CALLOUT INTERFACE |
4402 | |
4403 | During matching, when PCRE reaches a callout point, the external func- |
4404 | tion defined by pcre_callout or pcre[16|32]_callout is called (if it is |
4405 | set). This applies to both normal and DFA matching. The only argument |
4406 | to the callout function is a pointer to a pcre_callout or |
4407 | pcre[16|32]_callout block. These structures contains the following |
4408 | fields: |
4409 | |
4410 | int version; |
4411 | int callout_number; |
4412 | int *offset_vector; |
4413 | const char *subject; (8-bit version) |
4414 | PCRE_SPTR16 subject; (16-bit version) |
4415 | PCRE_SPTR32 subject; (32-bit version) |
4416 | int subject_length; |
4417 | int start_match; |
4418 | int current_position; |
4419 | int capture_top; |
4420 | int capture_last; |
4421 | void *callout_data; |
4422 | int pattern_position; |
4423 | int next_item_length; |
4424 | const unsigned char *mark; (8-bit version) |
4425 | const PCRE_UCHAR16 *mark; (16-bit version) |
4426 | const PCRE_UCHAR32 *mark; (32-bit version) |
4427 | |
4428 | The version field is an integer containing the version number of the |
4429 | block format. The initial version was 0; the current version is 2. The |
4430 | version number will change again in future if additional fields are |
4431 | added, but the intention is never to remove any of the existing fields. |
4432 | |
4433 | The callout_number field contains the number of the callout, as com- |
4434 | piled into the pattern (that is, the number after ?C for manual call- |
4435 | outs, and 255 for automatically generated callouts). |
4436 | |
4437 | The offset_vector field is a pointer to the vector of offsets that was |
4438 | passed by the caller to the matching function. When pcre_exec() or |
4439 | pcre[16|32]_exec() is used, the contents can be inspected, in order to |
4440 | extract substrings that have been matched so far, in the same way as |
4441 | for extracting substrings after a match has completed. For the DFA |
4442 | matching functions, this field is not useful. |
4443 | |
4444 | The subject and subject_length fields contain copies of the values that |
4445 | were passed to the matching function. |
4446 | |
4447 | The start_match field normally contains the offset within the subject |
4448 | at which the current match attempt started. However, if the escape se- |
4449 | quence \K has been encountered, this value is changed to reflect the |
4450 | modified starting point. If the pattern is not anchored, the callout |
4451 | function may be called several times from the same point in the pattern |
4452 | for different starting points in the subject. |
4453 | |
4454 | The current_position field contains the offset within the subject of |
4455 | the current match pointer. |
4456 | |
4457 | When the pcre_exec() or pcre[16|32]_exec() is used, the capture_top |
4458 | field contains one more than the number of the highest numbered cap- |
4459 | tured substring so far. If no substrings have been captured, the value |
4460 | of capture_top is one. This is always the case when the DFA functions |
4461 | are used, because they do not support captured substrings. |
4462 | |
4463 | The capture_last field contains the number of the most recently cap- |
4464 | tured substring. However, when a recursion exits, the value reverts to |
4465 | what it was outside the recursion, as do the values of all captured |
4466 | substrings. If no substrings have been captured, the value of cap- |
4467 | ture_last is -1. This is always the case for the DFA matching func- |
4468 | tions. |
4469 | |
4470 | The callout_data field contains a value that is passed to a matching |
4471 | function specifically so that it can be passed back in callouts. It is |
4472 | passed in the callout_data field of a pcre_extra or pcre[16|32]_extra |
4473 | data structure. If no such data was passed, the value of callout_data |
4474 | in a callout block is NULL. There is a description of the pcre_extra |
4475 | structure in the pcreapi documentation. |
4476 | |
4477 | The pattern_position field is present from version 1 of the callout |
4478 | structure. It contains the offset to the next item to be matched in the |
4479 | pattern string. |
4480 | |
4481 | The next_item_length field is present from version 1 of the callout |
4482 | structure. It contains the length of the next item to be matched in the |
4483 | pattern string. When the callout immediately precedes an alternation |
4484 | bar, a closing parenthesis, or the end of the pattern, the length is |
4485 | zero. When the callout precedes an opening parenthesis, the length is |
4486 | that of the entire subpattern. |
4487 | |
4488 | The pattern_position and next_item_length fields are intended to help |
4489 | in distinguishing between different automatic callouts, which all have |
4490 | the same callout number. However, they are set for all callouts. |
4491 | |
4492 | The mark field is present from version 2 of the callout structure. In |
4493 | callouts from pcre_exec() or pcre[16|32]_exec() it contains a pointer |
4494 | to the zero-terminated name of the most recently passed (*MARK), |
4495 | (*PRUNE), or (*THEN) item in the match, or NULL if no such items have |
4496 | been passed. Instances of (*PRUNE) or (*THEN) without a name do not |
4497 | obliterate a previous (*MARK). In callouts from the DFA matching func- |
4498 | tions this field always contains NULL. |
4499 | |
4500 | |
4501 | RETURN VALUES |
4502 | |
4503 | The external callout function returns an integer to PCRE. If the value |
4504 | is zero, matching proceeds as normal. If the value is greater than |
4505 | zero, matching fails at the current point, but the testing of other |
4506 | matching possibilities goes ahead, just as if a lookahead assertion had |
4507 | failed. If the value is less than zero, the match is abandoned, the |
4508 | matching function returns the negative value. |
4509 | |
4510 | Negative values should normally be chosen from the set of PCRE_ER- |
4511 | ROR_xxx values. In particular, PCRE_ERROR_NOMATCH forces a standard "no |
4512 | match" failure. The error number PCRE_ERROR_CALLOUT is reserved for |
4513 | use by callout functions; it will never be used by PCRE itself. |
4514 | |
4515 | |
4516 | AUTHOR |
4517 | |
4518 | Philip Hazel |
4519 | University Computing Service |
4520 | Cambridge CB2 3QH, England. |
4521 | |
4522 | |
4523 | REVISION |
4524 | |
4525 | Last updated: 12 November 2013 |
4526 | Copyright (c) 1997-2013 University of Cambridge. |
4527 | ------------------------------------------------------------------------------ |
4528 | |
4529 | |
4530 | PCRECOMPAT(3) Library Functions Manual PCRECOMPAT(3) |
4531 | |
4532 | |
4533 | |
4534 | NAME |
4535 | PCRE - Perl-compatible regular expressions |
4536 | |
4537 | DIFFERENCES BETWEEN PCRE AND PERL |
4538 | |
4539 | This document describes the differences in the ways that PCRE and Perl |
4540 | handle regular expressions. The differences described here are with re- |
4541 | spect to Perl versions 5.10 and above. |
4542 | |
4543 | 1. PCRE has only a subset of Perl's Unicode support. Details of what it |
4544 | does have are given in the pcreunicode page. |
4545 | |
4546 | 2. PCRE allows repeat quantifiers only on parenthesized assertions, but |
4547 | they do not mean what you might think. For example, (?!a){3} does not |
4548 | assert that the next three characters are not "a". It just asserts that |
4549 | the next character is not "a" three times (in principle: PCRE optimizes |
4550 | this to run the assertion just once). Perl allows repeat quantifiers on |
4551 | other assertions such as \b, but these do not seem to have any use. |
4552 | |
4553 | 3. Capturing subpatterns that occur inside negative lookahead asser- |
4554 | tions are counted, but their entries in the offsets vector are never |
4555 | set. Perl sometimes (but not always) sets its numerical variables from |
4556 | inside negative assertions. |
4557 | |
4558 | 4. Though binary zero characters are supported in the subject string, |
4559 | they are not allowed in a pattern string because it is passed as a nor- |
4560 | mal C string, terminated by zero. The escape sequence \0 can be used in |
4561 | the pattern to represent a binary zero. |
4562 | |
4563 | 5. The following Perl escape sequences are not supported: \l, \u, \L, |
4564 | \U, and \N when followed by a character name or Unicode value. (\N on |
4565 | its own, matching a non-newline character, is supported.) In fact these |
4566 | are implemented by Perl's general string-handling and are not part of |
4567 | its pattern matching engine. If any of these are encountered by PCRE, |
4568 | an error is generated by default. However, if the PCRE_JAVASCRIPT_COM- |
4569 | PAT option is set, \U and \u are interpreted as JavaScript interprets |
4570 | them. |
4571 | |
4572 | 6. The Perl escape sequences \p, \P, and \X are supported only if PCRE |
4573 | is built with Unicode character property support. The properties that |
4574 | can be tested with \p and \P are limited to the general category prop- |
4575 | erties such as Lu and Nd, script names such as Greek or Han, and the |
4576 | derived properties Any and L&. PCRE does support the Cs (surrogate) |
4577 | property, which Perl does not; the Perl documentation says "Because |
4578 | Perl hides the need for the user to understand the internal representa- |
4579 | tion of Unicode characters, there is no need to implement the somewhat |
4580 | messy concept of surrogates." |
4581 | |
4582 | 7. PCRE does support the \Q...\E escape for quoting substrings. Charac- |
4583 | ters in between are treated as literals. This is slightly different |
4584 | from Perl in that $ and @ are also handled as literals inside the |
4585 | quotes. In Perl, they cause variable interpolation (but of course PCRE |
4586 | does not have variables). Note the following examples: |
4587 | |
4588 | Pattern PCRE matches Perl matches |
4589 | |
4590 | \Qabc$xyz\E abc$xyz abc followed by the |
4591 | contents of $xyz |
4592 | \Qabc\$xyz\E abc\$xyz abc\$xyz |
4593 | \Qabc\E\$\Qxyz\E abc$xyz abc$xyz |
4594 | |
4595 | The \Q...\E sequence is recognized both inside and outside character |
4596 | classes. |
4597 | |
4598 | 8. Fairly obviously, PCRE does not support the (?{code}) and (??{code}) |
4599 | constructions. However, there is support for recursive patterns. This |
4600 | is not available in Perl 5.8, but it is in Perl 5.10. Also, the PCRE |
4601 | "callout" feature allows an external function to be called during pat- |
4602 | tern matching. See the pcrecallout documentation for details. |
4603 | |
4604 | 9. Subpatterns that are called as subroutines (whether or not recur- |
4605 | sively) are always treated as atomic groups in PCRE. This is like |
4606 | Python, but unlike Perl. Captured values that are set outside a sub- |
4607 | routine call can be reference from inside in PCRE, but not in Perl. |
4608 | There is a discussion that explains these differences in more detail in |
4609 | the section on recursion differences from Perl in the pcrepattern page. |
4610 | |
4611 | 10. If any of the backtracking control verbs are used in a subpattern |
4612 | that is called as a subroutine (whether or not recursively), their ef- |
4613 | fect is confined to that subpattern; it does not extend to the sur- |
4614 | rounding pattern. This is not always the case in Perl. In particular, |
4615 | if (*THEN) is present in a group that is called as a subroutine, its |
4616 | action is limited to that group, even if the group does not contain any |
4617 | | characters. Note that such subpatterns are processed as anchored at |
4618 | the point where they are tested. |
4619 | |
4620 | 11. If a pattern contains more than one backtracking control verb, the |
4621 | first one that is backtracked onto acts. For example, in the pattern |
4622 | A(*COMMIT)B(*PRUNE)C a failure in B triggers (*COMMIT), but a failure |
4623 | in C triggers (*PRUNE). Perl's behaviour is more complex; in many cases |
4624 | it is the same as PCRE, but there are examples where it differs. |
4625 | |
4626 | 12. Most backtracking verbs in assertions have their normal actions. |
4627 | They are not confined to the assertion. |
4628 | |
4629 | 13. There are some differences that are concerned with the settings of |
4630 | captured strings when part of a pattern is repeated. For example, |
4631 | matching "aba" against the pattern /^(a(b)?)+$/ in Perl leaves $2 un- |
4632 | set, but in PCRE it is set to "b". |
4633 | |
4634 | 14. PCRE's handling of duplicate subpattern numbers and duplicate sub- |
4635 | pattern names is not as general as Perl's. This is a consequence of the |
4636 | fact the PCRE works internally just with numbers, using an external ta- |
4637 | ble to translate between numbers and names. In particular, a pattern |
4638 | such as (?|(?<a>A)|(?<b>B), where the two capturing parentheses have |
4639 | the same number but different names, is not supported, and causes an |
4640 | error at compile time. If it were allowed, it would not be possible to |
4641 | distinguish which parentheses matched, because both names map to cap- |
4642 | turing subpattern number 1. To avoid this confusing situation, an error |
4643 | is given at compile time. |
4644 | |
4645 | 15. Perl recognizes comments in some places that PCRE does not, for ex- |
4646 | ample, between the ( and ? at the start of a subpattern. If the /x mod- |
4647 | ifier is set, Perl allows white space between ( and ? (though current |
4648 | Perls warn that this is deprecated) but PCRE never does, even if the |
4649 | PCRE_EXTENDED option is set. |
4650 | |
4651 | 16. Perl, when in warning mode, gives warnings for character classes |
4652 | such as [A-\d] or [a-[:digit:]]. It then treats the hyphens as liter- |
4653 | als. PCRE has no warning features, so it gives an error in these cases |
4654 | because they are almost certainly user mistakes. |
4655 | |
4656 | 17. In PCRE, the upper/lower case character properties Lu and Ll are |
4657 | not affected when case-independent matching is specified. For example, |
4658 | \p{Lu} always matches an upper case letter. I think Perl has changed in |
4659 | this respect; in the release at the time of writing (5.16), \p{Lu} and |
4660 | \p{Ll} match all letters, regardless of case, when case independence is |
4661 | specified. |
4662 | |
4663 | 18. PCRE provides some extensions to the Perl regular expression facil- |
4664 | ities. Perl 5.10 includes new features that are not in earlier ver- |
4665 | sions of Perl, some of which (such as named parentheses) have been in |
4666 | PCRE for some time. This list is with respect to Perl 5.10: |
4667 | |
4668 | (a) Although lookbehind assertions in PCRE must match fixed length |
4669 | strings, each alternative branch of a lookbehind assertion can match a |
4670 | different length of string. Perl requires them all to have the same |
4671 | length. |
4672 | |
4673 | (b) If PCRE_DOLLAR_ENDONLY is set and PCRE_MULTILINE is not set, the $ |
4674 | meta-character matches only at the very end of the string. |
4675 | |
4676 | (c) If PCRE_EXTRA is set, a backslash followed by a letter with no spe- |
4677 | cial meaning is faulted. Otherwise, like Perl, the backslash is quietly |
4678 | ignored. (Perl can be made to issue a warning.) |
4679 | |
4680 | (d) If PCRE_UNGREEDY is set, the greediness of the repetition quanti- |
4681 | fiers is inverted, that is, by default they are not greedy, but if fol- |
4682 | lowed by a question mark they are. |
4683 | |
4684 | (e) PCRE_ANCHORED can be used at matching time to force a pattern to be |
4685 | tried only at the first matching position in the subject string. |
4686 | |
4687 | (f) The PCRE_NOTBOL, PCRE_NOTEOL, PCRE_NOTEMPTY, PCRE_NOTEMPTY_ATSTART, |
4688 | and PCRE_NO_AUTO_CAPTURE options for pcre_exec() have no Perl equiva- |
4689 | lents. |
4690 | |
4691 | (g) The \R escape sequence can be restricted to match only CR, LF, or |
4692 | CRLF by the PCRE_BSR_ANYCRLF option. |
4693 | |
4694 | (h) The callout facility is PCRE-specific. |
4695 | |
4696 | (i) The partial matching facility is PCRE-specific. |
4697 | |
4698 | (j) Patterns compiled by PCRE can be saved and re-used at a later time, |
4699 | even on different hosts that have the other endianness. However, this |
4700 | does not apply to optimized data created by the just-in-time compiler. |
4701 | |
4702 | (k) The alternative matching functions (pcre_dfa_exec(), |
4703 | pcre16_dfa_exec() and pcre32_dfa_exec(),) match in a different way and |
4704 | are not Perl-compatible. |
4705 | |
4706 | (l) PCRE recognizes some special sequences such as (*CR) at the start |
4707 | of a pattern that set overall options that cannot be changed within the |
4708 | pattern. |
4709 | |
4710 | |
4711 | AUTHOR |
4712 | |
4713 | Philip Hazel |
4714 | University Computing Service |
4715 | Cambridge CB2 3QH, England. |
4716 | |
4717 | |
4718 | REVISION |
4719 | |
4720 | Last updated: 10 November 2013 |
4721 | Copyright (c) 1997-2013 University of Cambridge. |
4722 | ------------------------------------------------------------------------------ |
4723 | |
4724 | |
4725 | PCREPATTERN(3) Library Functions Manual PCREPATTERN(3) |
4726 | |
4727 | |
4728 | |
4729 | NAME |
4730 | PCRE - Perl-compatible regular expressions |
4731 | |
4732 | PCRE REGULAR EXPRESSION DETAILS |
4733 | |
4734 | The syntax and semantics of the regular expressions that are supported |
4735 | by PCRE are described in detail below. There is a quick-reference syn- |
4736 | tax summary in the pcresyntax page. PCRE tries to match Perl syntax and |
4737 | semantics as closely as it can. PCRE also supports some alternative |
4738 | regular expression syntax (which does not conflict with the Perl syn- |
4739 | tax) in order to provide some compatibility with regular expressions in |
4740 | Python, .NET, and Oniguruma. |
4741 | |
4742 | Perl's regular expressions are described in its own documentation, and |
4743 | regular expressions in general are covered in a number of books, some |
4744 | of which have copious examples. Jeffrey Friedl's "Mastering Regular Ex- |
4745 | pressions", published by O'Reilly, covers regular expressions in great |
4746 | detail. This description of PCRE's regular expressions is intended as |
4747 | reference material. |
4748 | |
4749 | This document discusses the patterns that are supported by PCRE when |
4750 | one its main matching functions, pcre_exec() (8-bit) or |
4751 | pcre[16|32]_exec() (16- or 32-bit), is used. PCRE also has alternative |
4752 | matching functions, pcre_dfa_exec() and pcre[16|32_dfa_exec(), which |
4753 | match using a different algorithm that is not Perl-compatible. Some of |
4754 | the features discussed below are not available when DFA matching is |
4755 | used. The advantages and disadvantages of the alternative functions, |
4756 | and how they differ from the normal functions, are discussed in the |
4757 | pcrematching page. |
4758 | |
4759 | |
4760 | SPECIAL START-OF-PATTERN ITEMS |
4761 | |
4762 | A number of options that can be passed to pcre_compile() can also be |
4763 | set by special items at the start of a pattern. These are not Perl-com- |
4764 | patible, but are provided to make these options accessible to pattern |
4765 | writers who are not able to change the program that processes the pat- |
4766 | tern. Any number of these items may appear, but they must all be to- |
4767 | gether right at the start of the pattern string, and the letters must |
4768 | be in upper case. |
4769 | |
4770 | UTF support |
4771 | |
4772 | The original operation of PCRE was on strings of one-byte characters. |
4773 | However, there is now also support for UTF-8 strings in the original |
4774 | library, an extra library that supports 16-bit and UTF-16 character |
4775 | strings, and a third library that supports 32-bit and UTF-32 character |
4776 | strings. To use these features, PCRE must be built to include appropri- |
4777 | ate support. When using UTF strings you must either call the compiling |
4778 | function with the PCRE_UTF8, PCRE_UTF16, or PCRE_UTF32 option, or the |
4779 | pattern must start with one of these special sequences: |
4780 | |
4781 | (*UTF8) |
4782 | (*UTF16) |
4783 | (*UTF32) |
4784 | (*UTF) |
4785 | |
4786 | (*UTF) is a generic sequence that can be used with any of the li- |
4787 | braries. Starting a pattern with such a sequence is equivalent to set- |
4788 | ting the relevant option. How setting a UTF mode affects pattern match- |
4789 | ing is mentioned in several places below. There is also a summary of |
4790 | features in the pcreunicode page. |
4791 | |
4792 | Some applications that allow their users to supply patterns may wish to |
4793 | restrict them to non-UTF data for security reasons. If the |
4794 | PCRE_NEVER_UTF option is set at compile time, (*UTF) etc. are not al- |
4795 | lowed, and their appearance causes an error. |
4796 | |
4797 | Unicode property support |
4798 | |
4799 | Another special sequence that may appear at the start of a pattern is |
4800 | (*UCP). This has the same effect as setting the PCRE_UCP option: it |
4801 | causes sequences such as \d and \w to use Unicode properties to deter- |
4802 | mine character types, instead of recognizing only characters with codes |
4803 | less than 128 via a lookup table. |
4804 | |
4805 | Disabling auto-possessification |
4806 | |
4807 | If a pattern starts with (*NO_AUTO_POSSESS), it has the same effect as |
4808 | setting the PCRE_NO_AUTO_POSSESS option at compile time. This stops |
4809 | PCRE from making quantifiers possessive when what follows cannot match |
4810 | the repeated item. For example, by default a+b is treated as a++b. For |
4811 | more details, see the pcreapi documentation. |
4812 | |
4813 | Disabling start-up optimizations |
4814 | |
4815 | If a pattern starts with (*NO_START_OPT), it has the same effect as |
4816 | setting the PCRE_NO_START_OPTIMIZE option either at compile or matching |
4817 | time. This disables several optimizations for quickly reaching "no |
4818 | match" results. For more details, see the pcreapi documentation. |
4819 | |
4820 | Newline conventions |
4821 | |
4822 | PCRE supports five different conventions for indicating line breaks in |
4823 | strings: a single CR (carriage return) character, a single LF (line- |
4824 | feed) character, the two-character sequence CRLF, any of the three pre- |
4825 | ceding, or any Unicode newline sequence. The pcreapi page has further |
4826 | discussion about newlines, and shows how to set the newline convention |
4827 | in the options arguments for the compiling and matching functions. |
4828 | |
4829 | It is also possible to specify a newline convention by starting a pat- |
4830 | tern string with one of the following five sequences: |
4831 | |
4832 | (*CR) carriage return |
4833 | (*LF) linefeed |
4834 | (*CRLF) carriage return, followed by linefeed |
4835 | (*ANYCRLF) any of the three above |
4836 | (*ANY) all Unicode newline sequences |
4837 | |
4838 | These override the default and the options given to the compiling func- |
4839 | tion. For example, on a Unix system where LF is the default newline se- |
4840 | quence, the pattern |
4841 | |
4842 | (*CR)a.b |
4843 | |
4844 | changes the convention to CR. That pattern matches "a\nb" because LF is |
4845 | no longer a newline. If more than one of these settings is present, the |
4846 | last one is used. |
4847 | |
4848 | The newline convention affects where the circumflex and dollar asser- |
4849 | tions are true. It also affects the interpretation of the dot metachar- |
4850 | acter when PCRE_DOTALL is not set, and the behaviour of \N. However, it |
4851 | does not affect what the \R escape sequence matches. By default, this |
4852 | is any Unicode newline sequence, for Perl compatibility. However, this |
4853 | can be changed; see the description of \R in the section entitled "New- |
4854 | line sequences" below. A change of \R setting can be combined with a |
4855 | change of newline convention. |
4856 | |
4857 | Setting match and recursion limits |
4858 | |
4859 | The caller of pcre_exec() can set a limit on the number of times the |
4860 | internal match() function is called and on the maximum depth of recur- |
4861 | sive calls. These facilities are provided to catch runaway matches that |
4862 | are provoked by patterns with huge matching trees (a typical example is |
4863 | a pattern with nested unlimited repeats) and to avoid running out of |
4864 | system stack by too much recursion. When one of these limits is |
4865 | reached, pcre_exec() gives an error return. The limits can also be set |
4866 | by items at the start of the pattern of the form |
4867 | |
4868 | (*LIMIT_MATCH=d) |
4869 | (*LIMIT_RECURSION=d) |
4870 | |
4871 | where d is any number of decimal digits. However, the value of the set- |
4872 | ting must be less than the value set (or defaulted) by the caller of |
4873 | pcre_exec() for it to have any effect. In other words, the pattern |
4874 | writer can lower the limits set by the programmer, but not raise them. |
4875 | If there is more than one setting of one of these limits, the lower |
4876 | value is used. |
4877 | |
4878 | |
4879 | EBCDIC CHARACTER CODES |
4880 | |
4881 | PCRE can be compiled to run in an environment that uses EBCDIC as its |
4882 | character code rather than ASCII or Unicode (typically a mainframe sys- |
4883 | tem). In the sections below, character code values are ASCII or Uni- |
4884 | code; in an EBCDIC environment these characters may have different code |
4885 | values, and there are no code points greater than 255. |
4886 | |
4887 | |
4888 | CHARACTERS AND METACHARACTERS |
4889 | |
4890 | A regular expression is a pattern that is matched against a subject |
4891 | string from left to right. Most characters stand for themselves in a |
4892 | pattern, and match the corresponding characters in the subject. As a |
4893 | trivial example, the pattern |
4894 | |
4895 | The quick brown fox |
4896 | |
4897 | matches a portion of a subject string that is identical to itself. When |
4898 | caseless matching is specified (the PCRE_CASELESS option), letters are |
4899 | matched independently of case. In a UTF mode, PCRE always understands |
4900 | the concept of case for characters whose values are less than 128, so |
4901 | caseless matching is always possible. For characters with higher val- |
4902 | ues, the concept of case is supported if PCRE is compiled with Unicode |
4903 | property support, but not otherwise. If you want to use caseless |
4904 | matching for characters 128 and above, you must ensure that PCRE is |
4905 | compiled with Unicode property support as well as with UTF support. |
4906 | |
4907 | The power of regular expressions comes from the ability to include al- |
4908 | ternatives and repetitions in the pattern. These are encoded in the |
4909 | pattern by the use of metacharacters, which do not stand for themselves |
4910 | but instead are interpreted in some special way. |
4911 | |
4912 | There are two different sets of metacharacters: those that are recog- |
4913 | nized anywhere in the pattern except within square brackets, and those |
4914 | that are recognized within square brackets. Outside square brackets, |
4915 | the metacharacters are as follows: |
4916 | |
4917 | \ general escape character with several uses |
4918 | ^ assert start of string (or line, in multiline mode) |
4919 | $ assert end of string (or line, in multiline mode) |
4920 | . match any character except newline (by default) |
4921 | [ start character class definition |
4922 | | start of alternative branch |
4923 | ( start subpattern |
4924 | ) end subpattern |
4925 | ? extends the meaning of ( |
4926 | also 0 or 1 quantifier |
4927 | also quantifier minimizer |
4928 | * 0 or more quantifier |
4929 | + 1 or more quantifier |
4930 | also "possessive quantifier" |
4931 | { start min/max quantifier |
4932 | |
4933 | Part of a pattern that is in square brackets is called a "character |
4934 | class". In a character class the only metacharacters are: |
4935 | |
4936 | \ general escape character |
4937 | ^ negate the class, but only if the first character |
4938 | - indicates character range |
4939 | [ POSIX character class (only if followed by POSIX |
4940 | syntax) |
4941 | ] terminates the character class |
4942 | |
4943 | The following sections describe the use of each of the metacharacters. |
4944 | |
4945 | |
4946 | BACKSLASH |
4947 | |
4948 | The backslash character has several uses. Firstly, if it is followed by |
4949 | a character that is not a number or a letter, it takes away any special |
4950 | meaning that character may have. This use of backslash as an escape |
4951 | character applies both inside and outside character classes. |
4952 | |
4953 | For example, if you want to match a * character, you write \* in the |
4954 | pattern. This escaping action applies whether or not the following |
4955 | character would otherwise be interpreted as a metacharacter, so it is |
4956 | always safe to precede a non-alphanumeric with backslash to specify |
4957 | that it stands for itself. In particular, if you want to match a back- |
4958 | slash, you write \\. |
4959 | |
4960 | In a UTF mode, only ASCII numbers and letters have any special meaning |
4961 | after a backslash. All other characters (in particular, those whose |
4962 | codepoints are greater than 127) are treated as literals. |
4963 | |
4964 | If a pattern is compiled with the PCRE_EXTENDED option, most white |
4965 | space in the pattern (other than in a character class), and characters |
4966 | between a # outside a character class and the next newline, inclusive, |
4967 | are ignored. An escaping backslash can be used to include a white space |
4968 | or # character as part of the pattern. |
4969 | |
4970 | If you want to remove the special meaning from a sequence of charac- |
4971 | ters, you can do so by putting them between \Q and \E. This is differ- |
4972 | ent from Perl in that $ and @ are handled as literals in \Q...\E se- |
4973 | quences in PCRE, whereas in Perl, $ and @ cause variable interpolation. |
4974 | Note the following examples: |
4975 | |
4976 | Pattern PCRE matches Perl matches |
4977 | |
4978 | \Qabc$xyz\E abc$xyz abc followed by the |
4979 | contents of $xyz |
4980 | \Qabc\$xyz\E abc\$xyz abc\$xyz |
4981 | \Qabc\E\$\Qxyz\E abc$xyz abc$xyz |
4982 | |
4983 | The \Q...\E sequence is recognized both inside and outside character |
4984 | classes. An isolated \E that is not preceded by \Q is ignored. If \Q |
4985 | is not followed by \E later in the pattern, the literal interpretation |
4986 | continues to the end of the pattern (that is, \E is assumed at the |
4987 | end). If the isolated \Q is inside a character class, this causes an |
4988 | error, because the character class is not terminated. |
4989 | |
4990 | Non-printing characters |
4991 | |
4992 | A second use of backslash provides a way of encoding non-printing char- |
4993 | acters in patterns in a visible manner. There is no restriction on the |
4994 | appearance of non-printing characters, apart from the binary zero that |
4995 | terminates a pattern, but when a pattern is being prepared by text |
4996 | editing, it is often easier to use one of the following escape se- |
4997 | quences than the binary character it represents. In an ASCII or Uni- |
4998 | code environment, these escapes are as follows: |
4999 | |
5000 | \a alarm, that is, the BEL character (hex 07) |
5001 | \cx "control-x", where x is any ASCII character |
5002 | \e escape (hex 1B) |
5003 | \f form feed (hex 0C) |
5004 | \n linefeed (hex 0A) |
5005 | \r carriage return (hex 0D) |
5006 | \t tab (hex 09) |
5007 | \0dd character with octal code 0dd |
5008 | \ddd character with octal code ddd, or back reference |
5009 | \o{ddd..} character with octal code ddd.. |
5010 | \xhh character with hex code hh |
5011 | \x{hhh..} character with hex code hhh.. (non-JavaScript mode) |
5012 | \uhhhh character with hex code hhhh (JavaScript mode only) |
5013 | |
5014 | The precise effect of \cx on ASCII characters is as follows: if x is a |
5015 | lower case letter, it is converted to upper case. Then bit 6 of the |
5016 | character (hex 40) is inverted. Thus \cA to \cZ become hex 01 to hex 1A |
5017 | (A is 41, Z is 5A), but \c{ becomes hex 3B ({ is 7B), and \c; becomes |
5018 | hex 7B (; is 3B). If the data item (byte or 16-bit value) following \c |
5019 | has a value greater than 127, a compile-time error occurs. This locks |
5020 | out non-ASCII characters in all modes. |
5021 | |
5022 | When PCRE is compiled in EBCDIC mode, \a, \e, \f, \n, \r, and \t gener- |
5023 | ate the appropriate EBCDIC code values. The \c escape is processed as |
5024 | specified for Perl in the perlebcdic document. The only characters that |
5025 | are allowed after \c are A-Z, a-z, or one of @, [, \, ], ^, _, or ?. |
5026 | Any other character provokes a compile-time error. The sequence \c@ en- |
5027 | codes character code 0; after \c the letters (in either case) encode |
5028 | characters 1-26 (hex 01 to hex 1A); [, \, ], ^, and _ encode characters |
5029 | 27-31 (hex 1B to hex 1F), and \c? becomes either 255 (hex FF) or 95 |
5030 | (hex 5F). |
5031 | |
5032 | Thus, apart from \c?, these escapes generate the same character code |
5033 | values as they do in an ASCII environment, though the meanings of the |
5034 | values mostly differ. For example, \cG always generates code value 7, |
5035 | which is BEL in ASCII but DEL in EBCDIC. |
5036 | |
5037 | The sequence \c? generates DEL (127, hex 7F) in an ASCII environment, |
5038 | but because 127 is not a control character in EBCDIC, Perl makes it |
5039 | generate the APC character. Unfortunately, there are several variants |
5040 | of EBCDIC. In most of them the APC character has the value 255 (hex |
5041 | FF), but in the one Perl calls POSIX-BC its value is 95 (hex 5F). If |
5042 | certain other characters have POSIX-BC values, PCRE makes \c? generate |
5043 | 95; otherwise it generates 255. |
5044 | |
5045 | After \0 up to two further octal digits are read. If there are fewer |
5046 | than two digits, just those that are present are used. Thus the se- |
5047 | quence \0\x\015 specifies two binary zeros followed by a CR character |
5048 | (code value 13). Make sure you supply two digits after the initial zero |
5049 | if the pattern character that follows is itself an octal digit. |
5050 | |
5051 | The escape \o must be followed by a sequence of octal digits, enclosed |
5052 | in braces. An error occurs if this is not the case. This escape is a |
5053 | recent addition to Perl; it provides way of specifying character code |
5054 | points as octal numbers greater than 0777, and it also allows octal |
5055 | numbers and back references to be unambiguously specified. |
5056 | |
5057 | For greater clarity and unambiguity, it is best to avoid following \ by |
5058 | a digit greater than zero. Instead, use \o{} or \x{} to specify charac- |
5059 | ter numbers, and \g{} to specify back references. The following para- |
5060 | graphs describe the old, ambiguous syntax. |
5061 | |
5062 | The handling of a backslash followed by a digit other than 0 is compli- |
5063 | cated, and Perl has changed in recent releases, causing PCRE also to |
5064 | change. Outside a character class, PCRE reads the digit and any follow- |
5065 | ing digits as a decimal number. If the number is less than 8, or if |
5066 | there have been at least that many previous capturing left parentheses |
5067 | in the expression, the entire sequence is taken as a back reference. A |
5068 | description of how this works is given later, following the discussion |
5069 | of parenthesized subpatterns. |
5070 | |
5071 | Inside a character class, or if the decimal number following \ is |
5072 | greater than 7 and there have not been that many capturing subpatterns, |
5073 | PCRE handles \8 and \9 as the literal characters "8" and "9", and oth- |
5074 | erwise re-reads up to three octal digits following the backslash, using |
5075 | them to generate a data character. Any subsequent digits stand for |
5076 | themselves. For example: |
5077 | |
5078 | \040 is another way of writing an ASCII space |
5079 | \40 is the same, provided there are fewer than 40 |
5080 | previous capturing subpatterns |
5081 | \7 is always a back reference |
5082 | \11 might be a back reference, or another way of |
5083 | writing a tab |
5084 | \011 is always a tab |
5085 | \0113 is a tab followed by the character "3" |
5086 | \113 might be a back reference, otherwise the |
5087 | character with octal code 113 |
5088 | \377 might be a back reference, otherwise |
5089 | the value 255 (decimal) |
5090 | \81 is either a back reference, or the two |
5091 | characters "8" and "1" |
5092 | |
5093 | Note that octal values of 100 or greater that are specified using this |
5094 | syntax must not be introduced by a leading zero, because no more than |
5095 | three octal digits are ever read. |
5096 | |
5097 | By default, after \x that is not followed by {, from zero to two hexa- |
5098 | decimal digits are read (letters can be in upper or lower case). Any |
5099 | number of hexadecimal digits may appear between \x{ and }. If a charac- |
5100 | ter other than a hexadecimal digit appears between \x{ and }, or if |
5101 | there is no terminating }, an error occurs. |
5102 | |
5103 | If the PCRE_JAVASCRIPT_COMPAT option is set, the interpretation of \x |
5104 | is as just described only when it is followed by two hexadecimal dig- |
5105 | its. Otherwise, it matches a literal "x" character. In JavaScript |
5106 | mode, support for code points greater than 256 is provided by \u, which |
5107 | must be followed by four hexadecimal digits; otherwise it matches a |
5108 | literal "u" character. |
5109 | |
5110 | Characters whose value is less than 256 can be defined by either of the |
5111 | two syntaxes for \x (or by \u in JavaScript mode). There is no differ- |
5112 | ence in the way they are handled. For example, \xdc is exactly the same |
5113 | as \x{dc} (or \u00dc in JavaScript mode). |
5114 | |
5115 | Constraints on character values |
5116 | |
5117 | Characters that are specified using octal or hexadecimal numbers are |
5118 | limited to certain values, as follows: |
5119 | |
5120 | 8-bit non-UTF mode less than 0x100 |
5121 | 8-bit UTF-8 mode less than 0x10ffff and a valid codepoint |
5122 | 16-bit non-UTF mode less than 0x10000 |
5123 | 16-bit UTF-16 mode less than 0x10ffff and a valid codepoint |
5124 | 32-bit non-UTF mode less than 0x100000000 |
5125 | 32-bit UTF-32 mode less than 0x10ffff and a valid codepoint |
5126 | |
5127 | Invalid Unicode codepoints are the range 0xd800 to 0xdfff (the so- |
5128 | called "surrogate" codepoints), and 0xffef. |
5129 | |
5130 | Escape sequences in character classes |
5131 | |
5132 | All the sequences that define a single character value can be used both |
5133 | inside and outside character classes. In addition, inside a character |
5134 | class, \b is interpreted as the backspace character (hex 08). |
5135 | |
5136 | \N is not allowed in a character class. \B, \R, and \X are not special |
5137 | inside a character class. Like other unrecognized escape sequences, |
5138 | they are treated as the literal characters "B", "R", and "X" by de- |
5139 | fault, but cause an error if the PCRE_EXTRA option is set. Outside a |
5140 | character class, these sequences have different meanings. |
5141 | |
5142 | Unsupported escape sequences |
5143 | |
5144 | In Perl, the sequences \l, \L, \u, and \U are recognized by its string |
5145 | handler and used to modify the case of following characters. By de- |
5146 | fault, PCRE does not support these escape sequences. However, if the |
5147 | PCRE_JAVASCRIPT_COMPAT option is set, \U matches a "U" character, and |
5148 | \u can be used to define a character by code point, as described in the |
5149 | previous section. |
5150 | |
5151 | Absolute and relative back references |
5152 | |
5153 | The sequence \g followed by an unsigned or a negative number, option- |
5154 | ally enclosed in braces, is an absolute or relative back reference. A |
5155 | named back reference can be coded as \g{name}. Back references are dis- |
5156 | cussed later, following the discussion of parenthesized subpatterns. |
5157 | |
5158 | Absolute and relative subroutine calls |
5159 | |
5160 | For compatibility with Oniguruma, the non-Perl syntax \g followed by a |
5161 | name or a number enclosed either in angle brackets or single quotes, is |
5162 | an alternative syntax for referencing a subpattern as a "subroutine". |
5163 | Details are discussed later. Note that \g{...} (Perl syntax) and |
5164 | \g<...> (Oniguruma syntax) are not synonymous. The former is a back |
5165 | reference; the latter is a subroutine call. |
5166 | |
5167 | Generic character types |
5168 | |
5169 | Another use of backslash is for specifying generic character types: |
5170 | |
5171 | \d any decimal digit |
5172 | \D any character that is not a decimal digit |
5173 | \h any horizontal white space character |
5174 | \H any character that is not a horizontal white space character |
5175 | \s any white space character |
5176 | \S any character that is not a white space character |
5177 | \v any vertical white space character |
5178 | \V any character that is not a vertical white space character |
5179 | \w any "word" character |
5180 | \W any "non-word" character |
5181 | |
5182 | There is also the single sequence \N, which matches a non-newline char- |
5183 | acter. This is the same as the "." metacharacter when PCRE_DOTALL is |
5184 | not set. Perl also uses \N to match characters by name; PCRE does not |
5185 | support this. |
5186 | |
5187 | Each pair of lower and upper case escape sequences partitions the com- |
5188 | plete set of characters into two disjoint sets. Any given character |
5189 | matches one, and only one, of each pair. The sequences can appear both |
5190 | inside and outside character classes. They each match one character of |
5191 | the appropriate type. If the current matching point is at the end of |
5192 | the subject string, all of them fail, because there is no character to |
5193 | match. |
5194 | |
5195 | For compatibility with Perl, \s did not used to match the VT character |
5196 | (code 11), which made it different from the the POSIX "space" class. |
5197 | However, Perl added VT at release 5.18, and PCRE followed suit at re- |
5198 | lease 8.34. The default \s characters are now HT (9), LF (10), VT (11), |
5199 | FF (12), CR (13), and space (32), which are defined as white space in |
5200 | the "C" locale. This list may vary if locale-specific matching is tak- |
5201 | ing place. For example, in some locales the "non-breaking space" char- |
5202 | acter (\xA0) is recognized as white space, and in others the VT charac- |
5203 | ter is not. |
5204 | |
5205 | A "word" character is an underscore or any character that is a letter |
5206 | or digit. By default, the definition of letters and digits is con- |
5207 | trolled by PCRE's low-valued character tables, and may vary if locale- |
5208 | specific matching is taking place (see "Locale support" in the pcreapi |
5209 | page). For example, in a French locale such as "fr_FR" in Unix-like |
5210 | systems, or "french" in Windows, some character codes greater than 127 |
5211 | are used for accented letters, and these are then matched by \w. The |
5212 | use of locales with Unicode is discouraged. |
5213 | |
5214 | By default, characters whose code points are greater than 127 never |
5215 | match \d, \s, or \w, and always match \D, \S, and \W, although this may |
5216 | vary for characters in the range 128-255 when locale-specific matching |
5217 | is happening. These escape sequences retain their original meanings |
5218 | from before Unicode support was available, mainly for efficiency rea- |
5219 | sons. If PCRE is compiled with Unicode property support, and the |
5220 | PCRE_UCP option is set, the behaviour is changed so that Unicode prop- |
5221 | erties are used to determine character types, as follows: |
5222 | |
5223 | \d any character that matches \p{Nd} (decimal digit) |
5224 | \s any character that matches \p{Z} or \h or \v |
5225 | \w any character that matches \p{L} or \p{N}, plus underscore |
5226 | |
5227 | The upper case escapes match the inverse sets of characters. Note that |
5228 | \d matches only decimal digits, whereas \w matches any Unicode digit, |
5229 | as well as any Unicode letter, and underscore. Note also that PCRE_UCP |
5230 | affects \b, and \B because they are defined in terms of \w and \W. |
5231 | Matching these sequences is noticeably slower when PCRE_UCP is set. |
5232 | |
5233 | The sequences \h, \H, \v, and \V are features that were added to Perl |
5234 | at release 5.10. In contrast to the other sequences, which match only |
5235 | ASCII characters by default, these always match certain high-valued |
5236 | code points, whether or not PCRE_UCP is set. The horizontal space char- |
5237 | acters are: |
5238 | |
5239 | U+0009 Horizontal tab (HT) |
5240 | U+0020 Space |
5241 | U+00A0 Non-break space |
5242 | U+1680 Ogham space mark |
5243 | U+180E Mongolian vowel separator |
5244 | U+2000 En quad |
5245 | U+2001 Em quad |
5246 | U+2002 En space |
5247 | U+2003 Em space |
5248 | U+2004 Three-per-em space |
5249 | U+2005 Four-per-em space |
5250 | U+2006 Six-per-em space |
5251 | U+2007 Figure space |
5252 | U+2008 Punctuation space |
5253 | U+2009 Thin space |
5254 | U+200A Hair space |
5255 | U+202F Narrow no-break space |
5256 | U+205F Medium mathematical space |
5257 | U+3000 Ideographic space |
5258 | |
5259 | The vertical space characters are: |
5260 | |
5261 | U+000A Linefeed (LF) |
5262 | U+000B Vertical tab (VT) |
5263 | U+000C Form feed (FF) |
5264 | U+000D Carriage return (CR) |
5265 | U+0085 Next line (NEL) |
5266 | U+2028 Line separator |
5267 | U+2029 Paragraph separator |
5268 | |
5269 | In 8-bit, non-UTF-8 mode, only the characters with codepoints less than |
5270 | 256 are relevant. |
5271 | |
5272 | Newline sequences |
5273 | |
5274 | Outside a character class, by default, the escape sequence \R matches |
5275 | any Unicode newline sequence. In 8-bit non-UTF-8 mode \R is equivalent |
5276 | to the following: |
5277 | |
5278 | (?>\r\n|\n|\x0b|\f|\r|\x85) |
5279 | |
5280 | This is an example of an "atomic group", details of which are given be- |
5281 | low. This particular group matches either the two-character sequence |
5282 | CR followed by LF, or one of the single characters LF (linefeed, |
5283 | U+000A), VT (vertical tab, U+000B), FF (form feed, U+000C), CR (car- |
5284 | riage return, U+000D), or NEL (next line, U+0085). The two-character |
5285 | sequence is treated as a single unit that cannot be split. |
5286 | |
5287 | In other modes, two additional characters whose codepoints are greater |
5288 | than 255 are added: LS (line separator, U+2028) and PS (paragraph sepa- |
5289 | rator, U+2029). Unicode character property support is not needed for |
5290 | these characters to be recognized. |
5291 | |
5292 | It is possible to restrict \R to match only CR, LF, or CRLF (instead of |
5293 | the complete set of Unicode line endings) by setting the option |
5294 | PCRE_BSR_ANYCRLF either at compile time or when the pattern is matched. |
5295 | (BSR is an abbrevation for "backslash R".) This can be made the default |
5296 | when PCRE is built; if this is the case, the other behaviour can be re- |
5297 | quested via the PCRE_BSR_UNICODE option. It is also possible to spec- |
5298 | ify these settings by starting a pattern string with one of the follow- |
5299 | ing sequences: |
5300 | |
5301 | (*BSR_ANYCRLF) CR, LF, or CRLF only |
5302 | (*BSR_UNICODE) any Unicode newline sequence |
5303 | |
5304 | These override the default and the options given to the compiling func- |
5305 | tion, but they can themselves be overridden by options given to a |
5306 | matching function. Note that these special settings, which are not |
5307 | Perl-compatible, are recognized only at the very start of a pattern, |
5308 | and that they must be in upper case. If more than one of them is |
5309 | present, the last one is used. They can be combined with a change of |
5310 | newline convention; for example, a pattern can start with: |
5311 | |
5312 | (*ANY)(*BSR_ANYCRLF) |
5313 | |
5314 | They can also be combined with the (*UTF8), (*UTF16), (*UTF32), (*UTF) |
5315 | or (*UCP) special sequences. Inside a character class, \R is treated as |
5316 | an unrecognized escape sequence, and so matches the letter "R" by de- |
5317 | fault, but causes an error if PCRE_EXTRA is set. |
5318 | |
5319 | Unicode character properties |
5320 | |
5321 | When PCRE is built with Unicode character property support, three addi- |
5322 | tional escape sequences that match characters with specific properties |
5323 | are available. When in 8-bit non-UTF-8 mode, these sequences are of |
5324 | course limited to testing characters whose codepoints are less than |
5325 | 256, but they do work in this mode. The extra escape sequences are: |
5326 | |
5327 | \p{xx} a character with the xx property |
5328 | \P{xx} a character without the xx property |
5329 | \X a Unicode extended grapheme cluster |
5330 | |
5331 | The property names represented by xx above are limited to the Unicode |
5332 | script names, the general category properties, "Any", which matches any |
5333 | character (including newline), and some special PCRE properties (de- |
5334 | scribed in the next section). Other Perl properties such as "InMusi- |
5335 | calSymbols" are not currently supported by PCRE. Note that \P{Any} does |
5336 | not match any characters, so always causes a match failure. |
5337 | |
5338 | Sets of Unicode characters are defined as belonging to certain scripts. |
5339 | A character from one of these sets can be matched using a script name. |
5340 | For example: |
5341 | |
5342 | \p{Greek} |
5343 | \P{Han} |
5344 | |
5345 | Those that are not part of an identified script are lumped together as |
5346 | "Common". The current list of scripts is: |
5347 | |
5348 | Arabic, Armenian, Avestan, Balinese, Bamum, Bassa_Vah, Batak, Bengali, |
5349 | Bopomofo, Brahmi, Braille, Buginese, Buhid, Canadian_Aboriginal, Car- |
5350 | ian, Caucasian_Albanian, Chakma, Cham, Cherokee, Common, Coptic, Cunei- |
5351 | form, Cypriot, Cyrillic, Deseret, Devanagari, Duployan, Egyptian_Hiero- |
5352 | glyphs, Elbasan, Ethiopic, Georgian, Glagolitic, Gothic, Grantha, |
5353 | Greek, Gujarati, Gurmukhi, Han, Hangul, Hanunoo, Hebrew, Hiragana, Im- |
5354 | perial_Aramaic, Inherited, Inscriptional_Pahlavi, Inscrip- |
5355 | tional_Parthian, Javanese, Kaithi, Kannada, Katakana, Kayah_Li, |
5356 | Kharoshthi, Khmer, Khojki, Khudawadi, Lao, Latin, Lepcha, Limbu, Lin- |
5357 | ear_A, Linear_B, Lisu, Lycian, Lydian, Mahajani, Malayalam, Mandaic, |
5358 | Manichaean, Meetei_Mayek, Mende_Kikakui, Meroitic_Cursive, Meroitic_Hi- |
5359 | eroglyphs, Miao, Modi, Mongolian, Mro, Myanmar, Nabataean, New_Tai_Lue, |
5360 | Nko, Ogham, Ol_Chiki, Old_Italic, Old_North_Arabian, Old_Permic, |
5361 | Old_Persian, Old_South_Arabian, Old_Turkic, Oriya, Osmanya, Pa- |
5362 | hawh_Hmong, Palmyrene, Pau_Cin_Hau, Phags_Pa, Phoenician, |
5363 | Psalter_Pahlavi, Rejang, Runic, Samaritan, Saurashtra, Sharada, Sha- |
5364 | vian, Siddham, Sinhala, Sora_Sompeng, Sundanese, Syloti_Nagri, Syriac, |
5365 | Tagalog, Tagbanwa, Tai_Le, Tai_Tham, Tai_Viet, Takri, Tamil, Telugu, |
5366 | Thaana, Thai, Tibetan, Tifinagh, Tirhuta, Ugaritic, Vai, Warang_Citi, |
5367 | Yi. |
5368 | |
5369 | Each character has exactly one Unicode general category property, spec- |
5370 | ified by a two-letter abbreviation. For compatibility with Perl, nega- |
5371 | tion can be specified by including a circumflex between the opening |
5372 | brace and the property name. For example, \p{^Lu} is the same as |
5373 | \P{Lu}. |
5374 | |
5375 | If only one letter is specified with \p or \P, it includes all the gen- |
5376 | eral category properties that start with that letter. In this case, in |
5377 | the absence of negation, the curly brackets in the escape sequence are |
5378 | optional; these two examples have the same effect: |
5379 | |
5380 | \p{L} |
5381 | \pL |
5382 | |
5383 | The following general category property codes are supported: |
5384 | |
5385 | C Other |
5386 | Cc Control |
5387 | Cf Format |
5388 | Cn Unassigned |
5389 | Co Private use |
5390 | Cs Surrogate |
5391 | |
5392 | L Letter |
5393 | Ll Lower case letter |
5394 | Lm Modifier letter |
5395 | Lo Other letter |
5396 | Lt Title case letter |
5397 | Lu Upper case letter |
5398 | |
5399 | M Mark |
5400 | Mc Spacing mark |
5401 | Me Enclosing mark |
5402 | Mn Non-spacing mark |
5403 | |
5404 | N Number |
5405 | Nd Decimal number |
5406 | Nl Letter number |
5407 | No Other number |
5408 | |
5409 | P Punctuation |
5410 | Pc Connector punctuation |
5411 | Pd Dash punctuation |
5412 | Pe Close punctuation |
5413 | Pf Final punctuation |
5414 | Pi Initial punctuation |
5415 | Po Other punctuation |
5416 | Ps Open punctuation |
5417 | |
5418 | S Symbol |
5419 | Sc Currency symbol |
5420 | Sk Modifier symbol |
5421 | Sm Mathematical symbol |
5422 | So Other symbol |
5423 | |
5424 | Z Separator |
5425 | Zl Line separator |
5426 | Zp Paragraph separator |
5427 | Zs Space separator |
5428 | |
5429 | The special property L& is also supported: it matches a character that |
5430 | has the Lu, Ll, or Lt property, in other words, a letter that is not |
5431 | classified as a modifier or "other". |
5432 | |
5433 | The Cs (Surrogate) property applies only to characters in the range |
5434 | U+D800 to U+DFFF. Such characters are not valid in Unicode strings and |
5435 | so cannot be tested by PCRE, unless UTF validity checking has been |
5436 | turned off (see the discussion of PCRE_NO_UTF8_CHECK, |
5437 | PCRE_NO_UTF16_CHECK and PCRE_NO_UTF32_CHECK in the pcreapi page). Perl |
5438 | does not support the Cs property. |
5439 | |
5440 | The long synonyms for property names that Perl supports (such as |
5441 | \p{Letter}) are not supported by PCRE, nor is it permitted to prefix |
5442 | any of these properties with "Is". |
5443 | |
5444 | No character that is in the Unicode table has the Cn (unassigned) prop- |
5445 | erty. Instead, this property is assumed for any code point that is not |
5446 | in the Unicode table. |
5447 | |
5448 | Specifying caseless matching does not affect these escape sequences. |
5449 | For example, \p{Lu} always matches only upper case letters. This is |
5450 | different from the behaviour of current versions of Perl. |
5451 | |
5452 | Matching characters by Unicode property is not fast, because PCRE has |
5453 | to do a multistage table lookup in order to find a character's prop- |
5454 | erty. That is why the traditional escape sequences such as \d and \w do |
5455 | not use Unicode properties in PCRE by default, though you can make them |
5456 | do so by setting the PCRE_UCP option or by starting the pattern with |
5457 | (*UCP). |
5458 | |
5459 | Extended grapheme clusters |
5460 | |
5461 | The \X escape matches any number of Unicode characters that form an |
5462 | "extended grapheme cluster", and treats the sequence as an atomic group |
5463 | (see below). Up to and including release 8.31, PCRE matched an ear- |
5464 | lier, simpler definition that was equivalent to |
5465 | |
5466 | (?>\PM\pM*) |
5467 | |
5468 | That is, it matched a character without the "mark" property, followed |
5469 | by zero or more characters with the "mark" property. Characters with |
5470 | the "mark" property are typically non-spacing accents that affect the |
5471 | preceding character. |
5472 | |
5473 | This simple definition was extended in Unicode to include more compli- |
5474 | cated kinds of composite character by giving each character a grapheme |
5475 | breaking property, and creating rules that use these properties to de- |
5476 | fine the boundaries of extended grapheme clusters. In releases of PCRE |
5477 | later than 8.31, \X matches one of these clusters. |
5478 | |
5479 | \X always matches at least one character. Then it decides whether to |
5480 | add additional characters according to the following rules for ending a |
5481 | cluster: |
5482 | |
5483 | 1. End at the end of the subject string. |
5484 | |
5485 | 2. Do not end between CR and LF; otherwise end after any control char- |
5486 | acter. |
5487 | |
5488 | 3. Do not break Hangul (a Korean script) syllable sequences. Hangul |
5489 | characters are of five types: L, V, T, LV, and LVT. An L character may |
5490 | be followed by an L, V, LV, or LVT character; an LV or V character may |
5491 | be followed by a V or T character; an LVT or T character may be follwed |
5492 | only by a T character. |
5493 | |
5494 | 4. Do not end before extending characters or spacing marks. Characters |
5495 | with the "mark" property always have the "extend" grapheme breaking |
5496 | property. |
5497 | |
5498 | 5. Do not end after prepend characters. |
5499 | |
5500 | 6. Otherwise, end the cluster. |
5501 | |
5502 | PCRE's additional properties |
5503 | |
5504 | As well as the standard Unicode properties described above, PCRE sup- |
5505 | ports four more that make it possible to convert traditional escape se- |
5506 | quences such as \w and \s to use Unicode properties. PCRE uses these |
5507 | non-standard, non-Perl properties internally when PCRE_UCP is set. How- |
5508 | ever, they may also be used explicitly. These properties are: |
5509 | |
5510 | Xan Any alphanumeric character |
5511 | Xps Any POSIX space character |
5512 | Xsp Any Perl space character |
5513 | Xwd Any Perl "word" character |
5514 | |
5515 | Xan matches characters that have either the L (letter) or the N (num- |
5516 | ber) property. Xps matches the characters tab, linefeed, vertical tab, |
5517 | form feed, or carriage return, and any other character that has the Z |
5518 | (separator) property. Xsp is the same as Xps; it used to exclude ver- |
5519 | tical tab, for Perl compatibility, but Perl changed, and so PCRE fol- |
5520 | lowed at release 8.34. Xwd matches the same characters as Xan, plus un- |
5521 | derscore. |
5522 | |
5523 | There is another non-standard property, Xuc, which matches any charac- |
5524 | ter that can be represented by a Universal Character Name in C++ and |
5525 | other programming languages. These are the characters $, @, ` (grave |
5526 | accent), and all characters with Unicode code points greater than or |
5527 | equal to U+00A0, except for the surrogates U+D800 to U+DFFF. Note that |
5528 | most base (ASCII) characters are excluded. (Universal Character Names |
5529 | are of the form \uHHHH or \UHHHHHHHH where H is a hexadecimal digit. |
5530 | Note that the Xuc property does not match these sequences but the char- |
5531 | acters that they represent.) |
5532 | |
5533 | Resetting the match start |
5534 | |
5535 | The escape sequence \K causes any previously matched characters not to |
5536 | be included in the final matched sequence. For example, the pattern: |
5537 | |
5538 | foo\Kbar |
5539 | |
5540 | matches "foobar", but reports that it has matched "bar". This feature |
5541 | is similar to a lookbehind assertion (described below). However, in |
5542 | this case, the part of the subject before the real match does not have |
5543 | to be of fixed length, as lookbehind assertions do. The use of \K does |
5544 | not interfere with the setting of captured substrings. For example, |
5545 | when the pattern |
5546 | |
5547 | (foo)\Kbar |
5548 | |
5549 | matches "foobar", the first substring is still set to "foo". |
5550 | |
5551 | Perl documents that the use of \K within assertions is "not well de- |
5552 | fined". In PCRE, \K is acted upon when it occurs inside positive asser- |
5553 | tions, but is ignored in negative assertions. Note that when a pattern |
5554 | such as (?=ab\K) matches, the reported start of the match can be |
5555 | greater than the end of the match. |
5556 | |
5557 | Simple assertions |
5558 | |
5559 | The final use of backslash is for certain simple assertions. An asser- |
5560 | tion specifies a condition that has to be met at a particular point in |
5561 | a match, without consuming any characters from the subject string. The |
5562 | use of subpatterns for more complicated assertions is described below. |
5563 | The backslashed assertions are: |
5564 | |
5565 | \b matches at a word boundary |
5566 | \B matches when not at a word boundary |
5567 | \A matches at the start of the subject |
5568 | \Z matches at the end of the subject |
5569 | also matches before a newline at the end of the subject |
5570 | \z matches only at the end of the subject |
5571 | \G matches at the first matching position in the subject |
5572 | |
5573 | Inside a character class, \b has a different meaning; it matches the |
5574 | backspace character. If any other of these assertions appears in a |
5575 | character class, by default it matches the corresponding literal char- |
5576 | acter (for example, \B matches the letter B). However, if the PCRE_EX- |
5577 | TRA option is set, an "invalid escape sequence" error is generated in- |
5578 | stead. |
5579 | |
5580 | A word boundary is a position in the subject string where the current |
5581 | character and the previous character do not both match \w or \W (i.e. |
5582 | one matches \w and the other matches \W), or the start or end of the |
5583 | string if the first or last character matches \w, respectively. In a |
5584 | UTF mode, the meanings of \w and \W can be changed by setting the |
5585 | PCRE_UCP option. When this is done, it also affects \b and \B. Neither |
5586 | PCRE nor Perl has a separate "start of word" or "end of word" metase- |
5587 | quence. However, whatever follows \b normally determines which it is. |
5588 | For example, the fragment \ba matches "a" at the start of a word. |
5589 | |
5590 | The \A, \Z, and \z assertions differ from the traditional circumflex |
5591 | and dollar (described in the next section) in that they only ever match |
5592 | at the very start and end of the subject string, whatever options are |
5593 | set. Thus, they are independent of multiline mode. These three asser- |
5594 | tions are not affected by the PCRE_NOTBOL or PCRE_NOTEOL options, which |
5595 | affect only the behaviour of the circumflex and dollar metacharacters. |
5596 | However, if the startoffset argument of pcre_exec() is non-zero, indi- |
5597 | cating that matching is to start at a point other than the beginning of |
5598 | the subject, \A can never match. The difference between \Z and \z is |
5599 | that \Z matches before a newline at the end of the string as well as at |
5600 | the very end, whereas \z matches only at the end. |
5601 | |
5602 | The \G assertion is true only when the current matching position is at |
5603 | the start point of the match, as specified by the startoffset argument |
5604 | of pcre_exec(). It differs from \A when the value of startoffset is |
5605 | non-zero. By calling pcre_exec() multiple times with appropriate argu- |
5606 | ments, you can mimic Perl's /g option, and it is in this kind of imple- |
5607 | mentation where \G can be useful. |
5608 | |
5609 | Note, however, that PCRE's interpretation of \G, as the start of the |
5610 | current match, is subtly different from Perl's, which defines it as the |
5611 | end of the previous match. In Perl, these can be different when the |
5612 | previously matched string was empty. Because PCRE does just one match |
5613 | at a time, it cannot reproduce this behaviour. |
5614 | |
5615 | If all the alternatives of a pattern begin with \G, the expression is |
5616 | anchored to the starting match position, and the "anchored" flag is set |
5617 | in the compiled regular expression. |
5618 | |
5619 | |
5620 | CIRCUMFLEX AND DOLLAR |
5621 | |
5622 | The circumflex and dollar metacharacters are zero-width assertions. |
5623 | That is, they test for a particular condition being true without con- |
5624 | suming any characters from the subject string. |
5625 | |
5626 | Outside a character class, in the default matching mode, the circumflex |
5627 | character is an assertion that is true only if the current matching |
5628 | point is at the start of the subject string. If the startoffset argu- |
5629 | ment of pcre_exec() is non-zero, circumflex can never match if the |
5630 | PCRE_MULTILINE option is unset. Inside a character class, circumflex |
5631 | has an entirely different meaning (see below). |
5632 | |
5633 | Circumflex need not be the first character of the pattern if a number |
5634 | of alternatives are involved, but it should be the first thing in each |
5635 | alternative in which it appears if the pattern is ever to match that |
5636 | branch. If all possible alternatives start with a circumflex, that is, |
5637 | if the pattern is constrained to match only at the start of the sub- |
5638 | ject, it is said to be an "anchored" pattern. (There are also other |
5639 | constructs that can cause a pattern to be anchored.) |
5640 | |
5641 | The dollar character is an assertion that is true only if the current |
5642 | matching point is at the end of the subject string, or immediately be- |
5643 | fore a newline at the end of the string (by default). Note, however, |
5644 | that it does not actually match the newline. Dollar need not be the |
5645 | last character of the pattern if a number of alternatives are involved, |
5646 | but it should be the last item in any branch in which it appears. Dol- |
5647 | lar has no special meaning in a character class. |
5648 | |
5649 | The meaning of dollar can be changed so that it matches only at the |
5650 | very end of the string, by setting the PCRE_DOLLAR_ENDONLY option at |
5651 | compile time. This does not affect the \Z assertion. |
5652 | |
5653 | The meanings of the circumflex and dollar characters are changed if the |
5654 | PCRE_MULTILINE option is set. When this is the case, a circumflex |
5655 | matches immediately after internal newlines as well as at the start of |
5656 | the subject string. It does not match after a newline that ends the |
5657 | string. A dollar matches before any newlines in the string, as well as |
5658 | at the very end, when PCRE_MULTILINE is set. When newline is specified |
5659 | as the two-character sequence CRLF, isolated CR and LF characters do |
5660 | not indicate newlines. |
5661 | |
5662 | For example, the pattern /^abc$/ matches the subject string "def\nabc" |
5663 | (where \n represents a newline) in multiline mode, but not otherwise. |
5664 | Consequently, patterns that are anchored in single line mode because |
5665 | all branches start with ^ are not anchored in multiline mode, and a |
5666 | match for circumflex is possible when the startoffset argument of |
5667 | pcre_exec() is non-zero. The PCRE_DOLLAR_ENDONLY option is ignored if |
5668 | PCRE_MULTILINE is set. |
5669 | |
5670 | Note that the sequences \A, \Z, and \z can be used to match the start |
5671 | and end of the subject in both modes, and if all branches of a pattern |
5672 | start with \A it is always anchored, whether or not PCRE_MULTILINE is |
5673 | set. |
5674 | |
5675 | |
5676 | FULL STOP (PERIOD, DOT) AND \N |
5677 | |
5678 | Outside a character class, a dot in the pattern matches any one charac- |
5679 | ter in the subject string except (by default) a character that signi- |
5680 | fies the end of a line. |
5681 | |
5682 | When a line ending is defined as a single character, dot never matches |
5683 | that character; when the two-character sequence CRLF is used, dot does |
5684 | not match CR if it is immediately followed by LF, but otherwise it |
5685 | matches all characters (including isolated CRs and LFs). When any Uni- |
5686 | code line endings are being recognized, dot does not match CR or LF or |
5687 | any of the other line ending characters. |
5688 | |
5689 | The behaviour of dot with regard to newlines can be changed. If the |
5690 | PCRE_DOTALL option is set, a dot matches any one character, without ex- |
5691 | ception. If the two-character sequence CRLF is present in the subject |
5692 | string, it takes two dots to match it. |
5693 | |
5694 | The handling of dot is entirely independent of the handling of circum- |
5695 | flex and dollar, the only relationship being that they both involve |
5696 | newlines. Dot has no special meaning in a character class. |
5697 | |
5698 | The escape sequence \N behaves like a dot, except that it is not af- |
5699 | fected by the PCRE_DOTALL option. In other words, it matches any char- |
5700 | acter except one that signifies the end of a line. Perl also uses \N to |
5701 | match characters by name; PCRE does not support this. |
5702 | |
5703 | |
5704 | MATCHING A SINGLE DATA UNIT |
5705 | |
5706 | Outside a character class, the escape sequence \C matches any one data |
5707 | unit, whether or not a UTF mode is set. In the 8-bit library, one data |
5708 | unit is one byte; in the 16-bit library it is a 16-bit unit; in the |
5709 | 32-bit library it is a 32-bit unit. Unlike a dot, \C always matches |
5710 | line-ending characters. The feature is provided in Perl in order to |
5711 | match individual bytes in UTF-8 mode, but it is unclear how it can use- |
5712 | fully be used. Because \C breaks up characters into individual data |
5713 | units, matching one unit with \C in a UTF mode means that the rest of |
5714 | the string may start with a malformed UTF character. This has undefined |
5715 | results, because PCRE assumes that it is dealing with valid UTF strings |
5716 | (and by default it checks this at the start of processing unless the |
5717 | PCRE_NO_UTF8_CHECK, PCRE_NO_UTF16_CHECK or PCRE_NO_UTF32_CHECK option |
5718 | is used). |
5719 | |
5720 | PCRE does not allow \C to appear in lookbehind assertions (described |
5721 | below) in a UTF mode, because this would make it impossible to calcu- |
5722 | late the length of the lookbehind. |
5723 | |
5724 | In general, the \C escape sequence is best avoided. However, one way of |
5725 | using it that avoids the problem of malformed UTF characters is to use |
5726 | a lookahead to check the length of the next character, as in this pat- |
5727 | tern, which could be used with a UTF-8 string (ignore white space and |
5728 | line breaks): |
5729 | |
5730 | (?| (?=[\x00-\x7f])(\C) | |
5731 | (?=[\x80-\x{7ff}])(\C)(\C) | |
5732 | (?=[\x{800}-\x{ffff}])(\C)(\C)(\C) | |
5733 | (?=[\x{10000}-\x{1fffff}])(\C)(\C)(\C)(\C)) |
5734 | |
5735 | A group that starts with (?| resets the capturing parentheses numbers |
5736 | in each alternative (see "Duplicate Subpattern Numbers" below). The as- |
5737 | sertions at the start of each branch check the next UTF-8 character for |
5738 | values whose encoding uses 1, 2, 3, or 4 bytes, respectively. The char- |
5739 | acter's individual bytes are then captured by the appropriate number of |
5740 | groups. |
5741 | |
5742 | |
5743 | SQUARE BRACKETS AND CHARACTER CLASSES |
5744 | |
5745 | An opening square bracket introduces a character class, terminated by a |
5746 | closing square bracket. A closing square bracket on its own is not spe- |
5747 | cial by default. However, if the PCRE_JAVASCRIPT_COMPAT option is set, |
5748 | a lone closing square bracket causes a compile-time error. If a closing |
5749 | square bracket is required as a member of the class, it should be the |
5750 | first data character in the class (after an initial circumflex, if |
5751 | present) or escaped with a backslash. |
5752 | |
5753 | A character class matches a single character in the subject. In a UTF |
5754 | mode, the character may be more than one data unit long. A matched |
5755 | character must be in the set of characters defined by the class, unless |
5756 | the first character in the class definition is a circumflex, in which |
5757 | case the subject character must not be in the set defined by the class. |
5758 | If a circumflex is actually required as a member of the class, ensure |
5759 | it is not the first character, or escape it with a backslash. |
5760 | |
5761 | For example, the character class [aeiou] matches any lower case vowel, |
5762 | while [^aeiou] matches any character that is not a lower case vowel. |
5763 | Note that a circumflex is just a convenient notation for specifying the |
5764 | characters that are in the class by enumerating those that are not. A |
5765 | class that starts with a circumflex is not an assertion; it still con- |
5766 | sumes a character from the subject string, and therefore it fails if |
5767 | the current pointer is at the end of the string. |
5768 | |
5769 | In UTF-8 (UTF-16, UTF-32) mode, characters with values greater than 255 |
5770 | (0xffff) can be included in a class as a literal string of data units, |
5771 | or by using the \x{ escaping mechanism. |
5772 | |
5773 | When caseless matching is set, any letters in a class represent both |
5774 | their upper case and lower case versions, so for example, a caseless |
5775 | [aeiou] matches "A" as well as "a", and a caseless [^aeiou] does not |
5776 | match "A", whereas a caseful version would. In a UTF mode, PCRE always |
5777 | understands the concept of case for characters whose values are less |
5778 | than 128, so caseless matching is always possible. For characters with |
5779 | higher values, the concept of case is supported if PCRE is compiled |
5780 | with Unicode property support, but not otherwise. If you want to use |
5781 | caseless matching in a UTF mode for characters 128 and above, you must |
5782 | ensure that PCRE is compiled with Unicode property support as well as |
5783 | with UTF support. |
5784 | |
5785 | Characters that might indicate line breaks are never treated in any |
5786 | special way when matching character classes, whatever line-ending se- |
5787 | quence is in use, and whatever setting of the PCRE_DOTALL and PCRE_MUL- |
5788 | TILINE options is used. A class such as [^a] always matches one of |
5789 | these characters. |
5790 | |
5791 | The minus (hyphen) character can be used to specify a range of charac- |
5792 | ters in a character class. For example, [d-m] matches any letter be- |
5793 | tween d and m, inclusive. If a minus character is required in a class, |
5794 | it must be escaped with a backslash or appear in a position where it |
5795 | cannot be interpreted as indicating a range, typically as the first or |
5796 | last character in the class, or immediately after a range. For example, |
5797 | [b-d-z] matches letters in the range b to d, a hyphen character, or z. |
5798 | |
5799 | It is not possible to have the literal character "]" as the end charac- |
5800 | ter of a range. A pattern such as [W-]46] is interpreted as a class of |
5801 | two characters ("W" and "-") followed by a literal string "46]", so it |
5802 | would match "W46]" or "-46]". However, if the "]" is escaped with a |
5803 | backslash it is interpreted as the end of range, so [W-\]46] is inter- |
5804 | preted as a class containing a range followed by two other characters. |
5805 | The octal or hexadecimal representation of "]" can also be used to end |
5806 | a range. |
5807 | |
5808 | An error is generated if a POSIX character class (see below) or an es- |
5809 | cape sequence other than one that defines a single character appears at |
5810 | a point where a range ending character is expected. For example, |
5811 | [z-\xff] is valid, but [A-\d] and [A-[:digit:]] are not. |
5812 | |
5813 | Ranges operate in the collating sequence of character values. They can |
5814 | also be used for characters specified numerically, for example |
5815 | [\000-\037]. Ranges can include any characters that are valid for the |
5816 | current mode. |
5817 | |
5818 | If a range that includes letters is used when caseless matching is set, |
5819 | it matches the letters in either case. For example, [W-c] is equivalent |
5820 | to [][\\^_`wxyzabc], matched caselessly, and in a non-UTF mode, if |
5821 | character tables for a French locale are in use, [\xc8-\xcb] matches |
5822 | accented E characters in both cases. In UTF modes, PCRE supports the |
5823 | concept of case for characters with values greater than 128 only when |
5824 | it is compiled with Unicode property support. |
5825 | |
5826 | The character escape sequences \d, \D, \h, \H, \p, \P, \s, \S, \v, \V, |
5827 | \w, and \W may appear in a character class, and add the characters that |
5828 | they match to the class. For example, [\dABCDEF] matches any hexadeci- |
5829 | mal digit. In UTF modes, the PCRE_UCP option affects the meanings of |
5830 | \d, \s, \w and their upper case partners, just as it does when they ap- |
5831 | pear outside a character class, as described in the section entitled |
5832 | "Generic character types" above. The escape sequence \b has a different |
5833 | meaning inside a character class; it matches the backspace character. |
5834 | The sequences \B, \N, \R, and \X are not special inside a character |
5835 | class. Like any other unrecognized escape sequences, they are treated |
5836 | as the literal characters "B", "N", "R", and "X" by default, but cause |
5837 | an error if the PCRE_EXTRA option is set. |
5838 | |
5839 | A circumflex can conveniently be used with the upper case character |
5840 | types to specify a more restricted set of characters than the matching |
5841 | lower case type. For example, the class [^\W_] matches any letter or |
5842 | digit, but not underscore, whereas [\w] includes underscore. A positive |
5843 | character class should be read as "something OR something OR ..." and a |
5844 | negative class as "NOT something AND NOT something AND NOT ...". |
5845 | |
5846 | The only metacharacters that are recognized in character classes are |
5847 | backslash, hyphen (only where it can be interpreted as specifying a |
5848 | range), circumflex (only at the start), opening square bracket (only |
5849 | when it can be interpreted as introducing a POSIX class name, or for a |
5850 | special compatibility feature - see the next two sections), and the |
5851 | terminating closing square bracket. However, escaping other non-al- |
5852 | phanumeric characters does no harm. |
5853 | |
5854 | |
5855 | POSIX CHARACTER CLASSES |
5856 | |
5857 | Perl supports the POSIX notation for character classes. This uses names |
5858 | enclosed by [: and :] within the enclosing square brackets. PCRE also |
5859 | supports this notation. For example, |
5860 | |
5861 | [01[:alpha:]%] |
5862 | |
5863 | matches "0", "1", any alphabetic character, or "%". The supported class |
5864 | names are: |
5865 | |
5866 | alnum letters and digits |
5867 | alpha letters |
5868 | ascii character codes 0 - 127 |
5869 | blank space or tab only |
5870 | cntrl control characters |
5871 | digit decimal digits (same as \d) |
5872 | graph printing characters, excluding space |
5873 | lower lower case letters |
5874 | print printing characters, including space |
5875 | punct printing characters, excluding letters and digits and space |
5876 | space white space (the same as \s from PCRE 8.34) |
5877 | upper upper case letters |
5878 | word "word" characters (same as \w) |
5879 | xdigit hexadecimal digits |
5880 | |
5881 | The default "space" characters are HT (9), LF (10), VT (11), FF (12), |
5882 | CR (13), and space (32). If locale-specific matching is taking place, |
5883 | the list of space characters may be different; there may be fewer or |
5884 | more of them. "Space" used to be different to \s, which did not include |
5885 | VT, for Perl compatibility. However, Perl changed at release 5.18, and |
5886 | PCRE followed at release 8.34. "Space" and \s now match the same set |
5887 | of characters. |
5888 | |
5889 | The name "word" is a Perl extension, and "blank" is a GNU extension |
5890 | from Perl 5.8. Another Perl extension is negation, which is indicated |
5891 | by a ^ character after the colon. For example, |
5892 | |
5893 | [12[:^digit:]] |
5894 | |
5895 | matches "1", "2", or any non-digit. PCRE (and Perl) also recognize the |
5896 | POSIX syntax [.ch.] and [=ch=] where "ch" is a "collating element", but |
5897 | these are not supported, and an error is given if they are encountered. |
5898 | |
5899 | By default, characters with values greater than 128 do not match any of |
5900 | the POSIX character classes. However, if the PCRE_UCP option is passed |
5901 | to pcre_compile(), some of the classes are changed so that Unicode |
5902 | character properties are used. This is achieved by replacing certain |
5903 | POSIX classes by other sequences, as follows: |
5904 | |
5905 | [:alnum:] becomes \p{Xan} |
5906 | [:alpha:] becomes \p{L} |
5907 | [:blank:] becomes \h |
5908 | [:digit:] becomes \p{Nd} |
5909 | [:lower:] becomes \p{Ll} |
5910 | [:space:] becomes \p{Xps} |
5911 | [:upper:] becomes \p{Lu} |
5912 | [:word:] becomes \p{Xwd} |
5913 | |
5914 | Negated versions, such as [:^alpha:] use \P instead of \p. Three other |
5915 | POSIX classes are handled specially in UCP mode: |
5916 | |
5917 | [:graph:] This matches characters that have glyphs that mark the page |
5918 | when printed. In Unicode property terms, it matches all char- |
5919 | acters with the L, M, N, P, S, or Cf properties, except for: |
5920 | |
5921 | U+061C Arabic Letter Mark |
5922 | U+180E Mongolian Vowel Separator |
5923 | U+2066 - U+2069 Various "isolate"s |
5924 | |
5925 | |
5926 | [:print:] This matches the same characters as [:graph:] plus space |
5927 | characters that are not controls, that is, characters with |
5928 | the Zs property. |
5929 | |
5930 | [:punct:] This matches all characters that have the Unicode P (punctua- |
5931 | tion) property, plus those characters whose code points are |
5932 | less than 128 that have the S (Symbol) property. |
5933 | |
5934 | The other POSIX classes are unchanged, and match only characters with |
5935 | code points less than 128. |
5936 | |
5937 | |
5938 | COMPATIBILITY FEATURE FOR WORD BOUNDARIES |
5939 | |
5940 | In the POSIX.2 compliant library that was included in 4.4BSD Unix, the |
5941 | ugly syntax [[:<:]] and [[:>:]] is used for matching "start of word" |
5942 | and "end of word". PCRE treats these items as follows: |
5943 | |
5944 | [[:<:]] is converted to \b(?=\w) |
5945 | [[:>:]] is converted to \b(?<=\w) |
5946 | |
5947 | Only these exact character sequences are recognized. A sequence such as |
5948 | [a[:<:]b] provokes error for an unrecognized POSIX class name. This |
5949 | support is not compatible with Perl. It is provided to help migrations |
5950 | from other environments, and is best not used in any new patterns. Note |
5951 | that \b matches at the start and the end of a word (see "Simple asser- |
5952 | tions" above), and in a Perl-style pattern the preceding or following |
5953 | character normally shows which is wanted, without the need for the as- |
5954 | sertions that are used above in order to give exactly the POSIX behav- |
5955 | iour. |
5956 | |
5957 | |
5958 | VERTICAL BAR |
5959 | |
5960 | Vertical bar characters are used to separate alternative patterns. For |
5961 | example, the pattern |
5962 | |
5963 | gilbert|sullivan |
5964 | |
5965 | matches either "gilbert" or "sullivan". Any number of alternatives may |
5966 | appear, and an empty alternative is permitted (matching the empty |
5967 | string). The matching process tries each alternative in turn, from left |
5968 | to right, and the first one that succeeds is used. If the alternatives |
5969 | are within a subpattern (defined below), "succeeds" means matching the |
5970 | rest of the main pattern as well as the alternative in the subpattern. |
5971 | |
5972 | |
5973 | INTERNAL OPTION SETTING |
5974 | |
5975 | The settings of the PCRE_CASELESS, PCRE_MULTILINE, PCRE_DOTALL, and |
5976 | PCRE_EXTENDED options (which are Perl-compatible) can be changed from |
5977 | within the pattern by a sequence of Perl option letters enclosed be- |
5978 | tween "(?" and ")". The option letters are |
5979 | |
5980 | i for PCRE_CASELESS |
5981 | m for PCRE_MULTILINE |
5982 | s for PCRE_DOTALL |
5983 | x for PCRE_EXTENDED |
5984 | |
5985 | For example, (?im) sets caseless, multiline matching. It is also possi- |
5986 | ble to unset these options by preceding the letter with a hyphen, and a |
5987 | combined setting and unsetting such as (?im-sx), which sets PCRE_CASE- |
5988 | LESS and PCRE_MULTILINE while unsetting PCRE_DOTALL and PCRE_EXTENDED, |
5989 | is also permitted. If a letter appears both before and after the hy- |
5990 | phen, the option is unset. |
5991 | |
5992 | The PCRE-specific options PCRE_DUPNAMES, PCRE_UNGREEDY, and PCRE_EXTRA |
5993 | can be changed in the same way as the Perl-compatible options by using |
5994 | the characters J, U and X respectively. |
5995 | |
5996 | When one of these option changes occurs at top level (that is, not in- |
5997 | side subpattern parentheses), the change applies to the remainder of |
5998 | the pattern that follows. An option change within a subpattern (see be- |
5999 | low for a description of subpatterns) affects only that part of the |
6000 | subpattern that follows it, so |
6001 | |
6002 | (a(?i)b)c |
6003 | |
6004 | matches abc and aBc and no other strings (assuming PCRE_CASELESS is not |
6005 | used). By this means, options can be made to have different settings |
6006 | in different parts of the pattern. Any changes made in one alternative |
6007 | do carry on into subsequent branches within the same subpattern. For |
6008 | example, |
6009 | |
6010 | (a(?i)b|c) |
6011 | |
6012 | matches "ab", "aB", "c", and "C", even though when matching "C" the |
6013 | first branch is abandoned before the option setting. This is because |
6014 | the effects of option settings happen at compile time. There would be |
6015 | some very weird behaviour otherwise. |
6016 | |
6017 | Note: There are other PCRE-specific options that can be set by the ap- |
6018 | plication when the compiling or matching functions are called. In some |
6019 | cases the pattern can contain special leading sequences such as (*CRLF) |
6020 | to override what the application has set or what has been defaulted. |
6021 | Details are given in the section entitled "Newline sequences" above. |
6022 | There are also the (*UTF8), (*UTF16),(*UTF32), and (*UCP) leading se- |
6023 | quences that can be used to set UTF and Unicode property modes; they |
6024 | are equivalent to setting the PCRE_UTF8, PCRE_UTF16, PCRE_UTF32 and the |
6025 | PCRE_UCP options, respectively. The (*UTF) sequence is a generic ver- |
6026 | sion that can be used with any of the libraries. However, the applica- |
6027 | tion can set the PCRE_NEVER_UTF option, which locks out the use of the |
6028 | (*UTF) sequences. |
6029 | |
6030 | |
6031 | SUBPATTERNS |
6032 | |
6033 | Subpatterns are delimited by parentheses (round brackets), which can be |