trunk/doc/pcreprecompile.3

.TH PCREPRECOMPILE 3 "12 November 2013" "PCRE 8.34"
.SH NAME
PCRE - Perl-compatible regular expressions
.SH "SAVING AND RE-USING PRECOMPILED PCRE PATTERNS"
.rs
.sp
If you are running an application that uses a large number of regular
expression patterns, it may be useful to store them in a precompiled form
instead of having to compile them every time the application is run.
If you are not using any private character tables (see the
.\" HREF
\fBpcre_maketables()\fP
.\"
documentation), this is relatively straightforward. If you are using private
tables, it is a little bit more complicated. However, if you are using the
just-in-time optimization feature, it is not possible to save and reload the
JIT data.
.P
If you save compiled patterns to a file, you can copy them to a different host
and run them there. If the two hosts have different endianness (byte order),
you should run the \fBpcre[16|32]_pattern_to_host_byte_order()\fP function on the
new host before trying to match the pattern. The matching functions return
PCRE_ERROR_BADENDIANNESS if they detect a pattern with the wrong endianness.
.P
Compiling regular expressions with one version of PCRE for use with a different
version is not guaranteed to work and may cause crashes, and saving and
restoring a compiled pattern loses any JIT optimization data.
.
.
.SH "SAVING A COMPILED PATTERN"
.rs
.sp
The value returned by \fBpcre[16|32]_compile()\fP points to a single block of
memory that holds the compiled pattern and associated data. You can find the
length of this block in bytes by calling \fBpcre[16|32]_fullinfo()\fP with an
argument of PCRE_INFO_SIZE. You can then save the data in any appropriate
manner. Here is sample code for the 8-bit library that compiles a pattern and
writes it to a file. It assumes that the variable \fIfd\fP refers to a file
that is open for output:
.sp
  int erroroffset, rc, size;
  char *error;
  pcre *re;
.sp
  re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL);
  if (re == NULL) { ... handle errors ... }
  rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size);
  if (rc < 0) { ... handle errors ... }
  rc = fwrite(re, 1, size, fd);
  if (rc != size) { ... handle errors ... }
.sp
In this example, the bytes that comprise the compiled pattern are copied
exactly. Note that this is binary data that may contain any of the 256 possible
byte values. On systems that make a distinction between binary and non-binary
data, be sure that the file is opened for binary output.
.P
If you want to write more than one pattern to a file, you will have to devise a
way of separating them. For binary data, preceding each pattern with its length
is probably the most straightforward approach. Another possibility is to write
out the data in hexadecimal instead of binary, one pattern to a line.
.P
Saving compiled patterns in a file is only one possible way of storing them for
later use. They could equally well be saved in a database, or in the memory of
some daemon process that passes them via sockets to the processes that want
them.
.P
If the pattern has been studied, it is also possible to save the normal study
data in a similar way to the compiled pattern itself. However, if the
PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot
be saved because it is too dependent on the current environment. When studying
generates additional information, \fBpcre[16|32]_study()\fP returns a pointer to a
\fBpcre[16|32]_extra\fP data block. Its format is defined in the
.\" HTML <a href="pcreapi.html#extradata">
.\" </a>
section on matching a pattern
.\"
in the
.\" HREF
\fBpcreapi\fP
.\"
documentation. The \fIstudy_data\fP field points to the binary study data, and
this is what you must save (not the \fBpcre[16|32]_extra\fP block itself). The
length of the study data can be obtained by calling \fBpcre[16|32]_fullinfo()\fP
with an argument of PCRE_INFO_STUDYSIZE. Remember to check that
\fBpcre[16|32]_study()\fP did return a non-NULL value before trying to save the
study data.
.
.
.SH "RE-USING A PRECOMPILED PATTERN"
.rs
.sp
Re-using a precompiled pattern is straightforward. Having reloaded it into main
memory, called \fBpcre[16|32]_pattern_to_host_byte_order()\fP if necessary, you
pass its pointer to \fBpcre[16|32]_exec()\fP or \fBpcre[16|32]_dfa_exec()\fP in
the usual way.
.P
However, if you passed a pointer to custom character tables when the pattern
was compiled (the \fItableptr\fP argument of \fBpcre[16|32]_compile()\fP), you
must now pass a similar pointer to \fBpcre[16|32]_exec()\fP or
\fBpcre[16|32]_dfa_exec()\fP, because the value saved with the compiled pattern
will obviously be nonsense. A field in a \fBpcre[16|32]_extra()\fP block is used
to pass this data, as described in the
.\" HTML <a href="pcreapi.html#extradata">
.\" </a>
section on matching a pattern
.\"
in the
.\" HREF
\fBpcreapi\fP
.\"
documentation.
.P
\fBWarning:\fP The tables that \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP use
must be the same as those that were used when the pattern was compiled. If this
is not the case, the behaviour is undefined.
.P
If you did not provide custom character tables when the pattern was compiled,
the pointer in the compiled pattern is NULL, which causes the matching
functions to use PCRE's internal tables. Thus, you do not need to take any
special action at run time in this case.
.P
If you saved study data with the compiled pattern, you need to create your own
\fBpcre[16|32]_extra\fP data block and set the \fIstudy_data\fP field to point
to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in
the \fIflags\fP field to indicate that study data is present. Then pass the
\fBpcre[16|32]_extra\fP block to the matching function in the usual way. If the
pattern was studied for just-in-time optimization, that data cannot be saved,
and so is lost by a save/restore cycle.
.
.
.SH "COMPATIBILITY WITH DIFFERENT PCRE RELEASES"
.rs
.sp
In general, it is safest to recompile all saved patterns when you update to a
new PCRE release, though not all updates actually require this.
.
.
.
.SH AUTHOR
.rs
.sp
.nf
Philip Hazel
University Computing Service
Cambridge CB2 3QH, England.
.fi
.
.
.SH REVISION
.rs
.sp
.nf
Last updated: 12 November 2013
Copyright (c) 1997-2013 University of Cambridge.
.fi
1	.TH PCREPRECOMPILE 3 "12 November 2013" "PCRE 8.34"
2	.SH NAME
3	PCRE - Perl-compatible regular expressions
4	.SH "SAVING AND RE-USING PRECOMPILED PCRE PATTERNS"
5	.rs
6	.sp
7	If you are running an application that uses a large number of regular
8	expression patterns, it may be useful to store them in a precompiled form
9	instead of having to compile them every time the application is run.
10	If you are not using any private character tables (see the
11	.\" HREF
12	\fBpcre_maketables()\fP
13	.\"
14	documentation), this is relatively straightforward. If you are using private
15	tables, it is a little bit more complicated. However, if you are using the
16	just-in-time optimization feature, it is not possible to save and reload the
17	JIT data.
18	.P
19	If you save compiled patterns to a file, you can copy them to a different host
20	and run them there. If the two hosts have different endianness (byte order),
21	you should run the \fBpcre[16\|32]_pattern_to_host_byte_order()\fP function on the
22	new host before trying to match the pattern. The matching functions return
23	PCRE_ERROR_BADENDIANNESS if they detect a pattern with the wrong endianness.
24	.P
25	Compiling regular expressions with one version of PCRE for use with a different
26	version is not guaranteed to work and may cause crashes, and saving and
27	restoring a compiled pattern loses any JIT optimization data.
28	.
29	.
30	.SH "SAVING A COMPILED PATTERN"
31	.rs
32	.sp
33	The value returned by \fBpcre[16\|32]_compile()\fP points to a single block of
34	memory that holds the compiled pattern and associated data. You can find the
35	length of this block in bytes by calling \fBpcre[16\|32]_fullinfo()\fP with an
36	argument of PCRE_INFO_SIZE. You can then save the data in any appropriate
37	manner. Here is sample code for the 8-bit library that compiles a pattern and
38	writes it to a file. It assumes that the variable \fIfd\fP refers to a file
39	that is open for output:
40	.sp
41	int erroroffset, rc, size;
42	char *error;
43	pcre *re;
44	.sp
45	re = pcre_compile("my pattern", 0, &error, &erroroffset, NULL);
46	if (re == NULL) { ... handle errors ... }
47	rc = pcre_fullinfo(re, NULL, PCRE_INFO_SIZE, &size);
48	if (rc < 0) { ... handle errors ... }
49	rc = fwrite(re, 1, size, fd);
50	if (rc != size) { ... handle errors ... }
51	.sp
52	In this example, the bytes that comprise the compiled pattern are copied
53	exactly. Note that this is binary data that may contain any of the 256 possible
54	byte values. On systems that make a distinction between binary and non-binary
55	data, be sure that the file is opened for binary output.
56	.P
57	If you want to write more than one pattern to a file, you will have to devise a
58	way of separating them. For binary data, preceding each pattern with its length
59	is probably the most straightforward approach. Another possibility is to write
60	out the data in hexadecimal instead of binary, one pattern to a line.
61	.P
62	Saving compiled patterns in a file is only one possible way of storing them for
63	later use. They could equally well be saved in a database, or in the memory of
64	some daemon process that passes them via sockets to the processes that want
65	them.
66	.P
67	If the pattern has been studied, it is also possible to save the normal study
68	data in a similar way to the compiled pattern itself. However, if the
69	PCRE_STUDY_JIT_COMPILE was used, the just-in-time data that is created cannot
70	be saved because it is too dependent on the current environment. When studying
71	generates additional information, \fBpcre[16\|32]_study()\fP returns a pointer to a
72	\fBpcre[16\|32]_extra\fP data block. Its format is defined in the
73	.\" HTML <a href="pcreapi.html#extradata">
74	.\" </a>
75	section on matching a pattern
76	.\"
77	in the
78	.\" HREF
79	\fBpcreapi\fP
80	.\"
81	documentation. The \fIstudy_data\fP field points to the binary study data, and
82	this is what you must save (not the \fBpcre[16\|32]_extra\fP block itself). The
83	length of the study data can be obtained by calling \fBpcre[16\|32]_fullinfo()\fP
84	with an argument of PCRE_INFO_STUDYSIZE. Remember to check that
85	\fBpcre[16\|32]_study()\fP did return a non-NULL value before trying to save the
86	study data.
87	.
88	.
89	.SH "RE-USING A PRECOMPILED PATTERN"
90	.rs
91	.sp
92	Re-using a precompiled pattern is straightforward. Having reloaded it into main
93	memory, called \fBpcre[16\|32]_pattern_to_host_byte_order()\fP if necessary, you
94	pass its pointer to \fBpcre[16\|32]_exec()\fP or \fBpcre[16\|32]_dfa_exec()\fP in
95	the usual way.
96	.P
97	However, if you passed a pointer to custom character tables when the pattern
98	was compiled (the \fItableptr\fP argument of \fBpcre[16\|32]_compile()\fP), you
99	must now pass a similar pointer to \fBpcre[16\|32]_exec()\fP or
100	\fBpcre[16\|32]_dfa_exec()\fP, because the value saved with the compiled pattern
101	will obviously be nonsense. A field in a \fBpcre[16\|32]_extra()\fP block is used
102	to pass this data, as described in the
103	.\" HTML <a href="pcreapi.html#extradata">
104	.\" </a>
105	section on matching a pattern
106	.\"
107	in the
108	.\" HREF
109	\fBpcreapi\fP
110	.\"
111	documentation.
112	.P
113	\fBWarning:\fP The tables that \fBpcre_exec()\fP and \fBpcre_dfa_exec()\fP use
114	must be the same as those that were used when the pattern was compiled. If this
115	is not the case, the behaviour is undefined.
116	.P
117	If you did not provide custom character tables when the pattern was compiled,
118	the pointer in the compiled pattern is NULL, which causes the matching
119	functions to use PCRE's internal tables. Thus, you do not need to take any
120	special action at run time in this case.
121	.P
122	If you saved study data with the compiled pattern, you need to create your own
123	\fBpcre[16\|32]_extra\fP data block and set the \fIstudy_data\fP field to point
124	to the reloaded study data. You must also set the PCRE_EXTRA_STUDY_DATA bit in
125	the \fIflags\fP field to indicate that study data is present. Then pass the
126	\fBpcre[16\|32]_extra\fP block to the matching function in the usual way. If the
127	pattern was studied for just-in-time optimization, that data cannot be saved,
128	and so is lost by a save/restore cycle.
129	.
130	.
131	.SH "COMPATIBILITY WITH DIFFERENT PCRE RELEASES"
132	.rs
133	.sp
134	In general, it is safest to recompile all saved patterns when you update to a
135	new PCRE release, though not all updates actually require this.
136	.
137	.
138	.
139	.SH AUTHOR
140	.rs
141	.sp
142	.nf
143	Philip Hazel
144	University Computing Service
145	Cambridge CB2 3QH, England.
146	.fi
147	.
148	.
149	.SH REVISION
150	.rs
151	.sp
152	.nf
153	Last updated: 12 November 2013
154	Copyright (c) 1997-2013 University of Cambridge.
155	.fi
Name	Value
svn:eol-style	native
svn:keywords	"Author Date Id Revision Url"