26 
<something> <something else> <something further> 
<something> <something else> <something further> 
27 
.sp 
.sp 
28 
there are three possible answers. The standard algorithm finds only one of 
there are three possible answers. The standard algorithm finds only one of 
29 
them, whereas the DFA algorithm finds all three. 
them, whereas the alternative algorithm finds all three. 
30 
. 
. 
31 
.SH "REGULAR EXPRESSIONS AS TREES" 
.SH "REGULAR EXPRESSIONS AS TREES" 
32 
.rs 
.rs 
35 
as a tree structure. An unlimited repetition in the pattern makes the tree of 
as a tree structure. An unlimited repetition in the pattern makes the tree of 
36 
infinite size, but it is still a tree. Matching the pattern to a given subject 
infinite size, but it is still a tree. Matching the pattern to a given subject 
37 
string (from a given starting point) can be thought of as a search of the tree. 
string (from a given starting point) can be thought of as a search of the tree. 
38 
There are two standard ways to search a tree: depthfirst and breadthfirst, 
There are two ways to search a tree: depthfirst and breadthfirst, and these 
39 
and these correspond to the two matching algorithms provided by PCRE. 
correspond to the two matching algorithms provided by PCRE. 
40 
. 
. 
41 
.SH "THE STANDARD MATCHING ALGORITHM" 
.SH "THE STANDARD MATCHING ALGORITHM" 
42 
.rs 
.rs 
43 
.sp 
.sp 
44 
In the terminology of Jeffrey Friedl's book \fIMastering Regular 
In the terminology of Jeffrey Friedl's book "Mastering Regular 
45 
Expressions\fP, the standard algorithm is an "NFA algorithm". It conducts a 
Expressions", the standard algorithm is an "NFA algorithm". It conducts a 
46 
depthfirst search of the pattern tree. That is, it proceeds along a single 
depthfirst search of the pattern tree. That is, it proceeds along a single 
47 
path through the tree, checking that the subject matches what is required. When 
path through the tree, checking that the subject matches what is required. When 
48 
there is a mismatch, the algorithm tries any alternatives at the current point, 
there is a mismatch, the algorithm tries any alternatives at the current point, 
63 
matched by portions of the pattern in parentheses. This provides support for 
matched by portions of the pattern in parentheses. This provides support for 
64 
capturing parentheses and back references. 
capturing parentheses and back references. 
65 
. 
. 
66 
.SH "THE DFA MATCHING ALGORITHM" 
.SH "THE ALTERNATIVE MATCHING ALGORITHM" 
67 
.rs 
.rs 
68 
.sp 
.sp 
69 
DFA stands for "deterministic finite automaton", but you do not need to 
This algorithm conducts a breadthfirst search of the tree. Starting from the 
70 
understand the origins of that name. This algorithm conducts a breadthfirst 
first matching point in the subject, it scans the subject string from left to 
71 
search of the tree. Starting from the first matching point in the subject, it 
right, once, character by character, and as it does this, it remembers all the 
72 
scans the subject string from left to right, once, character by character, and 
paths through the tree that represent valid matches. In Friedl's terminology, 
73 
as it does this, it remembers all the paths through the tree that represent 
this is a kind of "DFA algorithm", though it is not implemented as a 
74 
valid matches. 
traditional finite state machine (it keeps multiple states active 
75 

simultaneously). 
76 
.P 
.P 
77 
The scan continues until either the end of the subject is reached, or there are 
The scan continues until either the end of the subject is reached, or there are 
78 
no more unterminated paths. At this point, terminated paths represent the 
no more unterminated paths. At this point, terminated paths represent the 
93 
matches that start at later positions. 
matches that start at later positions. 
94 
.P 
.P 
95 
There are a number of features of PCRE regular expressions that are not 
There are a number of features of PCRE regular expressions that are not 
96 
supported by the DFA matching algorithm. They are as follows: 
supported by the alternative matching algorithm. They are as follows: 
97 
.P 
.P 
98 
1. Because the algorithm finds all possible matches, the greedy or ungreedy 
1. Because the algorithm finds all possible matches, the greedy or ungreedy 
99 
nature of repetition quantifiers is not relevant. Greedy and ungreedy 
nature of repetition quantifiers is not relevant. Greedy and ungreedy 
100 
quantifiers are treated in exactly the same way. 
quantifiers are treated in exactly the same way. However, possessive 
101 

quantifiers can make a difference when what follows could also match what is 
102 

quantified, for example in a pattern like this: 
103 

.sp 
104 

^a++\ew! 
105 

.sp 
106 

This pattern matches "aaab!" but not "aaa!", which would be matched by a 
107 

nonpossessive quantifier. Similarly, if an atomic group is present, it is 
108 

matched as if it were a standalone pattern at the current point, and the 
109 

longest match is then "locked in" for the rest of the overall pattern. 
110 
.P 
.P 
111 
2. When dealing with multiple paths through the tree simultaneously, it is not 
2. When dealing with multiple paths through the tree simultaneously, it is not 
112 
straightforward to keep track of captured substrings for the different matching 
straightforward to keep track of captured substrings for the different matching 
117 
not supported, and cause errors if encountered. 
not supported, and cause errors if encountered. 
118 
.P 
.P 
119 
4. For the same reason, conditional expressions that use a backreference as the 
4. For the same reason, conditional expressions that use a backreference as the 
120 
condition are not supported. 
condition or test for a specific group recursion are not supported. 
121 
.P 
.P 
122 
5. Callouts are supported, but the value of the \fIcapture_top\fP field is 
5. Callouts are supported, but the value of the \fIcapture_top\fP field is 
123 
always 1, and the value of the \fIcapture_last\fP field is always 1. 
always 1, and the value of the \fIcapture_last\fP field is always 1. 
124 
.P 
.P 
125 
6. 
6. 
126 
The \eC escape sequence, which (in the standard algorithm) matches a single 
The \eC escape sequence, which (in the standard algorithm) matches a single 
127 
byte, even in UTF8 mode, is not supported because the DFA algorithm moves 
byte, even in UTF8 mode, is not supported because the alternative algorithm 
128 
through the subject string one character at a time, for all active paths 
moves through the subject string one character at a time, for all active paths 
129 
through the tree. 
through the tree. 
130 
. 
. 
131 
.SH "ADVANTAGES OF THE DFA ALGORITHM" 
.SH "ADVANTAGES OF THE ALTERNATIVE ALGORITHM" 
132 
.rs 
.rs 
133 
.sp 
.sp 
134 
Using the DFA matching algorithm provides the following advantages: 
Using the alternative matching algorithm provides the following advantages: 
135 
.P 
.P 
136 
1. All possible matches (at a single point in the subject) are automatically 
1. All possible matches (at a single point in the subject) are automatically 
137 
found, and in particular, the longest match is found. To find more than one 
found, and in particular, the longest match is found. To find more than one 
140 
.P 
.P 
141 
2. There is much better support for partial matching. The restrictions on the 
2. There is much better support for partial matching. The restrictions on the 
142 
content of the pattern that apply when using the standard algorithm for partial 
content of the pattern that apply when using the standard algorithm for partial 
143 
matching do not apply to the DFA algorithm. For nonanchored patterns, the 
matching do not apply to the alternative algorithm. For nonanchored patterns, 
144 
starting position of a partial match is available. 
the starting position of a partial match is available. 
145 
.P 
.P 
146 
3. Because the DFA algorithm scans the subject string just once, and never 
3. Because the alternative algorithm scans the subject string just once, and 
147 
needs to backtrack, it is possible to pass very long subject strings to the 
never needs to backtrack, it is possible to pass very long subject strings to 
148 
matching function in several pieces, checking for partial matching each time. 
the matching function in several pieces, checking for partial matching each 
149 

time. 
150 
. 
. 
151 
.SH "DISADVANTAGES OF THE DFA ALGORITHM" 
.SH "DISADVANTAGES OF THE ALTERNATIVE ALGORITHM" 
152 
.rs 
.rs 
153 
.sp 
.sp 
154 
The DFA algorithm suffers from a number of disadvantages: 
The alternative algorithm suffers from a number of disadvantages: 
155 
.P 
.P 
156 
1. It is substantially slower than the standard algorithm. This is partly 
1. It is substantially slower than the standard algorithm. This is partly 
157 
because it has to search for all possible matches, but is also because it is 
because it has to search for all possible matches, but is also because it is 
159 
.P 
.P 
160 
2. Capturing parentheses and back references are not supported. 
2. Capturing parentheses and back references are not supported. 
161 
.P 
.P 
162 
3. The "atomic group" feature of PCRE regular expressions is supported, but 
3. Although atomic groups are supported, their use does not provide the 
163 
does not provide the advantage that it does for the standard algorithm. 
performance advantage that it does for the standard algorithm. 
164 
.P 
. 
165 
.in 0 
. 
166 
Last updated: 28 February 2005 
.SH AUTHOR 
167 
.br 
.rs 
168 
Copyright (c) 19972005 University of Cambridge. 
.sp 
169 

.nf 
170 

Philip Hazel 
171 

University Computing Service 
172 

Cambridge CB2 3QH, England. 
173 

.fi 
174 

. 
175 

. 
176 

.SH REVISION 
177 

.rs 
178 

.sp 
179 

.nf 
180 

Last updated: 06 March 2007 
181 

Copyright (c) 19972007 University of Cambridge. 
182 

.fi 