Overview
MatchesRegex is a code-based evaluator class in the arize-phoenix-evals package that checks whether text output matches a specified regular expression pattern. It extends the base Evaluator class and returns a binary score (1.0 for match, 0.0 for no match) along with an optional explanation of which substrings matched.
Description
The MatchesRegex evaluator provides a deterministic, code-based approach to evaluating text output against regex patterns. It is useful for verifying that model outputs contain expected patterns such as URLs, email addresses, code snippets, or structured formats.
The evaluator accepts either a raw regex string or a pre-compiled Pattern object. Internally, string patterns are compiled via re.compile(). The findall() method is used for matching, which means the evaluator checks for the presence of any substring matching the pattern, not whether the entire output matches.
| Parameter |
Type |
Default |
Description
|
pattern |
Union[str, Pattern[str]] |
(required) |
The regular expression pattern to match against. Can be a string or compiled re.Pattern.
|
name |
Optional[str] |
"matches_regex" |
Custom name for the evaluator.
|
include_explanation |
bool |
True |
Whether to include an explanation in the Score output.
|
Usage
from phoenix.evals.metrics import MatchesRegex
Code Reference
Key Methods
| Method |
Description
|
__init__(pattern, name, include_explanation) |
Compiles the regex pattern (if a string), stores the raw pattern string for explanations, and delegates to the parent Evaluator.__init__.
|
_evaluate(eval_input) |
Extracts the "output" field, runs pattern.findall(output), and returns a single-element list containing a Score.
|
_async_evaluate(eval_input) |
Delegates to _evaluate synchronously (regex matching is CPU-bound and fast).
|
Input Schema
Defined by the inner class InputSchema(BaseModel):
| Field |
Type |
Description
|
output |
str |
The text string to evaluate against the regex pattern.
|
I/O Contract
Input
| Field |
Type |
Required |
Description
|
output |
str |
Yes |
The text to evaluate against the regex pattern.
|
Output
Returns a list containing one Score object with the following fields:
| Field |
Description
|
name |
The evaluator name (default: "matches_regex", or custom name).
|
score |
1.0 if any substring matches the pattern, 0.0 otherwise.
|
explanation |
If include_explanation=True: a message indicating the match count and pattern, or a "no match" message. If False: None.
|
kind |
"code"
|
direction |
"maximize"
|
Usage Examples
URL Detection
import re
from phoenix.evals.metrics.matches_regex import MatchesRegex
pattern = re.compile(r"https?://[^\s]+")
contains_link = MatchesRegex(pattern=pattern)
eval_input = {"output": "Check out https://github.com/Arize-ai/phoenix!"}
scores = contains_link.evaluate(eval_input)
print(scores)
# [Score(name='matches_regex', score=1.0,
# explanation='There are 1 matches for the regex: https?://[^\s]+',
# kind='code', direction='maximize')]
Email Validation with Custom Name
from phoenix.evals.metrics import MatchesRegex
email_checker = MatchesRegex(
pattern=r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}",
name="contains_email",
)
eval_input = {"output": "Contact us at support@example.com for help."}
scores = email_checker.evaluate(eval_input)
# score=1.0, name='contains_email'
No Match Scenario
from phoenix.evals.metrics import MatchesRegex
number_checker = MatchesRegex(pattern=r"\d{3}-\d{4}")
eval_input = {"output": "No phone numbers here."}
scores = number_checker.evaluate(eval_input)
# score=0.0, explanation='No substrings matched the regex pattern \d{3}-\d{4}'
Disabling Explanations
from phoenix.evals.metrics import MatchesRegex
checker = MatchesRegex(pattern=r"\bJSON\b", include_explanation=False)
eval_input = {"output": "The response is in JSON format."}
scores = checker.evaluate(eval_input)
# score=1.0, explanation=None
Related Pages
