Implementation:Arize ai Phoenix PrecisionRecallFScore

Overview

PrecisionRecallFScore is a code evaluator class in the arize-phoenix-evals package that computes precision, recall, and F-beta scores for classification tasks. It extends the base Evaluator class and operates entirely without an LLM, performing deterministic metric computation over sequences of expected and predicted labels. It supports binary and multi-class classification with configurable averaging strategies (macro, micro, weighted).

Description

The PrecisionRecallFScore evaluator accepts two parallel sequences of labels (expected and output) and computes standard information retrieval metrics:

• Precision -- the fraction of predicted positives that are true positives.
• Recall -- the fraction of actual positives that were correctly predicted.
• F-beta score -- the weighted harmonic mean of precision and recall, parameterized by beta.

The evaluator constructs a per-class confusion matrix internally using the frozen dataclass _ClassCounts, which tracks true_positive, false_positive, and false_negative counts for each label. These counts are then aggregated according to the chosen averaging strategy to produce the final scores.

Parameter	Type	Default	Description
beta	float	1.0	Weight of recall relative to precision. Must be > 0. A value of 1.0 yields the standard F1 score; 0.5 weights precision higher; 2.0 weights recall higher.
average	Literal["macro", "micro", "weighted"]	"macro"	Aggregation strategy across classes. macro averages per-class metrics equally; micro aggregates counts globally; weighted averages per-class metrics weighted by class support.
zero_division	float	0.0	Value returned when a metric is undefined (division by zero).
positive_label	Optional[Union[str, int]]	None	When set, computes binary precision/recall/F for this label (one-vs-rest). If None and labels are {0, 1}, defaults to 1.

Usage

from phoenix.evals.metrics import PrecisionRecallFScore

The evaluator is instantiated with configuration parameters and then called with a dictionary containing expected and output sequences:

evaluator = PrecisionRecallFScore(beta=1.0, average="macro")
eval_input = {
    "expected": ["cat", "dog", "cat", "bird"],
    "output": ["cat", "cat", "cat", "bird"],
}
scores = evaluator(eval_input)

Code Reference

Property	Value
Source File	packages/phoenix-evals/src/phoenix/evals/metrics/precision_recall.py
Module	phoenix.evals.metrics.precision_recall
Class	PrecisionRecallFScore(Evaluator)
Lines	~400
Kind	"code"
Direction	"maximize"
Domain	LLM Evaluation, Metrics

Key Internal Components

• _ClassCounts -- A frozen dataclass holding true_positive, false_positive, and false_negative counts per class, with a support property that returns true_positive + false_negative.
• _format_beta_for_name(beta) -- A module-level helper that formats the beta value for use in score names (e.g., 1.0 becomes "f1", 0.5 becomes "f0_5").
• InputSchema -- A Pydantic BaseModel requiring expected: List[Union[str, int]] and output: List[Union[str, int]].

Key Methods

Method	Description
_evaluate(eval_input)	Main evaluation entry point. Validates inputs, collects labels, computes per-class counts, resolves binary vs. multi-class mode, aggregates metrics, and returns a list of three Score objects.
_collect_labels(expected, output)	Produces a stable-ordered list of all unique labels seen across both sequences.
_compute_counts(expected, output, labels)	Builds the per-label confusion matrix as a Dict[Hashable, _ClassCounts].
_safe_div(numerator, denominator)	Division with fallback to zero_division when denominator is zero.
_aggregate_precision_recall(counts_by_label)	Computes aggregated precision and recall using the configured averaging strategy (macro, micro, or weighted).
_compute_f_score(precision, recall, beta)	Computes the F-beta score from precision and recall values.
_resolve_positive_label(configured_positive, labels)	Determines whether to run in binary mode. Returns the positive label if explicitly set or if labels are {0, 1}; otherwise returns None for multi-class mode.

I/O Contract

Input

Field	Type	Required	Description
expected	List[Union[str, int]]	Yes	Sequence of ground-truth labels.
output	List[Union[str, int]]	Yes	Sequence of predicted labels. Must be the same length as expected.

Output

Returns a list of three Score objects:

Score Name	Description	Score Range
precision (or precision_{average})	The precision metric.	0.0 to 1.0
recall (or recall_{average})	The recall metric.	0.0 to 1.0
f{beta} (or f{beta}_{average})	The F-beta score (e.g., f1, f0_5, f2).	0.0 to 1.0

Each Score object contains metadata with beta, average, labels, and positive_label fields.

Note: The average suffix (e.g., _micro, _weighted) is only appended when a non-default averaging strategy is used. The default "macro" produces no suffix.

Usage Examples

Multi-class Evaluation (Macro Average)

from phoenix.evals.metrics import PrecisionRecallFScore

evaluator = PrecisionRecallFScore(beta=1.0, average="macro")
eval_input = {
    "expected": ["cat", "dog", "cat", "bird"],
    "output": ["cat", "cat", "cat", "bird"],
}
scores = evaluator(eval_input)
for s in scores:
    print(f"{s.name}: {s.score}")
# precision: ...
# recall: ...
# f1: ...

Binary Classification with Explicit Positive Label

from phoenix.evals.metrics import PrecisionRecallFScore

evaluator = PrecisionRecallFScore(beta=0.5, positive_label="spam")
eval_input = {
    "expected": ["spam", "ham", "spam"],
    "output": ["spam", "spam", "ham"],
}
scores = evaluator(eval_input)
for s in scores:
    print(f"{s.name}: {s.score}")
# precision: ...
# recall: ...
# f0_5: ...

Weighted Average for Imbalanced Classes

from phoenix.evals.metrics import PrecisionRecallFScore

evaluator = PrecisionRecallFScore(beta=1.0, average="weighted")
eval_input = {
    "expected": ["positive", "negative", "negative", "negative", "positive"],
    "output": ["positive", "negative", "positive", "negative", "negative"],
}
scores = evaluator(eval_input)
for s in scores:
    print(f"{s.name}: {s.score}")
# precision_weighted: ...
# recall_weighted: ...
# f1_weighted: ...

Related Pages

• Arize_ai_Phoenix_ExactMatch -- Simple exact string matching evaluator in the same metrics module.
• Arize_ai_Phoenix_MatchesRegex -- Regex-based code evaluator in the same metrics module.
• Arize_ai_Phoenix_Evals_Public_API -- The top-level phoenix.evals public API that re-exports this evaluator.
• Arize_ai_Phoenix_CorrectnessEvaluator -- LLM-based correctness evaluation for comparison with code-based precision/recall.