Implementation:Evidentlyai Evidently Legacy OOV Words Feature

Knowledge Sources	Evidentlyai_Evidently
Domains	ML Monitoring, NLP, Text Quality
Last Updated	2026-02-14 12:00 GMT

Overview

Computes the percentage of out-of-vocabulary (OOV) words in a text column by checking each word against the NLTK English dictionary with lemmatization.

Description

The OOVWordsPercentage class extends ApplyColumnGeneratedFeature to compute the percentage of words in a text column that are not found in the standard English dictionary. It uses NLTK's WordNetLemmatizer to normalize words before lookup and NLTK's words corpus as the reference vocabulary.

The computation works as follows:

1. For each cell value, the text is cleaned by removing all non-alphanumeric characters (except spaces) and split into individual words.
2. Each word is lowercased and checked against a configurable ignore_words tuple. If not in the ignore list, the word is lemmatized and checked against the NLTK English word set.
3. Words not found in the dictionary are counted as OOV. The final result is 100 * oov_count / total_word_count.
4. Null/NaN values and empty strings return 0.

The NLTK wordnet and words corpora are lazily downloaded on first use via generate_feature(), which initializes the lemmatizer and word set before delegating to the parent class's apply-based generation.

The feature type is Numerical and the default display name follows the template "OOV Words % for {column_name}".

Usage

Use this feature to monitor text quality in NLP pipelines, particularly when checking for gibberish, typos, or domain-specific jargon in generated text. A high OOV percentage may indicate low-quality or corrupted text output. The ignore_words parameter allows whitelisting domain-specific terms that would otherwise be flagged.

Code Reference

Source Location

• Repository: Evidentlyai_Evidently
• File: src/evidently/legacy/features/OOV_words_percentage_feature.py

Signature

class OOVWordsPercentage(ApplyColumnGeneratedFeature):
    class Config:
        type_alias = "evidently:feature:OOVWordsPercentage"

    __feature_type__: ClassVar = ColumnType.Numerical
    display_name_template: ClassVar = "OOV Words % for {column_name}"
    ignore_words: Tuple = ()

    def __init__(self, column_name: str, ignore_words=(), display_name: Optional[str] = None): ...
    def apply(self, value: Any): ...
    def generate_feature(self, data: pd.DataFrame, data_definition: DataDefinition) -> pd.DataFrame: ...

Import

from evidently.legacy.features.OOV_words_percentage_feature import OOVWordsPercentage

I/O Contract

Inputs

Name	Type	Required	Description
column_name	str	Yes	The name of the text column to analyze for OOV words.
ignore_words	Tuple	No	A tuple of words to exclude from OOV counting (domain-specific whitelist). Defaults to an empty tuple.
display_name	Optional[str]	No	Custom display name for the feature. Defaults to "OOV Words % for {column_name}".

Outputs

Name	Type	Description
return	pd.DataFrame	A single-column DataFrame with the OOV word percentage (0-100) for each row.

Usage Examples

from evidently.legacy.features.OOV_words_percentage_feature import OOVWordsPercentage

# Basic OOV percentage for a text column
oov_feature = OOVWordsPercentage(column_name="response_text")

# With domain-specific whitelist
oov_feature = OOVWordsPercentage(
    column_name="medical_notes",
    ignore_words=("covid", "mrna", "pcr"),
    display_name="OOV % (Medical)",
)

# Use within an Evidently column mapping or report
# result = oov_feature.generate_feature(data=df, data_definition=data_def)

Related Pages

• Environment:Evidentlyai_Evidently_Python_Core_Environment
• Implementation:Evidentlyai_Evidently_Legacy_Generated_Features - Base class ApplyColumnGeneratedFeature that OOVWordsPercentage extends.
• Implementation:Evidentlyai_Evidently_Legacy_Contains_Link_Feature - Another text quality feature using the same ApplyColumnGeneratedFeature pattern.