Implementation:Lm sys FastChat Filter Bad Conv Arena33k

Knowledge Sources	Lm_sys_FastChat
Domains	Data_Processing, Model_Evaluation
Last Updated	2026-02-07 06:00 GMT

Overview

Filters and classifies conversations for the Arena 33K dataset release by detecting bad formatting, anonymized content, redacted text, blocked words, and other quality issues.

Description

Filter Bad Conv Arena33k is a data quality gate used during the preparation of the Arena 33K public dataset release. It examines each conversation and assigns a TypeCode classification indicating whether the conversation is suitable for release or should be excluded for a specific reason. The module enforces multiple exclusion criteria to ensure the released dataset contains only clean, well-formatted, and appropriate conversations.

The TypeCode enum defines eight classification categories: CORRECT (suitable for release), ANONYMIZED (contains anonymized personal information markers), REDACTED (contains redacted content), BAD_FORMAT (malformed conversation structure), BLOCKED_WORD (contains prohibited terms), BLOCKED_MODEL (involves a model excluded from the release), TOO_SHORT (insufficient conversation length), and TOO_FREQUENT (likely a bot or automated query). Each conversation is passed through the detect_type function which applies these checks in sequence and returns the first matching TypeCode.

This module is part of a dataset release pipeline and is specifically tailored to the Arena 33K dataset. A separate but similar module exists for the LMSYS Chat 1M dataset with additional language handling capabilities.

Usage

Use this module when preparing the Arena 33K dataset for public release. Run it over the full corpus of conversations to classify each entry, then filter to retain only those with TypeCode.CORRECT. It should be used after initial data cleaning and before final dataset packaging.

Code Reference

Source Location

• Repository: Lm_sys_FastChat
• File: fastchat/serve/monitor/dataset_release_scripts/arena_33k/filter_bad_conv.py
• Lines: 1-155

Signature

class TypeCode(Enum):
    CORRECT = 0
    ANONYMIZED = 1
    REDACTED = 2
    BAD_FORMAT = 3
    BLOCKED_WORD = 4
    BLOCKED_MODEL = 5
    TOO_SHORT = 6
    TOO_FREQUENT = 7

def detect_type(conv: dict) -> TypeCode:
    """Classify a conversation by detecting quality issues and returning the appropriate TypeCode."""

Import

from fastchat.serve.monitor.dataset_release_scripts.arena_33k.filter_bad_conv import detect_type

I/O Contract

Inputs

Name	Type	Required	Description
conv	dict	Yes	A conversation dictionary containing messages, model identifiers, metadata, and conversation ID

Outputs

Name	Type	Description
type_code	TypeCode	An enum value indicating the classification of the conversation: CORRECT if it passes all checks, or the specific failure reason

TypeCode Classification

TypeCode	Value	Description
CORRECT	0	Conversation passes all quality checks and is suitable for release
ANONYMIZED	1	Conversation contains anonymization markers (e.g., "[NAME]", "[EMAIL]")
REDACTED	2	Conversation contains redacted content placeholders
BAD_FORMAT	3	Conversation has malformed structure or missing required fields
BLOCKED_WORD	4	Conversation contains prohibited words or phrases
BLOCKED_MODEL	5	Conversation involves a model excluded from the dataset release
TOO_SHORT	6	Conversation is too brief to be useful
TOO_FREQUENT	7	Conversation prompt appears too frequently, suggesting automated or bot traffic

Usage Examples

from fastchat.serve.monitor.dataset_release_scripts.arena_33k.filter_bad_conv import (
    detect_type,
    TypeCode,
)

# Classify a single conversation
conv = {
    "conversation_id": "abc123",
    "model": "gpt-4",
    "conversation": [
        {"role": "user", "content": "What is the capital of France?"},
        {"role": "assistant", "content": "The capital of France is Paris."},
    ],
}

result = detect_type(conv)
if result == TypeCode.CORRECT:
    print("Conversation is suitable for release")
else:
    print(f"Excluded: {result.name}")

# Batch filtering for dataset release
import json

clean_conversations = []
with open("arena_conversations.jsonl") as f:
    for line in f:
        conv = json.loads(line)
        if detect_type(conv) == TypeCode.CORRECT:
            clean_conversations.append(conv)

print(f"Retained {len(clean_conversations)} conversations for release")

Related Pages

• Principle:Lm_sys_FastChat_Conversation_Content_Filtering
• Implements: Principle:Lm_sys_FastChat_Conversation_Content_Filtering
• Lm_sys_FastChat_Filter_Bad_Conv_Chat1M - Similar filtering for the LMSYS Chat 1M dataset with Chinese language support
• Lm_sys_FastChat_Clean_Chat_Data - Upstream data cleaning and deduplication
• Lm_sys_FastChat_Deduplication - High-frequency prompt deduplication used to identify TOO_FREQUENT entries