Implementation:Ucbepic Docetl ScraperPage

Knowledge Sources	Ucbepic_Docetl
Domains	Frontend, React_UI
Last Updated	2026-02-08 00:00 GMT

Overview

Concrete tool for the AI-powered web scraper page in the DocETL frontend.

Description

This page component provides a full-featured scraper interface with a resizable three-panel layout: a chat panel for conversing with the AI scraper agent, a code execution panel showing Python scripts being run in sandboxed environments, and a dataset panel displaying collected results. It uses the Vercel AI SDK's useChat hook to communicate with the /api/scraper route and manages state for code executions, found links, sessions, and dataset refinement.

Usage

Rendered at the /scraper URL path. Users describe what data they want to collect, and the AI agent iteratively writes and executes scraping code, displaying results in real time.

Code Reference

Source Location

• Repository: Ucbepic_Docetl
• File: website/src/app/scraper/page.tsx
• Lines: 1-998

Signature

export default function ScraperPage(): JSX.Element

// Key interfaces:
interface CodeExecution {
  id: string;
  code: string;
  status: "running" | "completed" | "error";
  output?: string;
  error?: string;
  timestamp: Date;
}

interface FoundLink {
  title: string;
  url: string;
  snippet: string;
  score?: number;
  timestamp: Date;
}

Import

// This is a Next.js page component, auto-routed at /scraper

I/O Contract

Inputs (Props)

Name	Type	Required	Description
No props -- this is a page component

Outputs

Name	Type	Description
rendered	JSX.Element	Three-panel scraper UI with chat, code, and dataset views

Usage Examples

// Accessed via browser at /scraper
// The page manages its own state via useChat and useState hooks

Related Pages

• Environment:Ucbepic_Docetl_Frontend_Node_Environment