Implementation:CrewAIInc CrewAI BrightData Dataset Tool

Knowledge Sources	CrewAI
Domains	Data Extraction, Web Scraping, Tool Integration
Last Updated	2026-02-11 00:00 GMT

Overview

BrightDataDatasetTool is a CrewAI tool that extracts structured data from 30+ popular platforms (Amazon, Walmart, eBay, LinkedIn, Instagram, Facebook, TikTok, and others) using BrightData's pre-built dataset APIs.

Description

The tool provides cached, structured data retrieval from major e-commerce, social media, and business platforms without requiring agents to implement scraping logic, handle bot protection, or manage rate limits. It works by triggering BrightData dataset collection jobs via their API, polling for completion, and returning results in configurable formats (JSON, NDJSON, JSONL, CSV).

The module defines four key classes:

• BrightDataConfig -- Loads API URL, timeout, and polling interval from environment variables with sensible defaults (600s timeout, 1s polling interval).
• BrightDataDatasetToolException -- Custom exception with error code support for API failures.
• BrightDataDatasetToolSchema -- Pydantic schema requiring dataset_type and url, with optional format, zipcode, and additional_params.
• BrightDataDatasetTool -- The main tool class extending BaseTool, which manages the async scraping workflow.

The tool maintains a comprehensive registry of dataset configurations mapping friendly IDs (e.g., "amazon_product", "linkedin_person_profile") to BrightData dataset IDs and their required inputs. The async workflow uses aiohttp to trigger jobs, poll for status, and retrieve results.

Usage

Use this tool when agents need structured data from supported platforms such as Amazon product data, LinkedIn profiles, Instagram posts, Facebook marketplace listings, or TikTok shop information. It is particularly valuable for e-commerce analysis, social media monitoring, competitive intelligence, and lead generation workflows.

Code Reference

Source Location

• Repository: CrewAI
• File: lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_dataset.py
• Lines: 1-601

Signature

class BrightDataDatasetTool(BaseTool):
    name: str = "Bright Data Dataset Tool"
    description: str = "Scrapes structured data using Bright Data Dataset API from a URL and optional input parameters"
    args_schema: type[BaseModel] = BrightDataDatasetToolSchema
    dataset_type: str | None = None
    url: str | None = None
    format: str = "json"
    zipcode: str | None = None
    additional_params: dict[str, Any] | None = None
    env_vars: list[EnvVar]  # Requires BRIGHT_DATA_API_KEY

    def __init__(
        self,
        dataset_type: str | None = None,
        url: str | None = None,
        format: str = "json",
        zipcode: str | None = None,
        additional_params: dict[str, Any] | None = None,
        **kwargs: Any,
    ): ...

Import

from crewai_tools.tools.brightdata_tool.brightdata_dataset import BrightDataDatasetTool

I/O Contract

Inputs

Name	Type	Required	Description
dataset_type	str	Yes	The BrightData Dataset Type ID (e.g., "amazon_product", "linkedin_person_profile")
url	str	Yes	The URL from which structured data needs to be extracted
format	str	No	Response format: "json" (default), "ndjson", "jsonl", or "csv"
zipcode	str	No	Optional ZIP code to narrow down data geographically
additional_params	dict[str, Any]	No	Extra parameters for the BrightData API call

Outputs

Name	Type	Description
return	str	The structured dataset result from BrightData in the requested format, or an error message string if the operation fails

Supported Dataset Types

Category	Dataset IDs
E-commerce	amazon_product, amazon_product_reviews, amazon_product_search, walmart_product, walmart_seller, ebay_product, homedepot_products, zara_products, etsy_products, bestbuy_products
Social/Professional	linkedin_person_profile, linkedin_company_profile, linkedin_job_listings, linkedin_posts, linkedin_people_search, crunchbase_company, zoominfo_company_profile
Social Media	instagram_profiles, instagram_posts, instagram_reels, instagram_comments, facebook_posts, facebook_marketplace_listings, facebook_company_reviews, facebook_events, tiktok_profiles, tiktok_posts, tiktok_shop

Usage Examples

Basic Usage

import os
os.environ["BRIGHT_DATA_API_KEY"] = "your-api-key"

from crewai_tools.tools.brightdata_tool.brightdata_dataset import BrightDataDatasetTool

# Extract Amazon product data
tool = BrightDataDatasetTool(
    dataset_type="amazon_product",
    url="https://www.amazon.com/dp/B09V3KXJPB"
)
result = tool._run()

# Extract LinkedIn profile data with runtime parameters
tool = BrightDataDatasetTool()
result = tool._run(
    dataset_type="linkedin_person_profile",
    url="https://www.linkedin.com/in/example-user/",
    format="json"
)