Implementation:CrewAIInc CrewAI BrightData Unlocker Tool

Knowledge Sources	CrewAI
Domains	Tools, Web_Scraping, BrightData
Last Updated	2026-02-11 00:00 GMT

Overview

Concrete tool for scraping website content using Bright Data's Web Unlocker API provided by CrewAI.

Description

The BrightDataWebUnlockerTool class extends BaseTool to perform web scraping by routing requests through Bright Data's unlocking and proxy infrastructure, which can bypass bot protection mechanisms like CAPTCHAs, geo-restrictions, and anti-bot detection. The tool sends POST requests to the BrightData API endpoint (https://api.brightdata.com/request) with the target URL, zone identifier, response format (raw), and optional data format conversion. It supports two data formats: html (default) and markdown. The tool validates the BRIGHT_DATA_API_KEY and BRIGHT_DATA_ZONE environment variables during initialization, and includes comprehensive error handling that includes response text in error messages for debugging.

Usage

Use this tool when CrewAI agents need to reliably extract content from protected websites that employ anti-scraping measures, CAPTCHAs, or geo-restrictions, without implementing complex bot evasion logic.

Code Reference

Source Location

• Repository: CrewAI
• File: lib/crewai-tools/src/crewai_tools/tools/brightdata_tool/brightdata_unlocker.py
• Lines: 1-146

Signature

class BrightDataWebUnlockerTool(BaseTool):
    name: str = "Bright Data Web Unlocker Scraping"
    description: str = "Tool to perform web scraping using Bright Data Web Unlocker"
    args_schema: type[BaseModel] = BrightDataUnlockerToolSchema
    base_url: str = ""
    api_key: str = ""
    zone: str = ""
    url: str | None = None
    format: str = "raw"
    data_format: str = "markdown"

    def __init__(self, url=None, format="raw", data_format="markdown", **kwargs): ...
    def _run(self, url=None, format=None, data_format=None, **kwargs) -> Any: ...

Import

from crewai_tools import BrightDataWebUnlockerTool

I/O Contract

Inputs

Name	Type	Required	Description
url	str	Yes	Target URL to scrape (constructor or runtime)
format	str or None	No	Response format from BrightData (default "raw")
data_format	str or None	No	Response data format: "html" or "markdown" (default "markdown")

Outputs

Name	Type	Description
_run() returns	str	Scraped web page content in the requested format (HTML or markdown)

Usage Examples

Basic Usage

from crewai_tools import BrightDataWebUnlockerTool

tool = BrightDataWebUnlockerTool(data_format="markdown")
content = tool.run(url="https://example.com/protected-page")

Related Pages

• Principle:CrewAIInc_CrewAI_Built_In_Tool_Selection