Implementation:CrewAIInc CrewAI Jina Scrape Tool
| Knowledge Sources | |
|---|---|
| Domains | Tools, Web_Scraping, Jina |
| Last Updated | 2026-02-11 00:00 GMT |
Overview
Scrapes websites using Jina.ai's reader service and returns clean markdown content.
Description
JinaScrapeWebsiteTool extends BaseTool to provide lightweight web scraping through the Jina.ai reader API. It makes GET requests to https://r.jina.ai/{url} with a configurable 15-second timeout and returns the response as markdown text. The tool supports an optional website_url for fixed-URL mode or dynamic URL specification at runtime. Custom headers can be provided via the custom_headers constructor parameter, and an optional api_key is set as a Bearer token in the Authorization header. In fixed-URL mode, the tool description is automatically updated to reference the configured URL.
Usage
Use this tool when a CrewAI agent needs a simple, lightweight alternative to complex scraping solutions for extracting clean markdown content from web pages.
Code Reference
Source Location
- Repository: CrewAI
- File:
lib/crewai-tools/src/crewai_tools/tools/jina_scrape_website_tool/jina_scrape_website_tool.py - Lines: 1-50
Signature
class JinaScrapeWebsiteTool(BaseTool):
name: str = "JinaScrapeWebsiteTool"
description: str = "A tool that can be used to read a website content using Jina.ai reader ..."
args_schema: type[BaseModel] = JinaScrapeWebsiteToolInput
website_url: str | None = None
api_key: str | None = None
headers: dict = Field(default_factory=dict)
def __init__(self, website_url: str | None = None, api_key: str | None = None,
custom_headers: dict | None = None, **kwargs): ...
def _run(self, website_url: str | None = None) -> str: ...
Import
from crewai_tools import JinaScrapeWebsiteTool
I/O Contract
Inputs
| Name | Type | Required | Description |
|---|---|---|---|
| website_url | str | Yes (at init or runtime) | URL of the website to scrape |
| api_key | str | No | Jina.ai API key for authenticated requests (constructor param) |
| custom_headers | dict | No | Custom HTTP headers to include in requests (constructor param) |
Outputs
| Name | Type | Description |
|---|---|---|
| _run() returns | str | Markdown text content extracted from the web page |
Usage Examples
Dynamic URL
from crewai_tools import JinaScrapeWebsiteTool
tool = JinaScrapeWebsiteTool()
content = tool.run(website_url="https://example.com/article")
Fixed URL with API Key
from crewai_tools import JinaScrapeWebsiteTool
tool = JinaScrapeWebsiteTool(
website_url="https://example.com/docs",
api_key="your-jina-api-key",
)
content = tool.run()