crawl4ai

Repository

Crawls websites to extract Markdown and structured data, with browser automation for dynamic pages.

Overview

Use case: Source-backed content preparation

A web crawler and scraper that produces Markdown for language models, retrieval systems, and data pipelines. It also supports structured extraction, browser automation, and filtering that removes navigation and other page clutter.

Developers can run the Python library, use a Docker server, or call the hosted API. Browser controls, batch crawling, and schema extraction help turn dynamic websites into content that applications can process.

Source-based overview · Not hands-on tested.

Good for

Prepare website Markdown for retrieval pipelines
Extract page data with CSS or XPath schemas
Crawl pages that require JavaScript
Capture page screenshots and PDFs

Price

Cloud: 1,000 free credits a month; paid plans from $10 a month

Prices as stated in the README. Self-hosting costs are not stated.

Setup

Technical setup (inferred)
Install the Python package with pip, then run crawl4ai-setup to prepare its browser. Alternatively, obtain a key for the hosted API.

Change history

No substantive changes recorded. A successful source check alone is not a change.

Links work · Not hands-on tested · Recently active: source activity within 14 days · Updated: recorded change this week

Compare all candidates →

Licence, sources & verification

Listing facts

Apache-2.0

Listing facts
Source	https://github.com/unclecode/crawl4ai
Author credit	unclecode on GitHub
Licence	Apache-2.0
Use case / type	Source-backed content preparation · repository

Open action link

Evidence

Source-discovered · Not hands-on tested · Links not checked

Source activity date: 22 Oct 2026
Checked: 22 Oct 2026 · Imported from a source snapshot; not independently rechecked
Source-metadata observation: Not recorded

Source: Not checked; availability unconfirmed

Workflow context

Intended task: Prepare a draft, inspect its claims, and keep the receipts.
Setup: Technical setup (inferred)
Install the Python package with pip, then run crawl4ai-setup to prepare its browser. Alternatively, obtain a key for the hosted API.
Prerequisites: Review repository setup instructions, runtime, model access and API credentials; exact requirements are not established here.
Limitations: Source checking does not prove functionality. No hands-on test was performed. Capabilities and availability are unconfirmed.
Price: Cloud: 1,000 free credits a month; paid plans from $10 a month
Prices as stated in the README. Self-hosting costs are not stated.

Attribution & reuse

Source: unclecode on GitHub.

Repository licence: Apache-2.0 (check the repository for scope).

Changes made: summary paraphrased from the repository description with AI assistance, plus task tags; no README text, code or media reproduced.

Recorded modifications: summary written by TaskToTool in its own words from the repository's GitHub description; reviewed against the live source on 2026-10-22.

Editorial provenance

Summary written by TaskToTool from public source information; not hands-on tested.

Open-source crawler that turns websites into Markdown for LLMs and AI agents; run it yourself or through Crawl4AI Cloud.

Not hands-on tested.

AI-assisted drafting; reviewed by TaskToTool

Reviewed by TaskToTool Content Editor · 22 Oct 2026

How we write and verify listings: AI assistance, evidence labels and dates