CLI appResearch and knowledge

webclaw

Extract websites into clean Markdown, JSON, or LLM-ready context through CLI, MCP, or API.

Stars
1,729
Forks
181
License
AGPL-3.0
Updated
Updated Jul 13, 2026

Checking live repository facts…

Project overview

webclaw provides a local-first way to scrape, crawl, map, and extract websites for agents and RAG systems. Its core CLI and MCP workflow can run without an account for many pages, while a hosted API is available for bot-protected sites, JavaScript rendering, asynchronous crawl jobs, search, and production monitoring.

Repository facts

Primary language
Rust
License
AGPL-3.0
Repository updated
Jul 13, 2026
Default branch
main

Resource types

CLI appAPI or serverMCP serverSDK

Use cases

Research and knowledge

Platforms

Claude CodeCursorCodex

Runtime

Command lineCloudLocal

Protocols & integrations

Model Context ProtocolREST

Capabilities

Web search

Audience

Developers

Related projects

Browse more similar projects