webclaw MCP Server: Setup for Claude, Cursor and Other Apps

• Turns a web page into clean text

  Fetches one URL and returns its content as markdown or another format. It can also render the page and return the JSON a single-page app loads from its own API.

  Try: “Get the main text of https://example.com/pricing as markdown.”

  Network · scrape

• Reads a whole site from one starting page

  Follows links from a seed URL, breadth first, up to a depth and page limit you set. With a question, it follows the most likely links first and cuts each page to the parts that answer it.

  Try: “Crawl docs.example.com 2 levels deep and find how rate limits work.”

  Network · crawl

• Picks parts of a page by selector

  Fetches a page and returns only the elements you point to, or its tables. Pages that need JavaScript are rendered first when the server has browser support.

  Try: “Pull all tables from this page as JSON.”

  Network · select

• Saves a page as image, PDF or archive

  Opens the page in your local Chrome and saves a PNG or JPEG, a PDF up to 10 MiB, or an MHTML archive. Needs Chrome on your machine.

  Try: “Take a full-page screenshot of https://example.com.”

  Runs code · screenshot

• Lists the URLs of a website

  Finds a site's pages from robots.txt and sitemaps, and adds links from a short crawl when sitemaps are small. It can also use the public Common Crawl index.

  Try: “List all pages under example.com/blog.”

  Network · map

• Reads many pages at once

  Fetches a list of URLs at the same time and returns each result.

  Try: “Get the text of these 10 URLs.”

  Network · batch

• Pulls structured data out of a page

  Uses a language model to fill the fields you describe from a page. It needs a local Ollama or a provider key, and falls back to the webclaw cloud API when none is available.

  Try: “Extract product name and price from this page as JSON.”

  Network · extract

• Summarizes a web page

  Writes a short summary of a page with a language model. It falls back to the webclaw cloud API when no local model is available or the site blocks bots.

  Try: “Summarize https://example.com/report in five bullets.”

  Network · summarize

• Shows what changed on a page

  Compares a page with an earlier snapshot and reports the changes. It falls back to the cloud API when bot protection is detected.

  Try: “What changed on this pricing page since my last snapshot?”

  Network · diff

• Finds a site's brand details

  Reads a site and returns brand details such as name and colors. It falls back to the cloud API when bot protection is detected.

  Try: “Get the brand colors and logo of stripe.com.”

  Network · brand

• Runs a deep web research job

  Researches a question across the web using the webclaw cloud. It needs WEBCLAW_API_KEY, costs credits, and returns a cached answer for a repeated query.

  Try: “Research the main alternatives to Notion for teams.”

  Network · research

• Searches the web

  Runs a web search with your Serper key if SERPER_API_KEY is set, or with the webclaw cloud when WEBCLAW_API_KEY is set. With neither, it returns an error explaining both options.

  Try: “Search the web for the latest news about the Rust 2024 edition.”

  Network · search

• Reads a known site with its own reader

  Pulls clean data from a supported site using its site-specific reader. Some readers need WEBCLAW_API_KEY.

  Try: “Get the details of this GitHub repo.”

  Network · vertical_scrape

• Used by the assistant behind the scenes

  Lists site-specific readers

  Shows the built-in readers for specific sites such as Amazon, GitHub, Reddit, YouTube, npm and PyPI.

  Try: “Which sites have a special reader?”

  Read · list_extractors

Watch out for

• It downloads and runs a program on first use

  npx fetches a prebuilt webclaw-mcp binary from GitHub releases and caches it in ~/.cache/webclaw. Set WEBCLAW_MCP_BIN to a binary you built if you prefer. github.com

• Some tools send pages to the webclaw cloud

  extract, summarize, diff and brand fall back to the cloud API when no local model is set or a site blocks bots. Do not use them on private pages you do not want to leave your machine. github.com

• The research tool spends credits

  research needs a webclaw key and uses account credits. Repeated queries return the cached answer. github.com

Security advisories