I built a local, keyless Firecrawl for Claude Code - here's why published: false

Why I built it

A language model pays for every token of nav, ads, and footer it reads. That one sentence is the whole reason tearsheet exists.

I do a lot of research work inside Claude Code, and a huge chunk of it is "go read this page and tell me what it says." Firecrawl-style tools are genuinely great at that - but they're SaaS. API keys, quotas, a bill, and your URLs leaving your machine to get there. I wanted the same toolset, running entirely on my own laptop, feeding my agent clean content without any of that. So I built it.

It's called tearsheet, it's MIT-licensed, and this is the story of why.

tearsheet (n.): a page torn from a publication and filed as proof it ran. That definition is on the repo for a reason. The whole tool is built around the idea that what it hands back should be trustworthy enough to file as evidence. More on that below - it's the part I care about most.

Two things pushed me over the edge. Tokens are money. When you scrape a page and dump the raw HTML - or even a naive "readable" conversion - into a model's context, you're paying for the cookie banner, the mega-menu, the newsletter modal, and the six-deep footer. On a research run that fans out across dozens of pages, that waste compounds fast. I wanted a tool whose first principle was "return the least text that fully answers the question."

I didn't want a middleman. No API key to manage, no service to trust with my browsing, no telemetry, no rate limit that isn't mine. Just a thing on my machine that Claude Code can call. Everything tearsheet does - fetch, extract, cache, crawl - happens locally.

What it is

tearsheet is an MCP server exposing five tools. If you've used Firecrawl, these will feel familiar:

• scrape - one URL in, clean main-content markdown out.
• search - keyless metasearch (no search API key required).
• map - list every URL on a site without scraping it.
• crawl - walk a site and write each page to disk as markdown.
• extract - pull structured data (JSON-LD, OpenGraph, tables) as JSON, no LLM involved.

The intended flow is deliberately cheap: map → pick the URLs that matter → scrape those, or crawl → read the files it wrote. You never blast a whole site into context to find the three pages you actually needed.

Three design choices do the token-saving work:

• Main-content only. Extraction runs through trafilatura, so nav/ads/chrome are gone before anything reaches the model.
• Disk spillover instead of truncation-by-guessing. When a page is bigger than your limit, tearsheet writes the full copy to ~/.tearsheet/pages/ and prints the path. The model reads the file if it needs more, instead of you re-scraping with a bigger cap.
• Crawl returns an index, not bodies. A crawl never dumps page content into the tool output - it writes files and hands back a compact index (filename, ~token count, title, path). The consuming model decides what to actually read.

Under the hood it's fastmcp for the server, trafilatura for extraction, ddgs for keyless search, extruct + lxml for structured data, and a lazy-loaded Playwright Chromium fallback for the JavaScript-only pages that a plain fetch can't see.

The part I'm proudest of: it refuses to lie

Here's the thing that makes tearsheet different from a weekend scraper. Every extractor fails sometimes. The question is how. tearsheet's failure mode is omission, never fabrication - and every guard in it exists to make omission loud instead of silent.

A tool that quietly drops half a pricing table is far more dangerous than one that says "I couldn't read this cleanly," because you'll cite the clean-looking-but-wrong version without a second thought. So tearsheet is built to be noisy about its own blind spots:

• warning: lines in the header. Every scrape result starts with a header (url, title, token count, warnings) before the content. A warning: line means the extraction is known-unreliable - don't quote figures from it, re-run with raw=true.
• It won't cache junk. If a page serves a cookie/consent wall or a bot-protection challenge instead of content, tearsheet detects it, says so plainly, and refuses to cache it - so a poisoned result can't get replayed later.
• A price guard. Commercial and tabular pages are where extractors quietly bleed data - the markdown looks clean while dollar figures vanish. tearsheet counts the distinct money figures on the page versus in the extraction and warns you when too many went missing. This got calibrated on real pages that broke it: one pricing page came through with only 4 of its 24 prices and a flattened three-column matrix. That's exactly the silent failure the guard now catches.
• A raw=true escape hatch. When you don't trust the clean extraction, raw skips trafilatura entirely and hands back the visible page text so you can see for yourself.

The design goal - and the track record so far - is zero fabricated content. It omits, and it tells you when it did.

Install it and wire it into Claude Code

Fair warning: this is early. It's not on PyPI yet - that's deliberate, I'm still refining it and I'd rather ship it clean than ship it fast. For now it's git-clone-only. If you don't mind a little rough, here's the whole setup.

Clone and install (requires Python ≥ 3.12):

git clone https://github.com/Wynelson94/tearsheet.git
cd tearsheet
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"

Optional, for JavaScript-rendered pages:

playwright install chromium

(Without it, tearsheet degrades gracefully and tells you when a page needed rendering.)

Then register it with Claude Code as an MCP server:

claude mcp add --scope user tearsheet -- ~/Projects/tearsheet/.venv/bin/tearsheet-mcp

That's it. Restart Claude Code and you've got scrape, search, map, crawl, and extract available as tools.

Where it's still rough

I'd rather tell you the weak spots than let you find them. Commercial and tabular pages are the hard case - the whole price-guard system exists because that's where extraction is fragile. And some figures are invisible to any non-interactive fetch: prices rendered client-side in a JavaScript payload, literal "null" placeholders, or numbers drawn inside an image.

The rule I follow is procedural, not optimistic: any figure I'm going to quote gets raw or independent verification.

To keep myself honest about all this, there's a falsifiable eval harness in the repo (evals/) - a corpus weighted toward the commercial/tabular pages that are hardest, scored against an independent oracle, with a verdict that refuses to exist if the corpus isn't reachable. Neuter the guards and the verdict goes red; restore them and it goes green. The pages that once broke it are pinned as offline fixtures, so every future change has to answer to the original failures forever. There are 240+ tests, and CI runs fully offline on Python 3.12 and 3.13.

Clone it if you want it

That's tearsheet: a local, keyless, no-telemetry Firecrawl alternative built to feed Claude Code clean content cheaply - and built to be loud about the moments it can't.

• Repo: https://github.com/Wynelson94/tearsheet
• License: MIT
• Status: early, git-clone-only, not on PyPI yet

If you try it, I genuinely want the feedback - issues and bug reports especially. The whole point of the guards is that failures make the tool better, so if it breaks on a page, that's a finding, not a bug I'm embarrassed by.

Go tear some pages out of the internet.