BROWSER RENDERING IS THE BEST THING CLOUDFLARE SHIPPED IN 2025

I was evaluating Firecrawl. Then I found Cloudflare’s Browser Rendering API. The /markdown endpoint does what I needed. One REST call. URL in, clean markdown out. Ready for embedding.

I’ve been using it in two contexts. A RAG content pipeline that converts web pages to markdown for embedding into Vectorize. And an AI agent on Durable Objects that uses Browser Rendering for screenshots, content extraction, login flows, and authenticated scraping. Different patterns for different complexity levels. Both work well.

Worth walking through the architecture, the Firecrawl comparison, and the gaps I’ve hit in production.

How I use Browser Rendering today

The pipeline is straightforward. A URL comes in from a queue. Browser Rendering’s /markdown endpoint renders the page, strips the styling and scripts, returns clean markdown. That markdown gets chunked and embedded into Vectorize. One fetch call replaces what used to be a Firecrawl API call at a fraction of the cost.

# This is the entire Firecrawl replacement
curl -X POST 'https://api.cloudflare.com/client/v4/accounts/{account_id}/browser-rendering/markdown' \
  -H 'Authorization: Bearer {cf_token}' \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://example.com"}'

That “one fetch call” framing holds for static and server-rendered content. JavaScript-heavy SPAs with client-side rendering are less predictable. The results depend on whether the content has hydrated by the time extraction runs. The waitUntil and waitForSelector options help but you need to know they exist and tune them per site. For non-trivial SPAs it’s not quite as turnkey as the docs suggest.

The agent is more interesting. It has two browser tools that coexist based on task complexity.

browser_render is the stateless tool. It uses the BROWSER binding as a Fetcher. No Puppeteer import. No CDP. The Durable Object calls env.BROWSER.fetch() against the REST endpoints. /markdown, /screenshot, /content, /scrape, /json. Each call spins up a browser, does one thing, returns the result, browser dies. Covers 90% of what I need.

Stateless path:
Agent DO → env.BROWSER.fetch('/markdown') → Cloudflare Browser Rendering → result

browser_session is the stateful tool. It uses @cloudflare/puppeteer with puppeteer.launch(env.BROWSER) directly inside the Durable Object. The browser session stays alive across multiple tool calls within a conversation. The DO holds this.browser and this.page as instance properties. Twelve actions. Open, navigate, click, type, select, screenshot, content, text, evaluate, wait, scroll, close. Session lifecycle managed by DO alarms with a five minute idle timeout.

Stateful path:
Agent DO → puppeteer.launch(env.BROWSER) → persistent browser on CF edge
         → reuse across tool calls → close on conversation end

This replaced a container-to-Worker-to-browser CDP proxy chain. Three network hops collapsed to one. No container running 24/7 as the orchestrator. The DO talks directly to Browser Rendering via the Puppeteer binding.

The cost comparison against Firecrawl is significant. Browser Rendering bills at $0.09 per browser-hour after 10 free hours per month. At 10,000 pages per month through the pipeline that works out to roughly $0.36. Firecrawl’s Growth plan charges $99 for the same volume. At scale the gap widens further because Firecrawl’s per-page economics don’t improve linearly while Browser Rendering’s browser-hour model does.

Managing concurrent browsers at scale

Paid accounts get 30 concurrent browser instances with a limit of 30 new instances per minute. For single-page scraping workflows that’s plenty of headroom. For crawl workloads pushing hundreds of URLs through a queue with multiple consumers, you need to think about how you manage it.

The good news is that 30 concurrent browsers is real headroom. The question is how you manage it across workloads. If you’re using the REST API via fetch() for your queue consumers, concurrency management happens at the queue level through max_concurrency in wrangler.jsonc, not per-message. If you’re using the Puppeteer binding directly, puppeteer.limits(env.BROWSER) returns allowedBrowserAcquisitions so you can check headroom before each acquisition.

For my pipeline the REST API path is simpler. The queue consumer just calls /markdown and lets queue-level concurrency do the throttling.

// wrangler.jsonc: "max_concurrency": 10 (leaving headroom for agent browser sessions)
export default {
  async queue(batch, env) {
    for (const msg of batch.messages) {
      const md = await fetch(`${BR_API}/markdown`, {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${token}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ url: msg.body.url }),
      });
      await env.R2.put(`crawl/${encodeURIComponent(msg.body.url)}.md`, await md.text());
    }
  }
};

The max_concurrency setting is the lever. Set it lower than 30 to leave room for other workloads that need browser instances, like the agent’s stateful sessions. The tradeoff is throughput versus headroom and you tune it based on your mix.

What would still help is clearer documentation on what happens when you exceed the limit. Do REST API calls queue silently? Fail immediately? Queue with a timeout? The answer matters for retry strategy and the docs don’t spell it out.

A /crawl endpoint would close the biggest gap

The /markdown endpoint replaces Firecrawl for single pages perfectly. But Firecrawl’s /crawl does something Browser Rendering can’t. Follow links, depth-limited, entire site returned as an array of markdown pages. No equivalent exists on Cloudflare.

Today I build it myself every time.

// Step 1: Get all links on the page, filtered to same domain
const linksResponse = await fetch(`${BR_API}/links`, {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${token}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ url: siteUrl, excludeExternalLinks: true }),
});
const { result: links } = await linksResponse.json();

// Step 2: Push to Cloudflare Queue
for (const url of links.slice(0, maxPages)) {
  await env.CRAWL_QUEUE.send({ url, depth: currentDepth + 1 });
}

// Step 3: Queue consumer calls /markdown for each
export default {
  async queue(batch, env) {
    for (const msg of batch.messages) {
      const md = await fetch(`${BR_API}/markdown`, {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${token}`,
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({ url: msg.body.url }),
      });
      await env.R2.put(`crawl/${encodeURIComponent(msg.body.url)}.md`, await md.text());
    }
  }
};

Glue code that everyone building on Browser Rendering writes independently. The infrastructure is already there. The /links endpoint discovers URLs. The excludeExternalLinks parameter handles domain filtering server-side. The /markdown endpoint converts pages. Queues handle orchestration. Wire them together under the hood and expose it as a single REST call.

# What this should look like
curl -X POST 'https://api.cloudflare.com/client/v4/accounts/{id}/browser-rendering/crawl' \
  -H 'Authorization: Bearer {cf_token}' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://docs.example.com",
    "maxDepth": 3,
    "limit": 100,
    "sameDomain": true,
    "format": "markdown"
  }'

The primitives exist. The composition is the same across every implementation. This is the textbook case for platformizing a common pattern.

Session management needs polish

Two issues here. Reconnection gives you no signal when it fails. And getting a session ID after launch requires a round trip that shouldn’t be necessary.

Starting with reconnection. puppeteer.connect(env.BROWSER, sessionId) to reconnect to an existing session is fragile. Sometimes the session is gone with no clear signal. No way to distinguish “session expired” from “session crashed” from “session was cleaned up by the platform.” You get a generic connection failure.

This is my current reconnection logic.

async ensureBrowser(env: AgentEnv): Promise<{ browser: Browser; page: Page }> {
  // Try to reuse active session
  if (this.browser?.isConnected()) {
    this.lastBrowserActivity = Date.now();
    return { browser: this.browser, page: this.page! };
  }

  // Try to reconnect to existing session
  if (this.browserSessionId) {
    try {
      this.browser = await puppeteer.connect(env.BROWSER, this.browserSessionId);
      const pages = await this.browser.pages();
      this.page = pages[0] || await this.browser.newPage();
      this.lastBrowserActivity = Date.now();
      return { browser: this.browser, page: this.page };
    } catch {
      // Why did it fail? No idea. Session expired? Crashed? Cleaned up?
      // Silent fallback to new session. State is lost.
      this.browserSessionId = null;
    }
  }

  // Launch new session
  this.browser = await puppeteer.launch(env.BROWSER, {
    keep_alive: this.BROWSER_KEEP_ALIVE_MS
  });
  this.page = await this.browser.newPage();
  this.lastBrowserActivity = Date.now();

  // Capture session ID for future reconnection
  // puppeteer.launch() doesn't return the session ID directly,
  // so we round-trip through sessions() to find ours.
  // Note: sessions() is account-scoped, not DO-scoped. We match on
  // connectionId which works when this DO owns one browser at a time.
  // At scale with many DOs each running browsers, this is a race.
  const sessions = await puppeteer.sessions(env.BROWSER);
  const mySession = sessions.find(s => s.connectionId);
  if (mySession) {
    this.browserSessionId = mySession.sessionId;
  }

  return { browser: this.browser, page: this.page };
}

Every reconnect attempt is wrapped in try/catch with a silent fallback. That means lost authentication state, lost navigation history, lost cookies. Zero visibility into why.

The session ID capture pattern illustrates the broader issue. You launch a browser, then immediately query the sessions list to find the one you just created because puppeteer.launch() doesn’t expose .sessionId on the returned browser object. In a DO where you own exactly one browser at a time, finding the session with a connectionId works. But sessions() is account-scoped, not DO-scoped. At scale with multiple DOs each running their own browser, you could match another DO’s session. It’s a race condition that’s unlikely at low concurrency but real at high concurrency. Another reason puppeteer.launch() should just return the session ID directly.

A status endpoint and distinct error codes would address the reconnection problem.

# Check session state before attempting reconnect
GET /sessions/{sessionId}/status
# Returns: { "status": "active" | "expired" | "crashed" | "not_found", "idle_ms": 45000 }

// Distinct error types on connect failure
try {
  this.browser = await puppeteer.connect(env.BROWSER, this.browserSessionId);
} catch (error) {
  if (error.code === 'SESSION_EXPIRED') {
    // Expected. Launch fresh.
  } else if (error.code === 'SESSION_CRASHED') {
    // Log for debugging. Launch fresh.
  } else if (error.code === 'SESSION_NOT_FOUND') {
    // Platform cleaned it up. Launch fresh.
  }
}

The difference between “session expired naturally” and “something crashed” matters for building reliable agents. The session management surface area in general needs work. Typed error codes on reconnect failure, a session ID on the launch return, and a status check endpoint. None of these are large changes. Together they’d make stateful browser automation on Cloudflare significantly more ergonomic.

Anti-bot bypass is a gap I understand

Vanilla headless Chrome gets blocked by basic bot detection on non-trivial sites. Firecrawl’s anti-detection layer is their one remaining technical advantage. Rotating proxies, fingerprint randomization, stealth plugins.

But I understand why Cloudflare doesn’t ship a first-party stealth mode. Cloudflare sells Bot Management to site owners. Shipping a tool that defeats bot detection, including potentially their own customers’ detection, is a product conflict that won’t clear legal review no matter how you frame it.

The realistic path is probably a plugin or extension ecosystem. Let third parties provide stealth capabilities that plug into Browser Rendering sessions, similar to how Puppeteer’s community maintains stealth plugins. Cloudflare provides the rendering infrastructure. The stealth layer is somebody else’s problem and somebody else’s liability.

In the meantime I keep Firecrawl’s free tier around as a fallback for domains that block headless Chrome. That works at my scale. At larger scale it’s the one gap that doesn’t have a clean workaround within the Cloudflare ecosystem.

The Firecrawl comparison

Feature	Browser Rendering	Firecrawl
Single page to markdown	/markdown endpoint	/scrape endpoint
Full site crawl	Build it yourself	/crawl endpoint
LLM structured extraction	/json with prompt	/scrape with extract
Anti-bot bypass	None (by design)	Built-in
Session management	DO + Puppeteer binding	N/A (stateless)
Concurrent browsers	30 per paid account	Managed by Firecrawl
Usage tracking	Dashboard + per-request headers	Dashboard
Cost at 10K pages/month	~$0.36	$99 (Growth plan)

The cost column is decisive. The feature columns show where the work remains. Every “build it yourself” item is an opportunity for Cloudflare to eliminate a reason someone keeps a Firecrawl subscription alongside their Workers plan.

On the LLM extraction row. Browser Rendering’s /json endpoint with a prompt parameter and Firecrawl’s /scrape with extract mode do roughly the same thing. Both use an LLM to pull structured data from rendered pages. If someone has comparison data I’d like to see it.

The point

Browser Rendering is already the right choice for most scraping and browser automation workloads on Cloudflare. The REST API is the simplest web scraping interface I’ve used. The Puppeteer binding with Durable Objects is architecturally better than container-based alternatives. The cost structure makes Firecrawl hard to justify for most use cases.

These aren’t complaints. This is a product that’s 80% of the way to making an entire category of SaaS tools unnecessary. The remaining 20% is what separates a great infrastructure primitive from the thing everyone defaults to.

A /crawl endpoint would close the biggest gap. Session management ergonomics need polish. The concurrency limits are workable but overflow behavior needs documentation. Everything else is incremental.