Streaming Long AI Jobs to the Browser: SSE Patterns From Building an Audit Tool

Why SSE and not WebSockets

Short version: the browser never needs to talk back mid-job. The client uploads a contract, then listens. That's exactly the shape SSE was built for: one-directional server-to-client over plain HTTP. No connection upgrade, no socket lifecycle management, and the browser's EventSource gives you automatic reconnection for free, which turns out to be the most valuable feature of the whole protocol. WebSockets would work, but you'd be maintaining bidirectional machinery to use ten percent of it.

Design the event vocabulary before writing any code

My first attempt streamed whatever the pipeline felt like emitting: raw model tokens, log lines, half-thoughts. The frontend became a parser for an undocumented format that changed whenever I touched the backend. Bad.

Second attempt: a fixed vocabulary of typed events, treated as a real API contract. Four types cover everything:

type AuditEvent =
  | { type: "progress"; step: string; current: number; total: number }
  | { type: "partial-finding"; finding: Finding } // validated, complete finding
  | { type: "done"; summary: AuditSummary }
  | { type: "error"; message: string; recoverable: boolean };

Decisions inside that shape that earned their place:

• progress carries semantic steps, not percentages. "Analyzing withdraw(), function 3 of 7" keeps a user at the screen through minute four. A bar crawling from 41% to 43% doesn't, and honest percentages are impossible anyway when you don't know how long each model call takes.
• partial-finding is the retention feature. The pipeline finds issues one at a time, so I ship each one the moment it's validated. Users start reading the first finding while the model is still chewing on the rest, and the perceived wait collapses even though total time is unchanged. If your long job produces incremental results, streaming them beats any progress bar you could design.
• Findings stream only after validation. Early on I forwarded findings as raw model output and occasionally streamed garbage straight into the UI. Now everything passes schema validation server-side first. Stream results, never stream your parsing problem.
• error distinguishes recoverable from fatal. A single model call failing means retrying and telling the user - the audit continues. Out of budget or malformed input means the job is dead. The frontend does very different things with those, so the event must say which it is.

Reconnection: the part everyone skips

Laptops sleep. Phones switch networks. Proxies kill idle connections. Over several minutes, disconnection is a certainty at scale, and EventSource will reconnect automatically, sending a Last-Event-ID header with the last event it received. That header is only useful if you built for it.

Two requirements: every event gets a monotonic ID, and the server keeps a replayable log of events per job, independent of any connection. Which forces the real architectural insight: the job must not live inside the HTTP request. The audit runs somewhere durable and appends events to a log. The SSE endpoint is just a cursor over that log:

// app/api/audits/[id]/events/route.ts
export async function GET(req: Request, { params }: { params: { id: string } }) {
  const lastId = Number(req.headers.get("last-event-id") ?? -1);
  const job = await getJob(params.id);
  const encoder = new TextEncoder();
  const stream = new ReadableStream({
    async start(controller) {
      const send = (id: number, e: AuditEvent) =>
        controller.enqueue(
          encoder.encode(`id: ${id}\nevent: ${e.type}\ndata: ${JSON.stringify(e)}\n\n`),
        );

      // 1. Replay whatever this client missed
      for (const { id, event } of await job.eventsAfter(lastId)) send(id, event);

      // 2. Then follow the live log
      const unsubscribe = job.subscribe((id, event) => {
        send(id, event);
        if (event.type === "done" || (event.type === "error" && !event.recoverable)) {
          unsubscribe();
          controller.close();
        }
      });

      // 3. Heartbeat comment so proxies don't kill an idle connection
      const heartbeat = setInterval(
        () => controller.enqueue(encoder.encode(`: hb\n\n`)),
        15_000,
      );

      req.signal.addEventListener("abort", () => {
        clearInterval(heartbeat);
        unsubscribe();
      });
    },
  });

  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache, no-transform",
      Connection: "keep-alive",
    },
  });
}

Replay-then-follow is the whole trick. A client that drops at event 12 reconnects, replays 13 through wherever the job is now, and continues live. Refresh the page mid-audit and you lose nothing. The heartbeat comment line (SSE ignores lines starting with :) keeps intermediaries from declaring the connection dead during a long model call, and no-transform stops well-meaning proxies from buffering your stream into uselessness.

Serverless will fight you on this

Everything above assumes the job outlives the request, and default serverless assumes the opposite. A function that dies at its time limit takes your four-minute audit with it, and "keep the function alive by streaming" only stretches the ceiling - it doesn't remove it - and now a client disconnect can kill the job for every other viewer of that audit.

What I landed on: separate the worker from the stream. The job runs in something with a long lifetime (a worker process, a queue consumer, a container, whatever your platform offers), writes events to shared storage, and the SSE function does nothing but read and forward. SSE endpoints become cheap and stateless, the job becomes durable, and reconnection falls out naturally because the log is the source of truth. If you're on a platform where a long-lived worker is genuinely unavailable, chunk the pipeline into resumable stages and accept the added complexity - but know that you're paying it to avoid a worker, not because streaming requires it.

Backpressure, or when the model outruns the browser

Local models on a decent GPU can emit events faster than a busy tab renders them, especially token-level progress. Two things saved me:

• Coalesce chatty events server-side. Nobody needs 40 progress updates a second. I batch progress events on a short interval and send only the latest state per tick. Findings are never coalesced - every one ships. Classify your events as "latest value wins" or "every one matters" and throttle only the first kind.
• Respect the stream's own signals. controller.enqueue piles into a buffer if the consumer is slow, and unbounded buffering on a multi-minute job is a slow memory leak. Check controller.desiredSize before enqueueing low-priority events and drop stale progress ticks when it goes negative. The client that skipped some progress frames catches up instantly at the next one, and nobody notices.

The pattern that ties all of this together

Treat the event log as the product and the SSE connection as a disposable view of it. Every hard problem (reconnection, serverless limits, backpressure, even multiple tabs watching one audit) gets easy once the connection stops being where state lives.

What's the longest-running job you've had to keep a browser honest about, and what broke first?