AI Fluency Series · Project 1: Streaming Chat Or: how I spent a day proving a bug I was causing myself.

Disclaimer

A dumb little start to leveling up my AI fluency. Nothing fancy, no grand architecture, no hot takes - just a journal of me trying to actually understand things instead of pretending to.

Quick filter: if you're on React 19, have actually enabled the compiler (it's an opt-in plugin, not something the version hands you for free), and you write pure components - close the tab, you already know the ending. If you're on React 19 and assumed that meant the compiler was on: stay. That assumption is a trap, and I fell in it.

Why chat is Project 1

Streaming tokens into a growing list that re-renders on every one is the substrate under every AI feature I'm about to build - RAG answers stream, agents stream, all of it. If I can't make token-by-token rendering behave, nothing downstream will.

So the rule I gave myself: build it by hand before reaching for the library. No useChat until I've felt the problem it solves. I did it in Next.js - partly because that's where I'm headed, partly as an excuse to learn it - but none of this is Next-specific; any framework does the same thing.

And I mocked the stream instead of calling a real model, because the subject is how the UI handles a stream, not the model. A real LLM would just be a slower way to make the same tokens.

The storm

The setup was deliberately dumb: append each token into one blob of state, map the whole array to the screen.

setChunks(prev => [...prev, chunk]); // new array, every token
{chunks.map((c, i) => <div key={i}>{c}</div>)} // whole list, every token

I wired it up, fired the request, watched the words stream in. Worked on the first try - which felt great for about ten seconds, until the first real problem surfaced: it re-rendered the entire message list on every single token.

That's the O(N²) trap in the flesh. Token 200 re-processes 200 things, token 400 re-processes 400. It's not slow because it's badly written - it's slow because it recomputes the settled past on every update.

I also added an input box to test typing mid-stream and found a second bug I'd built for free: the input's state lived in the same component as the list, so every keystroke re-rendered all 400 messages. Two unrelated things welded together because they shared a component.

Fix: move the input's state into its own component. Suddenly typing re-renders one input, not the whole conversation - the single most useful React habit, and it has nothing to do with streaming.

Then measuring got weird

I wanted numbers, not vibes, so I dropped a render counter and a console.log into the message component. Chunk 0 was rendering on every token - the full storm, right there in the console. The Profiler said the same thing in pictures: record one response and watch the whole .map re-run, every chunk lighting up in the flamegraph on every word.

Except the numbers wouldn't sit still. Dev said 872 renders; prod said 426 - React's Strict Mode double-invokes in dev, so half my "data" was a lie the dev server was telling me. (Lesson one: never trust a perf number from npm run dev.)

There was also a version goose-chase that ended in pure comedy:

TypeError: messages.some is not a function
🧠 "typo?" → 🧠🧠 "@ai-sdk/react is behind ai, upgrade React!" → 🧠🌌 "the whole SDK protocol changed, I must rewrite everything"
🤡 the actual fix: I forgot to type await

But the real head-scratcher: I knew the compiler was supposed to memoize my components - I'd enabled it, and DevTools showed the little ✨ badge. So why was chunk 0 still re-rendering every token?

The twist

Here's the thing that made the whole day worth it: My measurement was causing the bug.

The React Compiler only memoizes components it can prove are pure - same inputs, same output, no side effects during render. And I'd shoved a console.log and a ref mutation straight into the render body to measure the renders. Those are side effects.

So the compiler took one look, decided it couldn't safely optimize the component, and silently bailed - no warning, no error, it just quietly stopped memoizing. Which means the O(N²) storm I was so carefully measuring was, in part, manufactured by the act of measuring it. The probe changed the experiment. I was shining a flashlight into a dark room to see how dark it was, then writing down "huh, pretty bright in here."

// impure → compiler refuses to memoize → every Chunk re-renders
const Chunk = ({ text }) => {
  const r = useRef(0);
  r.current++; // ⛔ mutation in render
  console.log('render', r.current); // ⛔ side effect in render
  return <div>{text}</div>;
};

// pure → compiler memoizes it; measure from an effect instead
const Chunk = ({ id, text }) => {
  useEffect(() => console.log('committed', id), [id, text]);
  return <div>{text}</div>;
};

Pull the side effects out, gate the logging into a useEffect, and chunk 0 went quiet. The ✨ badges meant what they said. The storm collapsed to O(N) - and I hadn't written a single React.memo. The compiler did it, the moment I stopped poisoning the well.

(Full honesty, since this is a journal and not a press release: the airtight proof is a clean Profiler read on a late commit with zero instrumentation. I eyeballed the sparkles more than I formally captured that. I'm ~95% there; the last 5% is a thirty-second measurement I owe myself.)

Then I used the library

Only after all that did I swap in useChat - and building it by hand first meant I could see exactly what it does. The fetch, the reader, the accumulation, the state updates: the entire machine I'd hand-cranked, gone, replaced by three destructured values.

But the surprise, which probably shouldn't have been one: useChat doesn't fix the render cost. It owns when you render - every token, via its own state updates - and has zero opinion about how expensive each render is. Drop a console.log back into a message component and the O(N²) is right back, useChat and all.

The abstraction changed the plumbing. It did not change the physics.

What carried over

The React-specific trivia is fun, but the two keepers are bigger than React:

• Re-processing the settled past on every update is the cost that bites. Streaming chat, an append-only log, a game loop - same shape, same fix: isolate what changed from what didn't. Memoization is just React's dialect for it.

• Your instrument can change what you measure. I hit it three times in one day - dev double-renders, keystrokes polluting my counts, a console.log disabling the compiler - and I anchored on an exciting theory (version hell!) while the stack trace pointed at the boring truth the whole time. "Did my measurement perturb the thing?" is now a reflex, and it's a science habit, not a frontend one.

The 2026 punchline, and the reason for that snarky opener: you mostly don't do memoization anymore. Keep your components pure and let the compiler do it. The skill moved from "know where to sprinkle memo" to "write pure components - and know how to tell when the compiler quietly gave up on you."

Next up in the series: RAG. Where, I'm told, the actual AI begins.

• filed from a laptop that is now, finally, rendering only what changed.