A Tiny LLM Request Recorder I Use to Reproduce Production Failures

A Tiny LLM Request Recorder I Use to Reproduce Production Failures

Most LLM failures are easy to describe and surprisingly hard to reproduce. A user reports that the model returned an empty answer. A tool call disappeared halfway through a stream. One provider rejected a request that worked everywhere else.

Then I open the logs and find something like this: LLM request failed: 400 Bad Request. Technically true. Operationally useless.

The missing piece is usually the exact request shape: model, parameters, message roles, tool definitions, timeout behavior, and the raw provider response. I wanted something smaller than a full observability platform, so I built a request recorder around fetch. It stores enough information to inspect or replay a failed call without logging the API key.

What the recorder captures

For each request, I want:

• a unique request ID
• timestamp and duration
• URL and model
• sanitized request body
• HTTP status
• raw response body
• network or timeout errors

I deliberately do not record the Authorization header. Prompt content is also redacted by default. Full payload capture must be enabled explicitly because storing production prompts can create a much worse problem than the bug being investigated.

The recorder

This example runs on Node.js 18 or newer and has no external dependencies. Create recorded-fetch.mjs:

import { randomUUID } from "node:crypto";
import { mkdir, writeFile } from "node:fs/promises";
import path from "node:path";

function sanitize(value, captureContent) {
  if (Array.isArray(value)) {
    return value.map((item) => sanitize(item, captureContent));
  }
  if (!value || typeof value !== "object") {
    return value;
  }
  const result = {};
  for (const [key, child] of Object.entries(value)) {
    const normalizedKey = key.toLowerCase();
    if (
      normalizedKey.includes("api_key") ||
      normalizedKey.includes("apikey") ||
      normalizedKey.includes("authorization")
    ) {
      result[key] = "[REDACTED]";
      continue;
    }
    if (
      !captureContent &&
      (normalizedKey === "content" ||
        normalizedKey === "prompt" ||
        normalizedKey === "input")
    ) {
      result[key] = "[REDACTED]";
      continue;
    }
    result[key] = sanitize(child, captureContent);
  }
  return result;
}

async function saveRecording(directory, recording) {
  await mkdir(directory, { recursive: true });
  const filename = `${recording.startedAt.replaceAll(":", "-")}-${recording.id}.json`;
  const destination = path.join(directory, filename);
  await writeFile(destination, JSON.stringify(recording, null, 2), "utf8");
  return destination;
}

export function createRecordedFetch({
  directory = ".llm-recordings",
  captureContent = false,
} = {}) {
  return async function recordedFetch(url, options = {}) {
    const id = randomUUID();
    const startedAt = new Date().toISOString();
    const start = performance.now();
    let requestBody;
    try {
      requestBody =
        typeof options.body === "string"
          ? JSON.parse(options.body)
          : options.body;
    } catch {
      requestBody = "[UNPARSEABLE BODY]";
    }
    const recording = {
      id,
      startedAt,
      request: {
        url: String(url),
        method: options.method ?? "GET",
        body: sanitize(requestBody, captureContent),
      },
    };
    try {
      const response = await fetch(url, options);
      const rawResponse = await response.text();
      recording.durationMs = Math.round(performance.now() - start);
      recording.response = {
        status: response.status,
        statusText: response.statusText,
        headers: Object.fromEntries(response.headers.entries()),
        body: rawResponse,
      };
      const savedTo = await saveRecording(directory, recording);
      if (!response.ok) {
        throw new Error(
          `LLM request ${id} failed with ${response.status}. Recording: ${savedTo}`
        );
      }
      return { requestId: id, response, rawResponse, savedTo };
    } catch (error) {
      if (!recording.response) {
        recording.durationMs = Math.round(performance.now() - start);
        recording.error = {
          name: error.name,
          message: error.message,
        };
        await saveRecording(directory, recording);
      }
      throw error;
    }
  };
}

This wrapper reads the response body once and returns it as rawResponse. That is important because a fetch response body cannot normally be consumed twice.

Using it for an LLM request

Create example.mjs:

import { createRecordedFetch } from "./recorded-fetch.mjs";

const recordedFetch = createRecordedFetch({
  directory: ".llm-recordings",
  captureContent: false,
});

const baseUrl = process.env.LLM_BASE_URL ?? "https://api.openai.com/v1";
const apiKey = process.env.LLM_API_KEY;

if (!apiKey) {
  throw new Error("LLM_API_KEY is required");
}

const payload = {
  model: process.env.LLM_MODEL ?? "gpt-4.1-mini",
  messages: [
    {
      role: "user",
      content: "Explain idempotency in one paragraph.",
    },
  ],
  temperature: 0.2,
};

const result = await recordedFetch(`${baseUrl}/chat/completions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify(payload),
  signal: AbortSignal.timeout(30_000),
});

const data = JSON.parse(result.rawResponse);
console.log({
  requestId: result.requestId,
  recording: result.savedTo,
  answer: data.choices?.[0]?.message?.content,
});

Run it with:

LLM_API_KEY="your-api-key" node example.mjs

Every call now produces a JSON file under .llm-recordings. A failed request might look like this:

{
  "id": "9e50d5a9-4a0d-42a2-94c9-711d36b2057d",
  "startedAt": "2026-07-17T08:14:32.442Z",
  "request": {
    "url": "https://api.example.com/v1/chat/completions",
    "method": "POST",
    "body": {
      "model": "example-model",
      "messages": [
        {
          "role": "user",
          "content": "[REDACTED]"
        }
      ],
      "temperature": 0.2
    }
  },
  "durationMs": 418,
  "response": {
    "status": 400,
    "statusText": "Bad Request",
    "headers": {
      "content-type": "application/json"
    },
    "body": "{ \"error\":{ \"message\": \"Unsupported parameter: temperature\" }}"
  }
}

That is already more useful than a generic 400. It tells me the failure was probably caused by a provider compatibility difference, not the prompt, network, or model output.

Replaying a request

For local or staging environments, I can temporarily set:

const recordedFetch = createRecordedFetch({
  captureContent: true,
});

The resulting request body can then be replayed with a small script. Create replay.mjs:

import { readFile } from "node:fs/promises";

const recordingPath = process.argv[2];
if (!recordingPath) {
  throw new Error("Usage: node replay.mjs <recording.json>");
}

const recording = JSON.parse(await readFile(recordingPath, "utf8"));
const serializedBody = JSON.stringify(recording.request.body);

if (serializedBody.includes("[REDACTED]")) {
  throw new Error(
    "This recording contains redacted values and cannot be replayed."
  );
}

const apiKey = process.env.LLM_API_KEY;
if (!apiKey) {
  throw new Error("LLM_API_KEY is required");
}

const response = await fetch(recording.request.url, {
  method: recording.request.method,
  headers: {
    Authorization: `Bearer ${apiKey}`,
    "Content-Type": "application/json",
  },
  body: serializedBody,
  signal: AbortSignal.timeout(30_000),
});

console.log("Status:", response.status);
console.log(await response.text());

Then run:

LLM_API_KEY="your-api-key" \
  node replay.mjs .llm-recordings/example.json

I only enable full-content recording with synthetic or approved test data. I would not turn it on globally in production.

What this has helped me distinguish

The recorder is most useful when several failures look identical from the outside. It helps separate:

• Request-shape failures: Unsupported parameters, invalid tool schemas, wrong message roles, or model-specific limits.
• Transport failures: Timeouts, connection resets, interrupted streams, and responses that never reach the application.
• Provider failures: A structured error response, unexpected content type, or different interpretation of an OpenAI-compatible field.
• Application failures: The provider returned a valid response, but my parser, state machine, or tool executor mishandled it.

Those categories lead to very different fixes. Retrying all four of them is not a debugging strategy.

Production considerations

This recorder is intentionally small, so it leaves several decisions to the application using it. Before running something like this in production, I would add:

• a retention period for recordings
• sampling instead of recording every successful request
• encryption at rest
• size limits for request and response bodies
• stricter redaction for user and tool data
• access controls around the recording directory
• correlation with the application trace or operation ID

I would also record failed calls more aggressively than successful ones. Keeping every response forever is an expensive way to create a privacy problem.

Why I keep this wrapper small

There are good observability products that can collect richer traces, token usage, latency distributions, and model-level metrics. I still like having a tiny recorder close to the HTTP boundary. It gives me a provider-neutral artifact I can inspect before data passes through SDK abstractions, response parsers, retry policies, or agent frameworks.

When I test multiple OpenAI-compatible endpoints-including my work on TokenBay-that raw boundary is often where compatibility problems become easiest to see.

The goal is not to log everything. It is to make the next production failure reproducible enough that I do not have to debug it from a single error message.