Idempotency is not a key, it's a contract

Every payments API has an Idempotency-Key header. Far fewer have written down what it means, and the ones that haven't are usually running a SELECT … WHERE key = ? before the insert and calling the job done. A key is a token. A contract is a set of answers: what counts as the same request, how long the answer stays valid, what a caller gets when their retry lands while the first attempt is still running, and which failures are worth remembering. Skip those and you've made double charges rarer without making them impossible, which is the worst outcome available: now nobody is watching for them. The promise is "same outcome", not "skip the duplicate" HTTP already has idempotent methods. RFC 9110 defines PUT and DELETE that way: sending the request twice leaves the server in the same state as sending it once. POST is deliberately excluded, which is why the header exists at all. It's a way to bolt the property onto the one method that doesn't have it. Notice what the definition covers and what it doesn't. It constrains server state. It says nothing about what the caller sees, and the caller is the entire reason you're doing this. Here's the handler people write first: const seen = await db.keys.find(key); if (seen) return res.status(200).json({ status: "ok" }); State is protected. The client is not. They retried because they never saw the first response: the socket died, the load balancer timed out, the phone lost signal in a lift. What they need back is the charge id from attempt one. What they got is {"status":"ok"} , which tells them a charge exists somewhere with an id they will never learn. So the promise has two halves, and both matter: - The work happens at most once. - Every retry gets back the original response, with the same status and the same body, byte for byte. The second half is what turns your endpoint into something a client can safely retry in a loop. Without it, retries are safe for you and useless for them. What counts as "the same request" The key is not the identity of the request. It's a name the client picked for one. The identity is the key plus what was in the body. POST /payments Idempotency-Key: 7f3a… { "amount": 5000, "to": "acct_1" } POST /payments Idempotency-Key: 7f3a… { "amount": 9000, "to": "acct_1" } Replay the first response and you've silently refused to send $90 while telling the caller you sent it. Execute the second and the key bought you nothing. Neither is acceptable, which means there's only one correct answer: reject it. Store a fingerprint of the request alongside the key and compare on every hit. import { createHash } from "node:crypto"; /** Stable hash of the parts of a request that change what it does. */ function fingerprint(path: string, body: unknown): Buffer { return createHash("sha256").update(path).update("\0").update(canonicalJson(body)).digest(); } Stripe returns a 400 with an idempotency_error here. 422 is defensible too. What isn't defensible is picking one of the two bodies and proceeding. The trap in this section is canonicalJson . Fingerprint the raw bytes and you'll ship false rejections, because a retry is very often a re-serialisation, not a replay of the same buffer. Two things reliably bite: - Key order. A client that builds the payload from a hash map may emit fields in a different order the second time. Same request, different bytes. - Timestamps. A client_ts orrequested_at field filled in withDate.now() at send time changes on every attempt. Now every retry is a fingerprint mismatch and your safest clients get the most errors. Canonicalise (sort keys, normalise numbers), or fingerprint an explicit allowlist of the fields that carry meaning. Both work. Hashing the whole body verbatim does not. The race is the actual problem Check-then-insert is a time-of-check-to-time-of-use bug wearing a business shirt. Two retries arrive 4 ms apart, both SELECT and miss, both charge. This is not an exotic interleaving; it's the normal shape of a retry storm, because whatever made the client retry (a timeout) also tends to make it retry more than once. Let the database arbitrate. Insert first, on the way in. create table idempotency_keys ( account_id uuid not null, endpoint text not null, key text not null, fingerprint bytea not null, state text not null check (state in ('in_flight', 'succeeded', 'failed')), response_status int, response_body jsonb, resource_id uuid, locked_until timestamptz, created_at timestamptz not null default now(), primary key (account_id, endpoint, key) ); const claimed = await db.query( insert into idempotency_keys (account_id, endpoint, key, fingerprint, state, locked_until) values ($1, $2, $3, $4, 'in_flight', now() + interval '60 seconds') on conflict (account_id, endpoint, key) do nothing returning key, [accountId, endpoint, key, fp], ); if (claimed.rowCount === 1) { // We own this key. Do the work, then write the response back. } Zero rows means someone else owns it, and the existing row tells you what to do: | Stored state | What the caller gets | |---|---| | Fingerprint differs | 400 : same key, different request | succeeded | The stored status and body, replayed | failed , deterministic | The stored status and body, replayed | in_flight , lock live | 409 + Retry-After | in_flight , lock expired | Recovery. See below; do not just re-run it | The 409 is worth defending, because the tempting alternative is to block: wait on a row lock until the first attempt finishes, then return its response. It reads beautifully and it's a bad idea at load. Every waiter is a held connection, so a provider that's gone slow converts directly into pool exhaustion, and the requests you're holding are by definition from clients that already gave up once. Hand back 409 with a Retry-After and let the client's backoff do the waiting. It has a much better place to do it than your connection pool. The key has to outlive the process that made it Server-generated keys don't work. Fetching one is itself a network call that can time out, and now you need idempotency for your idempotency endpoint. So the client owns the key, and where the client stores it decides whether any of this functions. // Broken: a new key on every attempt. This is a plain retry loop with extra steps. for (const delay of backoff) { await post("/payments", body, { key: crypto.randomUUID() }); } // Also broken: survives the loop, not a crash. The retry after restart is a new key. const key = crypto.randomUUID(); The key belongs next to the intent, in whatever durable thing already represents "we mean to pay this": the row in your own database, the payload in the job queue. Generate it once, when the intent is created, and read it back on every attempt including the ones that happen after a deploy, a pod eviction, or somebody replaying a dead-letter queue by hand on Monday morning. The idempotency key is a property of the job, not of the attempt. If you take one line from this post, take that one. Most idempotency that fails in production fails here, in the client, in code nobody thought of as payment code. Scope it, or you'll replay someone else's response Make the key composite: (account_id, endpoint, key) . Global uniqueness on the key column alone has two failure modes, and one of them is a security incident. Two endpoints sharing a key namespace means a client that reuses order-8812 for both /payments and /refunds gets the payment response back from the refund call. Annoying. Two tenants sharing a namespace is worse. Keys are frequently derived from things that aren't secret (an order number, an invoice id, a checkout-2026-08-12-0001 ), so tenant B can arrive with a key tenant A already used and receive A's stored response body. You built a cross-tenant read out of a deduplication table. Scope by account and the whole class disappears. The window is a promise about time, and 24 hours is usually a guess Stripe expires keys after 24 hours. That's a reasonable default for a browser and a bad fit for a lot of what actually retries. Think about the longest path a retry of your endpoint can take. A queue with seven-day retention. An operator replaying Friday's failures when they get in on Monday. A mobile client that was in a tunnel, then a plane, then a country with roaming disabled. If your window is 24 hours and your job queue retries for 7 days, then on day two the same key is a new key and the retry is a fresh charge, with the idempotency system fully installed, monitored, and reporting green. The window must be at least as long as the longest retry horizon of any caller you have. When you can't bound that, split the record instead of expiring it: - The response body is the expensive part. JSONB of a full resource, times every write request. Expire that on the usual timescale. - The key, fingerprint and resource_id are 100 bytes. Keep them for a year. A retry that arrives after the body is gone but while the key survives is still recognisable. Return 409 and point at resource_id . "You already did this, it's payment pay_… , go look" is an unhelpful answer that leaves the caller with a lookup to do. It's also infinitely better than charging them again, which is the only other option once you've forgotten the key entirely. Idempotency is not atomicity This is the part that separates a key column from a system that works. const charge = await provider.charge(amount, card); // ← crash here await ledger.record(charge.id, amount); The process dies between those two lines. The provider has taken the money. Your ledger doesn't know. The key row says in_flight and will say so until the lock expires. The key prevented a duplicate request; it did nothing about partial work, because it was never the kind of thing that could. An expired in_flight lock is an unknown, not a failure. That distinction is the whole game. Treat it as a failure and re-run the handler and you have built a double-charge machine whose trigger is your own worst outage: the one where processes were dying mid-request. Two t