DEV Community

Make Multipart Upload Abort Idempotent Before Orphaned Parts Start Billing You

Problem: Multipart Abort Is Not Idempotent

Multipart finalization is not the only retry boundary. Abort can succeed in object storage while the HTTP response is lost, leaving your database convinced the upload is active.

Model abort as a state transition, not a button wired directly to an SDK call:

active β†’ aborting β†’ aborted
                └──→ cleanup_pending

Implementing an Idempotent Abort Endpoint

The endpoint should own an idempotency key and durable intent:

await db.transaction(async tx => {
  const upload = await tx.lockUpload(uploadId);
  if (upload.state === "aborted") return;
  await tx.markAborting(uploadId, idempotencyKey);
  await outbox.enqueue("abort-multipart", { uploadId, storageKey });
});

A worker calls storage. If a retry receives NoSuchUpload, do not blindly fail: reconcile whether the upload was already completed, already aborted, or expired. Only the β€œalready absent because abort succeeded” path may converge to aborted; completion requires a separate terminal state.

Required Invariants: Test This Sequence

Step Fault Required invariant
persist abort intent process dies outbox resumes work
storage abort succeeds response is lost retry does not reactivate upload
duplicate message delivered twice one terminal state
client retries same key same operation result
cleanup scan stale active row reconcile before deleting

Amazon S3 Considerations

Amazon S3’s AbortMultipartUpload API notes that in-progress part uploads may still succeed around an abort and recommends verifying that parts are gone. That makes a post-abort verification pass part of correctness, not optional housekeeping.

Expose abort_requested_at, attempt count, last storage result, and remaining-part count. Alert on age in aborting, then run a lifecycle cleanup policy as a backstop-not as a substitute for application reconciliation.

Comments

No comments yet. Start the discussion.