Europe-US Marketplace Reminder System Under Load (Delayed Jobs, Cron, Postgres, Node.js)
Short Answer
Choose a database-backed sweeper when reminder state already lives in Postgres and minute-level delivery is acceptable. Choose delayed jobs when each reminder needs tighter timing or independent backpressure. Use cron only to wake the system; don't let a cron invocation become the source of truth.
| Choice | Best fit | Retry ownership | Main catch |
|---|---|---|---|
| Postgres sweeper | Moderate volume, minute-level timing, existing relational state | Application and database | Polling adds load and timing granularity |
| Delayed job per occurrence | Tighter timing and queue-level isolation | Queue and worker | Recurrence still needs durable application state |
| Cron-only callback | Small, rebuildable housekeeping | Scheduler | Weak fit for per-user retry and deduplication |
Short answer: for a marketplace cleanup that expires stale reservations and creates user reminders, use Postgres as the ledger, a short recurring wake-up, and idempotent workers. Retry behavior and idempotency matter more than the cheapest-looking scheduler line item.
A one-person product should outsource undifferentiated wake-ups, keep business state inspectable, and preserve the option to ship weekly without maintaining a second scheduling model. Store intent, claim due work with a lease, and make the side effect deduplicable.
What Breaks When Reminders Span Europe and the US
Treat a reminder as durable business data, not as a timer held in a Node.js process. A process can restart. A deployment can overlap the previous version. Regional clocks disagree at daylight-saving boundaries.
The durable row needs the intended local time, the resolved UTC instant, the recipient, the event type, and an idempotency key. The worker may run in Europe or the US; the stored due instant must mean the same thing in either place.
A cron trigger is useful as a metronome. It asks, "Is anything due?" It should not attempt to encode every customer's recurrence in infrastructure configuration. Cloudflare Workers Cron Triggers, for example, execute on UTC time and configuration changes can take time to propagate. That makes them useful for waking a sweeper, while user-facing local-time rules remain application data.
Delayed queues invert the scan. Each occurrence becomes a message scheduled for later delivery. This removes most empty polling and gives each item an independent delivery lifecycle. The catch is recurrence: after a worker handles Tuesday's reminder, some component still has to calculate and durably record the next occurrence. A queue message by itself is not the subscription rule.
Postgres keeps that rule beside the marketplace records it depends on. A cleanup can atomically mark a stale reservation expired and insert an outbox reminder. That transaction boundary prevents the awkward state where inventory is released but the corresponding reminder intent disappears.
Polling is less precise and consumes database capacity, so it isn't suitable when reminders need sub-minute latency at high sustained volume. In that case, keep Postgres as the ledger and use a delayed queue for delivery pressure.
One detail is unresolved without measurements: the volume where polling becomes the wrong economic choice. I'm not sure anyone can name it from architecture alone. Query rate, index selectivity, reminder burstiness, regional round-trip time, and operator time all move the boundary. A week of production histograms will answer it better than a vendor calculator.
Model Each Reminder Occurrence as a Durable Fact
A naive worker selects every row with due_at <= now(), sends the notification, and marks the row complete. If it stops after the send but before the update, the next run sends again. If it marks complete before sending, an interrupted process loses the reminder. No arrangement of those two statements creates exactly-once external delivery.
Make the states explicit: pending, leased, sent, and dead. Claim a bounded batch in a transaction. Give each claim a lease deadline and increment its attempt counter. An interrupted worker leaves leased work behind; a later sweep can reclaim it after the lease expires. A completed worker records the accepted result against the same idempotency key.
This is the important bit. The key should identify the business occurrence, such as reservation-expired:<reservation-id>:<scheduled-instant>, rather than the worker attempt. Every retry carries the same key. The notification adapter must enforce uniqueness locally even if the downstream channel does not accept an idempotency header. For example, insert a delivery record under a unique constraint before dispatch, then retain the durable outcome needed by the retry policy. The database transaction can guarantee one local dispatch decision; it cannot prove that an arbitrary remote side effect happened exactly once.
Retries also need a ceiling. Transient failures return to pending with a later next_attempt_at; permanent recipient errors move to dead; successful delivery moves to sent. Exponential delay with jitter avoids making every failed worker wake at the same instant. Keep the policy in data so an operator can explain why a reminder is waiting.
Don't spin inside one invocation for ten minutes. AWS SQS FIFO queues illustrate two related boundaries. They support ordered processing within a message group and deduplication, but those properties don't replace the marketplace's durable recurrence rule or its own record of delivery intent. A FIFO queue is a strong runner-up when ordering by user or reservation matters and queue-managed redelivery is worth the extra system boundary.
Claim 50 Rows, Then Give Ownership an Expiry
The implementation below shows the shape, not a framework. It assumes a reminder_jobs table with indexed next_attempt_at, a unique idempotency_key, and lease columns. The SQL uses FOR UPDATE SKIP LOCKED so overlapping workers claim different rows. Keep batches small enough that a deployment can drain them quickly.
import { Pool , PoolClient } from "pg" ;
type ReminderJob = { id : string ; user_id : string ; idempotency_key : string ; payload : unknown ; attempts : number ; } ;
const pool = new Pool ( { connectionString : process . env . DATABASE_URL } ) ;
const workerId = crypto . randomUUID ( ) ;
async function claimDueJobs ( client : PoolClient ) : Promise < ReminderJob [ ] > { const result = await client . query < ReminderJob > ( ` WITH due AS ( SELECT id FROM reminder_jobs WHERE (state = 'pending' AND next_attempt_at <= now()) OR (state = 'leased' AND lease_until <= now()) ORDER BY next_attempt_at, id FOR UPDATE SKIP LOCKED LIMIT 50 ) UPDATE reminder_jobs AS job SET state = 'leased', leased_by = $1, lease_until = now() + interval '2 minutes', attempts = attempts + 1 FROM due WHERE job.id = due.id RETURNING job.id, job.user_id, job.idempotency_key, job.payload, job.attempts ` , [ workerId ] ) ; return result . rows ; }
async function claimBatch ( ) : Promise < ReminderJob [ ] > { const client = await pool . connect ( ) ; try { await client . query ( "BEGIN" ) ; const jobs = await claimDueJobs ( client ) ; await client . query ( "COMMIT" ) ; return jobs ; } catch ( error ) { await client . query ( "ROLLBACK" ) ; throw error ; } finally { client . release ( ) ; } }
Delivery belongs behind a narrow adapter. Its contract accepts the stable key and returns a classified outcome. The adapter can use a downstream idempotency feature when available, but the worker's transitions stay the same.
type DeliveryResult = | { kind : "sent" ; receipt : string } | { kind : "retry" ; retryAt : Date } | { kind : "permanent-failure" ; reason : string } ;
async function finish ( job : ReminderJob , result : DeliveryResult ) : Promise < void > { if ( result . kind === "sent" ) { await pool . query ( `UPDATE reminder_jobs SET state = 'sent', sent_at = now(), receipt = $2, leased_by = NULL, lease_until = NULL WHERE id = $1 AND state = 'leased' AND leased_by = $3` , [ job . id , result . receipt , workerId ] , ) ; return ; } if ( result . kind === "permanent-failure" || job . attempts >= 8 ) { await pool . query ( `UPDATE reminder_jobs SET state = 'dead', failure_reason = $2, leased_by = NULL, lease_until = NULL WHERE id = $1 AND state = 'leased' AND leased_by = $3` , [ job . id , result . kind === "permanent-failure" ? result . reason : "attempt limit" , workerId ] , ) ; return ; } await pool . query ( `UPDATE reminder_jobs SET state = 'pending', next_attempt_at = $2, leased_by = NULL, lease_until = NULL WHERE id = $1 AND state = 'leased' AND leased_by = $3` , [ job . id , result . retryAt , workerId ] , ) ; }
Notice the ownership check in every update. A slow worker whose two-minute lease expired cannot overwrite the result of a newer claimant. The delivery adapter should check the stable key before performing its side effect, too. That closes the common duplicate path without pretending distributed exactly-once delivery exists.
The wake-up handler can now stay boring: claim at most 50 jobs, process them with a concurrency cap, record outcomes, and exit. Boring ships.
Rehearse Lease Theft Before a Weekly Release
Unit tests for date arithmetic are necessary, especially around spring-forward gaps and fall-back overlaps, but they don't exercise ownership. Add integration tests with a real Postgres instance and two workers. Pause worker A after it claims a job, advance the clock beyond its lease, let worker B reclaim and finish, then resume A. The final ownership predicate must prevent A from changing B's result.
Also test a repeated wake-up. Run two scheduler invocations at the same instant and assert that each job is claimed once. Seed 51 due rows to check the batch boundary. Seed one permanently invalid recipient and verify that it reaches dead rather than cycling forever. For recurrence, generate the next occurrence from the user's stored time-zone rule, then assert both the local representation and resolved UTC instant around a daylight-saving transition.
Observability should follow the state machine. Track the age of the oldest pending job, claim count, retry count, dead count, lease reclamations, and time from due_at to sent_at. Alert on age, not merely on whether cron fired: a green scheduler with a blocked worker is still a broken reminder system. Log the job ID and idempotency key, but keep message content and personal data out of routine logs.
Deployment is another overlap test. Let the old and new versions run concurrently while leases and ownership checks protect work. Schema changes should be additive first: add nullable columns and indexes, deploy code that understands both shapes, backfill in bounded batches, then enforce constraints. A weekly shipping cadence leaves little room for a ceremonial scheduler migration every time reminder rules change.
Cost belongs in the same operational view. Count database scans, queue operations, worker duration, and engineering hours spent on replay tools. The cheapest architecture is the one with the lowest total drag at the required timing and volume. For a small marketplace, one indexed query per minute may be an easy trade. At larger scale, empty scans and hot indexes can justify pushing occurrences to a delayed queue. Your mileage may vary - measure the oldest-job lag and database time before moving the boundary.
The Queue Earns Its Place at the Pressure Boundary
Use delayed jobs ahead of a Postgres sweeper when timing must be tighter than the polling interval, bursts need queue-level backpressure, or workers must scale independently from the application database. Keep the database record of intent and the stable idempotency key. The queue becomes transport, not truth. AWS SQS FIFO is relevant when ordered processing within a message group and deduplication match the workload; its exact limits and semantics should be checked against the current documentation before design freeze.
Use a cron-only function when the cleanup is small, deterministic, quick, and safe to recompute from authoritative data. It is not suitable for a large set of individualized reminders with independent retry histories. Cloudflare Workers Cron Triggers are an example of a managed UTC wake-up, but placing every user's schedule into trigger configuration would couple business data to deployment configuration.
Stick with the Postgres sweeper when minute-level resolution works, transactional creation matters, and the team needs one inspectable operational ledger. Move away when sweep load competes with marketplace transactions or due-job lag rises despite sensible indexing and batch sizes.
There is no permanent winner. The decision rule stays compact: cron wakes, Postgres remembers, and workers retry idempotently; add a delayed queue when measured timing or backpressure demands it. That preserves the revenue-per-hour advantage of a simple first version without hiding the exit criteria.
Comments
No comments yet. Start the discussion.