DEV Community

How to Scale Live Polls by Publishing Aggregated Tallies Instead of Every Vote

Short answer: publish an aggregated tally to viewers, keep each accepted vote in an auditable write path, and let reconnecting clients fetch a versioned snapshot before resuming updates. Publishing every vote is justified only when the product must expose a public event-by-event stream and its fan-out budget has been measured against peak participation.

The bill is dominated by fan-out, not ballot ingestion

For a poll with V accepted votes, C connected viewers, and a batch size of B votes, publishing every vote creates roughly V * C outbound deliveries; publishing one tally per batch creates roughly ceil(V / B) * C. Those are workload arithmetic, not benchmark claims, but they reveal the lever that matters.

At 20,000 votes and 8,000 viewers, the first design schedules 160 million viewer deliveries, while batches of 100 schedule about 1.6 million. Payload size, protocol framing, regional replication, and retries still affect the actual bill. Do less fan-out.

That conclusion doesn't permit dropping the vote ledger. It separates the durable record, which supports idempotency and audit, from the public projection, which supports a fast screen. The rest of the design follows from that boundary.

What does each live poll scaling model actually retain?

Treat a vote as a command and the tally as a materialized view. The command path validates poll state, applies the marketplace's eligibility policy, deduplicates a stable submission identifier, appends the accepted decision, and advances the tally in one transactional boundary.

The display path never needs the voter-level record; it needs {poll_id, counts, version, updated_at}. This distinction matters wherever a shared marketplace workspace shows both who is online and the current poll result: presence is ephemeral, but a vote may be subject to dispute, reconciliation, or a retention policy.

Model Viewer message Delivery growth Reconnect source Deliberately retained
Every vote One event per accepted vote V * C Event cursor plus replay log Vote events and replay window
Aggregated tally One snapshot or delta per batch ceil(V / B) * C Current versioned snapshot Vote ledger and latest projection
Hybrid Sparse vote notices plus periodic tally Between the two Snapshot, then newer notices Ledger, projection, bounded notice log

The every-vote model has a precise but expensive property: a viewer can reconstruct the visible count by replaying the same public sequence, provided ordering, authorization, and the replay window all hold. It also leaks timing and activity detail that a tally need not reveal.

The aggregated model makes the projection authoritative for viewers; it reduces delivery work and reveals less individual activity, but a client cannot reconstruct intermediate moments that the server chose not to retain. This is the first compliance limit: an append-only audit record does not imply indefinite public replay. Retention, access control, purpose limitation, and deletion obligations belong in the data design before transport is chosen.

I would keep the authoritative vote record for the period required by the marketplace's policy, keep only the latest public tally plus a short version history for operations, and avoid retaining viewer-specific delivery receipts unless a concrete control requires them.

How should a live poll scale: publish every vote or an aggregated tally?

Use aggregated tallies by default when the user-visible question is β€œwhat is the result now?” Choose every-vote publication only when observers genuinely consume individual public events, such as a moderated deliberation in which event order is part of the product, and when exposing that granularity is allowed. A hybrid is useful when the interface needs a small amount of motion but correctness still comes from periodic snapshots.

The decision can be made with four quantities:

  • peak accepted votes per second
  • concurrent viewers
  • acceptable visual staleness
  • reconnect rate

If the poll accepts 2,000 votes per second and has 30,000 viewers, a naive per-vote broadcaster faces 60 million deliveries per second before retries. A 250-millisecond aggregation window caps publication at four tally versions per second, or about 120,000 viewer deliveries per second, while introducing up to one window of intentional display delay. Again, these figures are computed examples. Your mileage may vary because payload compression, network topology, connection churn, and regional routing change the measured result.

Don't choose the window from aesthetics alone. Start from a display service-level objective, then load-test the complete path at the expected reconnect distribution. A one-second window may be acceptable for a casual workspace poll and unacceptable for a timed auction-like decision. I'm not sure any universal threshold would survive contact with both cases; a trace of real arrival rates and a product-approved staleness budget would resolve that uncertainty.

The catch is that aggregation is not suitable when every public vote must be independently consumed in real time. Keep the event stream in that case, partition it by poll, define a stable ordering rule, and impose admission control rather than pretending multiplication by viewer count disappears. Conversely, stick with snapshot publication when clients only render totals. It has the narrower contract, which is usually easier to reconcile.

Implement the tally as an idempotent projection

Exactly-once delivery across a database, a broker, and an unreliable client connection is not a credible transport promise. An exactly-once effect is achievable at the vote boundary when the server records an idempotency key and updates the tally atomically; downstream publication can then be at least once because each projection carries a monotonically increasing version. A client applies a version once and ignores older copies.

The following Go program is a compact in-memory model of that contract. Production storage should enforce the same uniqueness and transaction invariants durably. The code deliberately publishes totals rather than voter identifiers.

package main
import (
"errors"
"fmt"
"sync"
"time"
)
type Vote struct {
PollID string
Choice string
IdempotencyKey string
}
type Tally struct {
PollID string
Counts map[string]uint64
Version uint64
UpdatedAt time.Time
}
type Projector struct {
mu sync.Mutex
seen map[string]struct{}
counts map[string]uint64
ver uint64
}
func NewProjector() *Projector {
return &Projector{seen: make(map[string]struct{}), counts: make(map[string]uint64)}
}
func (p *Projector) Apply(v Vote) (Tally, error) {
p.mu.Lock()
defer p.mu.Unlock()
if v.PollID == "" || v.Choice == "" || v.IdempotencyKey == "" {
return Tally{}, errors.New("poll, choice, and idempotency key are required")
}
key := v.PollID + ":" + v.IdempotencyKey
if _, duplicate := p.seen[key]; !duplicate {
p.seen[key] = struct{}{}
p.counts[v.Choice]++
p.ver++
}
copyOfCounts := make(map[string]uint64, len(p.counts))
for choice, count := range p.counts {
copyOfCounts[choice] = count
}
return Tally{PollID: v.PollID, Counts: copyOfCounts, Version: p.ver, UpdatedAt: time.Now().UTC()}, nil
}
func main() {
p := NewProjector()
vote := Vote{PollID: "workspace-42", Choice: "approve", IdempotencyKey: "submission-9001"}
first, _ := p.Apply(vote)
retry, _ := p.Apply(vote)
fmt.Println(first.Counts["approve"], retry.Counts["approve"], retry.Version)
}

The output is 1 1 1: retrying the same submission neither increments the count nor advances the projection version. A database implementation needs a unique constraint on (poll_id, idempotency_key) and a transaction containing both the accepted-vote insert and projection update. An outbox written in that transaction can carry the new projection to the broadcaster; if it is published twice, the version rule makes the duplicate harmless at the display boundary.

Auditability needs more than a row count. Record the policy decision, actor or pseudonymous subject as permitted, poll configuration version, server timestamp, idempotency key, and a correlation identifier. Protect that record with least-privilege access and explicit retention. A tally mismatch should be repairable by replaying retained accepted votes into a fresh projection and comparing versions and counts, without treating the browser's state as evidence.

Reconnect from a snapshot, then accept newer versions

A reconnecting viewer has missed an unknown interval, so making it guess from its last transient message is fragile. On connection, fetch the current tally snapshot over an authenticated request, record version N, establish the realtime channel, and apply only messages with versions greater than N. To close the race between snapshot and subscription, either subscribe before fetching and buffer messages, or let the server resume from a cursor that covers the snapshot boundary.

WebSocket provides a bidirectional channel after its opening handshake, while WebRTC data channels add peer-to-peer transport machinery whose behavior and topology should be evaluated separately. Neither protocol supplies application-level vote idempotency, projection versioning, or audit retention. Those remain server contracts.

Here is the viewer-side state machine in Go. Apply accepts duplicates and reordering without rolling the display backward; a version gap triggers a snapshot refresh because the client cannot prove that the newest message alone is sufficient under every delta format. Order matters.

package pollview
import "sync"
type Snapshot struct {
Counts map[string]uint64
Version uint64
}
type View struct {
mu sync.Mutex
counts map[string]uint64
version uint64
refresh func() (Snapshot, error)
}
func (v *View) Apply(next Snapshot) error {
v.mu.Lock()
defer v.mu.Unlock()
if next.Version <= v.version {
return nil
}
if v.version != 0 && next.Version > v.version+1 {
fresh, err := v.refresh()
if err != nil {
return err
}
v.counts, v.version = fresh.Counts, fresh.Version
return nil
}
v.counts, v.version = next.Counts, next.Version
return nil
}

For full snapshots, a gap can also be accepted directly if the server contract states that every message completely replaces prior state. I still prefer refreshing after a suspicious gap when an authorization change, poll closure, or schema transition may have occurred - correctness is cheaper than debugging a plausible-looking stale total.

Presence should reconnect independently: losing an β€œonline” heartbeat must not invalidate an accepted vote or mutate the durable ledger. Test the race deliberately. Disconnect a client at version 41, accept votes through version 57, reconnect it while version 58 is being published, and assert that it converges to the authoritative snapshot without double counting. Repeat with duplicate versions, reversed delivery, expired credentials, and a poll that closes during reconnect. The important assertion is final state plus an audit correlation, not the number of callbacks observed.

Operate the boundary and know what you gave up

Observe accepted commands, duplicate submissions, projection lag, publication batches, active connections, reconnects, snapshot refreshes, and version gaps as separate signals. A single β€œmessages sent” graph hides the failure mode that matters. Reconcile the durable accepted-vote count against the sum of the materialized tally on a schedule and after deployments; alert on the mismatch, not merely on transport latency.

Deployment needs compatibility discipline. Add fields to tally messages before requiring clients to read them, keep unknown-field handling permissive, and version a breaking schema explicitly. During a rolling release, old and new publishers must agree on monotonic ordering for the same poll. Load tests should include bursty arrivals and reconnect storms rather than a smooth average, because reconnect traffic adds snapshot reads exactly when the broadcaster is already busy.

There is a real cost to the recommended design. By retaining only a bounded public version history, you cannot later reproduce every animation a viewer saw, and you cannot offer arbitrary event-level replay from the display channel. Investigations rely on the protected vote ledger and publication metadata instead. If regulation or product policy requires proof of each individual public delivery, this architecture is the wrong fit; use a durable per-recipient delivery log, accept its storage and privacy burden, and have counsel define the retention period.

The practical rule is narrow: preserve accepted votes for reconciliation, publish the smallest authoritative view users need, and make reconnect a snapshot-and-version protocol. Everything else should earn its multiplication by concurrent viewers.

References

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.