Schrödinger's Service: Why Your Microservice Is Both Up and Down Until You Observe It
DEV Community

Schrödinger's Service: Why Your Microservice Is Both Up and Down Until You Observe It

The Thought Experiment

In 1935, Schrödinger described a cat in a box that is simultaneously alive and dead - until you open the box and observe it. The point wasn't the cat. The point was this: The act of observation determines the state.

In quantum mechanics, a system doesn't have a definite state until it's measured. Before measurement, it exists in a superposition of possible states. Your microservices do the same thing.

The Problem: You're Observing the Wrong Thing

A health check is an observation. But it's a very specific one. It tells you: "the service responded to this particular request, at this particular moment, on this particular endpoint."

It does not tell you:

  • whether the database connection pool is exhausted
  • whether downstream dependencies are timing out
  • whether the cache is serving stale data
  • whether a specific code path that runs once every 10 minutes is broken
  • whether memory is slowly leaking under load

The service is in superposition. It's up and down, depending on which dimension you measure. Your health check collapses that superposition - but it collapses it into the state you chose to observe, not the state your users experience.

What Observers Actually Collapse

In quantum mechanics, different measurements reveal different properties. You can measure position or momentum, but not both precisely at the same time (Heisenberg uncertainty principle - a topic for another article 😅).

In distributed systems, different probes reveal different failure modes:

Observation What it collapses What it misses
/health ping Process is alive Everything else
DB connection check Can open a connection Pool exhaustion, slow queries
Dependency ping Dependency responds Degraded responses, timeouts
Synthetic transaction Happy path works Edge cases, specific user data
Error rate metric Something is failing What, and for whom

No single observation gives you the full picture. The state of your service is not a single value. It's a probability distribution across many dimensions. And you're sampling only a few of them.

The Observer Effect

In physics, the observer effect means that measuring a system can change it. Shine light on an electron to locate it, and you transfer momentum. The measurement perturbs the system.

Your monitoring does this too. A health check that opens a database connection consumes a connection from the pool. Under normal conditions, negligible. Under load, with an already-exhausted pool, your health check is now competing with real traffic for resources.

More subtle: a service that runs fine under monitoring traffic - periodic, lightweight, predictable - can fail under production traffic - bursty, heavy, unpredictable. You've been observing a different system than the one your users are using.

Superposition in Practice

Here are three real-world patterns where a service is simultaneously healthy and broken:

  1. The warm path is fine, the cold path is broken - Your health check hits the same cached response every time. A rarely-executed code path - triggered only by specific user conditions - has been broken for two weeks. Your health check has no idea.

  2. The replica is fine, the primary is not - Your health check reads from a replica. Writes are failing silently. Replication lag is growing. GET /health → 200. Users → errors.

  3. The process is alive, the logic is not - The service is up. It's processing requests. But it started consuming a config value from the wrong environment. Every response it sends is subtly wrong. Not a crash. Not a timeout. Wrong. Health check: green. Reality: broken.

Collapsing Into the Right State

The fix isn't better health checks. It's accepting that you need multiple, independent observations that together describe the service more completely.

  • Liveness - Is the process alive and not deadlocked? Simple. Lightweight. Just a ping. This is the Schrödinger question: is it dead or alive?
  • Readiness - Can it serve traffic right now? Checks immediate dependencies, connection pool availability, circuit breaker states.
  • Saturation - Is it approaching its limits? Memory, connection counts, queue depth, thread pool utilization. The warning signs that appear before failure.
  • Correctness - Is it producing the right results? Synthetic transactions that exercise real logic with known inputs and expected outputs. This is the hardest one, and the one most teams skip.
  • User-facing signals - What are real users actually experiencing? Error rates by endpoint. Latency percentiles. Business metrics. The only observation that truly matters.

Each of these collapses the superposition differently. Together, they give you a much more complete picture of the actual state.

The Uncertainty Budget

You will never fully observe your system. There's always a dimension you haven't measured. A failure mode you haven't anticipated. This is the engineering equivalent of the uncertainty principle: you can reduce uncertainty, but not eliminate it.

What you can do is be explicit about your observability budget:

  • Which states do you observe?
  • Which states are you blind to?
  • What are the consequences if those unobserved states are broken?

Most teams don't answer these questions. They add a health check, mark the service as monitored, and move on. But an unobserved failure is still a failure. The cat doesn't care whether you've opened the box!

Practical Checklist

Before marking a service as "observable":

  • Liveness probe: Is the process alive?
  • Readiness probe: Can it serve traffic? Does it check real dependencies?
  • Saturation metrics: Memory, connections, queue depth - with alerting thresholds
  • Synthetic transaction: Does a real user flow actually work end-to-end?
  • User-facing error rate: Are real requests succeeding?
  • Rare code paths: When did you last test the paths your health check never touches?

If you can't answer all six, your service is still in superposition.

Conclusion

Schrödinger's cat isn't a paradox about cats. It's a statement about the limits of observation. Your microservice isn't healthy because your health check says so. It's healthy because you've constructed enough independent observations to be confident that the state your users experience matches the state you're measuring.

That's a much higher bar. Most services never reach it.

A 200 from your health check doesn't mean the service is up. It means the service was up, for that probe, at that moment. Everything else is superposition.

Comments

No comments yet. Start the discussion.