Your Agent Bills While It Waits. Here's the Fix.

The Core Problem

"Most of an agent's life is spent waiting - on a tool, a human, the next step - and the whole time you're holding a live process awake and billing for it." - Hamza Tahir, ZenML, AI Engineer 2026

That single sentence, dropped in an AI Engineer conference talk this week, names the unit-economics problem that no agent demo video ever mentions. The problem that decides which agent companies survive contact with a CFO.

Every agent product demo shows the same clip: the agent plans, calls tools, produces output - a continuous stream of productive work. What the demo never shows is what happens between those steps:

• The tool call that takes 8 seconds to return
• The human-approval gate where someone is in a meeting
• The retry backoff after a rate limit
• The polling loop waiting for a webhook
• The overnight batch job that parks a process for 6 hours

That is where your money goes. Not inference. Not model selection. The waiting.

The Numbers: Inference Is the Minority of Your Bill

A peer-reviewed study published this month - "The Harness Effect" - ran controlled experiments across enterprise agent deployments and found something that should alarm every engineering leader running agents in production:

• Model inference accounts for roughly 20% of total agent cost
• Infrastructure, orchestration, tooling, and governance eat the other 80%
• Switching orchestration layers (same model, different harness) reduced cost per task by 41% - from $0.21 to $0.12
• The orchestration layer moved cost per task more than switching between the cheapest and most expensive model did
• Quality per dollar rose 82% (3.71 to 6.75)
• Task-completions per million tokens increased 68%

CockroachDB's engineering team confirmed this pattern at scale: re-sent context alone - system prompts, tool definitions, and state history redundantly transmitted across model calls - accounts for 62% of total agent inference bills according to Stanford Digital Economy Lab research.

Goldman Sachs projects a 24-fold increase in token consumption by 2030, meaning these inefficiencies compound, not shrink.

The unit economics in one line: Agentic workflows consume 5-30x more tokens per task than standard chatbots. The multiplier is not from better reasoning - it is from orchestration overhead, retries, and idle context maintenance.

Anatomy of Agent Idle Time

Where does the waiting actually happen? Four categories dominate.

1. Tool Call Latency

Every external tool invocation is a network round-trip. A deep technical analysis of agent wall-clock budgets identified four unsynchronized clocks operating in parallel during a single tool call:

• The agent harness clock (starts at request dispatch)
• The model's clock (begins at first token arrival)
• The tool client clock (measures from tool call emission)
• The tool server clock (activates on actual request receipt)

Hidden latency accumulates before tool execution even starts: TTFT delays, streaming pipeline hops, MCP router overhead, tool worker queue backlog. A tool with an 8-second budget may have only 6.6 seconds from the agent's perspective after 1.4 seconds of transit overhead.

The failure modes are worse than the latency: orphaned successful results (tool finished after the agent gave up), duplicate tool calls from retry logic, and agents replanning despite correct responses that arrived a beat too late.

2. Human-in-the-Loop Approval Gates

The most expensive wait of all - and the most unpredictable. When an agent hits an approval step:

• Best case: 30 seconds (someone is watching)
• Typical: 5-30 minutes (approver is in a meeting)
• Worst case: Hours or days (overnight, weekend, vacation)

During all of this, a conventional agent harness holds a live process open - memory allocated, event loop spinning, connection pools warm. You are billing for sleep.

3. Retry Backoff and Rate Limits

When a tool returns a 429 or a transient error, the standard pattern is exponential backoff: wait 1s, then 2s, then 4s, then 8s. During each wait, the process is alive and metered.

Reddit practitioners report that agents retry-looping on bad tool outputs quietly burn 5-10x the expected token budget per task before a human notices.

4. Polling Loops and Event Waits

Agents waiting for async results - CI pipeline completion, deployment status, file processing - often implement polling. Every poll iteration is a model turn that consumes tokens, even when the answer is "not ready yet." Multiply by dozens of concurrent agents and you have a fleet burning budget on the computational equivalent of checking if the oven has preheated.

What Durable Execution Actually Changes

The fix is not "make agents faster." The fix is stop billing for the waiting.

Durable execution - the pattern implemented by Temporal, Inngest, Rivet Actors, and now Cloudflare Workflows - treats waiting as a continuation rather than a loop:

1. When an agent needs to wait (tool response, human approval, timer), the harness snapshots the full execution state - memory, call stack, pending continuations - to durable storage
2. The process terminates
3. Compute drops to zero
4. You pay nothing
5. When the signal arrives (tool response, approval webhook, timer fires), the harness restores the agent from the snapshot in milliseconds
6. Execution resumes from the exact point it left off, as if no time passed

The key insight from Inngest's engineering team: the waitForEvent call suspends the workflow entirely - no compute resources consumed while waiting. Their data shows human-in-the-loop suspend/resume patterns drop idle-pending cost by 60-80%.

Microsoft's Agent Framework, announced at BUILD 2026, built this principle in from day one: agents scale to zero, paying nothing while idle, and scale back up on the next request - with files, disk state, and session identity persisting across the scale-to-zero boundary.

Their CodeAct pattern goes further: instead of choosing a tool, waiting, choosing the next - the model writes a single Python program that calls tools via call_tool(), runs it once in a sandbox, and returns a consolidated result. One round-trip instead of ten.

The Emerging Stack: Who Solves What

The durable execution category for agents is not theoretical - it is shipping. Here is the current landscape.

Temporal - The Incumbent

Temporal's February 2026 Series D ($300M at $5B valuation, led by a16z) and its Replay 2026 release wave - Workflow Streams for LLM output, Serverless Workers, an OpenAI Agents SDK integration GA since March - signal a company repositioning hard around agents.

• Strengths: Seven language SDKs, deepest production track record, event-history replay semantics
• Trade-off: Operational complexity. You run the cluster (or pay for Temporal Cloud). Workflow versioning is genuinely hard

Inngest - The Developer Experience Play

Event-driven step functions with no stateful backend to operate. Their AgentKit is a first-party multi-agent framework with MCP tooling, backed by a $21M Series A (Altimeter, September 2025).

• Strengths: Zero infrastructure to manage. Step-function model maps cleanly to agent workflows. Built-in event triggers
• Trade-off: Less flexibility than Temporal for complex orchestration. Newer, less battle-tested at scale

Rivet Actors - Durable Execution as a Primitive

Rivet Actors are the lowest-level option: stateful workload primitives built explicitly for AI agents, collaborative apps, and durable execution. Open-source under Apache 2.0 - you own the infrastructure completely. Their companion project AgentOS runs coding agents inside isolated Linux VMs with an in-process operating system kernel. Everything runs inside the kernel; nothing executes on the host.

• Strengths: Maximum control. No vendor lock-in. Purpose-built for the agent use case
• Trade-off: You build more yourself. Less ecosystem than Temporal

The Observability Layer: SigNoz

You cannot optimize what you cannot see. SigNoz pivoted to agent-native observability - waterfall views of every model call, tool invocation, and reasoning step. Their Claude Agent SDK integration via OpenTelemetry gives you P99 latency on tool calls, token budgets per session, and alerts on loops that exceed cost thresholds.

This is the missing telemetry: which tool calls are slow, which approval gates are the bottleneck, where retry storms happen, and how much money each idle period actually costs.

The Real Cost Math: A Worked Example

Let us make this concrete. A production coding agent that:

• Runs 40 tasks per day
• Each task involves ~12 tool calls (file reads, shell commands, API calls)
• Average tool response time: 3 seconds
• Two tasks per day hit a human approval gate (average 15 minutes)
• Three tasks hit rate limits (average backoff: 12 seconds per retry, 3 retries)

Without durable execution:

• Tool waiting: 40 tasks x 12 calls x 3s = 24 minutes/day of pure waiting
• Approval gates: 2 x 15 min = 30 minutes/day
• Rate-limit backoff: 3 tasks x 3 retries x 12s = 108 seconds
• Total idle compute per day: ~56 minutes - all billed at active-process rates

Scale that to 20 agents and you have 18.7 hours of paid idle time per day.

With durable execution:

• Tool waits under 5s: handled inline (not worth the checkpoint overhead)
• Approval gates: suspended to zero compute. Resume on webhook. Cost: $0
• Rate-limit backoff >5s: suspended. Resume on timer. Cost: $0
• Idle compute eliminated: ~32 minutes/day per agent (the approval gate + long retries)

At typical cloud pricing, that is the difference between "agents are too expensive for anything but demos" and "agents pay for themselves."

What the Community Is Saying

The Hacker News thread "Improving 15 LLMs at Coding in One Afternoon - Only the Harness Changed" crystallized the consensus: the model and its harness form "a whole cybernetic system of feedback loops" where "the harness can make as much if not more of a difference, when improved, as improvements to the model itself."

Meanwhile, another HN discussion - "Effective harnesses for long-running agents" - dug into what long-running agents actually need: multi-agent judge setups, external memory, context management, and crucially, state persistence across crashes.

The thread "The agent harness belongs outside the sandbox" pushed even further: the harness itself - including its durable execution layer - must live outside any sandboxed environment, because the orchestration state needs to survive the sandbox tearing down.

On X, @naval summarized the convergence: "an agent harness that plays games, writes code, reasons like a physicist, and saturates the ARC-AGI-3 benchmark" - underscoring that the harness IS the product surface now, not the model inside it.

Practitioners are converging on the same conclusion from the cost side. The AI Engineer conference this week dumped four talks in a single day - feature flags, durable execution, signed receipts, and reward hacking - all arguing that agents lack the operational primitives web software got a decade ago.

The contrarian view: Durable execution adds real operational complexity - workflow versioning, replay semantics, state serialization, schema evolution. For teams running fewer than ~50 concurrent agents, the engineering overhead of Temporal may cost more in developer-hours than the idle compute it saves. The break-even is not zero. Know your scale before you adopt.

What This Means for You

If you are building agents today:

Instrument first. Before you optimize, you need to see where time goes. Add OpenTelemetry traces to every tool call. Track wall-clock time per step. SigNoz or any OTel-compatible backend will show you the idle-time breakdown.

Separate "fast waits" from "slow waits." Tool calls under 5 seconds are not worth checkpoint/restore overhead. Human approvals and retry storms over 10 seconds are where durable execution pays off immediately.

Choose your complexity budget. Temporal if you are enterprise-scale and have platform engineers. Inngest if you want minimal ops burden. Rivet if you want to own the stack. Cloudflare Workflows if you are already in their ecosystem.

Budget per agent, not per model. The Harness Effect paper showed blended-model cost per task falling 41% from harness optimization alone. Track cost at the task level, not the API-call level. That is where the real unit economics live.

Set circuit breakers. Reddit practitioners are unanimous: without MAX_LOOPS and per-task token ceilings, your agent will run until the billing alarm fires.

The prediction: Within 12 months, "how does your agent handle idle time?" will be a standard question in vendor evaluations and architecture reviews. The teams that treat it as an infrastructure problem - not a model-quality problem - will run 5-10x more agents at the same budget. That is the real competitive advantage of durable execution: not reliability (though you get that too), but unit economics that let you scale.

For more on harness engineering as the decisive layer, see our coverage of why the harness - not the model - determines agent reliability, memory as the new competitive moat, and observability for agent budgets.

Originally published at AgentConn