AutoGen's hidden token tax: why a 3-agent chat costs 15 what you expect

Cost-audit series, episode 2

This series began with an AI agent that burned 136M tokens overnight →.

AutoGen is Microsoft's multi-agent framework. It's genuinely good at orchestrating agents that hand off work to each other. But its default memory model has a cost shape that surprises almost every team that hits it in production. This audit shows you exactly where the tokens go, with line numbers.

The setup: a 3-agent RoundRobin chat

The canonical AutoGen pattern is a RoundRobinGroupChat with N agents taking turns on a task. Here's the minimal version from the docs:

from autogen_agentchat.agents import AssistantAgent
from autogen_agentchat.teams import RoundRobinGroupChat
from autogen_agentchat.conditions import MaxMessageTermination

planner = AssistantAgent("planner", model_client=client, system_message="You plan.")
coder = AssistantAgent("coder", model_client=client, system_message="You code.")
reviewer = AssistantAgent("reviewer", model_client=client, system_message="You review.")

team = RoundRobinGroupChat(
    [planner, coder, reviewer],
    termination_condition=MaxMessageTermination(max_messages=10),
)

await team.run(task="Build a web scraper for Hacker News.")

Three agents, 10 turns total (~3-4 turns each). Seems cheap. It isn't.

The default context: unbounded, per-agent

Every AssistantAgent gets its own UnboundedChatCompletionContext by default:

# autogen-agentchat/src/autogen_agentchat/agents/_assistant_agent.py, __init__ (L708)
if model_context is not None:
    self._model_context = model_context
else:
    self._model_context = UnboundedChatCompletionContext()

UnboundedChatCompletionContext.get_messages() returns self._messages - the full list, no cap, no truncation:

# autogen-core/.../model_context/_unbounded_chat_completion_context.py (a ~20-line file)
async def get_messages(self) -> List[LLMMessage]:
    """Get at most `buffer_size` recent messages."""
    return self._messages

(The docstring says "at most buffer_size" - that's a copy-paste artifact from BufferedChatCompletionContext. There is no buffer. It returns everything.)

The handoff tax: every agent sees every message

When an agent's turn arrives, on_messages_stream adds all incoming messages to its own context before calling the LLM:

# _assistant_agent.py, in on_messages_stream (STEP 1: "Add new user/handoff messages
# to the model context")
await self._add_messages_to_context(
    model_context=model_context,
    messages=messages,  # ← the full message_thread from the group manager
    ...
)

And _add_messages_to_context appends each one:

# _assistant_agent.py, static method _add_messages_to_context
await model_context.add_message(llm_msg)
...
await model_context.add_message(msg.to_model_message())

The group manager (BaseGroupChatManager) maintains a single _message_thread and appends every response to it:

# _base_group_chat_manager.py
self._message_thread: List[BaseAgentEvent | BaseChatMessage] = []
...
await self.update_message_thread(delta)  # called after every agent response

So at turn T, the agent receiving the baton gets T-1 messages added to its already-growing context. Its context now contains everything it has ever seen.

The math: O(N × T²) total tokens

Let's be precise. Define:

• T = total turns in the conversation
• N = number of agents
• m = average tokens per message (system prompt + response, ~300 tokens is realistic for a coding task)

Each agent speaks every N turns. When agent i speaks on turn t, its context contains all t-1 prior messages (because it has been accumulating them since turn 1). Tokens consumed by agent i on turn t: context_tokens(t) = (t - 1) × m

Total tokens for agent i across all its turns (it speaks at turns N, 2N, 3N, … up to T):
Σ (kN - 1) × m for k = 1 to T/N ≈ m × N × (T/N)² / 2 = m × T² / (2N)

Total tokens across all N agents: N × m × T² / (2N) = m × T² / 2

The N cancels. Total cost scales as T² regardless of how many agents you add.

Worked example: 10 turns, 3 agents, 300 tokens/message

Naïve expectation (10 calls × 300 tokens each): 3,000 tokens
Actual: ~16,500 tokens - 5.5× more.

At 20 turns it's ~63,000 tokens vs 6,000 expected - 10.5× more.
At 30 turns: ~139,500 tokens vs 9,000 - 15.5× more.

The multiplier grows linearly with T. This is the same O(T²) shape as ConversationBufferMemory in LangChain - but AutoGen's version is per-agent, so it's easy to miss in per-call logs.

Why per-call logs hide this

If you're watching your LLM provider's per-call token counts, you see something like:

call 1:   300 tokens ✓ cheap
call 2:   600 tokens ✓ fine
call 3:   900 tokens ✓ ok
...
call 10: 3,000 tokens ← this one looks expensive

Each call looks like a modest increase. The cumulative total - 16,500 - only shows up when you sum across the run. Most observability dashboards show per-call costs, not per-run totals. The runaway is invisible until the bill arrives.

The fix: cap the context

AutoGen ships two bounded alternatives, named in the AssistantAgent.__init__ docstring (around L1034 of _assistant_agent.py):

• BufferedChatCompletionContext (limits message count)
• TokenLimitedChatCompletionContext (limits tokens)

Option 1: BufferedChatCompletionContext (sliding window)

from autogen_core.model_context import BufferedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=BufferedChatCompletionContext(buffer_size=5),  # last 5 messages
)

Cost shape becomes O(T × buffer_size) - linear. For buffer_size=5 and 30 turns: ~42,000 tokens vs 139,500 unbounded. 3.3× cheaper.

Option 2: TokenLimitedChatCompletionContext (token budget)

from autogen_core.model_context import TokenLimitedChatCompletionContext

coder = AssistantAgent(
    "coder",
    model_client=client,
    model_context=TokenLimitedChatCompletionContext(token_limit=2000),
)

Caps the context at a fixed token budget. More predictable than a message count because message sizes vary.

Which to use?

Detecting this in CI before it hits production

The pattern is detectable statically: any file that instantiates AssistantAgent without a model_context= argument is using the unbounded default.

# Flag unbounded AssistantAgent instantiations
grep -rn "AssistantAgent(" src/ | grep -v "model_context="

For dynamic detection - measuring actual token growth across a run - this is exactly what tokenscope does: it instruments LLM calls, tracks per-run cumulative cost, and can block a CI build when a PR's token delta exceeds a threshold. The wartzar-bee/ci-guardrail GitHub Action wraps tokenscope into a one-line workflow addition:

- uses: wartzar-bee/ci-guardrail@v1
  with:
    token_threshold: 50000  # block if PR adds >50k tokens/run
    github_token: ${{ secrets.GITHUB_TOKEN }}

Summary

The default UnboundedChatCompletionContext is correct for short tasks and full-history use cases. It becomes a cost trap in long multi-agent conversations. The fix is one constructor argument - but you have to know to add it.

The broader pattern: every major agent framework defaults to unbounded context because it's the safest correctness choice. Cost is a second-class citizen in the default config. That's the gap this series documents.

Next in the series: CrewAI - the delegation overhead. How hierarchical agent trees multiply your token bill.

• tokenscope on npm
• wartzar-bee/ci-guardrail
• @wartzarbee on dev.to