Intelligence Is a Budget: The 4-Level Pyramid I Use to Run AI Agents in Production

Running a general-purpose agent harness on a large model burned 1,000+ tokens just to log a coffee.

The first version of my personal expense tracker ran inside a general-purpose AI agent harness backed by a large model smart enough to pass a financial analyst exam, even though its entire job was to log “coffee, 120”.

When I first wired the bot up in a messenger, I took the obvious shortcut: I connected incoming messages straight to a full agent harness with tool calling.

The problem was not just the size of the model. It was the combination of a heavy model and a broad, general-purpose agent harness. Because a general-purpose harness is built to handle almost any task, it is not specialized for any single workflow. In practice, it burns roughly the same baseline token overhead on logging a coffee as it does on investigating a DevOps incident.

Every message forced the harness to reload AGENTS.md instructions, the full catalog of available tools, tool-selection reasoning, and tool-call requests. Logging one coffee cost 1,000+ tokens, and whenever the agent inspected state or chained calls, a single entry reached tens of thousands of tokens for a parsing job that only needed a few hundred.

Too much freedom also hurt reliability. Within a week of running daily entries in one long chat session, context piled up, responses slowed down, and the model started drifting on repetitive entries. Because an unconstrained LLM is non-deterministic, sending the exact same expense item on different days produced slightly different category names or landed in different categories altogether.

That pushed me toward a simpler architecture: replace the open-ended agent with a deterministic backend application, keep just a drop of intelligence at the entry point to turn voice or text into structured data, and tune the application rules and the parsing prompt separately. Model cost dropped close to zero, latency disappeared, and the data became consistent.

Building and breaking that tracker shaped how I design all of my AI systems today as a Senior Backend Engineer and AI Solutions Architect at a private marketing company in Thailand.

A Continuous Balance, Not Rigid Walls

Coming from traditional backend engineering (PHP, Go, Python, SQL, Redis, RabbitMQ, and Elasticsearch), I am used to treating compute and memory as budgets. Combining heavy models with broad agent harnesses creates a huge space for optimization once I treat intelligence the same way.

I map my stack as a 4-Level Intelligence Pyramid with the Human at the apex. The levels of this pyramid are not rigid walls between isolated tools. They represent a continuous balance between autonomy and rules.

Moving up the pyramid adds more intelligence, broader freedom of action, higher token cost, and more variance. Moving down the pyramid tightens the rules, increases speed and determinism, and handles much higher volume.

At the very top sits the Human (ideas, product intent, and final accountability). Below that apex sit four execution layers, and any workflow can slide up or down between them depending on how much freedom it actually needs.

Level 1: Strategic Agents (Architecture and Direction)

Right below the human apex sit Level 1: Strategic Agents. This is where top reasoning models and broad exploration actually belong.

When I start a new service or redesign a workflow, I intentionally state my vision at a high level so low-level implementation details do not clutter the core idea. A Strategic Agent helps me expand that vision: weighing trade-offs across data models, service boundaries, and security rules, and deciding which parts of the system genuinely need model intelligence and which parts should be plain code.

Because architectural planning happens a few times per feature rather than thousands of times a day, paying for broad context and deep reasoning here makes sense.

A Level 1 agent never types production code or touches live servers. Its output is an approved technical brief for the level below.

Level 2: Dev-Team Agents (Full Agents Inside a Static Pipeline)

Once I approve the brief, the work drops to Level 2: Dev-Team Agents, where strong models turn that brief into working code.

When coding agents are given too much freedom inside an open-ended harness, two things happen: they make unrequested decisions outside their scope, and side problems pull them away from the original plan.

To prevent that, my development workflow is structured like an application with a strict, static pipeline:

• a lead agent splits the approved brief into small, atomic tasks,
• developer agents write the code,
• an independent review agent checks the diff against the spec,
• and automated test suites gate the merge.

Inside each step of that static pipeline, however, the worker is still a full AI agent. A developer or reviewer agent has real autonomy to inspect files, run commands, and solve problems, but it is constrained by three things:

• a narrow task,
• an isolated context,
• and a fixed set of tools matched to its specialty.

Splitting work into small, single-task sessions eliminates roughly 90% of context bloat while keeping every agent on track.

Level 3: Apps with a Drop of AI (Scripts Decide, Small Model Structures the Data)

It is easy to confuse Level 2 and Level 3 because both combine code and AI, but the degree of freedom is completely different:

• At Level 2, the outer pipeline is fixed, but each step runs a full agent that chooses which tools to call and how to solve its narrow task.
• At Level 3, there are no agents and no tool-calling choices at all.

Level 3 is essentially a Level 4 script with a drop of intelligence at the entry point: whenever incoming data is unstructured, a small model (optionally paired with speech-to-text) converts that raw input into a strict schema first, and then deterministic scripts execute the rest of the structured flow.

My rebuilt expense tracker lives here. When I send a voice note or text message, speech-to-text transcribes the audio and a small (~24B parameter) self-hostable model extracts amount, currency, and a category chosen strictly from a fixed list stored in my database. Standard backend scripts then show me a quick confirmation, write the validated record to the database, or run spending queries over any date range.

Whenever I want to improve the tracker, I can either adjust the deterministic application rules or tune the narrow parsing prompt independently.

When I build Level 3 services, I follow one rule: I don't hope the model won't fail. I assume it will. Because I assume the model can fail at any moment, the model never controls application flow or touches storage directly. Deterministic scripts wrap that single inference call on both sides and validate every field before anything is saved.

Level 4: Plain Scripts (Pure Tools, Already Structured Data)

At the base of the pyramid sits Level 4: Plain Scripts, where inputs are already structured and there is no unstructured data to interpret at all.

Level 4 serves two roles in my stack:

• Standalone static workflows that run on a schedule or a button press with zero AI.
• Hard-coded CLI tools for higher layers: when an agent at Level 2 or an app at Level 3 needs to trigger a critical action, it calls a Level 4 script with strict rules and predefined parameters so it cannot improvise flags or break the system.

My deployer is the clearest example. When I tried letting capable agents run deployments directly, they occasionally added extra steps or skipped checks. Now my deployer is a pure Level 4 tool built on standard deterministic scripts. Whether I trigger a release from a button or through a voice command, the script runs the exact same sequence every time: fast, cheap, and identical on every run.

Sliding Up and Down the Pyramid: Start Smart, Then Make It Dumber

Each level does only what it is best at. The smarter the level, the fewer the tasks.

As the pyramid diagram above shows, this architecture works in two ways at once.

First, a single project flows top-to-bottom through the layers: I start with an Idea at the apex, Level 1 turns it into an architectural Plan, Level 2 agents build the separate Parts inside a static pipeline, Level 3 apps Assemble unstructured runtime inputs into structured flows, and Level 4 scripts mechanically Polish, test, and deploy the result.

Second, individual workflows migrate down the pyramid over time as I learn their edge cases (start smart, then make it dumber):

1. Explore at Level 1 or Level 2: When a task is new and unpredictable, I use high-level agents to discover what real inputs look like and where the edge cases hide.
2. Tighten the pipeline at Level 2: Where agents still need to reason and use tools, I lock their outer workflow into a static state machine with narrow tasks and isolated contexts.
3. Compile into Level 3 and Level 4: Where intelligence is only needed to turn unstructured data into a schema, I replace the agent with a single small-model parse inside a Level 3 script. Where the input is already structured, I lock the logic into a pure Level 4 script.

This also answers how a system keeps its context when tasks move down to smaller models or scripts. Persistent memory does not have to stay in a raw chat history reloaded on every turn. In my setup, each lower level of the pyramid is the compiled context from the level above, materialized as static pipelines, fixed database categories, and deterministic scripts.

Spend reasoning tokens where real decisions happen, and lock repeatable patterns into rules and scripts. Each level of the pyramid does only what it is best at, and the smarter the level, the fewer tasks it handles.

Whenever a workflow becomes predictable, I dial down the open-ended intelligence and dial up the rules. Spend the intelligence budget where real decisions happen, and let deterministic code handle the rest.

I am Mykola (Nick) Skalenko, a Senior Backend Engineer and AI Solutions Architect building distributed backend systems (PHP, Go, Python) and multi-agent engineering workflows. More of my architecture notes are on LinkedIn and X (@NikolaySkalenko).