Building RestaurantOS AI: Observable Multi-Agent Restaurant Orchestration with OpenTelemetry and SigNoz

Learn how we built an observable AI-powered restaurant operating system using OpenTelemetry, SigNoz, Prisma, and multi-agent architecture.

Modern restaurants generate enormous amounts of operational data every day. Forecasting demand, tracking inventory, minimizing food waste, and purchasing ingredients are all interconnected decisions that traditionally require significant manual effort. To automate these processes, we built RestaurantOS AI - an AI-powered Restaurant Operating System driven by specialized autonomous agents.

But there was one major challenge. Large Language Model (LLM) agents don't behave like traditional software. They are probabilistic. They make decisions. They call multiple services. They generate intermediate reasoning. Without observability, debugging becomes nearly impossible. That is why OpenTelemetry and SigNoz became core components of our architecture - rather than optional monitoring tools.

In this article, we'll walk through how we built an observable multi-agent system that makes every AI decision transparent.

Why RestaurantOS AI?

Restaurant managers constantly answer questions like:

• How many customers should we expect today?
• Which ingredients will run out?
• Which food items are close to expiry?
• What should we purchase from suppliers?
• How can we reduce food waste?

Instead of solving these manually, RestaurantOS AI coordinates multiple specialized AI agents that collaborate together.

Multi-Agent Architecture

RestaurantOS AI uses five specialized agents orchestrated through a sequential pipeline.

+--------------------------------------+
|         User Request                 |
|   e.g. "Tomato supply shortages"     |
+------------------+-------------------+
                   |
                   v
+--------------------------------------+
|         Supervisor Agent             |
|  Routes, Orchestrates & Synthesizes  |
+------------------+-------------------+
                   |
                   v
┌────────────────┐┌─────────────────┐┌────────────────┐┌─────────────────┐┌────────────────┐
│   Stage 1      │ ---> │   Stage 2      │ ---> │   Stage 3      │ ---> │   Stage 4      │ ---> │  Final Output  │
└────────────────┘└─────────────────┘└────────────────┘└─────────────────┘└────────────────┘
        |               ||               |               ||               |
        v               vv               v               vv               v
┌────────────────┐┌─────────────────┐┌────────────────┐┌─────────────────┐┌────────────────┐
│  Demand Agent  │ ---> │ Inventory Agent│ ---> │  Waste Agent   │ ---> │ Purchase Agent │ ---> │    Response    │
│                │      │                │      │                │      │                │      │   Synthesis   │
│ Forecast POS   │      │ Detect Stock   │      │ Reduce Waste   │      │ Generate POs   │      │ Final Decision│
│ Sales Demand   │      │ Deficits       │      │ & Promotions   │      │ Supplier Logic │      │   Returned    │
└────────────────┘└─────────────────┘└────────────────┘└─────────────────┘└────────────────┘

Supervisor Agent acts as the orchestrator. Responsibilities:

• Understands user intent
• Selects workflow
• Coordinates agents
• Synthesizes final response

Demand Forecast Agent uses historical POS sales to estimate:

• Customer traffic
• Menu demand
• Peak hours

Inventory Intelligence Agent compares projected demand against current inventory. Detects:

• Ingredient shortages
• Overstock
• Critical stock levels

Waste Reduction Agent identifies ingredients approaching expiry. Suggests:

• Chef specials
• Promotional meals
• Smart inventory utilization

Purchase Decision Agent creates supplier purchase recommendations by considering:

• Stock deficits
• Vendor pricing
• Lead times
• Bulk purchasing

Integrating OpenTelemetry

To trace every component of our backend, we initialized the OpenTelemetry Node SDK before the Express application starts.

import { NodeSDK } from ' @opentelemetry/sdk-node ' ;
import { OTLPTraceExporter } from ' @opentelemetry/exporter-trace-otlp-http ' ;

const sdk = new NodeSDK ({
    traceExporter ,
    instrumentations : [
        new HttpInstrumentation (),
        new ExpressInstrumentation (),
        new PrismaInstrumentation (),
        getNodeAutoInstrumentations ()
    ]
});

sdk . start ();

This automatically traces:

• HTTP requests
• Express middleware
• Prisma queries
• External network calls

Custom Agent Instrumentation

Auto instrumentation isn't enough for AI systems. We created custom spans around every agent execution.

return tracer . startActiveSpan ( `Agent ${ agent . name } ` , async ( span ) => {
    span . setAttribute ( " agent.name " , agent . name );
    const result = await agent . execute ( input );
    span . setStatus ({ code : SpanStatusCode . OK });
    span . end ();
});

Each span records:

• Agent name
• Execution duration
• Success/failure
• Events
• Exceptions

This lets us visualize every agent independently inside SigNoz.

LLM Observability

LLM requests deserve their own telemetry. We adopted OpenTelemetry GenAI Semantic Conventions.

span . setAttribute ( " gen_ai.request.model " , model );
span . setAttribute ( " gen_ai.usage.total_tokens " , response . usage . total_tokens );
span . setAttribute ( " gen_ai.latency_ms " , durationMs );

Now every LLM request captures:

• Prompt tokens
• Completion tokens
• Total tokens
• Model
• Latency

Exactly what production AI applications need.

SigNoz MCP Integration

During development we also used the official SigNoz MCP Server. It enabled:

• Dashboard generation through natural language
• Trace exploration
• Log investigations
• Alert creation
• Querying ClickHouse metrics directly

Instead of manually configuring dashboards, the AI assistant generated many of them automatically.

End-to-End Trace Flow

When a user submits a request:

POST /ai/query
│
├── Demand Agent
│   └── LLM Request
│
├── Inventory Agent
│   ├── Prisma Query
│   └── LLM Request
│
├── Waste Agent
│   └── LLM Request
│
└── Purchase Agent
    ├── Prisma Query
    └── LLM Request

Inside SigNoz we can immediately identify:

• Slow database queries
• Slow LLM calls
• Token-heavy prompts
• Failed agents
• Error propagation

This transformed debugging from hours into minutes.

Real-world Problems We Solved

1. OpenTelemetry Loaded Before dotenv
   Since OpenTelemetry was initialized using Node's --import, it executed before our application loaded environment variables. As a result, the OTLP exporter ignored our cloud configuration.
   Solution

   import dotenv from " dotenv " ;
   dotenv . config ();
   

   Load environment variables before initializing OpenTelemetry.

2. PostgreSQL Port Conflict
   Native PostgreSQL on Windows occupied port 5432, causing Prisma to connect to the wrong instance.
   Solution
   Expose Docker PostgreSQL on 5435 instead.

   ports :
     - " 5435:5432"
   

3. Docker Volume Credential Caching
   Changing POSTGRES_PASSWORD had no effect because PostgreSQL initializes credentials only once.
   Solution
   Delete the existing volume.

   docker rm -f restaurantos-postgres
   docker volume rm restaurantos_ai_postgres_data
   docker compose up -d postgres
   

Key Learnings

✅ AI systems require domain-specific observability.
✅ OpenTelemetry makes every agent execution traceable.
✅ SigNoz provides a complete end-to-end visualization.
✅ Token usage should be monitored just like CPU or memory.
✅ Database spans reveal bottlenecks that AI latency often hides.

Most importantly: Observability turns AI from a black box into production software.

Conclusion

RestaurantOS AI combines specialized AI agents with production-grade observability. By integrating OpenTelemetry and SigNoz, every HTTP request, database query, LLM call, and autonomous decision becomes traceable.

Instead of wondering why an agent behaved a certain way, we can inspect the complete execution path in seconds. As AI systems continue becoming more autonomous, observability will become just as important as the models themselves.

If you're building AI applications for production, start instrumenting early - you'll thank yourself later. Happy Building! 🚀