I Built an Open-Source Studio for Building, Testing, and Deploying AI Agents

I Built an Open-Source Studio for Building and Shipping AI Agents Building an AI chatbot is easy. Shipping one that has tools, knowledge, memory, observability, evaluations, human handoff, multiple model providers, workflows, and an actual interface your users can interact with is a different problem. That gap is what led me to build Chatbot Studio. It's an open-source platform for building AI agents and then publishing those agents as website chatbots or connecting them to channels such as WhatsApp. This isn't a SaaS announcement or a sales pitch. The project is MIT licensed, self-hostable, and available on GitHub: GitHub: https://github.com/judejulius/ChatbotStudio The problem I wanted to solve A lot of AI projects begin with something like this: response = client.chat.completions.create( model="...", messages=[...] ) Then reality arrives. You need tools. Then retrieval. Then credentials. Then streaming. Then conversation state. Then rate limits. Then evaluations. Then traces because something went wrong in production. Then a UI. Then somebody asks: "Can we put this on the website?" And someone else asks: "Can customers talk to a human if the AI gets stuck?" At that point you're no longer building a prompt around an LLM. You're building an agent platform. That is the problem Chatbot Studio is trying to explore. The core design: the agent and the chatbot are different things One architectural decision became especially important while building this. An agent should not be the same object as its presentation layer. The agent owns things such as: - model provider - model - system instructions - tools - MCP servers - knowledge - memory - skills - guardrails - sandbox settings - human-in-the-loop behavior A published chatbot owns the things that belong to the channel: - appearance - allowed domains - usage limits - launcher configuration - welcome experience - suggested prompts - publishing state In other words: Model + Knowledge + Tools | v Agent | test / evaluate | v Published Chatbot | Website / Channel The chatbot isn't a duplicated agent configuration. It's a channel sitting on top of the agent. That means I can improve an agent's instructions, knowledge, model, or tools without recreating every chatbot using it. The agent remains the source of truth. Building agents The main Agent Studio lets you configure and test an agent before publishing it. Currently the platform supports provider families including: - OpenAI - Anthropic - Google Gemini - Groq - OpenRouter - Ollama - OpenAI-compatible endpoints An agent can then be connected to knowledge bases, tools, MCP servers, memory, skills, guardrails, and other runtime controls. The test chat runs against the saved agent configuration. That part matters. I didn't want a playground where the test environment was secretly different from the thing that eventually gets deployed. The goal is: configure -> test -> evaluate -> publish rather than: prototype -> rewrite everything -> deploy something different MCP support One part I've been particularly interested in is Model Context Protocol support. Chatbot Studio can connect agents to MCP servers using: - stdio - SSE - streamable HTTP This makes external capabilities much easier to attach to an agent without baking every integration directly into the application. The broader architecture becomes something like: +----------------+ | Knowledge Base | +-------+--------+ | +----------+ +------v------+ | MCP Tool +----->+ | +----------+ | Agent | | | +----------+ +------+------+ | API Tool +------------+ +----------+ | v Conversation For me, MCP is interesting because it moves agent tooling toward a more interoperable ecosystem instead of every project inventing its own tool interface. RAG and knowledge bases Agents can also be connected to reusable knowledge bases. The backend currently handles sources including: - text - URLs - DOCX - Markdown - CSV - JSON Documents are processed into chunks and embeddings that can be retrieved during conversations. The important idea here was making knowledge a reusable platform resource rather than stuffing documents directly into one chatbot implementation. One knowledge base can therefore become part of a broader agent configuration. The chatbot is an actual Web Component For deployment on websites, I wanted the exported chatbot to be as framework-independent as possible. So the primary integration is a browser-native custom element: The component uses a Shadow DOM so the host website's CSS doesn't unexpectedly destroy the chatbot UI-and the chatbot doesn't leak its styles back into the host application. The element also exposes a small JavaScript API: const chatbot = document.querySelector("chatbot-widget"); chatbot.open(); chatbot.send("I need help with an order"); chatbot.close(); chatbot.reset(); And it emits events such as: chatbot-ready chatbot-error chatbot-handoff Because the core is a Web Component, framework integrations can stay fairly thin. Chatbot Studio can generate integrations for: - native HTML - React - Vue - Angular - WordPress Instead of maintaining five completely different chatbot implementations, they all revolve around the same browser-native element. The visual chatbot customizer I also wanted customization to use the real renderer. The editor therefore mounts the same widget implementation that gets exported. You can customize things such as: - launcher - window - header - greeting - suggested prompts - agent message bubbles - visitor message bubbles - composer - loading indicator - scroll controls - footer - dark/light behavior - lead capture - knowledge citations - streamed reasoning - human handoff - usage limits - allowed domains This avoids a problem I've seen in visual builders where the editor preview looks one way and the actual embedded component behaves differently. The preview and the shipped widget share the same rendering path. Human handoff AI shouldn't have to pretend it can solve every problem. So Chatbot Studio also has human handoff. Instead of handing a conversation off to an undefined "human", conversations can be routed into queues such as: General Support Technical Support Sales Billing Administrators control which users belong to which queues. The AI can hand the conversation to an appropriate queue, after which a person can take ownership of it. Once handed over, the assistant stops trying to answer the conversation as though nothing happened. This required treating handoff as application state rather than just another tool response. Agent teams and visual workflows Not every task belongs inside one large agent. The project therefore also supports agent teams and visual workflows. The workflow side supports concepts including: - DAG execution - conditional paths - input mapping - human approval - persisted runs - scheduled execution - execution traces The UI uses XYFlow for the visual workflow editor. This lets you move from: User -> Agent -> Response toward workflows such as: -> Research Agent ----- / \ Input -> Classifier -> Final Agent \ / -> Internal Data Tool -- And because workflow runs are persisted and traced, there is something to inspect when an orchestration path doesn't behave as expected. Evals and observability One lesson from working with LLM applications is that manually chatting with an agent is not a sufficient testing strategy. You eventually need repeatable evaluations. Chatbot Studio includes evaluation suites so test cases can be run repeatedly instead of relying only on intuition. There is also observability around things such as: - token usage - cost - latency - tool calls - traces - errors - sessions - visitor feedback There is also a prompt optimization workflow where generated improvements can be reviewed before being accepted. I deliberately wanted that last step to stay human-controlled. An optimizer can propose a prompt change. It shouldn't silently decide that production needs a new personality at 3 AM. The project has also grown beyond browser chatbots. There is a WhatsApp channel implementation using a Node.js bridge around Baileys. It handles things including: - QR pairing - authentication persistence - inbound messages - outbound messages - quoted replies - typing state - debounce windows - audio transcription - optional generated voice replies So the architecture is increasingly becoming: +--> Website Widget | Provider -> Agent -----+--> WhatsApp | +-------------> Workflows | +-------------> Agent Teams The channel shouldn't define the intelligence. The agent should. Under the hood Chatbot Studio isn't built as one giant application. The main pieces are: frontend/ backend/ widget/ wa-bridge/ Frontend The Studio UI is built with: Next.js 16 React 19 TypeScript Tailwind CSS Zustand XYFlow Backend The API and agent runtime use: Python 3.12+ FastAPI Pydantic Motor MongoDB APScheduler MCP Widget The embeddable chatbot is a separate JavaScript package built with esbuild. WhatsApp bridge WhatsApp transport runs as a separate Node.js service. Infrastructure The default Docker setup brings together: Next.js FastAPI MongoDB WhatsApp bridge with an optional nginx SSL profile. This separation has made it much easier to reason about where functionality actually belongs. Running it locally The project is designed to be self-hosted. You'll need: Node.js 20+ Python 3.12+ uv MongoDB 7 Clone the repository: git clone https://github.com/judejulius/ChatbotStudio.git cd ChatbotStudio Create your environment configuration: cp .env.example .env Install everything: npm run install:all Start MongoDB: npm run mongo Then run the development stack: npm run dev The frontend will be available on: http://localhost:3000 and FastAPI's API documentation on: http://localhost:8000/docs There's also a Docker Compose setup: docker compose up --build Make sure you replace the placeholder secrets in .env before running a real deployment. Security became part of the architecture Once a platform stores model-provider credentials, MCP crede