RDAI: A Multi-Brain Python SDK for Self-Healing AI

RDAI: Building a Self-Healing Multi-Provider AI Orchestrator for Python One Python SDK. Any AI Provider. Automatic Failover. Modern AI applications are increasingly built on APIs from multiple LLM providers. Gemini. OpenAI. Groq. Claude. DeepSeek. New models appear constantly, and each provider brings different capabilities, pricing, latency, limits, and failure modes. But there is a problem that is easy to overlook: What happens when the AI provider your application depends on suddenly stops responding? That question became the starting point for RDAI. RDAI is an open-source Python AI orchestration SDK and CLI designed around a simple idea: An AI provider failure should not have to become an application failure. 🚨 The Problem: Your AI Provider Is a Dependency A typical application starts simply: Your Application | v One AI Provider | v Response Then reality happens. A provider can hit a rate limit, time out, become temporarily unavailable, reject a request, or experience an outage. Suddenly: Your Application | v One AI Provider | Provider fails | v Application fails The natural response is to add retries and another provider: try: response = gemini.generate(prompt) except: try: response = openai.generate(prompt) except: response = groq.generate(prompt) That may be enough for a prototype. But as the application grows, so does the surrounding infrastructure: - retry logic - timeout handling - provider selection - API-key discovery - health checks - fallback chains - recovery behavior - debugging and diagnostics I wanted that complexity to live in one reusable layer instead of being rewritten inside every application. That layer became RDAI. 🧠 From One Brain to Multiple Brains I like thinking about LLM providers as different brains. Each brain can have different strengths and availability. Instead of an application being permanently coupled to one provider, RDAI places an orchestration layer in the middle: YOUR APPLICATION | v +---------------+ | RDAI | | AI Orchestrator| +-------+-------+ | +-------------+-------------+ | | | v v v Gemini OpenAI Groq | | | +-------------+-------------+ | FAILOVER Your application talks to RDAI. RDAI handles the provider layer. That separation is the foundation of the project. ⚡ How RDAI Routes Requests RDAI provides two routing strategies. Smart Routing Smart routing can use request traits and provider capabilities to prefer a better-fit available provider. Conceptually: Prompt | v Intent / Trait Detection | v Provider Matching | v Best-Fit Available Brain | failure? v Next Available Brain The current implementation keeps this intentionally lightweight: request signals such as coding-oriented terms or speed-oriented terms can influence provider ordering, while each provider exposes traits that the router can match. The goal is not to make routing mysterious. The goal is to make the default behavior useful while keeping the architecture extensible. Manual Priority Sometimes developers already know exactly how their provider chain should work. For example: strategy: manual providers: - openai - groq - gemini This gives you a predictable preference order. If the preferred provider fails, other available providers remain eligible for fallback. OpenAI | | failure v Groq | | failure v Gemini No provider-specific fallback code is required in the application. 🛡️ Automatic Failover The most important part of RDAI is not the first response. It is what happens when that response cannot be produced. The core failover flow is: REQUEST | v Provider A | +------+------+ | | SUCCESS FAILURE | | v v RETURN Provider B | +------+------+ | | SUCCESS FAILURE | | v v RETURN Provider C RDAI also maintains circuit-breaker state so repeatedly failing providers can temporarily be skipped before being probed again. In practical terms: Route ↓ Call provider ↓ Failure? ├── No → return response └── Yes ↓ update failure state ↓ try next eligible brain Reliability becomes part of the application architecture instead of an afterthought. 🩺 A CLI That Treats AI Infrastructure Like Infrastructure I did not want RDAI to be just another Python wrapper. The project also includes a CLI for setup, diagnostics, configuration, and benchmarking. rdai init Start with: pip install rdai Then: rdai init The interactive wizard lets developers choose providers and a routing strategy, then creates the local configuration files. The intended workflow is: Install ↓ Configure ↓ Verify ↓ Generate rdai doctor Before shipping, you need to know which brains are actually reachable. Run: rdai doctor The diagnostic flow checks configured provider credentials and performs live checks, reporting status and measured latency. Conceptually: Provider Key Status Live Check Latency ------------------------------------------------------ Gemini ✔ DETECTED 🟢 ALIVE 420ms Groq ✔ DETECTED 🟢 ALIVE 180ms OpenAI ✔ DETECTED 🔴 ERROR - That turns a vague question like: "Why isn't my AI application working?" into a much more actionable question: "Which provider is failing, and is it authentication, connectivity, rate limiting, timeout, or another provider-side error?" 🔐 Configuration Without Hardcoding Secrets RDAI separates routing configuration from credentials. Provider credentials can be discovered from the process environment or a local .env file, while routing policy can live in rdai.yaml . For example: GEMINI_API_KEY=your_key_here GROQ_API_KEY=your_key_here OPENAI_API_KEY=your_key_here And: strategy: smart providers: - gemini - openai - groq Environment values take precedence over .env values. The result is a cleaner separation: Secrets ↓ Environment / .env Routing Policy ↓ rdai.yaml Application Logic ↓ RDAI SDK 💻 One Python API Across Multiple Providers Without an orchestration layer, applications often end up learning several provider SDKs. With RDAI, application code can stay focused on the task: from rdai import AI ai = AI() response = ai.generate( "Explain dependency inversion in simple words." ) print(response) The public API stays small while provider-specific adapters live underneath it. 🎯 Overriding Default Models Different applications may prefer different models. RDAI supports model overrides without changing the overall orchestration API: from rdai import AI ai = AI( models={ "gemini": "gemini-1.5-flash", "groq": "llama3-8b-8192", } ) response = ai.generate("Hello world!") The architecture keeps provider selection separate from the application-facing API. 🔌 Bring Your Own Model The AI ecosystem moves quickly. A useful orchestration layer should not have to know every provider that will ever exist. That is why RDAI exposes a BaseProvider abstraction for custom adapters. from rdai.providers.base import BaseProvider from rdai import AI class CustomProvider(BaseProvider): def generate(self, prompt, **kwargs): return "Private model response" ai = AI( providers=[CustomProvider(api_key="your_key")] ) print(ai.generate("Hello custom engine!")) This opens the door to private endpoints, internal models, and future providers without changing the application-level API. 🏗️ The Architecture At a high level, RDAI separates responsibilities into distinct layers: Environment (.env / vars) | v Credential Discovery | v Provider Registry | v Smart / Manual Router | v Failover Engine | v Provider Adapters | v Response That separation matters. The application should not need to know how Gemini, OpenAI, Groq, Claude, or a custom provider implements its API. The adapter handles that boundary. The router decides what to try. The failover layer decides what happens when a provider fails. 🌍 Why This Matters for AI Applications There is a larger architectural shift happening in AI development. We are moving from: One Application | One Model | One Provider toward: One Application | Orchestration Layer | Multiple Models | Multiple Providers That does not mean every application needs ten providers. It means the application can be designed so that provider choice is not an irreversible architectural decision. That is the space RDAI is designed to explore. 🧪 Building RDAI: From Prototype to Hardened Release The first versions focused on proving the core idea: - multi-provider adapters - routing - automatic failover - configuration discovery - an interactive CLI - diagnostics Then real testing exposed places where the implementation needed to become stronger. The 1.0.2 release focused on hardening the foundation: unifying the routing/failover architecture, resolving a missing runtime dependency, adding explicit REST-provider timeouts, and improving diagnostic error classification. That process reinforced an important lesson: Resilience is defined by the failure path, not just the happy path. A demo can prove that an AI response works. A real developer tool must also explain what happens when it doesn't. 🧭 RDAI vs. Building the Fallback Layer Yourself There are two broad approaches. Build it inside every application Application | +-- Provider A logic +-- Provider B logic +-- Retry logic +-- Timeout logic +-- Health checks +-- Fallback logic +-- Diagnostics This can work, but the infrastructure becomes part of every application. Put the orchestration behind an SDK Application | v RDAI | +-- Routing +-- Failover +-- Provider adapters +-- Diagnostics +-- CLI The application stays smaller, while the resilience layer can evolve independently. That is the problem RDAI is trying to solve. ✨ The Developer Experience Matters A technically capable system can still be frustrating to use. I wanted the RDAI workflow to feel simple: pip install rdai then: rdai init then: rdai doctor then: from rdai import AI ai = AI() response = ai.generate("Build a resilient AI workflow.") The goal is not to make developers think about orchestration every time they make an AI call. The goal is to make the resilient path the easy path. 🔭 What's Next? The next direction for RDAI is moving from resilience toward visibility. The roadmap includes live streaming-oriented features such as: Live Streaming | +--