My Engineering Passion: Why I built a sub-15ms proxy to stop LLM agent crashes

This is a submission for Weekend Challenge: Passion Edition What I Built

I'm honestly obsessed with system reliability. Nothing frustrates me more than seeing a brilliantly engineered autonomous AI agent completely crash just because of a random 502 Bad Gateway or 429 Too Many Requests from an upstream provider like OpenAI. I became so passionate about solving this specific edge case that I spent the last few months building Selixes - a high-performance edge proxy that sits in front of your LLM calls.

It intercepts the data stream, and if the primary provider spikes or drops the connection mid-stream, it instantly fails over to a backup model in under 15ms. It also uses atomic Redis operations to set hard spend caps, so if your agent gets stuck in a loop, it cuts the connection before you accidentally burn hundreds of dollars in API credits.

Demo

You can check out the live project and documentation here: https://selixes.com (Since it's an infrastructure routing layer, the magic happens invisibly on the backend, but you can swap out your base URL in 2 lines of code to test it!)

Code

Because it exposes a standard OpenAI-compatible endpoint, you literally don't have to change your agent's backend logic. Just point it to the gateway:

import OpenAI from 'openai';

// Before: Direct connection (vulnerable to outages)
// const client = new OpenAI({ apiKey: 'sk-...' });

// After: Proxied through Selixes with Google AI fallback
const client = new OpenAI({
  baseURL: 'https://gateway.selixes.com/v1',
  apiKey: 'slx_...'
});

How I Built It

The hardest part of this project was achieving mid-stream failover without adding massive latency for the end user. If OpenAI accepts a connection but drops it mid-generation, catching that error and spinning up a brand new connection to a backup model usually causes a huge lag spike. Here's the architecture I went with to solve it:

• Atomic Spend Caps: Every request does a lightning-fast lookup using Redis INCRBYFLOAT. It checks if the session has exceeded its cost limit in under 1ms. If the cap is blown, it returns a 429 instantly.
• Optimistic TCP Pre-warming: I set up a rolling health window for providers using a Redis sliding window. When the primary provider's latency spikes above the 95th percentile, the gateway initiates a parallel background TCP connection to the fallback provider.
• The Google AI Integration: I integrated Google Gemini 1.5 Pro via the Google AI ecosystem as the default fallback engine. Why? Because to achieve sub-15ms failover, the fallback model has to be incredibly fast and highly available globally. Because Google's infrastructure is so robust, when OpenAI drops the ball, my gateway pipes the conversational context straight to Gemini. The user's agent loop stays alive, and they never even notice the primary provider went down.

Prize Categories

Best Use of Google AI

I'm submitting this for the Google AI category! Instead of just building a standard wrapper app, I specifically utilized Google AI (Gemini) as the backbone for an enterprise-grade failover infrastructure, relying on its speed to keep other systems online.