Open LLM Gateway: self-hosted LLM control plane (proxy + analytics + admin UI, no seat cap)

Problem with Multi-Cloud LLM Management

Our team used multiple LLM providers from different clouds: Google, Azure, and AWS. Every app and project had its own API key. We could not see which key belonged to which app, team, environment, etc. We also had no clear view of token usage and tracking. Later, finance asked which product spent a lot of money. We could not answer.

Previous Approach: LiteLLM

First I tried LiteLLM. It is a good tool if you need to quickly setup your llm gateway. However, LiteLLM has limitations: LiteLLM Admin UI SSO is free for only 5 users. After 5 users, you need the Enterprise plan. Org admin and SCIM are also Enterprise.

Solution: Open LLM Gateway

So I built Open LLM Gateway and released the full stack as Apache 2.0. This includes the admin UI, not only the proxy.

Repository Components

• OpenAI /openai/* and Anthropic /anthropic/* proxy
• Encrypted master keys; apps use child keys (sk_…)
• Portal: organizations, roles, invites, optional Google login, key rotate / tag / disable
• Cost and token analytics by provider, model, and tag
• Deploy on Cloudflare Workers and Vercel

Key Distinctions

What it is not: it is not LiteLLM if you only need limited seat without enterprise features. It is not OpenRouter if you want a hosted service and you do not want to run it yourself.

Recommendation

If your company wants a self-hosted LLM gateway, with no seat cap, and with the admin UI fully open source, Open LLM Gateway may be a good fit.

Source: https://github.com/Nelsonlin0321/llm-gateway
Site: https://llm-gateway.io