How I Built a $0 LLM Production Stack with 46 Free APIs

How I Built a $0 LLM Production Stack with 46 Free APIs And why every "free LLM API list" goes stale within a week The Problem I needed LLMs for a side project but couldn't justify $50-500/month for API costs. Every "free LLM API" list I found had the same problems: - Stale within days - providers change limits, add credit card requirements, deprecate models - Markdown tables - not machine-readable, can't plug into code - No verification - no way to know if a provider actually works today - No deployment path - just a list, no config for Hermes/LiteLLM/Portkey So I built free-llm-atlas: 46 free LLM API providers, auto-probed daily, structured JSON + gateway configs. The Zero-Cost Stack After testing 40+ endpoints, my production fallback chain: Layer 1 (Speed): Groq - 300+ tok/s, 14.4K req/day Layer 2 (Multimodal): Google AI Studio - 2M context, vision/audio, 1.5K req/day Layer 3 (Reasoning): NVIDIA NIM - Nemotron Ultra 1M ctx, function calling, 40 RPM Layer 4 (Chinese): Z.AI GLM-4 - 1M ctx, strong Chinese, 60 RPM Covers 95% of workloads at $0. The 17 Permanent Free Providers (No Credit Card) | Provider | Best For | Rate Limit | Context | |---|---|---|---| | Groq | Speed | 14.4K req/day | 131K | | Google AI Studio | Multimodal / Long context | 1.5K req/day | 2M | | NVIDIA NIM | Reasoning / Function calling | 40 RPM | 1M | | Cerebras | Extreme speed | 2.6K tok/s | 1M | | Cloudflare Workers AI | Edge inference | 10K neurons/day | 128K | | Cohere | RAG / Embeddings | 1K req/month | 16K | | Mistral | EU data residency | 1 RPS | 32K | | HuggingFace | Model variety | $0.10/mo credits | Varies | | GitHub Models | GPT-5, o4-mini free | 150 req/day | 128K | | OpenRouter | 14 free models | 50 RPD | 1M (Nemotron) | | Nebius | New free tier | 100 RPM | 128K | | OVHcloud | EU GDPR | Anonymous 2 RPM | 4K | | Inference.net | New permanent free | Unknown | Unknown | | LLM7.io | Claude/GPT access | 30 RPM | 200K | | Requesty | Router | 200 RPM | Varies | | Z.AI | Chinese | 60 RPM | 1M | | Coze | Agent platform | 100/day | 128K | Why Daily Probing Matters Static lists rot. Providers: - Change rate limits (Groq dropped from 14.4K → 1K RPD in 2026) - Add credit card requirements overnight - Deprecate models without notice - Shut down endpoints silently free-llm-atlas runs GitHub Actions daily at 06:00 UTC - every provider, every endpoint, every day. probe.py → test /models + /chat/completions → measure latency, success, tokens/sec, context → update providers.json → commit if changed → Git history = uptime dashboard Usage: 30 Seconds to Production git clone https://github.com/happyyboxx/free-llm-atlas cd free-llm-atlas # Find free providers needing NO credit card python3 -c " import json d = json.load(open('data/providers.json')) for p in d['providers']: if p['tier']=='permanent_free' and not p.get('requires_card'): print(f'✅ {p["name"]}: {p.get("rate_limit","N/A")}') " # Probe all 46 providers pip install httpx pyyaml python3 scripts/probe.py --all # Export gateway config python3 scripts/probe.py --export-config litellm > config.yaml Gateway Configs: Drop-in Replacement # Auto-generated litellm.yaml model_list: - model_name: llama-3.1-70b-groq litellm_params: model: groq/llama-3.1-70b-versatile api_base: https://api.groq.com/openai/v1 max_tokens: 8192 fallback: [gemini-flash, nim-nemotron, z-ai-glm] One command switches entire stack. Zero code changes when a provider fails. The Hidden Limits Nobody Talks About Everyone compares daily request limits. The real bottleneck is tokens/minute (TPM): | Provider | Daily Req | TPM | Real Limit | |---|---|---|---| | Groq | 14,400 | 6,000 | Concurrency: ~12 msg/min | | Together | 100 | 100,000 | Daily limit hits first | | NIM | ~2,400 | 40 RPM | Batch-friendly | | Google AI Studio | 1,500 | 1,000,000 | Effectively unlimited | | OpenRouter (free) | 50 | 20 RPM | Very low | TPM determines concurrency, not daily requests. Match provider to YOUR bottleneck. GitHub Actions = Free Infra The probe runs on GitHub Actions (free tier): - 0 infrastructure cost - Runs daily at 06:00 UTC - Updates providers.json with live status - Generates gateway configs - Commits changes → Git history = uptime dashboard Zero maintenance. Zero cost. Contributing The project aggregates from: - cheahjs/free-llm-api-resources (28K+ ⭐) - tashfeenahmed/freellmapi (17K+ ⭐) - mnfst/awesome-free-llm-apis (6K+ ⭐) - nejib1/Free-LLM PRs welcome for new providers, probe fixes, or doc improvements. Try It git clone https://github.com/happyyboxx/free-llm-atlas cd free-llm-atlas python3 scripts/probe.py --all python3 scripts/probe.py --export-config litellm Star ⭐ if this saves you money on LLM inference. Built because I was tired of paying for inference during development. Now my entire LLM stack costs $0. Top comments (0)