Rate Limiting Isn't One Layer: What LogicVisor and Titan Actually Do Differently

"Add rate limiting" sounds like one task. It's actually three separate layers, each defending against a different failure mode, with different trade-offs and different amounts of trust you can place in them. I only understood the distinction properly once I had to pick a strategy for two systems solving different problems: LogicVisor (a public AI tool anyone can hit anonymously) and Titan (a payments platform where the cost of a bad actor is different from the cost of someone burning through free AI credits). 1. Client-side: UX, not security Debouncing a search input, greying out a submit button after the first click, backing off exponentially after a 429. All of this makes an app feel considerate. None of it stops anyone. A malicious actor skips your JavaScript entirely and hits the endpoint directly with curl. Client-side limiting is worth doing (it saves you real traffic and prevents accidental double-submits), but it is not a security control. If it's the only thing standing between your API and abuse, you don't have rate limiting, you have a polite suggestion. 2. Server-side: the layer that actually protects you This is where LogicVisor and Titan diverge, because they're not defending against the same thing. LogicVisor: several checks before a single AI token gets spent LogicVisor is public. Anyone gets 3 free code reviews with no signup, which means the abuse surface is wide open by design. The submission route runs a stack of checks before it ever calls Gemini or Groq, because every AI call costs real money: // 1. Check if this exact code has already been reviewed by this model const cachedReview = await getCachedAIReview(preferred_model.id + "-" + canonicalHash); if (cachedReview) { return NextResponse.json({ success: true, data: cachedReview }, { status: 201 }); } // 2. Enforce the actual rate limit (IP + session based for anon users) const rateLimitResult = await enforceAIRateLimit(user.id, request); // 3. Slow down premium users intelligently instead of hard-blocking them const throttleDelay = await getThrottleDelay(user.id); if (throttleDelay > 0) { await new Promise((resolve) => setTimeout(resolve, throttleDelay)); } // 4. Under heavy load, degrade gracefully instead of rejecting outright const shouldDegrade = await shouldGracefullyDegrade(user.id, "ai_request"); if (shouldDegrade) { // return a basic, non-AI response instead of a 429 } A few things worth naming separately, because they get lumped together under "rate limiting" but aren't the same mechanism: - Abuse prevention: anonymous sessions are capped at 3 reviews, tracked by a UUID session (7-day expiry) plus a browser fingerprint (User-Agent + Accept-Language + Accept-Encoding). This is the classic server-side layer, IP and fingerprint based, closest to the "token bucket per identity" pattern. - Cost control via deduplication: before rate limiting even runs, the code is canonicalized (AST-based) and hashed. If someone (or the same person twice) submits identical logic, they get the cached review instead of triggering a new paid AI call. This isn't rate limiting in the strict sense, it's closer to database-level duplicate detection, but it does the same job of protecting a scarce resource. - Graceful degradation over hard rejection: instead of a flat 429 when usage spikes, authenticated users can get a stripped-down, non-AI response. It costs nothing and still gives the user something, rather than a wall. None of this is a single algorithm out of a textbook. It's several cheap, layered checks, ordered so the most expensive resource (the AI call) is the last thing hit, not the first. Titan: NestJS Throttler backed by Redis Titan's rate limiting is intentionally boring by comparison, and that's the correct choice for what it is. It's @nestjs/throttler wired up as a global guard, with Redis swapped in as the storage backend instead of the default in-memory store: ThrottlerModule.forRootAsync({ imports: [ConfigModule], inject: [ConfigService], useFactory: (configService: ConfigService) => ({ throttlers: [ { ttl: 60000, limit: 10, }, ], storage: new ThrottlerStorageRedisService( configService.get ('REDIS_URL'), ), }), }), // registered alongside the JWT guard as a global APP_GUARD { provide: APP_GUARD, useClass: ThrottlerGuard, }, 10 requests per 60 seconds, globally, enforced before a request even reaches a controller. No per-route tuning yet, no per-user tiers, just a flat ceiling applied everywhere via APP_GUARD . The reason Redis matters here isn't exotic: an in-memory counter only knows about requests hitting that one process. The moment you run more than one instance behind a load balancer, in-memory rate limiting stops being true rate limiting, each instance is independently under-counting. Swapping the storage to ThrottlerStorageRedisService gives every instance a shared, consistent view of who's made how many requests, which is the actual requirement once you're not running a single box. LogicVisor's layered, identity-aware approach makes sense for a system where the main threat is "someone is farming free AI reviews." Titan's flat, Redis-backed throttle makes sense for a system where the main threat is "someone is hammering an endpoint," and the priority is consistency across instances over nuance per user. 3. Database-level: the layer I'm deliberately not using yet Postgres and Supabase both support rate limiting closer to the data layer (unlogged tables with triggers, PL/pgSQL token bucket functions, TTL-based counters in Mongo). I don't use this in either project, and I don't think I should yet. It's the right call for protecting specific high-value operations (financial writes, heavy analytical queries) where you need the database itself to be the source of truth and can't tolerate a race between the app layer and the DB. LogicVisor's canonical-hash cache lookup is adjacent to this idea (checking the data layer before doing expensive work), but it's a caching pattern, not a rate limiter, and I'd rather keep that distinction honest than dress it up as something it isn't. What actually mattered, in order - Client-side limiting is UX. Treat it as UX. It's not a line item in your security review. - Server-side is where the real decision lives, and the right shape of it depends on what you're protecting: identity-based abuse (LogicVisor) versus flat request volume across a distributed system (Titan). They are not the same problem and don't need the same solution. - The most expensive resource should be the last thing your code touches. LogicVisor checks a cache, then a rate limit, then a throttle delay, before it ever calls an LLM. Ordering is a rate limiting decision too, even when it's not framed as one. - Database-level limiting is a tool for a specific job, not a default. I'm not reaching for it until there's an operation that actually needs that guarantee. Cover photo by HsinKai Tai on Unsplash Top comments (0)