Nginx Log Woes: When $remote_addr Lies to You

Here's a fun one that bit me last week. You're running Nginx behind a reverse proxy or load balancer. You want to log the actual client IP for rate limiting. You check your logs and see... 127.0.0.1 for every single request. Your rate limiter is blocking localhost. Not ideal.

The Problem

When a request hits Nginx through a proxy, $remote_addr contains the proxy's IP, not the client's. Your config probably has something like:

log_format main '$remote_addr - $request';
access_log /var/log/nginx/access.log main;

That $remote_addr is your proxy. The real client IP is buried in a header.

The Fix

Your proxy should be forwarding the real client IP via X-Forwarded-For or X-Real-IP. Then in Nginx you use the right variable:

log_format main '$http_x_real_ip - $request';

Or if you trust the X-Forwarded-For chain:

log_format main '$http_x_forwarded_for - $request';

But be careful with X-Forwarded-For - it's a comma-separated list and can be spoofed if your proxy doesn't sanitize it.

The Rate Limiting Piece

For rate limiting, you need the actual client IP too:

real_ip_header X-Real-IP;
set_real_ip_from 10.0.0.0/8; # your proxy CIDR

This tells Nginx to trust that header from your known proxy range and rewrite $remote_addr accordingly. After reloading, check your logs. You should see real IPs now.

Quick Debug

If it's not working:

# See what Nginx actually sees
tail -f /var/log/nginx/access.log | awk '{print $1}'

Compare what hits your upstream directly vs through the proxy. One of those headers should populate.

This bites teams regularly when they first set up a reverse proxy. The logs look fine until you need to debug or rate-limit, and then you're wondering why everyone's coming from the same IP. Happened to a client last month - their Cloudflare setup wasn't passing CF-Connecting-IP, so rate limiting was a no-op.