Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost

I rented one RTX 3090 at $0.259/hour, served Qwen2.5-7B-Instruct with vLLM 0.27.1 and default flags, and kept it busy for 60 seconds at 1, 4, 16 and 64 requests at once.

Each request had a prompt of about 512 tokens and a 256-token answer.

Results

• 1 at once: 40 output tokens/s, answer in 6.3 s, $1.81 per 1M output tokens
• 4 at once: 147 tokens/s, 6.8 s, $0.49 per 1M
• 16 at once: 442 tokens/s, 9.1 s, $0.16 per 1M
• 64 at once: 878 tokens/s, 17.9 s, $0.08 per 1M

0 errors at every level.

What it means

The card costs the same per hour whatever you do with it. At 64 requests in flight it produced 22 times more tokens than at 1, so each token cost 22 times less. The price is waiting: each answer took about 2.9 times longer.

So "how much does my GPU cost per token" has no single answer. It depends on how many requests you keep in flight, and that is a choice you make with your latency target.

How the cost was counted

Cost = GPU price x the measured window. The window runs until the last in-flight request finishes (62.8 to 71.7 s here), because that time is paid for too.

Requests and tokens came from vLLM's own /metrics counters before and after each level, and a client-side meter counted the same things separately. They matched exactly at every level.

What this does not tell you

These costs assume the GPU is busy the whole time. A real server is not. If it serves requests 30% of the day, each request really costs about 3.3x these numbers, and only your own traffic shows that share.

Also: one card, one run per level, synthetic prompts. The 1-at-once window includes a one-time kernel compile.

Start-up was 122.9 s from launch to /health with the model already on disk. If you scale to zero, every wake pays that before the first answer: about $0.009 per wake at this price.

Report, raw /metrics and server log: https://github.com/qaisermehdi3-coder/qvunex/tree/main/benchmarks/data/planb-3090-2026-10-27

If you want this for your own model and settings and don't want to run anything on your servers, I rent the same GPU and run it for you, as paid work: qv*********@gmail.com