DEV Community

One Task.Run Quietly Killed the BackgroundService Startup Footgun

I sat down this week to write about one of my favorite .NET footguns: the BackgroundService that does synchronous warmup and freezes your entire app's startup. I've hit it in real projects, I've seen it in code reviews, and it makes a great demo. So I built the repro on .NET 10, ran it, and the bug refused to happen. I spent a solid ten minutes convinced my demo was broken. It wasn't. The footgun is gone, and I'd missed the memo. Here's the setup, the numbers from both runtimes, and the one-line source change that explains everything.

The bug, as it worked for years

Quick recap of why this bites. Hosted services start one at a time, in registration order, and the service that actually starts Kestrel is registered last. So your warmup service runs before the server starts listening. That's by design, and it's fine, as long as your warmup is actually async. The trap was in BackgroundService.StartAsync. It called your ExecuteAsync directly, which means your code ran synchronously on the startup path until it hit its first await. If your warmup looked like this, there was no first await:

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    Clock.Log($"warmup started on thread {Environment.CurrentManagedThreadId}");
    for (var segment = 0; segment < PriceCache.TotalSegments; segment++)
    {
        // Stand-in for sync I/O: a legacy pricing file, a blocking DB driver,
        // a vendor SDK without async APIs. No await in sight.
        Thread.Sleep(80);
        cache.LoadSegment(segment);
    }
    Clock.Log("warmup finished - 40 segments loaded");
}

I wrapped that in a minimal API that starts Kestrel, probes itself with an HttpClient, and stamps every event with a stopwatch. Nothing fancy, one file, no packages. On .NET 8 it does exactly what the folklore says:

[   4 ms] process started (mode=blocking, .NET 8.0.29)
[  89 ms] warmup started on thread 1
[3299 ms] warmup finished - 40 segments loaded
[3327 ms] ApplicationStarted - Kestrel is listening
[3452 ms] first HTTP response: {"sku":"sku-0-0","price":10.00,"warm":true}

Thread 1. The main thread. The server can't accept a connection until the last price segment loads. My fake warmup is 3.2 seconds; I've seen real ones take forty. Health checks time out, orchestrators kill the pod, everyone blames Kubernetes.

Same code, .NET 10

The project multi-targets, so this is literally the same binary logic on the newer runtime:

[   1 ms] process started (mode=blocking, .NET 10.0.10)
[  88 ms] warmup started on thread 6
[ 114 ms] ApplicationStarted - Kestrel is listening
[ 256 ms] first HTTP response: {"sku":"sku-0-0","price":10.00,"warm":false}
[3301 ms] warmup finished - 40 segments loaded

Kestrel is answering at 256 ms while the warmup is still chewing through segments on a thread pool thread. No Task.Yield(), no ServicesStartConcurrently, no tricks. The blocking version just doesn't block anymore. These numbers are from a Linux container, and the warmup is Thread.Sleep standing in for sync I/O. Not a lab. I care about the shape of the timeline, not the milliseconds, and the shape is unmissable.

What changed

I went digging in dotnet/runtime. Here's BackgroundService.StartAsync in .NET 8:

_executeTask = ExecuteAsync(_stoppingCts.Token);
if (_executeTask.IsCompleted)
{
    return _executeTask;
}
return Task.CompletedTask;

And in .NET 10:

_executeTask = Task.Run(() => ExecuteAsync(_stoppingCts.Token), _stoppingCts.Token);
return Task.CompletedTask;

That's the whole fix. ExecuteAsync now gets shipped off to the thread pool, so it can be as synchronous as it wants and the host keeps moving. Years of blog posts saying "put await Task.Yield() at the top of your ExecuteAsync" made obsolete by one Task.Run. The yield trick still works, by the way, and I'd keep it if you multi-target or run on .NET 8, which is in support until this November. It costs nothing on 10.

Honestly, I think this is the right call. The old contract, "your code runs inline until its first await", was too subtle. Whether your app started fast depended on where a compiler-generated state machine happened to split your method. That's not a contract, that's a coin flip with extra steps.

What still blocks

Before you delete your warmup workarounds: this only changed BackgroundService. A plain IHostedService with a synchronous StartAsync body still runs inline, sequentially, on .NET 10:

[   1 ms] process started (mode=startasync, .NET 10.0.10)
[  97 ms] StartAsync warmup started on thread 1
[3307 ms] StartAsync warmup finished - 40 segments loaded
[3341 ms] ApplicationStarted - Kestrel is listening

The host awaits each StartAsync before touching the next service, and a body with no awaits blocks that loop. Same as it ever was.

The part that might bite you

Look back at the two first responses. On .NET 8 the first response said "warm":true. On .NET 10 it says "warm":false. The old blocking behavior was accidentally gating traffic: no request could arrive before the cache was ready. On .NET 10, requests land at 256 ms against a cache that won't be warm for another three seconds. If your app leaned on that accident, and plenty do without knowing it, the upgrade doesn't just make startup faster. It changes what your first hundred requests see.

Failures move too. I tested an ExecuteAsync that throws immediately: on .NET 8 the host never starts and RunAsync surfaces the exception, so a bad deploy dies loudly at startup. On .NET 10 the app starts first, then logs BackgroundService failed and shuts down via the default StopHost behavior. Same outcome eventually, different moment, and your deploy pipeline might care about the difference.

So my advice is the boring kind: if you need traffic held until warmup finishes, say so explicitly. Do the gating in a plain IHostedService.StartAsync on purpose, or wire a real readiness check, and treat fail-fast validation the same way. Relying on a base class's threading behavior for correctness is how this mess started.

Full runnable sample (multi-targeted, all four modes):
https://github.com/ssukhpinder/dev-to-code-samples/tree/main/005-backgroundservice-startup-block

Did you know this changed? Better question: is anything in your app quietly depending on the old behavior, a warmup that doubles as a traffic gate, a deploy check that expects startup to fail? I'd check before the upgrade does it for you. Tell me what you find in the comments.

  • Sukhpinder, still pointing stopwatches at things nobody asked me to time

Comments

No comments yet. Start the discussion.