AI Wrote a Thread-Safe Counter. The CPU Made It 5x Slower.

Meet the cache line. I asked an AI assistant for a simple thing: per-thread counters. Four threads, each incrementing its own slot in an array. No shared variables. No locks needed. The code it wrote was clean and correct:

long[] counters = new long[threadCount];
// thread t does: counters[t]++

Every thread writes only to its own element. There is no race here. Any code review would pass it. Every test would pass too. Then I measured it against a version that does exactly the same work, and the correct code lost by 5x.

The numbers

This is from my machine (Apple Silicon, .NET 10, 4 threads, 200 million increments per thread):

• Round 1: adjacent (one cache line): 464 ms / padded (line per thread): 85 ms / ratio: 5.4x
• Round 2: adjacent (one cache line): 397 ms / padded (line per thread): 85 ms / ratio: 4.7x

Same loop. Same number of increments. Same "each thread touches only its own counter". The only difference between the two versions is where the counters live in memory.

What the hardware actually does

A CPU never reads one byte from memory. It moves data in fixed blocks called cache lines - 64 bytes on x86, 128 bytes on Apple Silicon. Ask for one long and the whole block it lives in travels into the core's cache.

Think of a cook whose ingredients are in a basement fridge: going downstairs is expensive, so you never carry one carrot - you carry the whole crate. (Don't take my word for the 128: run sysctl hw.cachelinesize on an M-series Mac.)

Usually this works for you. Array elements sit side by side, so scanning an array is fast: you touch one element and the next fifteen arrive in the same crate for free.

But with multiple cores there is a rule: to write into a cache line, a core must own it exclusively. The moment core 1 writes, every other core's copy of that line is declared stale. And the unit of ownership is not your variable. It is the whole line.

Now look at my four counters. Four long values, 8 bytes each, side by side - 32 bytes. They all fit in one cache line. Thread 1 increments its counter and takes ownership of the line. A nanosecond later thread 2 increments its own, different counter - and has to rip the same line back. The line ping-pongs between cores on every single write. Four threads that share nothing in the code are fighting over one crate in the hardware.

This is called false sharing. False, because no data is actually shared. The fight is real anyway. Four counters in one cache line vs. one counter per line - same code, 5x difference.

The demo fix is one attribute

Give every thread its own cache line. Pad each counter so the next one starts in a different line:

[StructLayout(LayoutKind.Explicit, Size = 128)]
struct PaddedCounter
{
    [FieldOffset(64)]
    public long Value;
}

That is the entire difference between 464 ms and 85 ms. We pay a little memory - 128 bytes per counter instead of 8 - and get back the parallelism we thought we already had. (I use 128, not 64, for two reasons: Apple Silicon lines are 128 bytes, and on x86 the adjacent-line prefetcher likes to drag neighboring lines along.)

Run it yourself

The full demo is about 70 lines, no project file needed - with the .NET 10 SDK you can run a single file directly:

dotnet run -c Release FalseSharingDemo.cs

// FalseSharingDemo.cs - .NET 10 file-based app
using System.Diagnostics;
using System.Runtime.InteropServices;

const long Iterations = 200_000_000;
int threadCount = Math.Min(Environment.ProcessorCount, 4);

for (int round = 0; round < 2; round++)
{
    var slow = MeasureAdjacent(threadCount);
    var fast = MeasurePadded(threadCount);
    Console.WriteLine($"adjacent: {slow.TotalMilliseconds:F0} ms  padded: {fast.TotalMilliseconds:F0} ms  ratio: {slow / fast:F1}x");
}

static TimeSpan MeasureAdjacent(int threadCount)
{
    long[] counters = new long[threadCount];
    return RunThreads(threadCount, t =>
    {
        for (long i = 0; i < Iterations; i++) counters[t]++;
        return counters[t];
    });
}

static TimeSpan MeasurePadded(int threadCount)
{
    var counters = new PaddedCounter[threadCount];
    return RunThreads(threadCount, t =>
    {
        for (long i = 0; i < Iterations; i++) counters[t].Value++;
        return counters[t].Value;
    });
}

static TimeSpan RunThreads(int threadCount, Func<int, long> body)
{
    long sink = 0;
    var threads = new Thread[threadCount];
    var sw = Stopwatch.StartNew();
    for (int t = 0; t < threadCount; t++)
    {
        int id = t;
        threads[t] = new Thread(() => Interlocked.Add(ref sink, body(id)));
        threads[t].Start();
    }
    foreach (var th in threads) th.Join();
    sw.Stop();
    GC.KeepAlive(sink);
    return sw.Elapsed;
}

[StructLayout(LayoutKind.Explicit, Size = 128)]
struct PaddedCounter
{
    [FieldOffset(64)]
    public long Value;
}

Where this hides in real code

You will not write four counters in a loop at work. But you will write, or an AI will write for you:

• Cache statistics. Almost every in-memory cache keeps hits, misses, evictions. The natural implementation is fields next to each other, or a long[] with one slot per shard, updated with Interlocked.Increment from every thread. Fields next to each other means one cache line. This is literally the demo above, running in your production service.
• Sharded counters. The cruel version: you sharded a counter specifically to make it parallel, put the shards in one array - and they still share lines. You did the architecture work and the hardware quietly undid it.
• LRU metadata. A compact long[] lastAccessTicks per cache slot means every cache read becomes a write into a hot shared array. A read-heavy cache that is slow because of writes is a fun thing to debug.
• And sometimes the problem ships inside the library. ConcurrentDictionary - the base of most homemade .NET caches - internally keeps a counter per lock stripe in a plain array (_countPerLock in the source). Under very hot multi-threaded writes those neighbors can end up sharing lines. I have not benchmarked the real-world impact - but the layout is right there in the source, and now you know what to look for.

When you should NOT care

Honesty section. False sharing hurts when the writes are hot - millions of updates per second from several threads. If your cache updates its stats a thousand times per second, you will never notice, and padding everything "just in case" is cargo cult. The rule is the same as always: measure first. The demo above is the measurement; adapt it to your data layout and see if your ratio is 1.0x or 5x.

The actual point

The AI-generated code was not wrong. It compiled, it was race-free, it passed every test I could write for its correctness. An entire code review process could bless it. The 5x was invisible at every layer we normally check.

That is what changed with AI-assisted coding, and it is why hardware fundamentals became more valuable, not less. The model will happily generate a thread-safe counter, a sharded cache, an LRU eviction policy - and none of its correctness guarantees say anything about cache lines. Correctness and mechanical sympathy are different layers. Tests catch the first. Only understanding catches the second.

You do not need to memorize cache sizes. You need to know the crate exists. Keep data that is used together close. Keep data that is written by different threads apart. That one rule, read in both directions, is most of "cache-aware" programming.

The tools write the code now. Knowing why it is slow is still our job.

Originally published on vasyl.blog.