How long a serverless function stays warm on DigitalOcean

Every serverless platform has cold starts, and every write-up about them gives you a number for how long a function "stays warm". I wanted that number for DigitalOcean Functions, so I measured it.

There isn't one number. What decided whether my function was still warm had less to do with how long it had been idle and more to do with how long it had been alive before that.

The setup

Four functions, all web functions in the fra1 region, all with 512 MB of memory:

• pysmall: Python 3.12, no dependencies
• nodesmall: Node 22, no dependencies
• nodeheavy: Node 22 with two AWS SDK v3 clients and lodash
• pyheavy: Python 3.12 with numpy

Three copies of each, 12 functions in total, so I could run several idle gaps in parallel without one copy's calls warming another.

Each function creates a random ID when its code loads and counts its calls. A call that returns a call count of 1 ran in a brand new container. So I didn't have to guess cold from warm by latency. The function tells me.

A Python client on my laptop in Bulgaria, a new HTTPS connection for every request. It calls a function, waits a set time, calls it again, and then makes two more calls straight away.

12 idle gaps, from 30 seconds to 60 minutes, in a different shuffled order for every copy. Each gap was tried once on each of the 12 copies. The run took 3 hours 19 minutes, made 444 calls, and got 444 HTTP 200s. They ran in 113 different containers.

How long it stays warm

Nothing survived 20 minutes. Up to 15 minutes, it's a gradual curve, not a cliff. One function went cold after 30 seconds, and two stayed warm after 15 minutes.

Twelve samples per row is not a lot, so I wouldn't read anything into 3 and 5 minutes coming out the same. The overall shape is clear enough.

What decided it

I expected the curve to come from some random timer. Then I looked at how old each container was when the idle time started. For the gaps between 3 and 15 minutes:

A container that had just been started and was then left alone was usually gone a few minutes later. A container that had been around for three minutes or more survived every idle gap in that range, including one of 15 minutes. The oldest container in the run served its last call 33 minutes after it started.

In my test most containers were young when the idle started, because most probes had just cold-started a new one. That's why the main table looks pessimistic.

I didn't design the run to test age, so this is a pattern I found in the data, not something I set out to prove. But it's a strong pattern, and it matches something you'd want a platform to do: get rid of containers that were started for one request and never used again, and keep the ones that are clearly busy.

The practical side: a function with steady traffic will stay warm through quiet minutes much better than a function that gets one request now and then. The occasional request is exactly the one that pays for the cold start.

What a cold start costs

Cold means the call that ran in a new container. Warm is the two calls made straight after another call. All times are measured on my laptop, so they include the network and a new TLS connection each time. The TCP and TLS setup alone took a median of 33 ms.

Warm calls were the same for all four, about 90 ms. Python and Node with no dependencies came out the same cold too, at about 0.9 seconds. Dependencies are where it changes.

Each heavy function also measured how long its imports took:

• nodeheavy: the AWS SDK and lodash imports took a median of 148 ms, but the cold start was 924 ms slower than nodesmall
• pyheavy: import numpy took a median of 355 ms, but the cold start was 2.4 seconds slower than pysmall

So most of the extra time was spent before my code started running, on getting the larger package into the new container. If you want faster cold starts, the import time your profiler shows you is the smaller part. The size of what you ship is the bigger one.

Two smaller things

The first warm call after an idle gap is slower. When a function was still warm, the first call after the gap had a median of 146 to 161 ms, depending on the function. The two calls right after it had a median of 89 ms. Same container, same code, about 60 ms slower after a pause.

A call right after a cold start can be cold again. Out of 288 calls made immediately after another one finished, 9 landed in a brand new container. In 7 of those, the call before had itself just been a cold start. Once, one of the nodeheavy copies got a new container on three calls in a row, about a second apart. I didn't find out why. It does mean a single cold start can cost you twice.

What I got wrong

My first version of the Python function imported numpy and pandas. The remote build failed with 413 Payload Too Large. Functions have a 48 MB limit on the built function, and I assume the two together were over it.

With numpy alone and the default 256 MB of memory, every call failed after about 31 seconds with a generic "There was an error processing your request." I assumed numpy's import was too slow. It wasn't. I made a version of the function that didn't import numpy at all, and it failed the same way.

The answer came from invoking it asynchronously, because DigitalOcean keeps a full activation record for async calls. The record had initTime: 30001 and "The function exhausted its memory and was aborted." The function ran out of memory before any of my code ran. At 512 MB it worked, so I moved all four functions to 512 MB to keep the comparison fair.

I also spent a while looking for my web calls in doctl serverless activations list and found nothing. The docs say activation records are kept for asynchronous invocations, so a normal synchronous web call doesn't leave one. That's why I had each function report its own container ID and call count instead.

And I expected to find a single keep-warm timeout to put in the title. There isn't one, at least not one you can see from outside.

Caveats

This is one region, one memory size, one morning, and 12 samples per idle gap. The platform might behave differently under more load, or later in the day, or tomorrow.

All latencies include the network from Bulgaria to Frankfurt, and the server clocks and my laptop's clock disagreed by about a second, so I couldn't split a cold start into its parts from the server side.

The finding about container age is a correlation from data I didn't collect for that purpose. I didn't measure the size of the deployed packages. I know numpy and pandas together were over the limit and numpy alone was under it.

The cost of the whole experiment was nothing. DigitalOcean Functions come with 90,000 GiB-seconds free every month, and 444 calls at 512 MB didn't make a line on my invoice. I deleted the namespace when I was done.

What I'd take from it

Don't plan around a fixed warm time. In this run, 2 minutes idle was usually fine, 15 minutes usually wasn't, and a fresh container could be gone in 3.

A function that's in steady use stays warm much more reliably than one that's called once in a while. Keep the deployed package small. For both heavy functions, most of the extra cold start came before the imports.

If a function needs a big library and fails with a generic error, give it more memory before you start profiling. When something fails and the response tells you nothing, invoke the function asynchronously and read the activation record.