5 Docker Containers You Need for a Home AI Lab

Learn which five Docker containers make a reliable home AI lab, from Ollama and Open WebUI to Qdrant, n8n and Portainer CE, what job each one does, and the first mistake to avoid with each.

The first weekend goes well. A model runs locally, a chat window answers questions, and the laptop fan finally has a reason to exist.

The third weekend is where home AI labs die. Three containers sit on ports nobody wrote down. A folder of PDFs never got indexed, a cron job lives in someone's shell history, and an update wiped the models because they were stored inside the container. Not one of those failures is an AI problem; every one of them is a lab problem.

A useful home lab is a small set of containers with clear jobs, not one giant AI box. KDnuggets took the same layered view in its look at the local AI stack. Model serving, an interface, retrieval memory, automation and management are separate layers, and each one is a container you can replace without touching the others.

The five below are picked for the friction they remove rather than for how much AI is in the name. Two of them have no AI in them at all.

1. Ollama: The Model Runtime

Ollama is the engine room. It pulls and runs local models and exposes an HTTP endpoint on port 11434. Every other container in the lab talks to that endpoint, so nothing else needs to know how a model is loaded.

The official Docker Hub image ships three routes: CPU only, NVIDIA GPUs through the NVIDIA Container Toolkit with --gpus=all, and AMD GPUs through the rocm tag.

The install detail that matters most is the volume. Models are large, and a container recreated without a named volume downloads them all again. Worse, it can lose a model you pulled from a tag that has since moved.

The documented run command is:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Drop --gpus=all for the CPU route.

One honest caveat: GPU pass-through depends on the host, not on Ollama. The documented Docker GPU routes are Linux and Windows with WSL2. Docker Desktop on macOS does not hand containers an NVIDIA GPU, so a Mac lab either runs Ollama natively for speed or accepts CPU inference inside Docker.

2. Open WebUI: The Interface Layer

A terminal is fine for testing a model and useless for living with one. Open WebUI puts a browser interface on the lab: chats, users, file uploads, and connections to Ollama and to any other provider you add.

Docker is the officially recommended install, and the quick start is one command:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data -e WEBUI_SECRET_KEY=your-secret-key --name open-webui --restart always ghcr.io/open-webui/open-webui:main

Two of those flags earn their place. The volume at /app/backend/data holds users, chats and uploaded files. Without it, every recreate starts the lab from a blank login page.

WEBUI_SECRET_KEY signs sessions, so set it once with something like openssl rand -hex 32 and keep it, because a container that regenerates the key logs everyone out.

The :cuda and :ollama image variants exist for GPU acceleration and for bundling Ollama inside the same container. Both are options rather than the default, since a separate Ollama container is easier to upgrade and to share between tools.

3. Qdrant: Retrieval Memory for the Lab

Do you need a vector database at all? For casual chat, no, and skipping it is a legitimate decision. The moment the lab is supposed to answer questions from your own documents, notes or project files, those files have to be embedded and searched.

That is what Qdrant does: it stores vectors and finds the nearest ones. It is the memory the AI consults, which is a different thing from the AI itself.

The local quick start runs Qdrant with two ports, 6333 for the REST API and dashboard and 6334 for gRPC. The quick start also mounts a storage path so collections survive restarts.

The dashboard at localhost:6333/dashboard is worth opening once, because seeing a collection fill up is the fastest way to confirm an ingestion pipeline is doing anything.

A model alone is not a useful system, a point the earlier KDnuggets piece on self-hosted LLMs in the real world makes at length. Retrieval is usually the layer that turns a demo into something you reach for.

4. n8n: The Automation Spine

Every lab eventually needs glue. Files land in a folder and need embedding, model endpoints need calling on a schedule, and summaries need pushing somewhere a human will read them. n8n is a workflow tool that runs those jobs visually.

Its Docker Compose install is the sensible route for a lab, because Compose is how the rest of the stack should be described anyway. It serves on localhost:5678 and defaults to SQLite, with Postgres as the option once workflows multiply.

Three workflows show what it is for:

• Nightly ingestion. A scheduled job watches a folder, chunks new files and writes them into Qdrant.
• Summarize on demand. A webhook takes a URL or a PDF, sends it through Ollama for a summary and posts the result to a notes file.
• Weekly review. A job asks the model to read the week's lab notes and flag anything unfinished.

The caution is the obvious one. n8n holds credentials and workflow definitions, so its volume needs a backup, and its port has no business being exposed beyond the home network.

5. Portainer CE: The Control Panel

After four containers, the boring work becomes the work: restarts, logs, volume checks, spotting an image with an update, cleaning up the six stopped containers from last month. Portainer CE gives all of that a browser UI.

The official install runs on HTTPS port 9443 and keeps its own state in a portainer_data volume. The install also mounts the Docker socket at /var/run/docker.sock.

The socket mount is the part to respect. Whatever can talk to the Docker socket can control every container on the host. That is what makes Portainer useful, and it is also why it belongs on a trusted machine behind a home network, never on a public port. Treat it as the admin console it is.

The lab won't need it on day one; it will need it the first time someone asks why the chat interface stopped answering and nobody can remember which container to restart.

Starting With Two Containers and Adding the Rest When the Lab Demands Them

The five jobs are serve, talk, remember, automate and manage. Ollama and Open WebUI cover the first two and make a working lab in an afternoon.

Add Qdrant when you catch yourself pasting the same document into a chat for the third time. Add n8n when a manual step has happened often enough to be annoying. Add Portainer before the stack becomes opaque, which is usually the week after n8n arrives.

Describe all of it in one Compose file from the start. Compose gives every service a name the others can use as a hostname, so Open WebUI reaches the model at http://ollama:11434 with no IP addresses involved.

Named volumes are the difference between "tried this once" and a lab that survives updates. Pin image versions for anything other people rely on, because :main and :latest tags move without asking.

When the stack is stable and the itch to tune it arrives, the earlier KDnuggets piece on practical Docker configurations is the next read. Until then, five containers with clear jobs beat fifteen with clever names.

Nahla Davies is a software developer and tech writer. Before devoting her work full time to technical writing, she managed - among other intriguing things - to serve as a lead programmer at an Inc. 5,000 experiential branding organization whose clients include Samsung, Time Warner, Netflix, and Sony.