Build Your Own Kubernetes: A Small Orchestrator That Teaches the Real System

A container can be started with one command. Keeping the right number of containers running after machines fail, requests race, and processes restart is a different problem. That problem is the useful starting point for building your own Kubernetes. This article first explains why Kubernetes exists and how its real architecture works. It then uses that foundation to build a small container orchestrator. You will learn the roles of the API server, etcd, scheduler, controller manager, and worker components, follow an application from deployment to network traffic, and implement recovery through reconciliation. The target reader is a developer who can run containers and build a web API, but has not implemented a control plane. The practical goal is a local learning system that accepts a job, places it on a node, starts a container, observes its result, and recovers after its own processes restart. You do not need to reproduce the Kubernetes API to learn its central engineering ideas. 1. Why Kubernetes exists From one application to many machines Imagine your Node.js API running in one container. When traffic grows, you start three copies. Now you must decide where they run, replace failed copies, route requests to healthy instances, and deploy a new image without taking every copy offline. Containers package the application, but somebody must coordinate these decisions across machines. Kubernetes gives you a declarative way to express those requirements. You describe the intended workload and let controllers continually work toward that description. This helps when you operate multiple applications, need repeatable deployment policies, or must recover from machine failures. It also adds operational work, so a small application on one host may be better served by simpler tooling. Scaling is not automatic just because an application runs on Kubernetes. You can change a replica count yourself or configure an autoscaler with appropriate metrics and policies. The platform provides mechanisms. You still choose behavior, capacity, and acceptable failure modes. Docker packages an application and starts a container on a host. It does not, on its own, provide a cluster-wide answer to questions such as which machine has capacity, whether a lost machine's work should be replaced, or which running instances should receive traffic. Kubernetes coordinates independent components that continuously move the cluster toward a declared state. Its API stores an object's desired spec separately from its observed status . Kubernetes objects and controllers describe this model. The important insight is that an API request is not a promise that work has finished. A successful request may only mean that intent has been accepted. Later, controllers and node agents make progress and report what actually happened. | Question | Single-host container runner | Small learning orchestrator | Kubernetes cluster | |---|---|---|---| | Where should work run? | Caller chooses a host | Scheduler selects from known nodes | Scheduler places Pods using constraints and available resources | | What survives an API restart? | Often only runtime state | Desired state in a durable database | API objects persisted through the control plane | | Who repairs drift? | Operator or restart policy | Reconciliation loop | Controllers and node agents | | How do clients find changing instances? | Fixed address or manual proxy | Optional later feature | Services plus cluster networking implementation | | Who owns data after a container dies? | Host or external system | Explicitly out of scope at first | Persistent storage integrations, when configured | How Kubernetes developed Kubernetes grew from Google's experience with cluster management, including Borg. Google open sourced Kubernetes in 2014. The donation to the newly formed CNCF accompanied the 1.0 release in 2015, rather than the initial 2014 announcement. Borg provided design experience, but Kubernetes is a separate project. Kubernetes history and the Borg paper provide the historical record. Containers made applications easier to package and distribute. Cluster managers such as Kubernetes address the next set of responsibilities: deciding where workloads run, keeping them available, and coordinating changes across machines. 2. Understand the real Kubernetes architecture first Figure 1: The API server connects control plane decisions with execution on worker nodes. A cluster has a control plane that manages cluster state and nodes that run workloads. These are roles rather than a rule that every component needs a separate machine. Learning clusters may combine them. Production layouts depend on availability and isolation requirements. Before naming the processes, understand the objects those processes manage. | Object | Meaning | Example | |---|---|---| | Node | A registered machine that can run Pods | A Linux VM with a kubelet and container runtime | | Pod | One or more containers scheduled together with shared networking and optional shared volumes | An API container with a supporting sidecar | | Deployment | Desired rollout and replica configuration for an application | Three copies of version 2 of an API | | ReplicaSet | A desired population of matching Pods | The three Pods for a Deployment revision | | Service | A stable discovery and access abstraction for backends | One name and address for the API's changing Pods | Pods are replaceable. A Deployment manages ReplicaSets, whose controllers maintain the requested Pod population. Replacing a Pod creates a new object with a new identity. It does not move the old Pod to another node. Read more in the Pod and ReplicaSet documentation. API server: the shared entry point kube-apiserver exposes the Kubernetes HTTP API. kubectl , controllers, the scheduler, and node agents use that API to read and update cluster objects. Requests pass through access checks and applicable validation before accepted changes are persisted. This shared interface keeps components coordinated without having each component write directly to the database. For example, the scheduler records a Pod's node assignment through the API. It does not SSH into that node to launch a process. The Kubernetes API overview describes this boundary. etcd: durable cluster state etcd is the consistent key-value store backing Kubernetes API data. It persists records such as workload specifications, configuration, and reported status. Its contents describe the cluster, while the actual processes run on nodes. Container images belong in registries or node caches, and application databases require their own storage. This distinction matters during failure recovery. Restoring etcd restores saved cluster state. It does not by itself restore the files in your application's database. Protect etcd access, maintain backups, and test restoration. Ordinary cluster clients should use the API server rather than editing etcd directly. See operating etcd for Kubernetes. Scheduler: choose a node for an unscheduled Pod kube-scheduler watches for Pods without an assigned node, filters nodes against constraints and resource requests, then selects a suitable placement. It records that decision through the API. If no node fits, the Pod remains pending with scheduling information that helps diagnose the problem. Adding replicas cannot create machine capacity. The scheduler's job ends at placement, and the node handles execution. The later scheduling exercise implements this distinction on a smaller scale. Controller manager: keep objects converging kube-controller-manager hosts multiple control loops. Each reconciles a particular kind of responsibility, such as Deployments, ReplicaSets, Jobs, or node lifecycle. A Deployment controller manages ReplicaSets. A ReplicaSet controller creates or removes Pods to match its requested population. Suppose three replicas are requested and a Pod is deleted. The ReplicaSet controller can create a replacement Pod object. The scheduler then places it, and the selected node's kubelet arranges execution. Those are separate steps with separate failure modes. The controller manager reference lists the built-in controllers. Kubelet and runtime: make assigned Pods run The kubelet is the agent on each node. It observes assigned Pod specifications, coordinates their local execution, and reports status. It talks to the container runtime through the Container Runtime Interface, or CRI. Common runtime implementations include containerd and CRI-O. CRI is an interface, not the name of a runtime. A successful node assignment can still be followed by an image pull failure, startup failure, or an unhealthy container. The scheduler cannot solve those execution errors. Read more about the kubelet and CRI. Services, DNS, and network routing Figure 2: A typical ClusterIP Service provides stable discovery while network rules route requests to ready backends. Pod addresses can change when Pods are replaced. A typical ClusterIP Service gives clients a stable virtual IP and, with cluster DNS, a stable name. A label selector identifies relevant Pods, and EndpointSlices represent the discovered backends and their conditions. kube-proxy typically configures node network rules for Service traffic. Some network implementations replace that function. DNS resolves a name, and the network data plane delivers application traffic. Normal application requests do not pass through etcd or the API server. A Service also does not restart failed Pods. | Service type | Main use | Boundary | |---|---|---| | ClusterIP | Internal access through a virtual IP | Does not automatically expose an app to the internet | | NodePort | Access through a port on nodes | Still depends on reachable nodes and network policy | | LoadBalancer | Request an external load balancer integration | Requires a supporting implementation | This diagram shows ordinary ready backends. Headless Services and special endpoint policies behave differently. See the Service r