Installing Longhorn on Kubernetes

Longhorn is an open-source, lightweight distributed block storage system for Kubernetes that provides persistent, highly available storage for stateful applications running in a cluster. It runs a CSI driver, an instance manager on every node, a storage engine that replicates data and creates snapshots/backups, and a web UI for managing volumes. This guide walks through installing Longhorn on a Kubernetes cluster, creating ReadWriteOnce (RWO) and ReadWriteMany (RWX) StorageClasses, provisioning and testing PVCs with sample pods, exposing the Longhorn UI externally through an Nginx Ingress controller, and configuring volume backups to S3-compatible object storage. By the end, you'll have Longhorn running as your cluster's distributed storage layer, with tested RWO and RWX volumes, a browser-accessible dashboard, and automated backups to external object storage. Prerequisites: a Kubernetes cluster with at least 3 nodes (4 vCPUs each), S3-compatible object storage with a bucket (e.g. longhorn ) for volume backups, a Linux management workstation with SSH access as a non-root sudo user, andkubectl installed and configured to reach your cluster. 1. Install Longhorn on Kubernetes You can install Longhorn with Helm or kubectl. The steps below use kubectl with a pinned release manifest. 1. Download the latest Longhorn release manifest: $ wget https://github.com/longhorn/longhorn/releases/download/v1.6.1/longhorn.yaml This downloads Longhorn v1.6.1 - check the GitHub repository for the latest version before deploying. 2. Verify the file downloaded: $ ls Output: longhorn.yaml 3. Deploy Longhorn to your cluster: $ kubectl apply -f longhorn.yaml A successful install creates CRDs, RBAC roles, services (longhorn-backend , longhorn-frontend , etc.), a longhorn-manager DaemonSet, and longhorn-driver-deployer /longhorn-ui deployments. 4. Verify all Longhorn resources are running in the new longhorn-system namespace: $ kubectl get all -n longhorn-system Example output: NAME READY STATUS RESTARTS AGE pod/csi-attacher-57689cc84b-tlzpb 1/1 Running 0 73s pod/longhorn-csi-plugin-lp562 3/3 Running 1 (15s ago) 73s pod/longhorn-ui-655b65f7f9-lgllp 1/1 Running 0 98s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/longhorn-admission-webhook ClusterIP 10.100.203.235 9502/TCP 103s service/longhorn-backend ClusterIP 10.110.4.220 9500/TCP 104s NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/engine-image-ei-5cefaf2b 3 3 3 3 3 84s daemonset.apps/longhorn-csi-plugin 3 3 3 3 3 74s daemonset.apps/longhorn-manager 3 3 3 3 3 100s 5. Install the Longhorn NFS package to enable ReadWriteMany (RWX) volumes: $ kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.6.1/deploy/prerequisite/longhorn-nfs-installation.yaml 6. Confirm the NFS installation pods are running: $ kubectl get pods | grep longhorn-nfs-installation Output: longhorn-nfs-installation-2kd6d 1/1 Running 0 43s longhorn-nfs-installation-52kgg 1/1 Running 0 43s longhorn-nfs-installation-qsst9 1/1 Running 0 43s 2. Create Longhorn Storage Classes Longhorn ships with a default longhorn StorageClass, but you'll typically create dedicated classes per access mode so PVCs can target the right one. ReadWriteOnce (RWO) StorageClass RWO mounts a volume with read-write access on a single pod at a time - suitable for workloads like databases that need consistent writes to a single writer. 1. Create the StorageClass file: $ nano rwo-storageclass.yaml 2. Add the following configuration: kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: longhorn-prod provisioner: driver.longhorn.io allowVolumeExpansion: true parameters: numberOfReplicas: "3" staleReplicaTimeout: "2880" # 48 hours in minutes fromBackup: "" fsType: "ext4" reclaimPolicy: Retain This creates an expandable StorageClass longhorn-prod with a Retain reclaim policy so the volume survives PVC deletion. Key fields: - provisioner -driver.longhorn.io routes provisioning through the Longhorn engine. A cloud block-storage CSI driver value here instead would create volumes that Longhorn can't manage. - allowVolumeExpansion - lets you grow a volume's capacity later (e.g. 10Gi → 20Gi). - numberOfReplicas - number of Longhorn volume replicas spread across nodes. - staleReplicaTimeout - marks a replica dormant after this many idle minutes. - fromBackup - restores the volume from a backup source (e.g. your object storage bucket). - fsType - filesystem format for the volume. - reclaimPolicy -Delete orRetain behavior when the bound PVC/PV is removed. 3. Apply the StorageClass: $ kubectl apply -f rwo-storageclass.yaml 4. Verify it's available: $ kubectl get storageclass Output: NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE longhorn (default) driver.longhorn.io Delete Immediate true 35m longhorn-prod driver.longhorn.io Retain Immediate true 4s standard block.csi.example.com Delete Immediate true 46m (The standard row is just an example of a cluster's pre-existing default StorageClass - run kubectl get storageclass on your own cluster and substitute the real name/provisioner it shows.) ReadWriteMany (RWX) StorageClass RWX lets multiple pods mount the same volume with read-write access simultaneously. 1. Create the StorageClass file: $ nano rwx-storageclass.yaml 2. Add the following configuration: kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: longhorn-rwx provisioner: driver.longhorn.io allowVolumeExpansion: true reclaimPolicy: Delete volumeBindingMode: Immediate parameters: numberOfReplicas: "3" staleReplicaTimeout: "2880" fromBackup: "" fsType: "ext4" nfsOptions: "vers=4.2,noresvport,softerr,timeo=600,retrans=5,rw,hard" reclaimPolicy: Retain This creates the RWX StorageClass longhorn-rwx , which uses Longhorn's NFS share-manager pods to support simultaneous multi-pod read-write access. nfsOptions is the main addition that enables the extra NFS mount options RWX volumes need. 3. Apply the StorageClass: $ kubectl apply -f rwx-storageclass.yaml 4. Verify it's available: $ kubectl get storageclass Output: NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE longhorn (default) driver.longhorn.io Delete Immediate true 36m longhorn-prod driver.longhorn.io Retain Immediate true 46s longhorn-rwx driver.longhorn.io Retain Immediate true 9s standard block.csi.example.com Delete Immediate true 47m 3. Create Persistent Volume Claims (PVCs) Longhorn dynamically provisions a volume whenever a PVC references one of its StorageClasses. ReadWriteOnce (RWO) PVC 1. Create the PVC file: $ nano rwo-pvc.yaml 2. Add the following configuration: apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-rwo spec: accessModes: - ReadWriteOnce storageClassName: longhorn-prod resources: requests: storage: 10Gi This creates a PVC test-rwo referencing the longhorn-prod StorageClass, which provisions a 10Gi Longhorn volume once mounted. 3. Apply the PVC: $ kubectl apply -f rwo-pvc.yaml 4. Verify it's Bound : $ kubectl get pvc Output: NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE test-rwo Bound pvc-4026f8d2-873c-429b-8068-208b26a75692 10Gi RWO longhorn-prod 4s 5. Verify the underlying PV: $ kubectl get pv Output: NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE pvc-4026f8d2-873c-429b-8068-208b26a75692 10Gi RWO Retain Bound default/test-rwo longhorn-prod 8s Only a single pod can mount and write to this volume at a time. ReadWriteMany (RWX) PVC 1. Create the PVC file: $ nano rwx-pvc.yaml 2. Add the following configuration: apiVersion: v1 kind: PersistentVolumeClaim metadata: name: rwx-pvc spec: accessModes: - ReadWriteMany storageClassName: longhorn-rwx resources: requests: storage: 20Gi This creates a PVC rwx-pvc referencing longhorn-rwx , provisioning a 20Gi volume that multiple pods can mount simultaneously. 3. Apply the PVC: $ kubectl apply -f rwx-pvc.yaml 4. Verify both PVCs are Bound : $ kubectl get pvc Output: NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE rwx-pvc Bound pvc-509f254e-255d-4aee-bdc1-8c10c6cc49f5 20Gi RWX longhorn-rwx 7s test-rwo Bound pvc-4026f8d2-873c-429b-8068-208b26a75692 10Gi RWO longhorn-prod 3m10s 5. Verify the PVs: $ kubectl get pv Output: NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS VOLUMEATTRIBUTESCLASS REASON AGE pvc-4026f8d2-873c-429b-8068-208b26a75692 10Gi RWO Retain Bound default/test-rwo longhorn-prod 3m31s pvc-509f254e-255d-4aee-bdc1-8c10c6cc49f5 20Gi RWX Retain Bound default/rwx-pvc longhorn-rwx 28s All pods that mount rwx-pvc can read and write to it at the same time. 4. Test the PVCs ReadWriteOnce (RWO) 1. Create an Nginx pod that mounts the RWO PVC: $ nano nginx-rwo-pod.yaml apiVersion: v1 kind: Pod metadata: name: nginx-webserver spec: containers: - name: rwo-container image: nginx:latest volumeMounts: - name: rwo-volume mountPath: /data volumes: - name: rwo-volume persistentVolumeClaim: claimName: test-rwo 2. Apply it: $ kubectl apply -f nginx-rwo-pod.yaml 3. Verify the pod is running: $ kubectl get pods Output: NAME READY STATUS RESTARTS AGE nginx-webserver 1/1 Running 0 41s 4. Open a shell in the pod: $ kubectl exec -it nginx-webserver -- /bin/bash root@nginx-webserver:/# 5. Write a test page to the mounted volume: $ echo " Hello World! Nginx Writes to the RWO Longhorn Volume " > /usr/share/nginx/html/index.html 6. Confirm Nginx serves it: $ curl 127.0.0.1 Hello World! Nginx Writes to the RWO Longhorn Volume Only this one pod can write to the volume at a time - others could mount it read-only, but not write concurrently. 7. Exit the pod: $ exit ReadWriteMany (RWX) RWX is suited to workloads with separate reader/writer processes needing shared storage. 1. Create a backend pod that writes to the shared volume: $ nano log-backend.yaml apiVersion: v1 kind: Pod metadata: name: log-writer spec: containers: - name: log-writer-container image: busybox:latest command: ["/bin/sh", "