Skip to content

How To Setup NFS Storage for Kubernetes

For workloads that require massive storage (like media servers) or need to be accessible from any node in the cluster, local SSDs (local-path-provisioner) won't work. The data must live on the network.

A classic and highly effective homelab solution is to convert one of your worker nodes (the one with the largest HDDs) into an NFS Server and expose that storage directly to the Kubernetes cluster.

Step 1: Configure the NFS Server

Log into the node that houses your large hard drives (e.g., k8s-worker-02).

  1. Install the NFS Kernel Server: 

    sudo apt-get update
sudo apt-get install -y nfs-kernel-server

  2. Create the Shared Directory: 

    sudo mkdir -p /mnt/media
sudo chmod 0777 /mnt/media # Permissive mode to avoid permission headaches during initial ingestion

  3. Export the Directory to the Subnet: Edit /etc/exports and add a line explicitly allowing your local subnet (e.g., 192.168.1.0/24) to mount the drive: 

    echo "/mnt/media 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)" | sudo tee -a /etc/exports
    (Note: no_root_squash allows containers running as root to write to the share as root. This is generally required for many Docker containers, though less secure in enterprise environments).

  4. Apply and Restart: 

    sudo exportfs -a
sudo systemctl restart nfs-kernel-server

Step 1.5: Prepare the Cluster Nodes (The Client-Side)

Kubernetes does not have its own built-in NFS client. When a Pod requests an NFS volume, the underlying physical node it is scheduled on uses the host OS's native mount.nfs utility to attach the drive.

If this utility is missing from the node, the Pod will be permanently stuck in ContainerCreating with a cryptic fsconfig() failed: NFS: mount program didn't pass remote address error.

You must install the client package on every worker and control plane node in the cluster:

sudo apt-get update
sudo apt-get install -y nfs-common

Step 2: Bind the NFS Share in Kubernetes

Unlike dynamic provisioners (like Longhorn or AWS EBS), a raw NFS share doesn't automatically carve out chunks of data for you. You must manually map the physical network share to a Kubernetes PersistentVolume (PV), and then claim it with a PersistentVolumeClaim (PVC).

Create a file named nfs-storage.yaml:

---
# 1. The PersistentVolume defines the PHYSICAL network location
apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-media-pv
spec:
  capacity:
    storage: 1000Gi # This is arbitrary for NFS, it just defines the upper bound
  accessModes:
    - ReadWriteMany # NFS supports multiple pods reading/writing at once!
  persistentVolumeReclaimPolicy: Retain # If the PVC is deleted, DO NOT delete my media files!
  nfs:
    server: 192.168.1.53 # The static IP of k8s-worker-02
    path: /mnt/media

---
# 2. The PersistentVolumeClaim is what your Pod actually requests
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: nfs-media-pvc
  namespace: media # Must match your application's namespace
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1000Gi
  volumeName: nfs-media-pv # Explicitly bind to the PV we created above

Apply it to the cluster: 

kubectl apply -f nfs-storage.yaml

Step 3: Mount the PVC in your Pod

Finally, inside your application's Deployment manifest, mount the PVC exactly as you would a local disk:

      volumes:
      - name: media-storage
        persistentVolumeClaim:
          claimName: nfs-media-pvc
      containers:
      - name: jellyfin
        image: linuxserver/jellyfin
        volumeMounts:
        - name: media-storage
          mountPath: /data/media # This path inside the container maps to /mnt/media on the NFS server!

Any media files you drop into /mnt/media on k8s-worker-02 will instantly appear inside the Jellyfin container at /data/media, regardless of which physical node the Jellyfin pod is currently running on.

Step 4: Advanced - Dynamic Provisioning with Helm

Manually creating PVs and PVCs for every application gets tedious. Modern Kubernetes uses Dynamic Provisioning to automatically create PVs (and underlying folders on the NFS server) whenever an application asks for a PVC.

To set this up, we use Helm, the package manager for Kubernetes.

  1. Install Helm on your workstation (e.g., Mac): 

    brew install helm

  2. Add the NFS Provisioner repository: 

    helm repo add nfs-subdir-external-provisioner https://kubernetes-sigs.github.io/nfs-subdir-external-provisioner/

  3. Install the Provisioner: 

    helm install nfs-subdir-external-provisioner nfs-subdir-external-provisioner/nfs-subdir-external-provisioner \
    --set nfs.server=192.168.1.53 \
    --set nfs.path=/mnt/media \
    --set storageClass.name=nfs-client \
    --set storageClass.defaultClass=true \
    -n kube-system

Now, instead of manually creating a PersistentVolume, you only need to create a PersistentVolumeClaim, and the provisioner will automatically handle the rest!