Tag: persistent

How to rsync files to a Kubernetes Persistent Volume (PV)?

I need to rsync a recursive directory tree to a specific PV in a Kubernetes cluster.
Unfortunately, kubectl cp command does not support rsync behavior at the moment. So we have to do a simple hacking to do it.
rsync behavior to copy files from pod to pod, perhaps for kubectl cp · Issue #551 · kubernetes/kubectl · GitHub

TLDR;

We can use a rsync script to rsync files to a Kubernetes pod which mount specific PV as a volume and rsync is executable.

The same idea can be found here
rsync files to a kubernetes pod – Server Fault

Let’s say we have a recursive folder sample-dir and a PV named sample-pv which Persistent Volume Claim (PVC) named sample-pvc

Prepare working pod

The working pod has to (1) mount PVC sample-pvc as a volume; (2) has rsync is executable.

We will create a YAML file for working pod which mounts PVC sample-pvc as a volume. And for (2), we will use a rsync executable Docker image eeacms/rsync.

working-pod.yml

apiVersion: v1
kind: Pod
metadata:
    name: working-pod
spec:
    volumes:
    - name: sample-volume
      persistentVolumeClaim:
        claimName: sample-pvc # mount pvc as volume
    containers:
    - name: operator
      image: eeacms/rsync # rsync executable Docker image
      command: ['sh', '-c', 'sleep 3600'] # sleep 3600 secs for files copy
      volumeMounts:
      - mountPath: /var/files # mount volume to path /var/files
        name: sample-volume

Prepare rsync bash script

We will create a bash script name rsync-helper.sh for execute rsync on working pod.

#!/bin/bash
pod=$1;shift;kubectl exec -i $pod -- "$@"

Make sure rsync-helper.sh is executable.

$ chmod +x ./rsync-helper.sh

Deploy working pod to Kubernetes cluster

In next step, we will deploy working pod to Kubernetes cluster.

$ kubectl apply -f ./working-pod.yml

Execute rsyn command on remote working pod

After status of wroking-pod pod become running, we will execute rsync-helper.sh script on it remotely.

$ rsync -av --delete-after --progress -e './rsync-helper.sh' sample-dir working-pod:/var/files/

building file list ...
 0 files...
19 files to consider
sample-dir/
sample-dir/new-file
           6 100%    0.00kB/s    0:00:00
           6 100%    0.00kB/s    0:00:00 (xfer#1, to-check=16/19)
sample-dir/updated
           5 100%    4.88kB/s    0:00:00
           5 100%    4.88kB/s    0:00:00 (xfer#2, to-check=15/19)
deleting sample_dir/deleted

sent 689 bytes  received 70 bytes  1518.00 bytes/sec
total size is 98957  speedup is 130.38

Note, you can add –dryrun option for debuging

$ rsync -av --dry-run --delete-after --progress -e './rsync-helper.sh' sample_dir working-pod:/var/files/

Teardown

We will delete working pod by bellow command

$ kubectl delete --grace-period=1 -f ./working-pod.yml

Conclusion

In this blog, we have a work around solution for synchronizing files and directories between local machine and PV on Kubernetes cluster.

Goodluck, happy Hacking!

Additional, Other Kubernetes related topics is here. https://tuantranf.me/tag/kubernetes/