[til] Prevent deleting running pods on Kubernetes

Kubernetes sends the preStop event immediately before the Container is terminated. Kubernetes’ management of the Container blocks until the preStop handler completes, unless the Pod’s grace period expires. For more details, see Termination of Pods.

So we can add a preStop event handler to prevent deleting a running pod.

pods/prestop-test.yaml
apiVersion: v1
kind: Pod
metadata:
  name: lifecycle-demo
spec:
  containers:
  - name: lifecycle-demo-container
    image: nginx
    lifecycle:
      postStart:
        exec:
          command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
      preStop:
        exec:
          # Stop Nginx process and wait for all processes be killed
          command: ["/bin/sh","-c","nginx -s quit; while killall -0 nginx; do sleep 1; done"]

Or just simple waiting until running job counter be delete as bellow sample

        lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "while [ -f \"/tmp/running-job-counter\" ]; do sleep 1; done"]

Container Lifecycle Hooks – Kubernetes

Attach Handlers to Container Lifecycle Events – Kubernetes