Clearing Evicted Pod in One or All Namespaces on Kubernetes Cluster

NAME       READY   STATUS    RESTARTS   AGE
myapp 0/1 Evicted 0 10s

You might have seen pods in the above state on Kubernetes cluster. Eviction is the process of terminating one or multiple pods on K8s nodes which could be because of lack of resources like CPU, memory, disk space, and filesystem inodes etc. It is mostly due to resource planning difficulty and pod needs to be rescheduled from high to low pressure node.

When any of the node in the cluster is under high pressure, it starts to cleanup unused Kubernetes resources like killed container, old container images etc. If that also could not sufficiently clean server resource of the node, further containers start to get killed.

The queue of getting killed in containers list comes on top which has not well defined cpu & memory requests, limits. This should be defined on pod manifest like this:

containers:
- name: myapp
image: nginx:alpine
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 768Mi

The evicted pods can be seen with simple kubectl get pods command.

To delete pods of all namespaces at once, run the following script:

kubectl get po --all-namespaces | awk '{if ($4 ~ /Evicted/) system ("kubectl -n " $1 " delete pods " $2)}'

Deleting Evicted Pods In Each Namespace

#For Default Namespace
kubectl get pods | grep Evicted | awk '{print $1}' | xargs kubectl delete pod
#For $NAMESPACEkubectl get pods -n $NAMESPACE | grep Evicted | awk '{print $1}' | xargs kubectl delete pod -n $NAMESPACE

--

--

DevOps | SRE | #GDE

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store