When we create redis cluster in kubernetes, sometimes, we would get this error:
root@redis-0:/data# redis-cli --cluster create --cluster-replicas 1 10.42.3.27:6379 10.42.4.56:6379 10.42.7.113:6379 10.42.5.200:6379 10.42.6.179:6379 10.42.5.201:6379 [ERR] Node 10.42.3.27:6379 is not empty. Either the node already knows other nodes (check with CLUSTER NODES) or contains some key in database 0.
- Kubernetes 1.19
You are using the old PVC in the newly created redis cluster statefulsets. If you just delete the statefulset like this:
kubectl delete statefulset xxx
Then the PVC bound to the statefulset is not deleted automatically by default.
According to the official kubernetes document :
Deleting the Pods in a StatefulSet will not delete the associated volumes. This is to ensure that you have the chance to copy data off the volume before deleting it. Deleting the PVC after the pods have terminated might trigger deletion of the backing Persistent Volumes depending on the storage class and reclaim policy. You should never assume ability to access a volume after claim deletion.
You can delete the PVC as following:
kubectl delete statefulset -l app=myapp kubectl delete pvc -l app=myapp
Please note the label “app=myapp” should be changed according to your statefulset app labels.