K8 Version:
Client Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:44:27Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"6", GitVersion:"v1.6.4", GitCommit:"d6f433224538d4f9ca2f7ae19b252e6fcb66a3ae", GitTreeState:"clean", BuildDate:"2017-05-19T18:33:17Z", GoVersion:"go1.7.5", Compiler:"gc", Platform:"linux/amd64"} I tried to launch spinnaker pods(yaml files here). I choose Flannel(kubectl apply -f kube-flannel.yml) while installing K8. Then I see the pods are not starting, it is struck in "ContainerCreating" status. I kubectl describe a pod, showing NetworkPlugin cni failed to set up pod
veeru@ubuntu:/opt/spinnaker/experimental/kubernetes/simple$ kubectl describe pod data-redis-master-v000-38j80 --namespace=spinnaker Name: data-redis-master-v000-38j80 Namespace: spinnaker Node: ubuntu/192.168.6.136 Start Time: Thu, 01 Jun 2017 02:54:14 -0700 Labels: load-balancer-data-redis-server=true replication-controller=data-redis-master-v000 Annotations: "kind":"SerializedReference","apiVersion":"v1","reference":{"kind":"ReplicaSet","namespace":"spinnaker","name":"data-redis-master-v000","uid":"43d4a44c-46b0-11e7-b0e1-000c29b... Status: Pending IP: Controllers: ReplicaSet/data-redis-master-v000 Containers: redis-master: Container ID: Image: Image ID: Port: 6379/TCP State: Waiting Reason: ContainerCreating Ready: False Restart Count: 0 Limits: cpu: 100m Requests: cpu: 100m Environment: MASTER: true Mounts: /redis-master-data from data (rw) /var/run/secrets/ from default-token-71p4q (ro) Conditions: Type Status Initialized True Ready False PodScheduled True Volumes: data: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: default-token-71p4q: Type: Secret (a volume populated by a Secret) SecretName: default-token-71p4q Optional: false QoS Class: Burstable Node-Selectors: <none> Tolerations: for 300s for 300s Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 45m 45m 1 default-scheduler Normal Scheduled Successfully assigned data-redis-master-v000-38j80 to ubuntu 43m 43m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: 8265d80732e7b73ebf8f1493d40403021064b61436c4c559b41330e7592fd47f" 43m 43m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: rpc error: code = 2 desc = Error: No such container: b972862d763e621e026728073deb9a304748c4ec4522982db0a168663ab59d36 42m 42m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: 72b39083a3a81c0da1d4b7fa65b5d6450b62a3562a05452c27b185bc33197327" 41m 41m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: d315511bfa9f6f09d7ef4cd277bde44e4885291ea566e3089460356c1ed34413" 40m 40m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: a03d776d2d7c5c4ae9c1ec31681b0b6e40759326a452916cff0e60c4d4e2c954" 40m 40m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: acf30a4aacda0c53bdbb8bc2d416704720bd1b623c43874052b4029f15950052" 39m 39m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: ea49f5f9428d585be7138f4ebce54f713eef549b16104a3d7aa728175b6ebc2a" 38m 38m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "447d302c-46b0-11e7-b0e1-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: ec2483435b4b22576c9bd7bffac5d67d53893c189c0cf26aca1ae6af79d09914" 38m 1m 39 kubelet, ubuntu Warning FailedSync (events with common reason combined) 45m 1s 448 kubelet, ubuntu Normal SandboxChanged Pod sandbox changed, it will be killed and re-created. 45m 0s 412 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "CreatePodSandbox" for "data-redis-master-v000-38j80_spinnaker(447d302c-46b0-11e7-b0e1-000c29b1270f)" with CreatePodSandboxError: "CreatePodSandbox for pod \"data-redis-master-v000-38j80_spinnaker(447d302c-46b0-11e7-b0e1-000c29b1270f)\" failed: rpc error: code = 2 desc = NetworkPlugin cni failed to set up pod \"data-redis-master-v000-38j80_spinnaker\" network: open /run/flannel/subnet.env: no such file or directory" How can I resolve above issue?
UPDATE-1
I have reinitialized K8 with kubeadm init --pod-network-cidr=10.244.0.0/16 and deployed sample nginx pod. Still getting same error
-----------------OUTPUT REMOVED------------------------------- Events: FirstSeen LastSeen Count From SubObjectPath Type Reason Message --------- -------- ----- ---- ------------- -------- ------ ------- 3m 3m 1 default-scheduler Normal Scheduled Successfully assigned nginx-622qj to ubuntu 1m 1m 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "0728fece-46fe-11e7-ae5d-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: 38250afd765f0108aeff6e31bbe5a642a60db99b97cbbf15711f810cbe8f3829" 24s 24s 1 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "KillPodSandbox" for "0728fece-46fe-11e7-ae5d-000c29b1270f" with KillPodSandboxError: "rpc error: code = 2 desc = NetworkPlugin cni failed to teardown pod \"_\" network: CNI failed to retrieve network namespace path: Error: No such container: 3bebcef02cb5f6645a65dcf06b2730144080f9d3c4fb18267feca5c5ce21031c" 2m 9s 33 kubelet, ubuntu Normal SandboxChanged Pod sandbox changed, it will be killed and re-created. 3m 7s 32 kubelet, ubuntu Warning FailedSync Error syncing pod, skipping: failed to "CreatePodSandbox" for "nginx-622qj_default(0728fece-46fe-11e7-ae5d-000c29b1270f)" with CreatePodSandboxError: "CreatePodSandbox for pod \"nginx-622qj_default(0728fece-46fe-11e7-ae5d-000c29b1270f)\" failed: rpc error: code = 2 desc = NetworkPlugin cni failed to set up pod \"nginx-622qj_default\" network: open /run/flannel/subnet.env: no such file or directory" 4 Answers
You error message shows flanel subnet.evn file is missing. you need to fix flannel configuration first. What version of kubernetes your using?
network: open /run/flannel/subnet.env: no such file or directory"
if your using kubernetes 1.6 and above, we can use below yaml file to configure the flannel container process.
kubectl create -f kubectl create -f Running the following command resolved my issues:
kubeadm init --pod-network-cidr=10.244.0.0/16 For flannel as cni the api server needs to have the argument --pod-network-cidr=... to be set to the overlay.
Here are the steps that fixed my issue.
Create a file called
subnet.envat location/run/flannel/inside your worker nodes.Add the below content in it.
FLANNEL_NETWORK=10.244.0.0/16
FLANNEL_SUBNET=10.244.0.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=trueSave the file and create the pod again. It should saw the runnign status now.
I was getting same error on one of node after disk pressure and some how it deleted file /run/flannel/subnet.env.
After creating file with content from other node resolve my issue.
