This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 1 failed / 9 succeeded
Started2020-06-30 10:47
Elapsed53m50s
Revisionrelease-1.17
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/7d19f7eb-a202-4842-96ab-89c4159640f7/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/7d19f7eb-a202-4842-96ab-89c4159640f7/targets/test

Test Failures


task-06-upgrade 15m0s

timeout. The task did not complete in less than 15m0s as expected
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 9 Passed Tests

Show 3 Skipped Tests

Error lines from build-log.txt

... skipping 299 lines ...
Created symlink /etc/systemd/system/multi-user.target.wants/kubelet.service → /kind/systemd/kubelet.service.
Created symlink /etc/systemd/system/kubelet.service → /kind/systemd/kubelet.service.
time="10:54:16" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 mkdir -p /etc/systemd/system/kubelet.service.d]"
time="10:54:18" level=info msg="Adding /etc/systemd/system/kubelet.service.d/10-kubeadm.conf to the image"
time="10:54:18" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 cp /alter/bits/systemd/10-kubeadm.conf /etc/systemd/system/kubelet.service.d/10-kubeadm.conf]"
time="10:54:21" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 chown -R root:root /etc/systemd/system/kubelet.service.d/10-kubeadm.conf]"
time="10:54:24" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 /bin/sh -c echo \"KUBELET_EXTRA_ARGS=--fail-swap-on=false\" >> /etc/default/kubelet]"
time="10:54:26" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 mkdir -p /kinder]"
time="10:54:29" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 rsync -r /alter/bits/upgrade /kinder]"
time="10:55:20" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 chown -R root:root /kinder/upgrade]"
time="10:55:23" level=info msg="Start CRI ..."
time="10:55:23" level=info msg="starting dockerd"
time="10:55:23" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 bash -c docker info &> /dev/null]"
... skipping 5 lines ...
time="2020-06-30T10:55:26.429068682Z" level=info msg="ccResolverWrapper: sending new addresses to cc: [{unix:///var/run/docker/containerd/containerd.sock 0  <nil>}]" module=grpc
time="2020-06-30T10:55:26.429099428Z" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2020-06-30T10:55:26.429169896Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4201a9bd0, CONNECTING" module=grpc
time="2020-06-30T10:55:27.063582066Z" level=info msg="starting containerd" revision=bb71b10fd8f58240ca47fbb579b9d1028eea7c84 version=1.2.5 
time="2020-06-30T10:55:27.065162469Z" level=info msg="loading plugin "io.containerd.content.v1.content"..." type=io.containerd.content.v1 
time="2020-06-30T10:55:27.065342559Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.btrfs"..." type=io.containerd.snapshotter.v1 
time="2020-06-30T10:55:27.065662537Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.btrfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
time="2020-06-30T10:55:27.065711309Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.aufs"..." type=io.containerd.snapshotter.v1 
time="2020-06-30T10:55:27.096137908Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.aufs" error="modprobe aufs failed: "modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: FATAL: Module aufs not found in directory /lib/modules/4.15.0-1044-gke\n": exit status 1" 
time="2020-06-30T10:55:27.096198374Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.native"..." type=io.containerd.snapshotter.v1 
time="2020-06-30T10:55:27.096365121Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.overlayfs"..." type=io.containerd.snapshotter.v1 
time="2020-06-30T10:55:27.096603398Z" level=info msg="loading plugin "io.containerd.snapshotter.v1.zfs"..." type=io.containerd.snapshotter.v1 
time="2020-06-30T10:55:27.098088833Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.zfs" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
time="2020-06-30T10:55:27.098113972Z" level=info msg="loading plugin "io.containerd.metadata.v1.bolt"..." type=io.containerd.metadata.v1 
time="2020-06-30T10:55:27.098207959Z" level=warning msg="could not use snapshotter btrfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs must be a btrfs filesystem to be used with the btrfs snapshotter" 
time="2020-06-30T10:55:27.098221448Z" level=warning msg="could not use snapshotter aufs in metadata plugin" error="modprobe aufs failed: "modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: FATAL: Module aufs not found in directory /lib/modules/4.15.0-1044-gke\n": exit status 1" 
time="2020-06-30T10:55:27.098233631Z" level=warning msg="could not use snapshotter zfs in metadata plugin" error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter" 
time="2020-06-30T10:55:27.247207080Z" level=info msg="loading plugin "io.containerd.differ.v1.walking"..." type=io.containerd.differ.v1 
time="2020-06-30T10:55:27.248095044Z" level=info msg="loading plugin "io.containerd.gc.v1.scheduler"..." type=io.containerd.gc.v1 
time="2020-06-30T10:55:27.248159141Z" level=info msg="loading plugin "io.containerd.service.v1.containers-service"..." type=io.containerd.service.v1 
time="2020-06-30T10:55:27.248175539Z" level=info msg="loading plugin "io.containerd.service.v1.content-service"..." type=io.containerd.service.v1 
time="2020-06-30T10:55:27.248190580Z" level=info msg="loading plugin "io.containerd.service.v1.diff-service"..." type=io.containerd.service.v1 
time="2020-06-30T10:55:27.248225371Z" level=info msg="loading plugin "io.containerd.service.v1.images-service"..." type=io.containerd.service.v1 
... skipping 18 lines ...
time="2020-06-30T10:55:27.249866754Z" level=info msg="loading plugin "io.containerd.grpc.v1.tasks"..." type=io.containerd.grpc.v1 
time="2020-06-30T10:55:27.249880423Z" level=info msg="loading plugin "io.containerd.grpc.v1.version"..." type=io.containerd.grpc.v1 
time="2020-06-30T10:55:27.249893152Z" level=info msg="loading plugin "io.containerd.grpc.v1.cri"..." type=io.containerd.grpc.v1 
time="2020-06-30T10:55:27.249962462Z" level=info msg="Start cri plugin with config {PluginConfig:{ContainerdConfig:{Snapshotter:overlayfs DefaultRuntime:{Type:io.containerd.runtime.v1.linux Engine: Root: Options:<nil>} UntrustedWorkloadRuntime:{Type: Engine: Root: Options:<nil>} Runtimes:map[] NoPivot:false} CniConfig:{NetworkPluginBinDir:/opt/cni/bin NetworkPluginConfDir:/etc/cni/net.d NetworkPluginConfTemplate:} Registry:{Mirrors:map[docker.io:{Endpoints:[https://registry-1.docker.io]}] Auths:map[]} StreamServerAddress:127.0.0.1 StreamServerPort:0 EnableSelinux:false SandboxImage:k8s.gcr.io/pause:3.1 StatsCollectPeriod:10 SystemdCgroup:false EnableTLSStreaming:false X509KeyPairStreaming:{TLSCertFile: TLSKeyFile:} MaxContainerLogLineSize:16384} ContainerdRootDir:/var/lib/docker/containerd/daemon ContainerdEndpoint:/var/run/docker/containerd/containerd.sock RootDir:/var/lib/docker/containerd/daemon/io.containerd.grpc.v1.cri StateDir:/var/run/docker/containerd/daemon/io.containerd.grpc.v1.cri}" 
time="2020-06-30T10:55:27.249988728Z" level=info msg="Connect containerd service" 
time="2020-06-30T10:55:27.250177122Z" level=info msg="Get image filesystem path "/var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.overlayfs"" 
time="2020-06-30T10:55:27.250382170Z" level=error msg="Failed to load cni during init, please check CRI plugin status before setting up network for pods" error="cni config load failed: no network config found in /etc/cni/net.d: cni plugin not initialized: failed to load cni config" 
time="2020-06-30T10:55:27.250614908Z" level=info msg="loading plugin "io.containerd.grpc.v1.introspection"..." type=io.containerd.grpc.v1 
time="2020-06-30T10:55:27.250909272Z" level=info msg=serving... address="/var/run/docker/containerd/containerd-debug.sock" 
time="2020-06-30T10:55:27.250979922Z" level=info msg=serving... address="/var/run/docker/containerd/containerd.sock" 
time="2020-06-30T10:55:27.250992524Z" level=info msg="containerd successfully booted in 0.190110s" 
time="2020-06-30T10:55:27.257344964Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4201a9bd0, READY" module=grpc
time="2020-06-30T10:55:27.272634124Z" level=info msg="Start subscribing containerd event" 
... skipping 15 lines ...
time="2020-06-30T10:55:27.409351016Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc420940130, READY" module=grpc
time="2020-06-30T10:55:27.629791887Z" level=info msg="Graph migration to content-addressability took 0.00 seconds"
time="2020-06-30T10:55:27.630275723Z" level=warning msg="Your kernel does not support swap memory limit"
time="2020-06-30T10:55:27.630344948Z" level=warning msg="Your kernel does not support cgroup rt period"
time="2020-06-30T10:55:27.630359327Z" level=warning msg="Your kernel does not support cgroup rt runtime"
time="2020-06-30T10:55:27.632376474Z" level=info msg="Loading containers: start."
time="2020-06-30T10:55:27.640778611Z" level=warning msg="Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: WARNING: Module bridge not found in directory /lib/modules/4.15.0-1044-gke\nmodprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: WARNING: Module br_netfilter not found in directory /lib/modules/4.15.0-1044-gke\n, error: exit status 1"
time="2020-06-30T10:55:27.644329739Z" level=warning msg="Running modprobe nf_nat failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: WARNING: Module nf_nat not found in directory /lib/modules/4.15.0-1044-gke`, error: exit status 1"
time="2020-06-30T10:55:27.681534326Z" level=warning msg="Running modprobe xt_conntrack failed with message: `modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.0-1044-gke/modules.dep.bin'\nmodprobe: WARNING: Module xt_conntrack not found in directory /lib/modules/4.15.0-1044-gke`, error: exit status 1"
time="2020-06-30T10:55:29.243991122Z" level=info msg="Default bridge (docker0) is assigned with an IP address 172.18.0.0/16. Daemon option --bip can be used to set a preferred IP address"
time="2020-06-30T10:55:31.767758360Z" level=info msg="Loading containers: done."
time="2020-06-30T10:55:38.243993696Z" level=info msg="Docker daemon" commit=d14af54 graphdriver(s)=overlay2 version=18.09.4
time="2020-06-30T10:55:38.244229354Z" level=info msg="Daemon has completed initialization"
time="2020-06-30T10:55:38.418825730Z" level=info msg="API listen on /var/run/docker.sock"
time="10:55:40" level=info msg="dockerd started"
... skipping 16 lines ...
time="10:58:20" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 docker save -o=/kinder/upgrade/v1.17.9-rc.0.11+9a1884e4f62b4a/etcd.tar k8s.gcr.io/etcd:3.4.3-0]"
time="10:59:05" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 bash -c docker pull k8s.gcr.io/coredns:1.6.5 > /dev/null]"
time="10:59:17" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 docker save -o=/kinder/upgrade/v1.17.9-rc.0.11+9a1884e4f62b4a/coredns.tar k8s.gcr.io/coredns:1.6.5]"
time="10:59:25" level=info msg="Stop CRI ..."
time="10:59:25" level=debug msg="Running: [docker exec kind-build-98e7d002-cb8c-426a-a346-7f5bd0de15c7 pkill -f dockerd]"
time="2020-06-30T10:59:27.535824082Z" level=info msg="Processing signal 'terminated'"
time="2020-06-30T10:59:27.537819689Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=moby
time="2020-06-30T10:59:27.537925387Z" level=info msg="stopping healthcheck following graceful shutdown" module=libcontainerd
time="2020-06-30T10:59:27.538032122Z" level=info msg="stopping event stream following graceful shutdown" error="context canceled" module=libcontainerd namespace=plugins.moby
time="2020-06-30T10:59:27.538535926Z" level=info msg="Stop CRI service" 
time="2020-06-30T10:59:27.576338545Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207694f0, TRANSIENT_FAILURE" module=grpc
time="2020-06-30T10:59:27.576398335Z" level=info msg="pickfirstBalancer: HandleSubConnStateChange: 0xc4207694f0, CONNECTING" module=grpc
time="10:59:28" level=info msg="Commit to kindest/node:test ..."
sha256:9270eda70d0d1a5ae1bf0aec908dd48c6a75dfaebe1c1d5cb4cbb2e8e4f2ad31
time="11:03:26" level=info msg="Image alter completed."
... skipping 174 lines ...
kinder-upgrade-control-plane-1:$ Preparing /kind/kubeadm.conf
time="11:09:00" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-1]"
time="11:09:02" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-1 kubeadm version -o=short]"
time="11:09:04" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:09:04" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:09:04" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:09:05" level=debug msg="generated config:\napiServer:\n  certSANs:\n  - localhost\n  - 172.17.0.2\napiVersion: kubeadm.k8s.io/v1beta2\nclusterName: kinder-upgrade\ncontrolPlaneEndpoint: 172.17.0.7:6443\ncontrollerManager:\n  extraArgs:\n    enable-hostpath-provisioner: \"true\"\nkind: ClusterConfiguration\nkubernetesVersion: v1.16.13-rc.0.5+24212641c73bc0\nnetworking:\n  podSubnet: 192.168.0.0/16\n  serviceSubnet: \"\"\nscheduler:\n  extraArgs: null\n---\napiVersion: kubeadm.k8s.io/v1beta2\nbootstrapTokens:\n- token: abcdef.0123456789abcdef\ncertificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\nkind: InitConfiguration\nlocalAPIEndpoint:\n  advertiseAddress: 172.17.0.2\n  bindPort: 6443\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.2\n---\napiVersion: kubelet.config.k8s.io/v1beta1\nevictionHard:\n  imagefs.available: 0%\n  nodefs.available: 0%\n  nodefs.inodesFree: 0%\nimageGCHighThresholdPercent: 100\nkind: KubeletConfiguration\n---\napiVersion: kubeproxy.config.k8s.io/v1alpha1\nkind: KubeProxyConfiguration\n"
time="11:09:05" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-1-123179675 kinder-upgrade-control-plane-1:/kind/kubeadm.conf]"

kinder-upgrade-lb:$ Updating load balancer configuration with 1 control plane backends
time="11:09:08" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-1]"
time="11:09:09" level=debug msg="Writing loadbalancer config on kinder-upgrade-lb..."
time="11:09:09" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-lb-427224382 kinder-upgrade-lb:/usr/local/etc/haproxy/haproxy.cfg]"
... skipping 32 lines ...
I0630 11:09:21.200475     861 checks.go:377] validating the presence of executable ebtables
I0630 11:09:21.200531     861 checks.go:377] validating the presence of executable ethtool
I0630 11:09:21.200585     861 checks.go:377] validating the presence of executable socat
I0630 11:09:21.200628     861 checks.go:377] validating the presence of executable tc
I0630 11:09:21.200691     861 checks.go:377] validating the presence of executable touch
I0630 11:09:21.200753     861 checks.go:521] running all checks
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0630 11:09:22.542368     861 checks.go:407] checking whether the given node name is reachable using net.LookupHost
I0630 11:09:22.542655     861 checks.go:619] validating kubelet version
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
... skipping 375 lines ...
kinder-upgrade-control-plane-2:$ Preparing /kind/kubeadm.conf
time="11:12:08" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-2]"
time="11:12:09" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm version -o=short]"
time="11:12:12" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:12:12" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:12:12" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:12:12" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ncontrolPlane:\n  certificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\n  localAPIEndpoint:\n    advertiseAddress: 172.17.0.6\n    bindPort: 6443\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.6\n"
time="11:12:12" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-2-633308667 kinder-upgrade-control-plane-2:/kind/kubeadm.conf]"
time="11:12:15" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm version -o=short]"

kinder-upgrade-control-plane-2:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="11:12:18" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-2 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0630 11:12:20.155510    1023 join.go:363] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 17 lines ...
I0630 11:12:22.539339    1023 checks.go:377] validating the presence of executable ebtables
I0630 11:12:22.540028    1023 checks.go:377] validating the presence of executable ethtool
I0630 11:12:22.540700    1023 checks.go:377] validating the presence of executable socat
I0630 11:12:22.541479    1023 checks.go:377] validating the presence of executable tc
I0630 11:12:22.542143    1023 checks.go:377] validating the presence of executable touch
I0630 11:12:22.544328    1023 checks.go:521] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0630 11:12:23.634283    1023 checks.go:407] checking whether the given node name is reachable using net.LookupHost
I0630 11:12:23.635741    1023 checks.go:619] validating kubelet version
I0630 11:12:24.142765    1023 checks.go:129] validating if the service is enabled and active
I0630 11:12:24.267584    1023 checks.go:202] validating availability of port 10250
I0630 11:12:24.272350    1023 checks.go:433] validating if the connectivity type is via proxy or direct
I0630 11:12:24.272418    1023 join.go:433] [preflight] Discovering cluster-info
... skipping 112 lines ...
I0630 11:12:41.189532    1023 local.go:136] Adding etcd member: https://172.17.0.6:2380
I0630 11:12:41.462810    1023 local.go:142] Updated etcd member list: [{kinder-upgrade-control-plane-2 https://172.17.0.6:2380} {kinder-upgrade-control-plane-1 https://172.17.0.2:2380}]
I0630 11:12:41.464496    1023 etcd.go:372] [etcd] attempting to see if all cluster endpoints ([https://172.17.0.2:2379 https://172.17.0.6:2379]) are available 1/8
[etcd] Announced new etcd member joining to the existing etcd cluster
[etcd] Creating static Pod manifest for "etcd"
[etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
{"level":"warn","ts":"2020-06-30T11:12:57.495Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"passthrough:///https://172.17.0.6:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
I0630 11:12:57.496373    1023 etcd.go:377] [etcd] Attempt timed out
I0630 11:12:57.496634    1023 etcd.go:369] [etcd] Waiting 5s until next retry
I0630 11:13:02.512601    1023 etcd.go:372] [etcd] attempting to see if all cluster endpoints ([https://172.17.0.2:2379 https://172.17.0.6:2379]) are available 2/8
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0630 11:13:03.380440    1023 round_trippers.go:443] POST https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps 409 Conflict in 772 milliseconds
I0630 11:13:03.520308    1023 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config 200 OK in 139 milliseconds
... skipping 68 lines ...
kinder-upgrade-control-plane-3:$ Preparing /kind/kubeadm.conf
time="11:14:01" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-control-plane-3]"
time="11:14:02" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm version -o=short]"
time="11:14:05" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:14:05" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:14:05" level=debug msg="Preparing automaticCopyCertsPatches for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:14:05" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ncontrolPlane:\n  certificateKey: \"0123456789012345678901234567890123456789012345678901234567890123\"\n  localAPIEndpoint:\n    advertiseAddress: 172.17.0.5\n    bindPort: 6443\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.5\n"
time="11:14:05" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-control-plane-3-211102181 kinder-upgrade-control-plane-3:/kind/kubeadm.conf]"
time="11:14:07" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm version -o=short]"

kinder-upgrade-control-plane-3:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="11:14:10" level=debug msg="Running: [docker exec kinder-upgrade-control-plane-3 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
I0630 11:14:12.980562    1134 join.go:363] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 17 lines ...
I0630 11:14:15.356732    1134 checks.go:377] validating the presence of executable ebtables
I0630 11:14:15.356834    1134 checks.go:377] validating the presence of executable ethtool
I0630 11:14:15.357064    1134 checks.go:377] validating the presence of executable socat
I0630 11:14:15.359812    1134 checks.go:377] validating the presence of executable tc
I0630 11:14:15.360057    1134 checks.go:377] validating the presence of executable touch
I0630 11:14:15.360373    1134 checks.go:521] running all checks
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
CGROUPS_CPUSET: enabled
CGROUPS_DEVICES: enabled
CGROUPS_FREEZER: enabled
CGROUPS_MEMORY: enabled
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0630 11:14:16.395602    1134 checks.go:407] checking whether the given node name is reachable using net.LookupHost
I0630 11:14:16.395924    1134 checks.go:619] validating kubelet version
I0630 11:14:17.115189    1134 checks.go:129] validating if the service is enabled and active
I0630 11:14:17.252143    1134 checks.go:202] validating availability of port 10250
I0630 11:14:17.252383    1134 checks.go:433] validating if the connectivity type is via proxy or direct
I0630 11:14:17.252425    1134 join.go:433] [preflight] Discovering cluster-info
... skipping 111 lines ...
I0630 11:14:34.917761    1134 local.go:136] Adding etcd member: https://172.17.0.5:2380
I0630 11:14:35.356313    1134 local.go:142] Updated etcd member list: [{kinder-upgrade-control-plane-2 https://172.17.0.6:2380} {kinder-upgrade-control-plane-3 https://172.17.0.5:2380} {kinder-upgrade-control-plane-1 https://172.17.0.2:2380}]
I0630 11:14:35.358026    1134 etcd.go:372] [etcd] attempting to see if all cluster endpoints ([https://172.17.0.6:2379 https://172.17.0.2:2379 https://172.17.0.5:2379]) are available 1/8
[etcd] Announced new etcd member joining to the existing etcd cluster
[etcd] Creating static Pod manifest for "etcd"
[etcd] Waiting for the new etcd member to join the cluster. This can take up to 40s
{"level":"warn","ts":"2020-06-30T11:14:46.865Z","caller":"clientv3/retry_interceptor.go:61","msg":"retrying of unary invoker failed","target":"passthrough:///https://172.17.0.5:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
I0630 11:14:46.877187    1134 etcd.go:377] [etcd] Attempt timed out
I0630 11:14:46.878605    1134 etcd.go:369] [etcd] Waiting 5s until next retry
I0630 11:14:51.879790    1134 etcd.go:372] [etcd] attempting to see if all cluster endpoints ([https://172.17.0.6:2379 https://172.17.0.2:2379 https://172.17.0.5:2379]) are available 2/8
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
I0630 11:14:52.180932    1134 round_trippers.go:443] POST https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps 409 Conflict in 102 milliseconds
I0630 11:14:52.207424    1134 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config 200 OK in 19 milliseconds
... skipping 69 lines ...

kinder-upgrade-worker-1:$ Preparing /kind/kubeadm.conf
time="11:15:53" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-worker-1]"
time="11:15:54" level=debug msg="Running: [docker exec kinder-upgrade-worker-1 kubeadm version -o=short]"
time="11:15:57" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:15:57" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:15:57" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.4\n"
time="11:15:57" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-worker-1-991870815 kinder-upgrade-worker-1:/kind/kubeadm.conf]"

kinder-upgrade-worker-1:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="11:16:01" level=debug msg="Running: [docker exec kinder-upgrade-worker-1 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
[preflight] Running pre-flight checks
I0630 11:16:03.193409    1228 join.go:363] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 16 lines ...
I0630 11:16:05.535935    1228 checks.go:377] validating the presence of executable ebtables
I0630 11:16:05.536151    1228 checks.go:377] validating the presence of executable ethtool
I0630 11:16:05.536383    1228 checks.go:377] validating the presence of executable socat
I0630 11:16:05.536583    1228 checks.go:377] validating the presence of executable tc
I0630 11:16:05.536835    1228 checks.go:377] validating the presence of executable touch
I0630 11:16:05.537989    1228 checks.go:521] running all checks
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0630 11:16:06.704558    1228 checks.go:407] checking whether the given node name is reachable using net.LookupHost
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
... skipping 95 lines ...

kinder-upgrade-worker-2:$ Preparing /kind/kubeadm.conf
time="11:17:34" level=debug msg="Running: /usr/bin/docker [docker inspect -f {{range .NetworkSettings.Networks}}{{.IPAddress}},{{.GlobalIPv6Address}}{{end}} kinder-upgrade-worker-2]"
time="11:17:35" level=debug msg="Running: [docker exec kinder-upgrade-worker-2 kubeadm version -o=short]"
time="11:17:37" level=debug msg="Preparing kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:17:37" level=debug msg="Preparing dockerPatch for kubeadm config v1beta2 (kubeadm version 1.16.13-rc.0.5+24212641c73bc0)"
time="11:17:37" level=debug msg="generated config:\napiVersion: kubeadm.k8s.io/v1beta2\ndiscovery:\n  bootstrapToken:\n    apiServerEndpoint: 172.17.0.7:6443\n    token: abcdef.0123456789abcdef\n    unsafeSkipCAVerification: true\nkind: JoinConfiguration\nnodeRegistration:\n  criSocket: /var/run/dockershim.sock\n  kubeletExtraArgs:\n    fail-swap-on: \"false\"\n    node-ip: 172.17.0.3\n"
time="11:17:37" level=debug msg="Running: [docker cp /tmp/kinder-upgrade-worker-2-972259634 kinder-upgrade-worker-2:/kind/kubeadm.conf]"

kinder-upgrade-worker-2:$ kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables
time="11:17:40" level=debug msg="Running: [docker exec kinder-upgrade-worker-2 kubeadm join --config=/kind/kubeadm.conf --v=6 --ignore-preflight-errors=Swap,SystemVerification,FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]"
[preflight] Running pre-flight checks
I0630 11:17:42.353313    1318 join.go:363] [preflight] found NodeName empty; using OS hostname as NodeName
... skipping 16 lines ...
I0630 11:17:44.870357    1318 checks.go:377] validating the presence of executable ebtables
I0630 11:17:44.870525    1318 checks.go:377] validating the presence of executable ethtool
I0630 11:17:44.870658    1318 checks.go:377] validating the presence of executable socat
I0630 11:17:44.870767    1318 checks.go:377] validating the presence of executable tc
I0630 11:17:44.870944    1318 checks.go:377] validating the presence of executable touch
I0630 11:17:44.871130    1318 checks.go:521] running all checks
	[WARNING SystemVerification]: failed to parse kernel config: unable to load kernel module: "configs", output: "modprobe: FATAL: Module configs not found in directory /lib/modules/4.15.0-1044-gke\n", err: exit status 1
I0630 11:17:46.044487    1318 checks.go:407] checking whether the given node name is reachable using net.LookupHost
[preflight] The system verification failed. Printing the output from the verification:
KERNEL_VERSION: 4.15.0-1044-gke
DOCKER_VERSION: 18.09.4
DOCKER_GRAPH_DRIVER: overlay2
OS: Linux
CGROUPS_CPU: enabled
CGROUPS_CPUACCT: enabled
... skipping 1431 lines ...
I0630 11:34:55.702240   29590 local.go:106] [etcd] get the member id from peer: https://172.17.0.2:2380
I0630 11:34:55.760485   29590 local.go:112] [etcd] removing etcd member: https://172.17.0.2:2380, id: 13322012572989635620
[reset] Stopping the kubelet service
I0630 11:34:55.970807   29590 local.go:117] [etcd] Updated etcd member list: [{kinder-upgrade-control-plane-2 https://172.17.0.6:2380} {kinder-upgrade-control-plane-3 https://172.17.0.5:2380}]
I0630 11:34:55.979164   29590 cleanupnode.go:57] [reset] Getting init system
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0630 11:34:56.341634   29590 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0630 11:34:56.802351   29590 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
[apiclient] Error getting Pods with label selector "component=kube-apiserver" [rpc error: code = Unavailable desc = transport is closing]
I0630 11:34:57.082097   15446 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver 500 Internal Server Error in 1075 milliseconds
[apiclient] Error getting Pods with label selector "component=kube-apiserver" [Get https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver: net/http: request canceled (Client.Timeout exceeded while awaiting headers)]
I0630 11:35:07.521280   15446 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver  in 10015 milliseconds
I0630 11:35:17.446646   15446 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver  in 9429 milliseconds
[apiclient] Error getting Pods with label selector "component=kube-apiserver" [Get https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver: unexpected EOF]
[upgrade/staticpods] Component "kube-apiserver" upgraded successfully!
[upgrade/staticpods] Preparing for "kube-controller-manager" upgrade
[upgrade/staticpods] Renewing controller-manager.conf certificate
I0630 11:35:17.848314   15446 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/pods?labelSelector=component%3Dkube-apiserver 200 OK in 353 milliseconds
I0630 11:35:17.850956   15446 loader.go:375] Config loaded from file:  /etc/kubernetes/controller-manager.conf
[upgrade/staticpods] Moved new manifest to "/etc/kubernetes/manifests/kube-controller-manager.yaml" and backed up old manifest to "/etc/kubernetes/tmp/kubeadm-backup-manifests-2020-06-30-11-33-54/kube-controller-manager.yaml"
... skipping 103 lines ...
I0630 11:35:43.981307   18274 local.go:106] [etcd] get the member id from peer: https://172.17.0.6:2380
I0630 11:35:44.190569   18274 local.go:112] [etcd] removing etcd member: https://172.17.0.6:2380, id: 8673597326142672285
[reset] Stopping the kubelet service
I0630 11:35:44.344981   18274 local.go:117] [etcd] Updated etcd member list: [{kinder-upgrade-control-plane-3 https://172.17.0.5:2380}]
I0630 11:35:44.345072   18274 cleanupnode.go:57] [reset] Getting init system
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0630 11:35:44.846895   18274 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0630 11:35:44.932408   18274 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
I0630 11:36:19.928410   18274 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
I0630 11:36:19.929810   18274 reset.go:218] [reset] Deleting content of /var/lib/etcd
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/etcd /var/lib/kubelet /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
... skipping 31 lines ...
I0630 11:36:22.771606   17627 uploadconfig.go:58] Removing APIEndpoint for Node "kinder-upgrade-control-plane-3"
I0630 11:36:22.861328   17627 round_trippers.go:443] PUT https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 88 milliseconds
I0630 11:36:22.862010   17627 removeetcdmember.go:54] [reset] Checking for etcd config
I0630 11:36:22.862051   17627 local.go:97] [etcd] creating etcd client that connects to etcd pods
I0630 11:36:22.876272   17627 round_trippers.go:443] GET https://172.17.0.7:6443/api/v1/namespaces/kube-system/configmaps/kubeadm-config?timeout=10s 200 OK in 14 milliseconds
I0630 11:36:22.877084   17627 etcd.go:108] etcd endpoints read from pods: 
W0630 11:36:22.877619   17627 removeetcdmember.go:61] [reset] failed to remove etcd member: error syncing endpoints with etc: etcdclient: no available endpoints
.Please manually remove this etcd member using etcdctl
I0630 11:36:22.877661   17627 cleanupnode.go:57] [reset] Getting init system
[reset] Stopping the kubelet service
W0630 11:36:23.331909   17627 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
[reset] Unmounting mounted directories in "/var/lib/kubelet"
I0630 11:36:23.732815   17627 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
I0630 11:36:58.858842   17627 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
I0630 11:36:58.864682   17627 reset.go:218] [reset] Deleting content of /var/lib/etcd
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
... skipping 22 lines ...
I0630 11:37:01.440079   14509 removeetcdmember.go:54] [reset] Checking for etcd config
W0630 11:37:01.440442   14509 removeetcdmember.go:79] [reset] No kubeadm config, using etcd pod spec to get data directory
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
I0630 11:37:01.441666   14509 cleanupnode.go:57] [reset] Getting init system
[reset] Stopping the kubelet service
W0630 11:37:01.951189   14509 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
[reset] Unmounting mounted directories in "/var/lib/kubelet"
I0630 11:37:02.233963   14509 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
I0630 11:37:22.163414   14509 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
I0630 11:37:22.163918   14509 reset.go:209] [reset] Deleting content of /var/lib/kubelet
I0630 11:37:22.169192   14509 reset.go:209] [reset] Deleting content of /etc/cni/net.d
... skipping 21 lines ...
I0630 11:37:25.352376   13798 cleanupnode.go:57] [reset] Getting init system
[preflight] Running pre-flight checks
[reset] No etcd config found. Assuming external etcd
[reset] Please, manually reset etcd to prevent further issues
[reset] Stopping the kubelet service
[reset] Unmounting mounted directories in "/var/lib/kubelet"
W0630 11:37:25.740983   13798 unmount_linux.go:42] [reset] Failed to unmount mounted directory in /var/lib/kubelet: /var/lib/kubelet
I0630 11:37:25.974833   13798 cleanupnode.go:79] [reset] Removing Kubernetes-managed containers
[reset] Deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] Deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf]
[reset] Deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes /var/lib/cni]
I0630 11:37:46.106794   13798 cleanupnode.go:87] [reset] Removing contents from the config and pki directories
I0630 11:37:46.107121   13798 reset.go:209] [reset] Deleting content of /var/lib/kubelet
... skipping 19 lines ...
Deleting cluster "kinder-upgrade" ...
time="11:37:48" level=debug msg="Running: /usr/bin/docker [docker ps -q -a --no-trunc --filter label=io.k8s.sigs.kind.cluster --format {{.Names}}\\t{{.Label \"io.k8s.sigs.kind.cluster\"}} --filter label=io.k8s.sigs.kind.cluster=kinder-upgrade]"
time="11:37:49" level=debug msg="Running: /usr/bin/docker [docker rm -f -v kinder-upgrade-lb kinder-upgrade-control-plane-3 kinder-upgrade-worker-1 kinder-upgrade-control-plane-2 kinder-upgrade-control-plane-1 kinder-upgrade-worker-2]"
 completed!

Ran 10 of 13 tasks in 0.000 seconds
FAIL! -- 9 tasks Passed | 1 Failed | 3 Skipped

see junit-runner.xml and task logs files for more details

Error: failed executing the workflow
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
... skipping 3 lines ...