This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 20 failed / 1184 succeeded
Started2020-10-25 12:31
Elapsed14h53m
Revision
Builderfdc7e5f8-16bd-11eb-b256-6ee25ea2e440
infra-commitd88010efd
job-versionv1.17.14-rc.0.15+54f3c097502f5b
master_os_imagecos-77-12371-175-0
node_os_imagecos-77-12371-175-0
revisionv1.17.14-rc.0.15+54f3c097502f5b

Test Failures


Cluster upgrade [sig-cloud-provider-gcp] cluster-upgrade 4m33s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Cluster\supgrade\s\[sig\-cloud\-provider\-gcp\]\scluster\-upgrade$'
Oct 25 12:45:22.087: Unexpected error:
    <*errors.errorString | 0xc00218a690>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-6hkr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-vlrh\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-xc35\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   8m19s   v1.18.11-rc.0.13+806617c8cf1f0d\\nbootstrap-e2e-minion-group-6hkr   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-vlrh   Ready                      <none>   8m20s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-xc35   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.11-rc.0.13+806617c8cf1f0d\\\"\\nname: \\\"bootstrap-e2e-minion-group-6hkr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-vlrh\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-xc35\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\n\", stderr \"Project: k8s-jkns-gci-gce-sd-log\\nNetwork Project: k8s-jkns-gci-gce-sd-log\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-6hkr bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-xc35\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.110.103; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.110.103  RUNNING\\nWarning: Permanently added 'compute.8446227793057467425' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-gci-gce-sd-log\\nNetwork Project: k8s-jkns-gci-gce-sd-log\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-6hkr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-vlrh\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-xc35\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   8m19s   v1.18.11-rc.0.13+806617c8cf1f0d\nbootstrap-e2e-minion-group-6hkr   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-vlrh   Ready                      <none>   8m20s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-xc35   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.11-rc.0.13+806617c8cf1f0d\"\nname: \"bootstrap-e2e-minion-group-6hkr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-vlrh\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-xc35\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\n", stderr "Project: k8s-jkns-gci-gce-sd-log\nNetwork Project: k8s-jkns-gci-gce-sd-log\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-6hkr bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-xc35\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.110.103; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.110.103  RUNNING\nWarning: Permanently added 'compute.8446227793057467425' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-gci-gce-sd-log\nNetwork Project: k8s-jkns-gci-gce-sd-log\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153 +0x132
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc001f891e8)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc0009d2b00, 0x799a240, 0xc, 0xc, 0xc000695440, 0xc0032eac30, 0x2, 0xc003476da0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:485 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:158 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00269e100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:125 +0x324
k8s.io/kubernetes/test/e2e.TestE2E(0xc00269e100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:111 +0x2b
testing.tRunner(0xc00269e100, 0x4af5d58)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] cluster upgrade should maintain a functioning cluster [Feature:ClusterUpgrade] 16m46s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\scluster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:ClusterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:142
Oct 25 12:45:22.087: Unexpected error:
    <*errors.errorString | 0xc00218a690>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-6hkr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-vlrh\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-xc35\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   8m19s   v1.18.11-rc.0.13+806617c8cf1f0d\\nbootstrap-e2e-minion-group-6hkr   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-vlrh   Ready                      <none>   8m20s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-xc35   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.11-rc.0.13+806617c8cf1f0d\\\"\\nname: \\\"bootstrap-e2e-minion-group-6hkr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-vlrh\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-xc35\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\n\", stderr \"Project: k8s-jkns-gci-gce-sd-log\\nNetwork Project: k8s-jkns-gci-gce-sd-log\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-6hkr bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-xc35\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.110.103; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.110.103  RUNNING\\nWarning: Permanently added 'compute.8446227793057467425' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-gci-gce-sd-log\\nNetwork Project: k8s-jkns-gci-gce-sd-log\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-6hkr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-vlrh\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-xc35\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   8m19s   v1.18.11-rc.0.13+806617c8cf1f0d\nbootstrap-e2e-minion-group-6hkr   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-vlrh   Ready                      <none>   8m20s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-xc35   Ready                      <none>   8m21s   v1.17.14-rc.0.15+54f3c097502f5b\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.11-rc.0.13+806617c8cf1f0d\"\nname: \"bootstrap-e2e-minion-group-6hkr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-vlrh\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-xc35\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\n", stderr "Project: k8s-jkns-gci-gce-sd-log\nNetwork Project: k8s-jkns-gci-gce-sd-log\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-6hkr bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-xc35\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.110.103; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gci-gce-sd-log/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.110.103  RUNNING\nWarning: Permanently added 'compute.8446227793057467425' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-gci-gce-sd-log\nNetwork Project: k8s-jkns-gci-gce-sd-log\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m13s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:57
Oct 26 00:32:01.671: Timed out after 60.003s.
Expected success, but got an error:
    <*errors.errorString | 0xc003a4b3d0>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-fcw9.1641636ae7b9a655  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fcw9.1641636ae7b9a655 62d89870-31bf-44a4-ac81-b8e3234d590f 36607 0 2020-10-26 00:30:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:53 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-fcw9 bootstrap-e2e-minion-group-fcw9   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-fcw9} 2020-10-26 00:30:53 +0000 UTC 2020-10-26 00:30:53 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fcw9.1641636ae7b9c564  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fcw9.1641636ae7b9c564 b3fa5202-f0fc-46d4-b8e9-e67bf5df4f07 36606 0 2020-10-26 00:30:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:53 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-fcw9 bootstrap-e2e-minion-group-fcw9   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-fcw9} 2020-10-26 00:30:53 +0000 UTC 2020-10-26 00:30:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164155a2924e30ae  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164155a2924e30ae 72b65b63-6f62-4d46-b37d-4b5ce3321a0a 35318 0 2020-10-25 23:11:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-26 00:06:28 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh 4be61eed-0711-4de1-963f-b91b892bc562 v1  } NodeNotReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeNotReady {node-controller } 2020-10-25 20:18:19 +0000 UTC 2020-10-26 00:06:28 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164155a915e34e86  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164155a915e34e86 4528a67d-ba77-496e-b063-3bef5955a698 35319 0 2020-10-25 23:12:00 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:06:30 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-25 20:18:47 +0000 UTC 2020-10-26 00:06:30 +0000 UTC 7 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168089b6a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168089b6a9 830e894a-e6aa-4c85-b76a-504e6d5c3eda 35320 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168dfe3a45  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168dfe3a45 8f3eeac3-a630-4701-906a-df9197af2e43 35324 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168e0004ef  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168e0004ef d25ed207-f6a8-4e01-8be8-51f8d0756a1f 35325 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168e0158d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168e0158d1 35a6cf36-05cf-4cac-b42e-c12695b81b41 35326 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162169432af71  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162169432af71 09a5fbc2-eea8-46b1-a43b-39f24f1447a0 35327 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeNotReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.16416216a9b1ba2f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.16416216a9b1ba2f a7267f2c-f4be-4349-8061-06e872bd362e 35328 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.16416216aeb47670  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.16416216aeb47670 69b7f83c-80ee-442a-afc7-2e55a94c2c08 35329 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeReady {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.1641636bb2d246da  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.1641636bb2d246da 695610df-fd7f-4318-88dd-3e59af6c9ded 36609 0 2020-10-26 00:30:56 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:56 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:30:56 +0000 UTC 2020-10-26 00:30:56 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.1641636bb2d2723f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.1641636bb2d2723f a1d845bb-9fdc-4747-ad9f-705219fbd3b5 36608 0 2020-10-26 00:30:56 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:56 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:30:56 +0000 UTC 2020-10-26 00:30:56 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641551333bfb93d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641551333bfb93d 37d6d45b-342a-445d-a14e-fc399ff47331 35171 0 2020-10-26 00:04:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-26 00:04:03 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 6924ba4b-8015-46e9-ad94-d0c798e9a30c v1  } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {node-controller } 2020-10-25 20:08:03 +0000 UTC 2020-10-26 00:04:02 +0000 UTC 5 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164155a81d2d81e0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164155a81d2d81e0 64324bb7-ab3f-492e-b9c1-e1b043583c2a 35172 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:04:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-xc35} 2020-10-25 20:18:42 +0000 UTC 2020-10-26 00:04:05 +0000 UTC 8 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a53fedf0b2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a53fedf0b2 a16f5356-e816-4ee9-bc00-47ab87dd530f 34986 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d5020ad  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d5020ad 83863cbe-ece1-4b75-bda4-a778779c5197 34987 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d505d7b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d505d7b 0178c76a-187c-4e1e-a336-6908c14f066f 34988 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d508bd5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d508bd5 296f6621-6beb-4f2f-afde-c5c24bf07e8a 34989 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a558ff44c3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a558ff44c3 a94cfbe1-ac1b-4964-ae88-b7c7df17d387 34990 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a55c88d389  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a55c88d389 b207d2a6-43a9-47b4-9c55-e4e77588dee8 34991 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a5617dccbf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a5617dccbf 1c16d21a-2d1c-41ab-958d-4912bd7bf8b0 34992 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c760e11081  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c760e11081 f8ba6e84-677d-4a70-a7ac-94d44c559d48 35113 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d31d8d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d31d8d1 6082cf4b-beff-47b3-82b4-aa789e4d6f95 35114 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d331ffc  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d331ffc 1c0eef8e-aa6f-4f76-9e9c-b463b011ede8 35115 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d344381  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d344381 391ebba1-2813-48fc-8724-f8488597fd11 35116 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c7814bc910  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c7814bc910 3b9c0970-18e7-4fe6-883a-8bf4140f89fe 35117 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c78eceeb70  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c78eceeb70 03ad10cd-a68b-4d70-a7e5-db9963ef548b 35118 0 2020-10-26 00:00:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:52 +0000 UTC 2020-10-26 00:00:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c79864f252  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c79864f252 8eb13472-76e7-4464-a87d-fb884a8374ef 35119 0 2020-10-26 00:00:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:52 +0000 UTC 2020-10-26 00:00:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4bb728d4b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4bb728d4b c266789c-b79b-4852-ab67-9aebcb2346e9 35173 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79ecac7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79ecac7 6103ea08-ed0b-4034-b85e-65328d5c6918 35177 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79f08fe  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79f08fe e0f5be78-6b46-4adb-b394-f52e2c4a697d 35178 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79f3367  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79f3367 a9a7af81-52b9-41ee-9489-1e7bc290aaf5 35179 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4d141334e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4d141334e f98c2213-4ad4-4e0d-b9d9-f351942ceb2d 35180 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4e5c6d901  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4e5c6d901 b2a7d0c1-eea1-446c-a917-34e89715b187 35181 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4edbb59aa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4edbb59aa ceff3e67-c902-4c14-b5b6-6da619e10d04 35182 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641636cdc2b28a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641636cdc2b28a3 8b17b525-0630-44b5-ad30-541599ef1173 36611 0 2020-10-26 00:31:01 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:31:01 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xc35} 2020-10-26 00:31:01 +0000 UTC 2020-10-26 00:31:01 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641636cdc2b47d2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641636cdc2b47d2 2e3a0aba-0922-49e5-b8a7-1fc2a7f03133 36610 0 2020-10-26 00:31:01 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:31:01 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xc35} 2020-10-26 00:31:01 +0000 UTC 2020-10-26 00:31:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-fcw9.1641636ae7b9a655  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fcw9.1641636ae7b9a655 62d89870-31bf-44a4-ac81-b8e3234d590f 36607 0 2020-10-26 00:30:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:53 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-fcw9 bootstrap-e2e-minion-group-fcw9   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-fcw9} 2020-10-26 00:30:53 +0000 UTC 2020-10-26 00:30:53 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fcw9.1641636ae7b9c564  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fcw9.1641636ae7b9c564 b3fa5202-f0fc-46d4-b8e9-e67bf5df4f07 36606 0 2020-10-26 00:30:53 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:53 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-fcw9 bootstrap-e2e-minion-group-fcw9   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-fcw9} 2020-10-26 00:30:53 +0000 UTC 2020-10-26 00:30:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164155a2924e30ae  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164155a2924e30ae 72b65b63-6f62-4d46-b37d-4b5ce3321a0a 35318 0 2020-10-25 23:11:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-26 00:06:28 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh 4be61eed-0711-4de1-963f-b91b892bc562 v1  } NodeNotReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeNotReady {node-controller } 2020-10-25 20:18:19 +0000 UTC 2020-10-26 00:06:28 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164155a915e34e86  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164155a915e34e86 4528a67d-ba77-496e-b063-3bef5955a698 35319 0 2020-10-25 23:12:00 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:06:30 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-25 20:18:47 +0000 UTC 2020-10-26 00:06:30 +0000 UTC 7 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168089b6a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168089b6a9 830e894a-e6aa-4c85-b76a-504e6d5c3eda 35320 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168dfe3a45  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168dfe3a45 8f3eeac3-a630-4701-906a-df9197af2e43 35324 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168e0004ef  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168e0004ef d25ed207-f6a8-4e01-8be8-51f8d0756a1f 35325 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162168e0158d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162168e0158d1 35a6cf36-05cf-4cac-b42e-c12695b81b41 35326 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-vlrh status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.164162169432af71  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.164162169432af71 09a5fbc2-eea8-46b1-a43b-39f24f1447a0 35327 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeNotReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.16416216a9b1ba2f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.16416216a9b1ba2f a7267f2c-f4be-4349-8061-06e872bd362e 35328 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.16416216aeb47670  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.16416216aeb47670 69b7f83c-80ee-442a-afc7-2e55a94c2c08 35329 0 2020-10-26 00:06:31 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:06:31 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } NodeReady Node bootstrap-e2e-minion-group-vlrh status is now: NodeReady {kubelet bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:06:31 +0000 UTC 2020-10-26 00:06:31 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.1641636bb2d246da  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.1641636bb2d246da 695610df-fd7f-4318-88dd-3e59af6c9ded 36609 0 2020-10-26 00:30:56 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:56 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:30:56 +0000 UTC 2020-10-26 00:30:56 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-vlrh.1641636bb2d2723f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-vlrh.1641636bb2d2723f a1d845bb-9fdc-4747-ad9f-705219fbd3b5 36608 0 2020-10-26 00:30:56 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:30:56 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-vlrh bootstrap-e2e-minion-group-vlrh   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-vlrh} 2020-10-26 00:30:56 +0000 UTC 2020-10-26 00:30:56 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641551333bfb93d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641551333bfb93d 37d6d45b-342a-445d-a14e-fc399ff47331 35171 0 2020-10-26 00:04:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-26 00:04:03 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 6924ba4b-8015-46e9-ad94-d0c798e9a30c v1  } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {node-controller } 2020-10-25 20:08:03 +0000 UTC 2020-10-26 00:04:02 +0000 UTC 5 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164155a81d2d81e0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164155a81d2d81e0 64324bb7-ab3f-492e-b9c1-e1b043583c2a 35172 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:04:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-xc35} 2020-10-25 20:18:42 +0000 UTC 2020-10-26 00:04:05 +0000 UTC 8 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a53fedf0b2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a53fedf0b2 a16f5356-e816-4ee9-bc00-47ab87dd530f 34986 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d5020ad  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d5020ad 83863cbe-ece1-4b75-bda4-a778779c5197 34987 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d505d7b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d505d7b 0178c76a-187c-4e1e-a336-6908c14f066f 34988 0 2020-10-25 23:58:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a54d508bd5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a54d508bd5 296f6621-6beb-4f2f-afde-c5c24bf07e8a 34989 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:24 +0000 UTC 2020-10-25 23:58:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a558ff44c3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a558ff44c3 a94cfbe1-ac1b-4964-ae88-b7c7df17d387 34990 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a55c88d389  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a55c88d389 b207d2a6-43a9-47b4-9c55-e4e77588dee8 34991 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161a5617dccbf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161a5617dccbf 1c16d21a-2d1c-41ab-958d-4912bd7bf8b0 34992 0 2020-10-25 23:58:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-25 23:58:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-25 23:58:25 +0000 UTC 2020-10-25 23:58:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c760e11081  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c760e11081 f8ba6e84-677d-4a70-a7ac-94d44c559d48 35113 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d31d8d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d31d8d1 6082cf4b-beff-47b3-82b4-aa789e4d6f95 35114 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d331ffc  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d331ffc 1c0eef8e-aa6f-4f76-9e9c-b463b011ede8 35115 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c76d344381  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c76d344381 391ebba1-2813-48fc-8724-f8488597fd11 35116 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c7814bc910  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c7814bc910 3b9c0970-18e7-4fe6-883a-8bf4140f89fe 35117 0 2020-10-26 00:00:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:51 +0000 UTC 2020-10-26 00:00:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c78eceeb70  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c78eceeb70 03ad10cd-a68b-4d70-a7e5-db9963ef548b 35118 0 2020-10-26 00:00:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:52 +0000 UTC 2020-10-26 00:00:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161c79864f252  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161c79864f252 8eb13472-76e7-4464-a87d-fb884a8374ef 35119 0 2020-10-26 00:00:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:00:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:00:52 +0000 UTC 2020-10-26 00:00:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4bb728d4b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4bb728d4b c266789c-b79b-4852-ab67-9aebcb2346e9 35173 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79ecac7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79ecac7 6103ea08-ed0b-4034-b85e-65328d5c6918 35177 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79f08fe  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79f08fe e0f5be78-6b46-4adb-b394-f52e2c4a697d 35178 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4c79f3367  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4c79f3367 a9a7af81-52b9-41ee-9489-1e7bc290aaf5 35179 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-xc35 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4d141334e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4d141334e f98c2213-4ad4-4e0d-b9d9-f351942ceb2d 35180 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeNotReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4e5c6d901  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4e5c6d901 b2a7d0c1-eea1-446c-a917-34e89715b187 35181 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.164161f4edbb59aa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.164161f4edbb59aa ceff3e67-c902-4c14-b5b6-6da619e10d04 35182 0 2020-10-26 00:04:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-26 00:04:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } NodeReady Node bootstrap-e2e-minion-group-xc35 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-xc35} 2020-10-26 00:04:06 +0000 UTC 2020-10-26 00:04:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641636cdc2b28a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641636cdc2b28a3 8b17b525-0630-44b5-ad30-541599ef1173 36611 0 2020-10-26 00:31:01 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:31:01 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xc35} 2020-10-26 00:31:01 +0000 UTC 2020-10-26 00:31:01 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xc35.1641636cdc2b47d2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xc35.1641636cdc2b47d2 2e3a0aba-0922-49e5-b8a7-1fc2a7f03133 36610 0 2020-10-26 00:31:01 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-26 00:31:01 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-xc35 bootstrap-e2e-minion-group-xc35   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xc35} 2020-10-26 00:31:01 +0000 UTC 2020-10-26 00:31:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:140