This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 21 failed / 1183 succeeded
Started2020-10-26 18:27
Elapsed14h43m
Revision
Buildere436ab1c-17b8-11eb-b256-6ee25ea2e440
infra-commit10707fb6e
job-versionv1.17.14-rc.0.15+54f3c097502f5b
master_os_imagecos-77-12371-175-0
node_os_imagecos-77-12371-175-0
revisionv1.17.14-rc.0.15+54f3c097502f5b

Test Failures


Cluster upgrade [sig-cloud-provider-gcp] cluster-upgrade 4m25s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Cluster\supgrade\s\[sig\-cloud\-provider\-gcp\]\scluster\-upgrade$'
Oct 26 18:42:22.101: Unexpected error:
    <*errors.errorString | 0xc00175dcb0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-3x8h\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-v0pd\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-wf2j\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE    VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m     v1.18.11-rc.0.13+806617c8cf1f0d\\nbootstrap-e2e-minion-group-3x8h   Ready                      <none>   9m     v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-v0pd   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-wf2j   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\ncontroller-manager   Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.11-rc.0.13+806617c8cf1f0d\\\"\\nname: \\\"bootstrap-e2e-minion-group-3x8h\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-v0pd\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-wf2j\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\n\", stderr \"Project: gce-gci-upg-1-3-lat-ctl-skew\\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-wf2j\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.197.48.170; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.197.48.170  RUNNING\\nWarning: Permanently added 'compute.5606181746614380268' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: gce-gci-upg-1-3-lat-ctl-skew\\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-3x8h\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-v0pd\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-wf2j\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE    VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m     v1.18.11-rc.0.13+806617c8cf1f0d\nbootstrap-e2e-minion-group-3x8h   Ready                      <none>   9m     v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-v0pd   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-wf2j   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \ncontroller-manager   Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.11-rc.0.13+806617c8cf1f0d\"\nname: \"bootstrap-e2e-minion-group-3x8h\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-v0pd\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-wf2j\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\n", stderr "Project: gce-gci-upg-1-3-lat-ctl-skew\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-wf2j\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.197.48.170; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.197.48.170  RUNNING\nWarning: Permanently added 'compute.5606181746614380268' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: gce-gci-upg-1-3-lat-ctl-skew\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153 +0x132
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc002fcd1e8)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc000d1e420, 0x799a240, 0xc, 0xc, 0xc00089db90, 0xc000a7a870, 0x2, 0xc002897fc0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:485 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:158 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001ee500)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:125 +0x324
k8s.io/kubernetes/test/e2e.TestE2E(0xc0001ee500)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:111 +0x2b
testing.tRunner(0xc0001ee500, 0x4af5d58)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] cluster upgrade should maintain a functioning cluster [Feature:ClusterUpgrade] 16m56s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\scluster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:ClusterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:142
Oct 26 18:42:22.101: Unexpected error:
    <*errors.errorString | 0xc00175dcb0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-3x8h\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-v0pd\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-wf2j\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE    VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m     v1.18.11-rc.0.13+806617c8cf1f0d\\nbootstrap-e2e-minion-group-3x8h   Ready                      <none>   9m     v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-v0pd   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\\nbootstrap-e2e-minion-group-wf2j   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\ncontroller-manager   Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.11-rc.0.13+806617c8cf1f0d\\\"\\nname: \\\"bootstrap-e2e-minion-group-3x8h\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-v0pd\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\nname: \\\"bootstrap-e2e-minion-group-wf2j\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.14-rc.0.15+54f3c097502f5b\\\"\\n\", stderr \"Project: gce-gci-upg-1-3-lat-ctl-skew\\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-wf2j\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.197.48.170; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.197.48.170  RUNNING\\nWarning: Permanently added 'compute.5606181746614380268' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: gce-gci-upg-1-3-lat-ctl-skew\\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.11-rc.0.13+806617c8cf1f0d]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-3x8h\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-v0pd\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-wf2j\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.11-rc.0.13+806617c8cf1f0d/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE    VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m     v1.18.11-rc.0.13+806617c8cf1f0d\nbootstrap-e2e-minion-group-3x8h   Ready                      <none>   9m     v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-v0pd   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\nbootstrap-e2e-minion-group-wf2j   Ready                      <none>   9m1s   v1.17.14-rc.0.15+54f3c097502f5b\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \ncontroller-manager   Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.11-rc.0.13+806617c8cf1f0d\"\nname: \"bootstrap-e2e-minion-group-3x8h\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-v0pd\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\nname: \"bootstrap-e2e-minion-group-wf2j\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.14-rc.0.15+54f3c097502f5b\"\n", stderr "Project: gce-gci-upg-1-3-lat-ctl-skew\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-wf2j\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.197.48.170; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/gce-gci-upg-1-3-lat-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.197.48.170  RUNNING\nWarning: Permanently added 'compute.5606181746614380268' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: gce-gci-upg-1-3-lat-ctl-skew\nNetwork Project: gce-gci-upg-1-3-lat-ctl-skew\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m10s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:57
Oct 27 06:07:25.901: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc0064e3e70>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-master.1641c172001165e9  default /api/v1/namespaces/default/events/bootstrap-e2e-master.1641c172001165e9 4290b615-70d1-4ff9-afbe-aaf93eb72dd1 34937 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-master 9725e6c4-d2c2-4704-aa88-32fc1de1a0cf v1  } RegisteredNode Node bootstrap-e2e-master event: Registered Node bootstrap-e2e-master in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641b21ae18f1ef5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641b21ae18f1ef5 26bfa8ff-2f2b-464b-af28-2bb8e2c1a674 36794 0 2020-10-27 00:32:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 05:41:04 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 00:32:50 +0000 UTC 2020-10-27 05:41:04 +0000 UTC 13 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c1720013461b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c1720013461b 3c584be0-6a50-47b6-a0ed-191955a4178f 34938 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bb3c22ec-ffb9-4a8c-a70a-b4eb194eaabe v1  } RegisteredNode Node bootstrap-e2e-minion-group-3x8h event: Registered Node bootstrap-e2e-minion-group-3x8h in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ec76312440  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ec76312440 6a25401c-2da7-4025-8b07-71b71fffd4de 36793 0 2020-10-27 05:41:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:41:03 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bb3c22ec-ffb9-4a8c-a70a-b4eb194eaabe v1  } NodeNotReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeNotReady {node-controller } 2020-10-27 05:41:03 +0000 UTC 2020-10-27 05:41:03 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ecf9e9ca7b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ecf9e9ca7b 810601b5-3671-476a-b3db-af22e1c26376 36795 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05cc7d9c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05cc7d9c dc675ee8-ded7-447c-b683-eaec4fd702de 36800 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05ccc0a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05ccc0a3 4a505ed1-6eca-43e3-b9df-9bfd94f215a9 36799 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05cd68d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05cd68d1 61d718d8-6e31-4832-a0e9-6cfaf8d43959 36801 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed12970354  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed12970354 9c8d929c-529f-49a9-b82e-921b86f596a2 36802 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeNotReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed17facbcb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed17facbcb 2823aae2-912e-4850-8024-51181e4983f8 36803 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed1f6ecec2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed1f6ecec2 9f2ec5ad-a615-4f09-b426-cb56ea2bc8ba 36804 0 2020-10-27 05:41:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeReady {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:06 +0000 UTC 2020-10-27 05:41:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c44d84d9182c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c44d84d9182c f42a0107-f984-4e5e-8559-15e95da2e1a3 37958 0 2020-10-27 06:06:19 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:19 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 06:06:19 +0000 UTC 2020-10-27 06:06:19 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c44d84d94441  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c44d84d94441 21ff4906-ac16-40b6-8fd2-6bd809ae426b 37957 0 2020-10-27 06:06:19 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:19 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 06:06:19 +0000 UTC 2020-10-27 06:06:19 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641b21b56f35e1b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641b21b56f35e1b 7c73581f-2b2f-4204-ac50-41a4cce9d9a9 37876 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:04:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 00:32:52 +0000 UTC 2020-10-27 06:04:50 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c172001380af  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c172001380af 3c506db0-bfdd-4d23-9ec4-b5bf91c0a664 34939 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd ae65edb8-7605-47b5-ab34-040571ca0d1d v1  } RegisteredNode Node bootstrap-e2e-minion-group-v0pd event: Registered Node bootstrap-e2e-minion-group-v0pd in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24e0dd12960  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24e0dd12960 1a3654d4-fe77-41b1-b6f5-29efed6a9d4e 37875 0 2020-10-27 05:29:42 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 06:04:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd ae65edb8-7605-47b5-ab34-040571ca0d1d v1  } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {node-controller } 2020-10-27 05:29:42 +0000 UTC 2020-10-27 06:04:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24ef3264e3e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24ef3264e3e 66fd7b26-dab9-46c0-a7e8-6c7bf442998d 35767 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:46 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0de5a8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0de5a8 358664c4-5438-41ca-b8e0-80bf1a853124 35771 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0e150d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0e150d b3be79f1-c247-44cd-a50c-0ed074caa0a5 35772 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0e5f4a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0e5f4a d0b48570-b1d9-4ddc-83d3-fd9fa8f65456 35773 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f097807a7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f097807a7 fcf863e0-2db5-41ef-bb37-234e3f087632 35774 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f157edf91  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f157edf91 2f458676-5286-41f5-bdc1-a84f9aaa15ec 35775 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f1d7b9fb5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f1d7b9fb5 c50627ec-15d4-4c45-8914-d9273e006e56 35776 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438e6b9d148  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438e6b9d148 3fee84c3-e629-460c-a826-d93ae3df5ff6 37877 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:50 +0000 UTC 2020-10-27 06:04:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa19422a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa19422a 47f79ba7-13f9-47d5-a4e2-28ca24dfd9d6 37882 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa1a94b3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa1a94b3 87018e3d-658c-4e83-8b10-664be3dcea91 37883 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa1bc78c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa1bc78c 2bfae24e-dc6d-48ed-87e2-e3d5e9001a48 37884 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c4390f331fab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c4390f331fab bbb1f96a-a695-4bfb-a673-686181390764 37881 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c43928781292  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c43928781292 02153154-0c31-4b3d-be43-409f8b471224 37885 0 2020-10-27 06:04:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:52 +0000 UTC 2020-10-27 06:04:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c43930a1a55d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c43930a1a55d bba2849a-b64c-4587-a067-082028e7b559 37886 0 2020-10-27 06:04:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:52 +0000 UTC 2020-10-27 06:04:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c44e3b3e5c65  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c44e3b3e5c65 0aa4de06-822c-44c2-b2d3-78f0aebe1474 37960 0 2020-10-27 06:06:22 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:22 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:06:22 +0000 UTC 2020-10-27 06:06:22 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c44e3b3e7dab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c44e3b3e7dab 8e3e8965-96bc-4cf9-9d32-83e8d0ec7c7a 37959 0 2020-10-27 06:06:22 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:22 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:06:22 +0000 UTC 2020-10-27 06:06:22 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c1720013b4a2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c1720013b4a2 92082b10-d7c7-4901-a3e5-f058c99a9da2 34936 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j 36eb8255-c294-49b3-b918-9a1e469ab98e v1  } RegisteredNode Node bootstrap-e2e-minion-group-wf2j event: Registered Node bootstrap-e2e-minion-group-wf2j in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c44eea7395ec  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c44eea7395ec 7c78104a-ae13-4fc8-9c20-0ab333d36098 37962 0 2020-10-27 06:06:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j bootstrap-e2e-minion-group-wf2j   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-wf2j} 2020-10-27 06:06:25 +0000 UTC 2020-10-27 06:06:25 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c44eea73bbb4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c44eea73bbb4 7dcaa262-c8fa-4d4c-b564-00539193661e 37961 0 2020-10-27 06:06:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j bootstrap-e2e-minion-group-wf2j   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-wf2j} 2020-10-27 06:06:25 +0000 UTC 2020-10-27 06:06:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-master.1641c172001165e9  default /api/v1/namespaces/default/events/bootstrap-e2e-master.1641c172001165e9 4290b615-70d1-4ff9-afbe-aaf93eb72dd1 34937 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-master 9725e6c4-d2c2-4704-aa88-32fc1de1a0cf v1  } RegisteredNode Node bootstrap-e2e-master event: Registered Node bootstrap-e2e-master in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641b21ae18f1ef5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641b21ae18f1ef5 26bfa8ff-2f2b-464b-af28-2bb8e2c1a674 36794 0 2020-10-27 00:32:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 05:41:04 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 00:32:50 +0000 UTC 2020-10-27 05:41:04 +0000 UTC 13 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c1720013461b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c1720013461b 3c584be0-6a50-47b6-a0ed-191955a4178f 34938 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bb3c22ec-ffb9-4a8c-a70a-b4eb194eaabe v1  } RegisteredNode Node bootstrap-e2e-minion-group-3x8h event: Registered Node bootstrap-e2e-minion-group-3x8h in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ec76312440  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ec76312440 6a25401c-2da7-4025-8b07-71b71fffd4de 36793 0 2020-10-27 05:41:03 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:41:03 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bb3c22ec-ffb9-4a8c-a70a-b4eb194eaabe v1  } NodeNotReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeNotReady {node-controller } 2020-10-27 05:41:03 +0000 UTC 2020-10-27 05:41:03 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ecf9e9ca7b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ecf9e9ca7b 810601b5-3671-476a-b3db-af22e1c26376 36795 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05cc7d9c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05cc7d9c dc675ee8-ded7-447c-b683-eaec4fd702de 36800 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05ccc0a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05ccc0a3 4a505ed1-6eca-43e3-b9df-9bfd94f215a9 36799 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed05cd68d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed05cd68d1 61d718d8-6e31-4832-a0e9-6cfaf8d43959 36801 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-3x8h status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed12970354  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed12970354 9c8d929c-529f-49a9-b82e-921b86f596a2 36802 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeNotReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed17facbcb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed17facbcb 2823aae2-912e-4850-8024-51181e4983f8 36803 0 2020-10-27 05:41:05 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:05 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:05 +0000 UTC 2020-10-27 05:41:05 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c2ed1f6ecec2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c2ed1f6ecec2 9f2ec5ad-a615-4f09-b426-cb56ea2bc8ba 36804 0 2020-10-27 05:41:06 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:41:06 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } NodeReady Node bootstrap-e2e-minion-group-3x8h status is now: NodeReady {kubelet bootstrap-e2e-minion-group-3x8h} 2020-10-27 05:41:06 +0000 UTC 2020-10-27 05:41:06 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c44d84d9182c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c44d84d9182c f42a0107-f984-4e5e-8559-15e95da2e1a3 37958 0 2020-10-27 06:06:19 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:19 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 06:06:19 +0000 UTC 2020-10-27 06:06:19 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-3x8h.1641c44d84d94441  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-3x8h.1641c44d84d94441 21ff4906-ac16-40b6-8fd2-6bd809ae426b 37957 0 2020-10-27 06:06:19 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:19 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-3x8h bootstrap-e2e-minion-group-3x8h   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-3x8h} 2020-10-27 06:06:19 +0000 UTC 2020-10-27 06:06:19 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641b21b56f35e1b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641b21b56f35e1b 7c73581f-2b2f-4204-ac50-41a4cce9d9a9 37876 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:04:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 00:32:52 +0000 UTC 2020-10-27 06:04:50 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c172001380af  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c172001380af 3c506db0-bfdd-4d23-9ec4-b5bf91c0a664 34939 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd ae65edb8-7605-47b5-ab34-040571ca0d1d v1  } RegisteredNode Node bootstrap-e2e-minion-group-v0pd event: Registered Node bootstrap-e2e-minion-group-v0pd in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24e0dd12960  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24e0dd12960 1a3654d4-fe77-41b1-b6f5-29efed6a9d4e 37875 0 2020-10-27 05:29:42 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 06:04:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd ae65edb8-7605-47b5-ab34-040571ca0d1d v1  } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {node-controller } 2020-10-27 05:29:42 +0000 UTC 2020-10-27 06:04:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24ef3264e3e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24ef3264e3e 66fd7b26-dab9-46c0-a7e8-6c7bf442998d 35767 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:46 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0de5a8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0de5a8 358664c4-5438-41ca-b8e0-80bf1a853124 35771 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0e150d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0e150d b3be79f1-c247-44cd-a50c-0ed074caa0a5 35772 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24eff0e5f4a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24eff0e5f4a d0b48570-b1d9-4ddc-83d3-fd9fa8f65456 35773 0 2020-10-27 05:29:46 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:46 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f097807a7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f097807a7 fcf863e0-2db5-41ef-bb37-234e3f087632 35774 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f157edf91  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f157edf91 2f458676-5286-41f5-bdc1-a84f9aaa15ec 35775 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c24f1d7b9fb5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c24f1d7b9fb5 c50627ec-15d4-4c45-8914-d9273e006e56 35776 0 2020-10-27 05:29:47 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 05:29:47 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 05:29:47 +0000 UTC 2020-10-27 05:29:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438e6b9d148  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438e6b9d148 3fee84c3-e629-460c-a826-d93ae3df5ff6 37877 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:50 +0000 UTC 2020-10-27 06:04:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa19422a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa19422a 47f79ba7-13f9-47d5-a4e2-28ca24dfd9d6 37882 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa1a94b3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa1a94b3 87018e3d-658c-4e83-8b10-664be3dcea91 37883 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c438fa1bc78c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c438fa1bc78c 2bfae24e-dc6d-48ed-87e2-e3d5e9001a48 37884 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-v0pd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c4390f331fab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c4390f331fab bbb1f96a-a695-4bfb-a673-686181390764 37881 0 2020-10-27 06:04:51 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:51 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeNotReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:51 +0000 UTC 2020-10-27 06:04:51 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c43928781292  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c43928781292 02153154-0c31-4b3d-be43-409f8b471224 37885 0 2020-10-27 06:04:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:52 +0000 UTC 2020-10-27 06:04:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c43930a1a55d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c43930a1a55d bba2849a-b64c-4587-a067-082028e7b559 37886 0 2020-10-27 06:04:52 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-27 06:04:52 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } NodeReady Node bootstrap-e2e-minion-group-v0pd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:04:52 +0000 UTC 2020-10-27 06:04:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c44e3b3e5c65  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c44e3b3e5c65 0aa4de06-822c-44c2-b2d3-78f0aebe1474 37960 0 2020-10-27 06:06:22 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:22 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:06:22 +0000 UTC 2020-10-27 06:06:22 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-v0pd.1641c44e3b3e7dab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-v0pd.1641c44e3b3e7dab 8e3e8965-96bc-4cf9-9d32-83e8d0ec7c7a 37959 0 2020-10-27 06:06:22 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:22 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-v0pd bootstrap-e2e-minion-group-v0pd   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-v0pd} 2020-10-27 06:06:22 +0000 UTC 2020-10-27 06:06:22 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c1720013b4a2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c1720013b4a2 92082b10-d7c7-4901-a3e5-f058c99a9da2 34936 0 2020-10-27 05:13:57 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-27 05:13:57 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j 36eb8255-c294-49b3-b918-9a1e469ab98e v1  } RegisteredNode Node bootstrap-e2e-minion-group-wf2j event: Registered Node bootstrap-e2e-minion-group-wf2j in Controller {node-controller } 2020-10-27 05:13:57 +0000 UTC 2020-10-27 05:13:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c44eea7395ec  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c44eea7395ec 7c78104a-ae13-4fc8-9c20-0ab333d36098 37962 0 2020-10-27 06:06:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j bootstrap-e2e-minion-group-wf2j   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-wf2j} 2020-10-27 06:06:25 +0000 UTC 2020-10-27 06:06:25 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-wf2j.1641c44eea73bbb4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-wf2j.1641c44eea73bbb4 7dcaa262-c8fa-4d4c-b564-00539193661e 37961 0 2020-10-27 06:06:25 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-27 06:06:25 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-wf2j bootstrap-e2e-minion-group-wf2j   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-wf2j} 2020-10-27 06:06:25 +0000 UTC 2020-10-27 06:06:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:140