This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 57 failed / 868 succeeded
Started2020-10-14 10:33
Elapsed15h12m
Revision
Builder9e09b7e7-0e08-11eb-9c65-0a69ced3f66e
infra-commitd8fda771c
job-versionv1.17.13-rc.0.25+4a28e05c652659
master_os_imagecos-77-12371-175-0
node_os_imagecos-77-12371-175-0
revisionv1.17.13-rc.0.25+4a28e05c652659

Test Failures


Cluster upgrade [sig-cloud-provider-gcp] cluster-upgrade 4m48s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Cluster\supgrade\s\[sig\-cloud\-provider\-gcp\]\scluster\-upgrade$'
Oct 14 10:51:29.134: Unexpected error:
    <*errors.errorString | 0xc00241ba40>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.10-rc.0.43+cc9a451a45036b]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-5msq\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-jlgt\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-kx4t\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.10-rc.0.43+cc9a451a45036b/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m24s   v1.18.10-rc.0.43+cc9a451a45036b\\nbootstrap-e2e-minion-group-5msq   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nbootstrap-e2e-minion-group-jlgt   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nbootstrap-e2e-minion-group-kx4t   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.10-rc.0.43+cc9a451a45036b\\\"\\nname: \\\"bootstrap-e2e-minion-group-5msq\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-jlgt\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-kx4t\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-serial-1-2\\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-kx4t\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.84.11; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.84.11  RUNNING\\nWarning: Permanently added 'compute.9155516370403420490' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-serial-1-2\\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.10-rc.0.43+cc9a451a45036b]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-5msq\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-jlgt\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-kx4t\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.10-rc.0.43+cc9a451a45036b/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m24s   v1.18.10-rc.0.43+cc9a451a45036b\nbootstrap-e2e-minion-group-5msq   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nbootstrap-e2e-minion-group-jlgt   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nbootstrap-e2e-minion-group-kx4t   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.10-rc.0.43+cc9a451a45036b\"\nname: \"bootstrap-e2e-minion-group-5msq\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-jlgt\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-kx4t\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\n", stderr "Project: k8s-jkns-e2e-gce-serial-1-2\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-kx4t\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.84.11; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.84.11  RUNNING\nWarning: Permanently added 'compute.9155516370403420490' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-serial-1-2\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153 +0x132
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc002d111e8)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc000c82b00, 0x799a240, 0xc, 0xc, 0xc00085ce10, 0xc0016125a0, 0x2, 0xc0026c2e00)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:485 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.3.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:158 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc00047a700)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:125 +0x324
k8s.io/kubernetes/test/e2e.TestE2E(0xc00047a700)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:111 +0x2b
testing.tRunner(0xc00047a700, 0x4af5d18)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] cluster upgrade should maintain a functioning cluster [Feature:ClusterUpgrade] 17m53s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\scluster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:ClusterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:142
Oct 14 10:51:29.134: Unexpected error:
    <*errors.errorString | 0xc00241ba40>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.10-rc.0.43+cc9a451a45036b]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-5msq\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-jlgt\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-kx4t\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.10-rc.0.43+cc9a451a45036b/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Applying the latest default CoreDNS configuration ==\\nserviceaccount/coredns unchanged\\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\\nconfigmap/coredns configured\\ndeployment.apps/coredns unchanged\\nservice/kube-dns unchanged\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m24s   v1.18.10-rc.0.43+cc9a451a45036b\\nbootstrap-e2e-minion-group-5msq   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nbootstrap-e2e-minion-group-jlgt   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nbootstrap-e2e-minion-group-kx4t   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.18.10-rc.0.43+cc9a451a45036b\\\"\\nname: \\\"bootstrap-e2e-minion-group-5msq\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-jlgt\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\nname: \\\"bootstrap-e2e-minion-group-kx4t\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.13-rc.0.25+4a28e05c652659\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-serial-1-2\\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-kx4t\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.84.11; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.84.11  RUNNING\\nWarning: Permanently added 'compute.9155516370403420490' (ED25519) to the list of known hosts.\\r\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-serial-1-2\\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.18.10-rc.0.43+cc9a451a45036b]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-5msq\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-jlgt\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-kx4t\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.18.10-rc.0.43+cc9a451a45036b/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Applying the latest default CoreDNS configuration ==\nserviceaccount/coredns unchanged\nclusterrole.rbac.authorization.k8s.io/system:coredns unchanged\nclusterrolebinding.rbac.authorization.k8s.io/system:coredns configured\nconfigmap/coredns configured\ndeployment.apps/coredns unchanged\nservice/kube-dns unchanged\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m24s   v1.18.10-rc.0.43+cc9a451a45036b\nbootstrap-e2e-minion-group-5msq   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nbootstrap-e2e-minion-group-jlgt   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nbootstrap-e2e-minion-group-kx4t   Ready                      <none>   9m26s   v1.17.13-rc.0.25+4a28e05c652659\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.18.10-rc.0.43+cc9a451a45036b\"\nname: \"bootstrap-e2e-minion-group-5msq\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-jlgt\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\nname: \"bootstrap-e2e-minion-group-kx4t\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.13-rc.0.25+4a28e05c652659\"\n", stderr "Project: k8s-jkns-e2e-gce-serial-1-2\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-kx4t\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.84.11; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-serial-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.84.11  RUNNING\nWarning: Permanently added 'compute.9155516370403420490' (ED25519) to the list of known hosts.\r\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-serial-1-2\nNetwork Project: k8s-jkns-e2e-gce-serial-1-2\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:153
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m9s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:57
Oct 14 12:26:50.705: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc002daf080>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66e9ddee  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66e9ddee 37a401c2-b615-4d28-9b7a-94518211c42f 1909 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-5msq status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:34:26 +0000 UTC 14 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66eb4940  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66eb4940 cbb1c12f-0e64-4683-a022-1ce6998e166c 1900 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-5msq status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:29:54 +0000 UTC 10 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66ec78d7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66ec78d7 001b8252-4be4-4b7a-9a18-093b97096559 1907 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-5msq status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:30:37 +0000 UTC 11 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd8600b48a1a4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd8600b48a1a4 153ad5ab-ba7d-4b46-9918-22fc34879283 1872 0 2020-10-14 11:29:09 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-14 11:29:09 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq 0176f60b-5bf3-4ab8-ac14-2b84165ac732 v1  } RemovingNode Node bootstrap-e2e-minion-group-5msq event: Removing Node bootstrap-e2e-minion-group-5msq from Controller {node-controller } 2020-10-14 11:29:09 +0000 UTC 2020-10-14 11:29:09 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd8b75d17da5f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd8b75d17da5f cff1fde9-ff39-4921-817e-8b31258fa93e 1927 0 2020-10-14 11:35:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-14 11:35:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq 4bfc0d9d-09a5-4343-9fd6-df0b8e3120c4 v1  } RegisteredNode Node bootstrap-e2e-minion-group-5msq event: Registered Node bootstrap-e2e-minion-group-5msq in Controller {node-controller } 2020-10-14 11:35:24 +0000 UTC 2020-10-14 11:35:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163ddb76cfecb2f6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163ddb76cfecb2f6 b5b0d2ba-1c40-4fc7-9cd4-ac687af7f1bf 3183 0 2020-10-14 12:25:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-5msq} 2020-10-14 12:25:45 +0000 UTC 2020-10-14 12:25:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163ddb76cfeccf28  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163ddb76cfeccf28 fd53cd12-4008-4136-b343-780e281651bf 3182 0 2020-10-14 12:25:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-5msq} 2020-10-14 12:25:45 +0000 UTC 2020-10-14 12:25:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-jlgt.163ddb7760aaa521  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-jlgt.163ddb7760aaa521 a90fcda1-d9f3-4af3-b193-1a9d8aacf191 3185 0 2020-10-14 12:25:48 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-jlgt   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-jlgt} 2020-10-14 12:25:48 +0000 UTC 2020-10-14 12:25:48 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-jlgt.163ddb7760aac907  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-jlgt.163ddb7760aac907 819f5ca4-01d4-412b-9a99-9607d1ac14c3 3184 0 2020-10-14 12:25:48 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-jlgt   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-jlgt} 2020-10-14 12:25:48 +0000 UTC 2020-10-14 12:25:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163dd5cd9d4ab2cd  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163dd5cd9d4ab2cd 71c23eab-197f-4a6b-92f2-3d58e362e92c 2940 0 2020-10-14 12:21:43 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:21:43 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 10:42:01 +0000 UTC 2020-10-14 12:21:43 +0000 UTC 4 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ea0e888ce  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ea0e888ce 1eed0753-f927-47d0-bcdb-696cb86541c6 2941 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc21f7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc21f7 7a01d83f-76eb-4f52-b9cc-9fbe62f85beb 2942 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc5844  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc5844 82e268c9-ed55-4dda-8f73-71eb72cdf992 2943 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc8562  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc8562 7fbc584f-53f1-4bfe-a97c-a03e789138ad 2944 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ebba336c4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ebba336c4 b2ed2a68-bb41-4217-8786-102c8f30040a 2945 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeNotReady Node bootstrap-e2e-minion-group-kx4t status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ecb29a17a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ecb29a17a b38c95f5-49fd-4894-a016-b7f5d3c91565 2946 0 2020-10-14 12:21:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:45 +0000 UTC 2020-10-14 12:21:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ed44b2b44  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ed44b2b44 ab74483e-4794-4e5f-9c24-8bbc8b25a3cc 2947 0 2020-10-14 12:21:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeReady Node bootstrap-e2e-minion-group-kx4t status is now: NodeReady {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:45 +0000 UTC 2020-10-14 12:21:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb77f036e674  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb77f036e674 ebe77d0f-c918-467f-8873-f58873bd4d9f 3187 0 2020-10-14 12:25:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:25:50 +0000 UTC 2020-10-14 12:25:50 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb77f0370757  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb77f0370757 9e21e461-3f7a-4be7-88cd-dddf26fe9286 3186 0 2020-10-14 12:25:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:25:50 +0000 UTC 2020-10-14 12:25:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66e9ddee  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66e9ddee 37a401c2-b615-4d28-9b7a-94518211c42f 1909 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-5msq status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:34:26 +0000 UTC 14 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66eb4940  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66eb4940 cbb1c12f-0e64-4683-a022-1ce6998e166c 1900 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-5msq status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:29:54 +0000 UTC 10 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd5cd66ec78d7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd5cd66ec78d7 001b8252-4be4-4b7a-9a18-093b97096559 1907 0 2020-10-14 10:42:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-5msq status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-5msq} 2020-10-14 10:42:00 +0000 UTC 2020-10-14 11:30:37 +0000 UTC 11 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd8600b48a1a4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd8600b48a1a4 153ad5ab-ba7d-4b46-9918-22fc34879283 1872 0 2020-10-14 11:29:09 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-14 11:29:09 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq 0176f60b-5bf3-4ab8-ac14-2b84165ac732 v1  } RemovingNode Node bootstrap-e2e-minion-group-5msq event: Removing Node bootstrap-e2e-minion-group-5msq from Controller {node-controller } 2020-10-14 11:29:09 +0000 UTC 2020-10-14 11:29:09 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163dd8b75d17da5f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163dd8b75d17da5f cff1fde9-ff39-4921-817e-8b31258fa93e 1927 0 2020-10-14 11:35:24 +0000 UTC <nil> <nil> map[] map[] [] []  [{kube-controller-manager Update v1 2020-10-14 11:35:24 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq 4bfc0d9d-09a5-4343-9fd6-df0b8e3120c4 v1  } RegisteredNode Node bootstrap-e2e-minion-group-5msq event: Registered Node bootstrap-e2e-minion-group-5msq in Controller {node-controller } 2020-10-14 11:35:24 +0000 UTC 2020-10-14 11:35:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163ddb76cfecb2f6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163ddb76cfecb2f6 b5b0d2ba-1c40-4fc7-9cd4-ac687af7f1bf 3183 0 2020-10-14 12:25:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-5msq} 2020-10-14 12:25:45 +0000 UTC 2020-10-14 12:25:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-5msq.163ddb76cfeccf28  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-5msq.163ddb76cfeccf28 fd53cd12-4008-4136-b343-780e281651bf 3182 0 2020-10-14 12:25:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-5msq bootstrap-e2e-minion-group-5msq   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-5msq} 2020-10-14 12:25:45 +0000 UTC 2020-10-14 12:25:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-jlgt.163ddb7760aaa521  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-jlgt.163ddb7760aaa521 a90fcda1-d9f3-4af3-b193-1a9d8aacf191 3185 0 2020-10-14 12:25:48 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-jlgt   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-jlgt} 2020-10-14 12:25:48 +0000 UTC 2020-10-14 12:25:48 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-jlgt.163ddb7760aac907  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-jlgt.163ddb7760aac907 819f5ca4-01d4-412b-9a99-9607d1ac14c3 3184 0 2020-10-14 12:25:48 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:48 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-jlgt bootstrap-e2e-minion-group-jlgt   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-jlgt} 2020-10-14 12:25:48 +0000 UTC 2020-10-14 12:25:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163dd5cd9d4ab2cd  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163dd5cd9d4ab2cd 71c23eab-197f-4a6b-92f2-3d58e362e92c 2940 0 2020-10-14 12:21:43 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:21:43 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 10:42:01 +0000 UTC 2020-10-14 12:21:43 +0000 UTC 4 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ea0e888ce  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ea0e888ce 1eed0753-f927-47d0-bcdb-696cb86541c6 2941 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc21f7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc21f7 7a01d83f-76eb-4f52-b9cc-9fbe62f85beb 2942 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc5844  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc5844 82e268c9-ed55-4dda-8f73-71eb72cdf992 2943 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3eabcc8562  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3eabcc8562 7fbc584f-53f1-4bfe-a97c-a03e789138ad 2944 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-kx4t status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ebba336c4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ebba336c4 b2ed2a68-bb41-4217-8786-102c8f30040a 2945 0 2020-10-14 12:21:44 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:44 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeNotReady Node bootstrap-e2e-minion-group-kx4t status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:44 +0000 UTC 2020-10-14 12:21:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ecb29a17a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ecb29a17a b38c95f5-49fd-4894-a016-b7f5d3c91565 2946 0 2020-10-14 12:21:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:45 +0000 UTC 2020-10-14 12:21:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb3ed44b2b44  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb3ed44b2b44 ab74483e-4794-4e5f-9c24-8bbc8b25a3cc 2947 0 2020-10-14 12:21:45 +0000 UTC <nil> <nil> map[] map[] [] []  [{kubelet Update v1 2020-10-14 12:21:45 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } NodeReady Node bootstrap-e2e-minion-group-kx4t status is now: NodeReady {kubelet bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:21:45 +0000 UTC 2020-10-14 12:21:45 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb77f036e674  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb77f036e674 ebe77d0f-c918-467f-8873-f58873bd4d9f 3187 0 2020-10-14 12:25:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:25:50 +0000 UTC 2020-10-14 12:25:50 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-kx4t.163ddb77f0370757  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-kx4t.163ddb77f0370757 9e21e461-3f7a-4be7-88cd-dddf26fe9286 3186 0 2020-10-14 12:25:50 +0000 UTC <nil> <nil> map[] map[] [] []  [{node-problem-detector Update v1 2020-10-14 12:25:50 +0000 UTC FieldsV1 &FieldsV1{Raw:*[123 34 102 58 99 111 117 110 116 34 58 123 125 44 34 102 58 102 105 114 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 105 110 118 111 108 118 101 100 79 98 106 101 99 116 34 58 123 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 44 34 102 58 108 97 115 116 84 105 109 101 115 116 97 109 112 34 58 123 125 44 34 102 58 109 101 115 115 97 103 101 34 58 123 125 44 34 102 58 114 101 97 115 111 110 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 102 58 99 111 109 112 111 110 101 110 116 34 58 123 125 44 34 102 58 104 111 115 116 34 58 123 125 125 44 34 102 58 116 121 112 101 34 58 123 125 125],}}]} {Node  bootstrap-e2e-minion-group-kx4t bootstrap-e2e-minion-group-kx4t   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-kx4t} 2020-10-14 12:25:50 +0000 UTC 2020-10-14 12:25:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:140
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] kubelet [k8s.io] [sig-node] Clean up pods on node kubelet should be able to delete 10 pods per node in 1m0s. 5m22s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\skubelet\s\[k8s\.io\]\s\[sig\-node\]\sClean\sup\spods\son\snode\skubelet\sshould\sbe\sable\sto\sdelete\s10\spods\sper\snode\sin\s1m0s\.$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/kubelet.go:338
Oct 14 23:27:44.068: Unexpected error:
    <*errors.errorString | 0xc0002913f0>: {
        s: "Only 25 pods started out of 30",
    }
    Only 25 pods started out of 30
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/kubelet.go:351