This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 17 failed / 819 succeeded
Started2020-03-30 00:14
Elapsed15h11m
Revision
Builder4c420607-721b-11ea-a0b0-0e43f1618cf3
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/f769ad4e-4fa1-490a-9e62-e39c9d60cdd5/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/f769ad4e-4fa1-490a-9e62-e39c9d60cdd5/targets/test
infra-commitfea5af139
job-versionv1.16.9-beta.0.17+bb616cba1cc983
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.17+bb616cba1cc983

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 17m14s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 30 00:29:02.864: Unexpected error:
    <*errors.errorString | 0xc0028f6630>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-9t0w\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-bqz5\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-m3lc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m40s   v1.17.5-beta.0.22+ea3eeb494a528e\\nbootstrap-e2e-minion-group-9t0w   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-bqz5   Ready                      <none>   9m43s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-m3lc   Ready                      <none>   9m43s   v1.16.9-beta.0.17+bb616cba1cc983\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\ncontroller-manager   Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.22+ea3eeb494a528e\\\"\\nname: \\\"bootstrap-e2e-minion-group-9t0w\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-bqz5\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-m3lc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\n\", stderr \"Project: k8s-jnks-gci-autoscaling\\nNetwork Project: k8s-jnks-gci-autoscaling\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-9t0w bootstrap-e2e-minion-group-bqz5 bootstrap-e2e-minion-group-m3lc\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 34.82.113.58; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jnks-gci-autoscaling/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jnks-gci-autoscaling/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.113.58  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jnks-gci-autoscaling\\nNetwork Project: k8s-jnks-gci-autoscaling\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-9t0w\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-bqz5\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-m3lc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m40s   v1.17.5-beta.0.22+ea3eeb494a528e\nbootstrap-e2e-minion-group-9t0w   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-bqz5   Ready                      <none>   9m43s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-m3lc   Ready                      <none>   9m43s   v1.16.9-beta.0.17+bb616cba1cc983\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\ncontroller-manager   Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.22+ea3eeb494a528e\"\nname: \"bootstrap-e2e-minion-group-9t0w\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-bqz5\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-m3lc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\n", stderr "Project: k8s-jnks-gci-autoscaling\nNetwork Project: k8s-jnks-gci-autoscaling\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-9t0w bootstrap-e2e-minion-group-bqz5 bootstrap-e2e-minion-group-m3lc\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 34.82.113.58; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jnks-gci-autoscaling/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jnks-gci-autoscaling/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.113.58  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jnks-gci-autoscaling\nNetwork Project: k8s-jnks-gci-autoscaling\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m18s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 30 06:37:59.084: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc003c79700>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-64hw.1600f1f4489f64fc  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600f1f4489f64fc 11fba924-a113-4f9f-9e00-a35dd10d26dd 14017 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 01:51:40 +0000 UTC 2020-03-30 06:32:24 +0000 UTC 10 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600fa3208041c28  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600fa3208041c28 1f537f3d-8cf7-4380-ace7-89fa6de1f4ad 14016 0 2020-03-30 05:54:37 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw 13116c7b-57ba-48a7-97cd-175f971897e8   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {node-controller } 2020-03-30 04:22:41 +0000 UTC 2020-03-30 06:32:23 +0000 UTC 3 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3708be8cf1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3708be8cf1 d6232f3e-5042-42a9-9c85-6a95d51f9941 12857 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:40 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff37184f2733  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff37184f2733 478b45fb-0f1d-4e05-8dcc-7870fed42cd7 12861 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3718545a25  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3718545a25 0c2434a9-2721-476f-81d3-cf4fa092594a 12862 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-64hw status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3718592d6d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3718592d6d ea8e2e6a-c945-4131-8dbc-b5c5c06921b2 12863 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3721d2fe8c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3721d2fe8c 1b5e5c05-a15c-4ff8-a345-83b92435390c 12864 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff373431a2fa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff373431a2fa 74aa834f-d50e-4132-97cd-ac96d97ad6fa 12865 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3735ca4ab1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3735ca4ab1 b7cbc950-a32c-4546-b695-6d60a029c23a 12866 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeReady Node bootstrap-e2e-minion-group-64hw status is now: NodeReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146484d3938  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146484d3938 8ff3da0b-4e3f-4efa-a8ac-176af5945090 14018 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507e9189  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507e9189 8c6d75df-639d-4cc5-8512-4d30005c4318 14022 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507ed220  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507ed220 30a251ce-1902-4127-b8c9-4547b03303ff 14024 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-64hw status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507f302c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507f302c 1e86387e-ff53-49e2-8dc8-0366c2448de8 14023 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1601014652aab56e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1601014652aab56e 801a2de3-d19e-4676-a337-2ce6cebd2f6c 14025 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101466e3ab244  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101466e3ab244 c1b41cf9-8a69-47ed-8905-c06beefde3fb 14026 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146713237d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146713237d1 cba5f617-5fc3-45b6-99c9-8a3234ea8226 14027 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeReady Node bootstrap-e2e-minion-group-64hw status is now: NodeReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101848d248f6e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101848d248f6e 5bfa3bad-a736-4613-8ef7-3bc0d1f182e8 14165 0 2020-03-30 06:36:52 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 06:36:52 +0000 UTC 2020-03-30 06:36:52 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101848d24bc8e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101848d24bc8e 680cfad4-b548-4301-804f-bf254d50ac67 14164 0 2020-03-30 06:36:52 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 06:36:52 +0000 UTC 2020-03-30 06:36:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.1600ecea5f22eeb7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.1600ecea5f22eeb7 40cb6ace-07e7-4748-ac20-eba8b4a68ad7 13672 0 2020-03-30 06:20:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 00:19:20 +0000 UTC 2020-03-30 06:20:35 +0000 UTC 7 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.1600f2046cc3e059  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.1600f2046cc3e059 740409e6-5771-4b68-bfb1-a1fa8a72c9f3 13671 0 2020-03-30 06:20:32 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc 683a95bd-d957-4a55-bd82-6b94778a272c   } NodeNotReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeNotReady {node-controller } 2020-03-30 01:52:49 +0000 UTC 2020-03-30 06:20:32 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a12999b6af  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a12999b6af 1b5d9b54-e1e0-47c2-95ab-5bf7a83a6fa8 13673 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:35 +0000 UTC 2020-03-30 06:20:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a13191c0a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a13191c0a9 76c7fa8b-8ed0-41fd-a6f1-71da52aaca5b 13677 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a1319207e1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a1319207e1 05ac9a41-8cdc-4a8e-ac48-206f02a5592f 13678 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a131923d3e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a131923d3e 5300c934-e04b-4f77-8616-82f3f09e439e 13679 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a1340181b8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a1340181b8 7100ef05-682b-4b41-9c77-270fe5804c1a 13680 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeNotReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a158d4639f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a158d4639f 913073a7-11cf-4555-842b-f73151947b14 13681 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a15cba480e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a15cba480e 4f0e0598-1a86-476a-9bb6-3c1032dbff6e 13682 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeReady {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.16010185240ed406  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.16010185240ed406 18959297-6000-4628-ba4f-f141ab680973 14167 0 2020-03-30 06:36:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:36:55 +0000 UTC 2020-03-30 06:36:55 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.16010185240ef79e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.16010185240ef79e 3d0eadb9-4b92-490f-a2ec-fca2cbc32c98 14166 0 2020-03-30 06:36:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:36:55 +0000 UTC 2020-03-30 06:36:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-shvs.16010185b73e4194  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-shvs.16010185b73e4194 41b1cdff-ac0b-47b7-9959-9777b6fd0aee 14169 0 2020-03-30 06:36:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-shvs bootstrap-e2e-minion-group-shvs   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-shvs} 2020-03-30 06:36:57 +0000 UTC 2020-03-30 06:36:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-shvs.16010185b73e60f6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-shvs.16010185b73e60f6 b78bc80f-897f-43d5-8fd6-a76c5f8a77e7 14168 0 2020-03-30 06:36:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-shvs bootstrap-e2e-minion-group-shvs   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-shvs} 2020-03-30 06:36:57 +0000 UTC 2020-03-30 06:36:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-64hw.1600f1f4489f64fc  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600f1f4489f64fc 11fba924-a113-4f9f-9e00-a35dd10d26dd 14017 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 01:51:40 +0000 UTC 2020-03-30 06:32:24 +0000 UTC 10 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600fa3208041c28  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600fa3208041c28 1f537f3d-8cf7-4380-ace7-89fa6de1f4ad 14016 0 2020-03-30 05:54:37 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw 13116c7b-57ba-48a7-97cd-175f971897e8   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {node-controller } 2020-03-30 04:22:41 +0000 UTC 2020-03-30 06:32:23 +0000 UTC 3 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3708be8cf1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3708be8cf1 d6232f3e-5042-42a9-9c85-6a95d51f9941 12857 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:40 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff37184f2733  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff37184f2733 478b45fb-0f1d-4e05-8dcc-7870fed42cd7 12861 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3718545a25  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3718545a25 0c2434a9-2721-476f-81d3-cf4fa092594a 12862 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-64hw status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3718592d6d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3718592d6d ea8e2e6a-c945-4131-8dbc-b5c5c06921b2 12863 0 2020-03-30 05:54:40 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:40 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3721d2fe8c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3721d2fe8c 1b5e5c05-a15c-4ff8-a345-83b92435390c 12864 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff373431a2fa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff373431a2fa 74aa834f-d50e-4132-97cd-ac96d97ad6fa 12865 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1600ff3735ca4ab1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1600ff3735ca4ab1 b7cbc950-a32c-4546-b695-6d60a029c23a 12866 0 2020-03-30 05:54:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeReady Node bootstrap-e2e-minion-group-64hw status is now: NodeReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 05:54:41 +0000 UTC 2020-03-30 05:54:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146484d3938  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146484d3938 8ff3da0b-4e3f-4efa-a8ac-176af5945090 14018 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507e9189  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507e9189 8c6d75df-639d-4cc5-8512-4d30005c4318 14022 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507ed220  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507ed220 30a251ce-1902-4127-b8c9-4547b03303ff 14024 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-64hw status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146507f302c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146507f302c 1e86387e-ff53-49e2-8dc8-0366c2448de8 14023 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-64hw status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.1601014652aab56e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.1601014652aab56e 801a2de3-d19e-4676-a337-2ce6cebd2f6c 14025 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeNotReady Node bootstrap-e2e-minion-group-64hw status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101466e3ab244  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101466e3ab244 c1b41cf9-8a69-47ed-8905-c06beefde3fb 14026 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.16010146713237d1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.16010146713237d1 cba5f617-5fc3-45b6-99c9-8a3234ea8226 14027 0 2020-03-30 06:32:25 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } NodeReady Node bootstrap-e2e-minion-group-64hw status is now: NodeReady {kubelet bootstrap-e2e-minion-group-64hw} 2020-03-30 06:32:25 +0000 UTC 2020-03-30 06:32:25 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101848d248f6e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101848d248f6e 5bfa3bad-a736-4613-8ef7-3bc0d1f182e8 14165 0 2020-03-30 06:36:52 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 06:36:52 +0000 UTC 2020-03-30 06:36:52 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-64hw.160101848d24bc8e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-64hw.160101848d24bc8e 680cfad4-b548-4301-804f-bf254d50ac67 14164 0 2020-03-30 06:36:52 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-64hw bootstrap-e2e-minion-group-64hw   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-64hw} 2020-03-30 06:36:52 +0000 UTC 2020-03-30 06:36:52 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.1600ecea5f22eeb7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.1600ecea5f22eeb7 40cb6ace-07e7-4748-ac20-eba8b4a68ad7 13672 0 2020-03-30 06:20:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 00:19:20 +0000 UTC 2020-03-30 06:20:35 +0000 UTC 7 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.1600f2046cc3e059  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.1600f2046cc3e059 740409e6-5771-4b68-bfb1-a1fa8a72c9f3 13671 0 2020-03-30 06:20:32 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc 683a95bd-d957-4a55-bd82-6b94778a272c   } NodeNotReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeNotReady {node-controller } 2020-03-30 01:52:49 +0000 UTC 2020-03-30 06:20:32 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a12999b6af  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a12999b6af 1b5d9b54-e1e0-47c2-95ab-5bf7a83a6fa8 13673 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:35 +0000 UTC 2020-03-30 06:20:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a13191c0a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a13191c0a9 76c7fa8b-8ed0-41fd-a6f1-71da52aaca5b 13677 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a1319207e1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a1319207e1 05ac9a41-8cdc-4a8e-ac48-206f02a5592f 13678 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a131923d3e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a131923d3e 5300c934-e04b-4f77-8616-82f3f09e439e 13679 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-m3lc status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a1340181b8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a1340181b8 7100ef05-682b-4b41-9c77-270fe5804c1a 13680 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeNotReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a158d4639f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a158d4639f 913073a7-11cf-4555-842b-f73151947b14 13681 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.160100a15cba480e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.160100a15cba480e 4f0e0598-1a86-476a-9bb6-3c1032dbff6e 13682 0 2020-03-30 06:20:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } NodeReady Node bootstrap-e2e-minion-group-m3lc status is now: NodeReady {kubelet bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:20:36 +0000 UTC 2020-03-30 06:20:36 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.16010185240ed406  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.16010185240ed406 18959297-6000-4628-ba4f-f141ab680973 14167 0 2020-03-30 06:36:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:36:55 +0000 UTC 2020-03-30 06:36:55 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m3lc.16010185240ef79e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m3lc.16010185240ef79e 3d0eadb9-4b92-490f-a2ec-fca2cbc32c98 14166 0 2020-03-30 06:36:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m3lc bootstrap-e2e-minion-group-m3lc   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-m3lc} 2020-03-30 06:36:55 +0000 UTC 2020-03-30 06:36:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-shvs.16010185b73e4194  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-shvs.16010185b73e4194 41b1cdff-ac0b-47b7-9959-9777b6fd0aee 14169 0 2020-03-30 06:36:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-shvs bootstrap-e2e-minion-group-shvs   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-shvs} 2020-03-30 06:36:57 +0000 UTC 2020-03-30 06:36:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-shvs.16010185b73e60f6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-shvs.16010185b73e60f6 b78bc80f-897f-43d5-8fd6-a76c5f8a77e7 14168 0 2020-03-30 06:36:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-shvs bootstrap-e2e-minion-group-shvs   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-shvs} 2020-03-30 06:36:57 +0000 UTC 2020-03-30 06:36:57 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [sig-apps] Daemon set [Serial] should not update pod when spec was updated and update strategy is OnDelete 5m24s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-apps\]\sDaemon\sset\s\[Serial\]\sshould\snot\supdate\spod\swhen\sspec\swas\supdated\sand\supdate\sstrategy\sis\sOnDelete$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/daemon_set.go:277
Mar 30 14:08:20.728: error waiting for daemon pod to start
Unexpected error:
    <*errors.errorString | 0xc0000d3000>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/apps/daemon_set.go:288