This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 7 failed / 912 succeeded
Started2020-03-21 21:27
Elapsed15h14m
Revision
Builderae6a30e5-6bba-11ea-85de-3e9df61a68c7
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/6e8edf7e-d8f9-415c-bab7-7d693d3604a3/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/6e8edf7e-d8f9-415c-bab7-7d693d3604a3/targets/test
infra-commit9d80002da
job-versionv1.16.9-beta.0.7+5116ee4b159565
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.7+5116ee4b159565

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 17m40s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 21 21:44:19.561: Unexpected error:
    <*errors.errorString | 0xc002db0770>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-md6c\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mql0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-pkzl\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE   VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   11m   v1.17.5-beta.0.11+4dfcd1cc87879f\\nbootstrap-e2e-minion-group-md6c   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-mql0   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-pkzl   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\ncontroller-manager   Healthy   ok                  \\nscheduler            Healthy   ok                  \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.11+4dfcd1cc87879f\\\"\\nname: \\\"bootstrap-e2e-minion-group-md6c\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mql0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-pkzl\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\n\", stderr \"Project: gce-up-glat-up-etcd\\nNetwork Project: gce-up-glat-up-etcd\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-pkzl\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 34.83.128.2; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-up-glat-up-etcd/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/gce-up-glat-up-etcd/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP  STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.83.128.2  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: gce-up-glat-up-etcd\\nNetwork Project: gce-up-glat-up-etcd\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-md6c\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mql0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-pkzl\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE   VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   11m   v1.17.5-beta.0.11+4dfcd1cc87879f\nbootstrap-e2e-minion-group-md6c   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-mql0   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-pkzl   Ready                      <none>   11m   v1.16.9-beta.0.7+5116ee4b159565\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\ncontroller-manager   Healthy   ok                  \nscheduler            Healthy   ok                  \netcd-1               Healthy   {\"health\":\"true\"}   \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.11+4dfcd1cc87879f\"\nname: \"bootstrap-e2e-minion-group-md6c\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mql0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-pkzl\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\n", stderr "Project: gce-up-glat-up-etcd\nNetwork Project: gce-up-glat-up-etcd\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-pkzl\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 34.83.128.2; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-up-glat-up-etcd/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/gce-up-glat-up-etcd/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP  STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.83.128.2  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: gce-up-glat-up-etcd\nNetwork Project: gce-up-glat-up-etcd\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m16s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 22 01:08:29.861: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc001b5b530>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-md6c.15fe716f87cb850a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe716f87cb850a aa276099-0b55-4eea-be59-68ec73ee02ff 6041 0 2020-03-22 00:20:10 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c a9aa5d3c-59ea-4525-9cb7-0dc0ac42d186   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {node-controller } 2020-03-21 22:14:02 +0000 UTC 2020-03-22 00:20:10 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe74955419a1ab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe74955419a1ab 83c8fb0e-33b7-40da-a6f7-31997227bf16 6233 0 2020-03-21 23:11:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-md6c} 2020-03-21 23:11:43 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe785210082228  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe785210082228 40fb2c44-3316-4817-a319-2a90f0a52a6f 6043 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:20:12 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4eaaf6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4eaaf6 e44ab450-8cf0-4784-9892-f1980c292d77 6145 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:24:37 +0000 UTC 7 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4ef79b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4ef79b bb4da8d4-f1ff-4b9d-a4c4-6defce6c0355 6146 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-md6c status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:24:37 +0000 UTC 7 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4f29ac  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4f29ac cce520f2-ae45-481f-ac67-ff7b50fedbcc 6158 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:25:57 +0000 UTC 11 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521e607d44  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521e607d44 2cf4b888-8cd3-49bc-afe7-c88b0fdd8688 6050 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:20:12 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe785231eb7c02  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe785231eb7c02 dc0943f0-5c26-4883-be2d-209ed6e1c84a 6051 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:13 +0000 UTC 2020-03-22 00:20:13 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7852345dddb0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7852345dddb0 658da4ce-39c0-4589-9018-e2ed904458d6 6052 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeReady Node bootstrap-e2e-minion-group-md6c status is now: NodeReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:13 +0000 UTC 2020-03-22 00:20:13 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7885d5fc13b4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7885d5fc13b4 362441c0-66f6-44e6-8367-07bf90cba428 6135 0 2020-03-22 00:23:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c a9aa5d3c-59ea-4525-9cb7-0dc0ac42d186   } RemovingNode Node bootstrap-e2e-minion-group-md6c event: Removing Node bootstrap-e2e-minion-group-md6c from Controller {node-controller } 2020-03-22 00:23:55 +0000 UTC 2020-03-22 00:23:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78dd27cdb8b0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78dd27cdb8b0 6d6dc0e8-1d5d-4d86-9e2c-fa58becbe4d0 6175 0 2020-03-22 00:30:10 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c 04d710cb-a974-4a78-b022-19a3d1a5e6b5   } RegisteredNode Node bootstrap-e2e-minion-group-md6c event: Registered Node bootstrap-e2e-minion-group-md6c in Controller {node-controller } 2020-03-22 00:30:10 +0000 UTC 2020-03-22 00:30:10 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79194c43fb76  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79194c43fb76 de51ff60-2791-4f48-9264-21185fbf6b89 6234 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195833ba56  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195833ba56 a5677067-32e7-4a46-b9de-71e265606919 6235 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195838135c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195838135c 231a1d91-295c-40cb-9792-bcd5626d671a 6236 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-md6c status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7919583c49ba  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7919583c49ba e301d33c-1bab-4564-8cfb-7682cb7d5719 6237 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195af1c656  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195af1c656 518165fa-bc47-48fd-a058-c38180b9282e 6238 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe791971305716  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe791971305716 4e4d7389-a4b3-462a-99a3-0b2ce0b84316 6239 0 2020-03-22 00:34:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:29 +0000 UTC 2020-03-22 00:34:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7919750a672e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7919750a672e 161173b0-d6ca-4b2e-a15f-78ba175e112c 6240 0 2020-03-22 00:34:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeReady Node bootstrap-e2e-minion-group-md6c status is now: NodeReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:29 +0000 UTC 2020-03-22 00:34:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7ae56ab6bb53  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7ae56ab6bb53 80ba3383-5814-49a2-82b7-21fef04e370e 7092 0 2020-03-22 01:07:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-md6c} 2020-03-22 01:07:24 +0000 UTC 2020-03-22 01:07:24 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7ae56ab6dad6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7ae56ab6dad6 22d9a999-89a7-412f-9ba8-6d64f66802cd 7091 0 2020-03-22 01:07:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-md6c} 2020-03-22 01:07:24 +0000 UTC 2020-03-22 01:07:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mql0.15fe7ae5ffe315d2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mql0.15fe7ae5ffe315d2 635c6895-4ae6-4a81-b300-520a016fc570 7094 0 2020-03-22 01:07:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-mql0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mql0} 2020-03-22 01:07:27 +0000 UTC 2020-03-22 01:07:27 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mql0.15fe7ae5ffe33a4d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mql0.15fe7ae5ffe33a4d 556023ae-2500-4650-bd17-6057898c53b0 7093 0 2020-03-22 01:07:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-mql0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mql0} 2020-03-22 01:07:27 +0000 UTC 2020-03-22 01:07:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe718a54e10953  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe718a54e10953 22c97b61-ea01-4f35-a753-551e9b2cd9d9 6962 0 2020-03-22 00:14:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl c446b3a7-e71f-406b-886c-9c45fc35859b   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {node-controller } 2020-03-21 22:15:57 +0000 UTC 2020-03-22 01:02:45 +0000 UTC 5 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7494c1047ae6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7494c1047ae6 b4a79ccd-3960-4ef1-b1e8-eb43444eed7c 6963 0 2020-03-21 23:11:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-21 23:11:41 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780645490757  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780645490757 cb80e9cd-37ba-4096-8fe3-bc5037e33321 5904 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780650245f00  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780650245f00 063a57a2-63ad-438e-b931-3b96838540ef 5909 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780650249c1a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780650249c1a 48f924ed-950e-4f46-9fa7-289da7e6d236 5908 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe78065024d0fa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe78065024d0fa 12fc63d3-2f34-467c-9bc6-e45bce2b7e14 5910 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780653074728  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780653074728 809096f0-ced4-4759-9ae9-b95eadc51231 5911 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780675206a7c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780675206a7c fb69c246-c7f9-41da-a282-730f9b8995fe 5912 0 2020-03-22 00:14:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:48 +0000 UTC 2020-03-22 00:14:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe78067ba6519b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe78067ba6519b 4b8b95f7-7557-4f9d-939a-2fc99723c746 5913 0 2020-03-22 00:14:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:48 +0000 UTC 2020-03-22 00:14:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7933fe5a6de9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7933fe5a6de9 035e7c31-d36f-4cae-a401-b5bc45889c7c 6261 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340877df66  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340877df66 dce8b070-c147-41d7-ae81-71174f9ee593 6266 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7934087ea8ae  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7934087ea8ae 3b6eb6bc-049f-41f6-aa94-c490bb18d242 6265 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340882ff9f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340882ff9f a6c40a2a-9552-4017-963c-8d69cc3cc7f7 6267 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340d055c5c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340d055c5c 5fbcc7a1-4209-4068-9f00-5f421dd88ee1 6268 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7934391c52e6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7934391c52e6 a27cc556-e2d1-4943-8136-5db44aa51f8d 6269 0 2020-03-22 00:36:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:24 +0000 UTC 2020-03-22 00:36:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79343a4c0ee3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79343a4c0ee3 e6f29f9d-1bee-4592-991c-bff4345ea043 6270 0 2020-03-22 00:36:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:24 +0000 UTC 2020-03-22 00:36:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4f4da2d0f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4f4da2d0f d7f7da7d-1023-41eb-a868-091b74e1da33 6964 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd6ce403  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd6ce403 f63d5d27-627e-4299-8185-f227f83b8979 6968 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd73fbf4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd73fbf4 9ca55a64-0bc5-4186-865a-d584ecf9b736 6969 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd7a94a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd7a94a9 c731d62b-6d30-4fba-9bfa-8380e4f00d17 6970 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4feed23e6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4feed23e6 f434aa54-47b2-4512-b57d-7a0cb2abb3f3 6971 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa51bb0b49a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa51bb0b49a 1ac479f8-9308-402d-89ca-2d9171046e4d 6972 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:48 +0000 UTC 2020-03-22 01:02:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa51da49123  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa51da49123 61216a94-7bd6-4329-bdb7-b1ea4880ede5 6973 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:48 +0000 UTC 2020-03-22 01:02:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7ae69431279c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7ae69431279c 6bf90686-ae69-462c-9ff0-6cd17c98298b 7096 0 2020-03-22 01:07:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:07:29 +0000 UTC 2020-03-22 01:07:29 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7ae694314b6a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7ae694314b6a e69f4906-a2f4-440a-a711-62aad599589f 7095 0 2020-03-22 01:07:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:07:29 +0000 UTC 2020-03-22 01:07:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-md6c.15fe716f87cb850a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe716f87cb850a aa276099-0b55-4eea-be59-68ec73ee02ff 6041 0 2020-03-22 00:20:10 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c a9aa5d3c-59ea-4525-9cb7-0dc0ac42d186   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {node-controller } 2020-03-21 22:14:02 +0000 UTC 2020-03-22 00:20:10 +0000 UTC 4 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe74955419a1ab  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe74955419a1ab 83c8fb0e-33b7-40da-a6f7-31997227bf16 6233 0 2020-03-21 23:11:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-md6c} 2020-03-21 23:11:43 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe785210082228  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe785210082228 40fb2c44-3316-4817-a319-2a90f0a52a6f 6043 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:20:12 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4eaaf6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4eaaf6 e44ab450-8cf0-4784-9892-f1980c292d77 6145 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:24:37 +0000 UTC 7 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4ef79b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4ef79b bb4da8d4-f1ff-4b9d-a4c4-6defce6c0355 6146 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-md6c status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:24:37 +0000 UTC 7 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521b4f29ac  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521b4f29ac cce520f2-ae45-481f-ac67-ff7b50fedbcc 6158 0 2020-03-22 00:20:12 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:25:57 +0000 UTC 11 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78521e607d44  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78521e607d44 2cf4b888-8cd3-49bc-afe7-c88b0fdd8688 6050 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:12 +0000 UTC 2020-03-22 00:20:12 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe785231eb7c02  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe785231eb7c02 dc0943f0-5c26-4883-be2d-209ed6e1c84a 6051 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:13 +0000 UTC 2020-03-22 00:20:13 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7852345dddb0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7852345dddb0 658da4ce-39c0-4589-9018-e2ed904458d6 6052 0 2020-03-22 00:20:13 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeReady Node bootstrap-e2e-minion-group-md6c status is now: NodeReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:20:13 +0000 UTC 2020-03-22 00:20:13 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7885d5fc13b4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7885d5fc13b4 362441c0-66f6-44e6-8367-07bf90cba428 6135 0 2020-03-22 00:23:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c a9aa5d3c-59ea-4525-9cb7-0dc0ac42d186   } RemovingNode Node bootstrap-e2e-minion-group-md6c event: Removing Node bootstrap-e2e-minion-group-md6c from Controller {node-controller } 2020-03-22 00:23:55 +0000 UTC 2020-03-22 00:23:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe78dd27cdb8b0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe78dd27cdb8b0 6d6dc0e8-1d5d-4d86-9e2c-fa58becbe4d0 6175 0 2020-03-22 00:30:10 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c 04d710cb-a974-4a78-b022-19a3d1a5e6b5   } RegisteredNode Node bootstrap-e2e-minion-group-md6c event: Registered Node bootstrap-e2e-minion-group-md6c in Controller {node-controller } 2020-03-22 00:30:10 +0000 UTC 2020-03-22 00:30:10 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79194c43fb76  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79194c43fb76 de51ff60-2791-4f48-9264-21185fbf6b89 6234 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195833ba56  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195833ba56 a5677067-32e7-4a46-b9de-71e265606919 6235 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195838135c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195838135c 231a1d91-295c-40cb-9792-bcd5626d671a 6236 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-md6c status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7919583c49ba  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7919583c49ba e301d33c-1bab-4564-8cfb-7682cb7d5719 6237 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-md6c status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe79195af1c656  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe79195af1c656 518165fa-bc47-48fd-a058-c38180b9282e 6238 0 2020-03-22 00:34:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeNotReady Node bootstrap-e2e-minion-group-md6c status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:28 +0000 UTC 2020-03-22 00:34:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe791971305716  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe791971305716 4e4d7389-a4b3-462a-99a3-0b2ce0b84316 6239 0 2020-03-22 00:34:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:29 +0000 UTC 2020-03-22 00:34:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7919750a672e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7919750a672e 161173b0-d6ca-4b2e-a15f-78ba175e112c 6240 0 2020-03-22 00:34:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } NodeReady Node bootstrap-e2e-minion-group-md6c status is now: NodeReady {kubelet bootstrap-e2e-minion-group-md6c} 2020-03-22 00:34:29 +0000 UTC 2020-03-22 00:34:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7ae56ab6bb53  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7ae56ab6bb53 80ba3383-5814-49a2-82b7-21fef04e370e 7092 0 2020-03-22 01:07:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-md6c} 2020-03-22 01:07:24 +0000 UTC 2020-03-22 01:07:24 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-md6c.15fe7ae56ab6dad6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-md6c.15fe7ae56ab6dad6 22d9a999-89a7-412f-9ba8-6d64f66802cd 7091 0 2020-03-22 01:07:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-md6c bootstrap-e2e-minion-group-md6c   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-md6c} 2020-03-22 01:07:24 +0000 UTC 2020-03-22 01:07:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mql0.15fe7ae5ffe315d2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mql0.15fe7ae5ffe315d2 635c6895-4ae6-4a81-b300-520a016fc570 7094 0 2020-03-22 01:07:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-mql0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mql0} 2020-03-22 01:07:27 +0000 UTC 2020-03-22 01:07:27 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mql0.15fe7ae5ffe33a4d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mql0.15fe7ae5ffe33a4d 556023ae-2500-4650-bd17-6057898c53b0 7093 0 2020-03-22 01:07:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mql0 bootstrap-e2e-minion-group-mql0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mql0} 2020-03-22 01:07:27 +0000 UTC 2020-03-22 01:07:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe718a54e10953  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe718a54e10953 22c97b61-ea01-4f35-a753-551e9b2cd9d9 6962 0 2020-03-22 00:14:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl c446b3a7-e71f-406b-886c-9c45fc35859b   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {node-controller } 2020-03-21 22:15:57 +0000 UTC 2020-03-22 01:02:45 +0000 UTC 5 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7494c1047ae6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7494c1047ae6 b4a79ccd-3960-4ef1-b1e8-eb43444eed7c 6963 0 2020-03-21 23:11:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-21 23:11:41 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780645490757  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780645490757 cb80e9cd-37ba-4096-8fe3-bc5037e33321 5904 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780650245f00  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780650245f00 063a57a2-63ad-438e-b931-3b96838540ef 5909 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780650249c1a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780650249c1a 48f924ed-950e-4f46-9fa7-289da7e6d236 5908 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe78065024d0fa  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe78065024d0fa 12fc63d3-2f34-467c-9bc6-e45bce2b7e14 5910 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780653074728  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780653074728 809096f0-ced4-4759-9ae9-b95eadc51231 5911 0 2020-03-22 00:14:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:47 +0000 UTC 2020-03-22 00:14:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe780675206a7c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe780675206a7c fb69c246-c7f9-41da-a282-730f9b8995fe 5912 0 2020-03-22 00:14:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:48 +0000 UTC 2020-03-22 00:14:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe78067ba6519b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe78067ba6519b 4b8b95f7-7557-4f9d-939a-2fc99723c746 5913 0 2020-03-22 00:14:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:14:48 +0000 UTC 2020-03-22 00:14:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7933fe5a6de9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7933fe5a6de9 035e7c31-d36f-4cae-a401-b5bc45889c7c 6261 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340877df66  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340877df66 dce8b070-c147-41d7-ae81-71174f9ee593 6266 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7934087ea8ae  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7934087ea8ae 3b6eb6bc-049f-41f6-aa94-c490bb18d242 6265 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340882ff9f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340882ff9f a6c40a2a-9552-4017-963c-8d69cc3cc7f7 6267 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79340d055c5c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79340d055c5c 5fbcc7a1-4209-4068-9f00-5f421dd88ee1 6268 0 2020-03-22 00:36:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:23 +0000 UTC 2020-03-22 00:36:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7934391c52e6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7934391c52e6 a27cc556-e2d1-4943-8136-5db44aa51f8d 6269 0 2020-03-22 00:36:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:24 +0000 UTC 2020-03-22 00:36:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe79343a4c0ee3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe79343a4c0ee3 e6f29f9d-1bee-4592-991c-bff4345ea043 6270 0 2020-03-22 00:36:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 00:36:24 +0000 UTC 2020-03-22 00:36:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4f4da2d0f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4f4da2d0f d7f7da7d-1023-41eb-a868-091b74e1da33 6964 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd6ce403  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd6ce403 f63d5d27-627e-4299-8185-f227f83b8979 6968 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd73fbf4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd73fbf4 9ca55a64-0bc5-4186-865a-d584ecf9b736 6969 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4fd7a94a9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4fd7a94a9 c731d62b-6d30-4fba-9bfa-8380e4f00d17 6970 0 2020-03-22 01:02:47 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-pkzl status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa4feed23e6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa4feed23e6 f434aa54-47b2-4512-b57d-7a0cb2abb3f3 6971 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeNotReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:47 +0000 UTC 2020-03-22 01:02:47 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa51bb0b49a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa51bb0b49a 1ac479f8-9308-402d-89ca-2d9171046e4d 6972 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:48 +0000 UTC 2020-03-22 01:02:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7aa51da49123  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7aa51da49123 61216a94-7bd6-4329-bdb7-b1ea4880ede5 6973 0 2020-03-22 01:02:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } NodeReady Node bootstrap-e2e-minion-group-pkzl status is now: NodeReady {kubelet bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:02:48 +0000 UTC 2020-03-22 01:02:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7ae69431279c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7ae69431279c 6bf90686-ae69-462c-9ff0-6cd17c98298b 7096 0 2020-03-22 01:07:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:07:29 +0000 UTC 2020-03-22 01:07:29 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-pkzl.15fe7ae694314b6a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-pkzl.15fe7ae694314b6a e69f4906-a2f4-440a-a711-62aad599589f 7095 0 2020-03-22 01:07:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-pkzl bootstrap-e2e-minion-group-pkzl   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-pkzl} 2020-03-22 01:07:29 +0000 UTC 2020-03-22 01:07:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [sig-storage] In-tree Volumes [Driver: gcepd] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works 26s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sIn\-tree\sVolumes\s\[Driver\:\sgcepd\]\s\[Testpattern\:\sDynamic\sPV\s\(default\sfs\)\(allowExpansion\)\]\svolume\-expand\sVerify\sif\soffline\sPVC\sexpansion\sworks$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:154
Mar 22 12:27:26.271: While updating pvc for more size
Unexpected error:
    <*errors.errorString | 0xc006297a20>: {
        s: "error fetching pvc \"gcepdqx9fc\" for resizing with persistentvolumeclaims \"gcepdqx9fc\" not found",
    }
    error fetching pvc "gcepdqx9fc" for resizing with persistentvolumeclaims "gcepdqx9fc" not found
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:178
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 8m6s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Mar 21 21:44:19.561: Unexpected error:
    <*errors.errorString | 0xc002db0770>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-md6c\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mql0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-pkzl\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==