This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 7 failed / 940 succeeded
Started2020-03-26 23:19
Elapsed15h12m
Revision
Builder2455ae36-6fb8-11ea-bd6a-f29ce8b0e481
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/b6375415-8983-4a6f-9f22-46d1be334b00/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/b6375415-8983-4a6f-9f22-46d1be334b00/targets/test
infra-commitd2e2e04d1
job-versionv1.16.9-beta.0.17+bb616cba1cc983
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.17+bb616cba1cc983

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 18m14s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 26 23:34:51.290: Unexpected error:
    <*errors.errorString | 0xc0029e6ac0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.20+12ff0e52eca73e]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-g277\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-xnw0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-z2c0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.20+12ff0e52eca73e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m     v1.17.5-beta.0.20+12ff0e52eca73e\\nbootstrap-e2e-minion-group-g277   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-xnw0   Ready                      <none>   9m59s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-z2c0   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.20+12ff0e52eca73e\\\"\\nname: \\\"bootstrap-e2e-minion-group-g277\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-xnw0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-z2c0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\n\", stderr \"Project: gce-cvm-upg-lat-1-4-ctl-skew\\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-g277 bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-z2c0\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 34.82.173.66; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.173.66  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: gce-cvm-upg-lat-1-4-ctl-skew\\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.20+12ff0e52eca73e]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-g277\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-xnw0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-z2c0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.20+12ff0e52eca73e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m     v1.17.5-beta.0.20+12ff0e52eca73e\nbootstrap-e2e-minion-group-g277   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-xnw0   Ready                      <none>   9m59s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-z2c0   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.20+12ff0e52eca73e\"\nname: \"bootstrap-e2e-minion-group-g277\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-xnw0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-z2c0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\n", stderr "Project: gce-cvm-upg-lat-1-4-ctl-skew\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-g277 bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-z2c0\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 34.82.173.66; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.173.66  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: gce-cvm-upg-lat-1-4-ctl-skew\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m17s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 27 12:47:44.104: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc00370ec80>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-701f.1600272f6e301545  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f6e301545 2d8780a7-b90f-4c19-b44d-318b0b05b037 25950 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f76331678  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f76331678 e19d68c8-0f59-49ec-ad80-06a7ad0a6e7c 25951 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-701f status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f763350e8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f763350e8 c86f7cfa-8fed-41f3-b102-1f4973668708 25952 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-701f status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f76342113  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f76342113 b20a2f10-1651-4072-bef9-2aea650dc509 25953 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-701f status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f7bdc3f95  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f7bdc3f95 53c89940-24b8-4a5c-9d57-249141315d30 25955 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f7dca3089  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f7dca3089 c3210f2c-2d55-467a-b1f7-7683cf5e0d5a 25959 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeReady Node bootstrap-e2e-minion-group-701f status is now: NodeReady {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272ff05af9d3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272ff05af9d3 3a707d80-35be-4dcf-a3e2-3424bfc630af 25964 0 2020-03-27 11:55:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } Starting Starting kube-proxy. {kube-proxy bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:55 +0000 UTC 2020-03-27 11:55:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273001d28868  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273001d28868 ccf5c3e1-c989-4ebb-816a-f6422d55e2c9 25965 0 2020-03-27 11:55:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } RegisteredNode Node bootstrap-e2e-minion-group-701f event: Registered Node bootstrap-e2e-minion-group-701f in Controller {node-controller } 2020-03-27 11:55:55 +0000 UTC 2020-03-27 11:55:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273073159cdf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273073159cdf a7bf98e0-2d5e-4521-a53f-61b4fb4c5b4e 25969 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } ContainerdStart Starting containerd container runtime... {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.160027307315e459  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.160027307315e459 14121bfb-91bd-4982-983a-ea6829ba1744 25971 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } DockerStart Starting Docker Application Container Engine... {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273075a4f355  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273075a4f355 44389136-1571-4252-8b15-85a5ceb948e8 25970 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.16002742a3394644  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.16002742a3394644 a4b4b4a2-698d-4dd1-9bac-9d016978078c 25990 0 2020-03-27 11:57:15 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } NodeNotReady Node bootstrap-e2e-minion-group-701f status is now: NodeNotReady {node-controller } 2020-03-27 11:57:15 +0000 UTC 2020-03-27 11:57:15 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600274d1d47d7b6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600274d1d47d7b6 0e7ac03a-faf9-424b-b480-c38b86b09325 25992 0 2020-03-27 11:58:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } Deleting node bootstrap-e2e-minion-group-701f because it does not exist in the cloud provider Node bootstrap-e2e-minion-group-701f event: DeletingNode {cloud-node-lifecycle-controller } 2020-03-27 11:58:00 +0000 UTC 2020-03-27 11:58:00 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600274d210e57ff  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600274d210e57ff a218b2f0-22b7-4dc4-9749-ef029afe5239 25993 0 2020-03-27 11:58:01 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } RemovingNode Node bootstrap-e2e-minion-group-701f event: Removing Node bootstrap-e2e-minion-group-701f from Controller {node-controller } 2020-03-27 11:58:01 +0000 UTC 2020-03-27 11:58:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.1600221312b9e02d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.1600221312b9e02d 76903db6-c4e5-4eb8-84f7-309ca6c6b6a4 27762 0 2020-03-27 10:22:14 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 10:22:14 +0000 UTC 2020-03-27 12:26:47 +0000 UTC 6 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028decff21ddf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028decff21ddf 4543ad9d-ac31-4c86-840e-e97657e77f90 27761 0 2020-03-27 12:26:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd a9df19ae-26b2-41ff-87fa-e4aa1a018622   } NodeNotReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeNotReady {node-controller } 2020-03-27 12:26:46 +0000 UTC 2020-03-27 12:26:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df3f3de4b8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df3f3de4b8 a108b05d-b649-495a-93e5-01a9e4bc48af 27763 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df4974132d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df4974132d 2f6e8f98-59b3-4496-b9f2-a8dc4477e8ab 27768 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df49744a4b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df49744a4b fd8935d4-4435-4cad-9d39-cb71a4fa53d9 27767 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df49747747  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df49747747 2915deb6-df24-4d97-9ba8-1ad5cf46c988 27769 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df4ae5b2bb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df4ae5b2bb 89dbd305-82c8-4b3d-bac0-a8cf3850282c 27770 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeNotReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df5873ada3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df5873ada3 13260f21-0ede-4e53-85f9-8b9c42bd1842 27771 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df5aef7fe9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df5aef7fe9 59559150-7fb1-4f4f-a94a-b7d12c86eaa1 27772 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160029f47fbb8d86  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160029f47fbb8d86 4737fc14-e84d-4536-b1e7-3d81c22906bf 28649 0 2020-03-27 12:46:38 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:46:38 +0000 UTC 2020-03-27 12:46:38 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160029f47fbbb2e8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160029f47fbbb2e8 410d905c-9b6e-4157-8abb-b0404017c265 28648 0 2020-03-27 12:46:38 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:46:38 +0000 UTC 2020-03-27 12:46:38 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xnw0.160029f50cb31af8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xnw0.160029f50cb31af8 1d12075d-ebaa-4c27-87ff-2c187ecfb3c5 28651 0 2020-03-27 12:46:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-xnw0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xnw0} 2020-03-27 12:46:41 +0000 UTC 2020-03-27 12:46:41 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xnw0.160029f50cb345b1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xnw0.160029f50cb345b1 eb8bfb4c-d550-4dd1-9694-a6bda9d65f65 28650 0 2020-03-27 12:46:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-xnw0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xnw0} 2020-03-27 12:46:41 +0000 UTC 2020-03-27 12:46:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-z2c0.160029f5a27b0ac4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-z2c0.160029f5a27b0ac4 5bc44517-a3ac-4415-815f-7df57056c60e 28653 0 2020-03-27 12:46:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-z2c0 bootstrap-e2e-minion-group-z2c0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-z2c0} 2020-03-27 12:46:43 +0000 UTC 2020-03-27 12:46:43 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-z2c0.160029f5a27b2c52  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-z2c0.160029f5a27b2c52 3a85d75c-f8ae-4a81-923e-fe3f4a0df0dc 28652 0 2020-03-27 12:46:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-z2c0 bootstrap-e2e-minion-group-z2c0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-z2c0} 2020-03-27 12:46:43 +0000 UTC 2020-03-27 12:46:43 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-701f.1600272f6e301545  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f6e301545 2d8780a7-b90f-4c19-b44d-318b0b05b037 25950 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f76331678  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f76331678 e19d68c8-0f59-49ec-ad80-06a7ad0a6e7c 25951 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-701f status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f763350e8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f763350e8 c86f7cfa-8fed-41f3-b102-1f4973668708 25952 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-701f status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f76342113  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f76342113 b20a2f10-1651-4072-bef9-2aea650dc509 25953 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-701f status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f7bdc3f95  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f7bdc3f95 53c89940-24b8-4a5c-9d57-249141315d30 25955 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272f7dca3089  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272f7dca3089 c3210f2c-2d55-467a-b1f7-7683cf5e0d5a 25959 0 2020-03-27 11:55:53 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } NodeReady Node bootstrap-e2e-minion-group-701f status is now: NodeReady {kubelet bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:53 +0000 UTC 2020-03-27 11:55:53 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600272ff05af9d3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600272ff05af9d3 3a707d80-35be-4dcf-a3e2-3424bfc630af 25964 0 2020-03-27 11:55:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } Starting Starting kube-proxy. {kube-proxy bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:55 +0000 UTC 2020-03-27 11:55:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273001d28868  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273001d28868 ccf5c3e1-c989-4ebb-816a-f6422d55e2c9 25965 0 2020-03-27 11:55:55 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } RegisteredNode Node bootstrap-e2e-minion-group-701f event: Registered Node bootstrap-e2e-minion-group-701f in Controller {node-controller } 2020-03-27 11:55:55 +0000 UTC 2020-03-27 11:55:55 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273073159cdf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273073159cdf a7bf98e0-2d5e-4521-a53f-61b4fb4c5b4e 25969 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } ContainerdStart Starting containerd container runtime... {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.160027307315e459  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.160027307315e459 14121bfb-91bd-4982-983a-ea6829ba1744 25971 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } DockerStart Starting Docker Application Container Engine... {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600273075a4f355  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600273075a4f355 44389136-1571-4252-8b15-85a5ceb948e8 25970 0 2020-03-27 11:55:57 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f bootstrap-e2e-minion-group-701f   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-701f} 2020-03-27 11:55:57 +0000 UTC 2020-03-27 11:55:57 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.16002742a3394644  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.16002742a3394644 a4b4b4a2-698d-4dd1-9bac-9d016978078c 25990 0 2020-03-27 11:57:15 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } NodeNotReady Node bootstrap-e2e-minion-group-701f status is now: NodeNotReady {node-controller } 2020-03-27 11:57:15 +0000 UTC 2020-03-27 11:57:15 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600274d1d47d7b6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600274d1d47d7b6 0e7ac03a-faf9-424b-b480-c38b86b09325 25992 0 2020-03-27 11:58:00 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } Deleting node bootstrap-e2e-minion-group-701f because it does not exist in the cloud provider Node bootstrap-e2e-minion-group-701f event: DeletingNode {cloud-node-lifecycle-controller } 2020-03-27 11:58:00 +0000 UTC 2020-03-27 11:58:00 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-701f.1600274d210e57ff  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-701f.1600274d210e57ff a218b2f0-22b7-4dc4-9749-ef029afe5239 25993 0 2020-03-27 11:58:01 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-701f c29ae661-3eea-4950-bddd-46371e7bc3c2   } RemovingNode Node bootstrap-e2e-minion-group-701f event: Removing Node bootstrap-e2e-minion-group-701f from Controller {node-controller } 2020-03-27 11:58:01 +0000 UTC 2020-03-27 11:58:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.1600221312b9e02d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.1600221312b9e02d 76903db6-c4e5-4eb8-84f7-309ca6c6b6a4 27762 0 2020-03-27 10:22:14 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 10:22:14 +0000 UTC 2020-03-27 12:26:47 +0000 UTC 6 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028decff21ddf  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028decff21ddf 4543ad9d-ac31-4c86-840e-e97657e77f90 27761 0 2020-03-27 12:26:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd a9df19ae-26b2-41ff-87fa-e4aa1a018622   } NodeNotReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeNotReady {node-controller } 2020-03-27 12:26:46 +0000 UTC 2020-03-27 12:26:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df3f3de4b8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df3f3de4b8 a108b05d-b649-495a-93e5-01a9e4bc48af 27763 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df4974132d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df4974132d 2f6e8f98-59b3-4496-b9f2-a8dc4477e8ab 27768 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df49744a4b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df49744a4b fd8935d4-4435-4cad-9d39-cb71a4fa53d9 27767 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df49747747  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df49747747 2915deb6-df24-4d97-9ba8-1ad5cf46c988 27769 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-qxvd status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df4ae5b2bb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df4ae5b2bb 89dbd305-82c8-4b3d-bac0-a8cf3850282c 27770 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeNotReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df5873ada3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df5873ada3 13260f21-0ede-4e53-85f9-8b9c42bd1842 27771 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160028df5aef7fe9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160028df5aef7fe9 59559150-7fb1-4f4f-a94a-b7d12c86eaa1 27772 0 2020-03-27 12:26:48 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } NodeReady Node bootstrap-e2e-minion-group-qxvd status is now: NodeReady {kubelet bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:26:48 +0000 UTC 2020-03-27 12:26:48 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160029f47fbb8d86  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160029f47fbb8d86 4737fc14-e84d-4536-b1e7-3d81c22906bf 28649 0 2020-03-27 12:46:38 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:46:38 +0000 UTC 2020-03-27 12:46:38 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxvd.160029f47fbbb2e8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxvd.160029f47fbbb2e8 410d905c-9b6e-4157-8abb-b0404017c265 28648 0 2020-03-27 12:46:38 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxvd bootstrap-e2e-minion-group-qxvd   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qxvd} 2020-03-27 12:46:38 +0000 UTC 2020-03-27 12:46:38 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xnw0.160029f50cb31af8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xnw0.160029f50cb31af8 1d12075d-ebaa-4c27-87ff-2c187ecfb3c5 28651 0 2020-03-27 12:46:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-xnw0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xnw0} 2020-03-27 12:46:41 +0000 UTC 2020-03-27 12:46:41 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xnw0.160029f50cb345b1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xnw0.160029f50cb345b1 eb8bfb4c-d550-4dd1-9694-a6bda9d65f65 28650 0 2020-03-27 12:46:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-xnw0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xnw0} 2020-03-27 12:46:41 +0000 UTC 2020-03-27 12:46:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-z2c0.160029f5a27b0ac4  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-z2c0.160029f5a27b0ac4 5bc44517-a3ac-4415-815f-7df57056c60e 28653 0 2020-03-27 12:46:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-z2c0 bootstrap-e2e-minion-group-z2c0   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-z2c0} 2020-03-27 12:46:43 +0000 UTC 2020-03-27 12:46:43 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-z2c0.160029f5a27b2c52  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-z2c0.160029f5a27b2c52 3a85d75c-f8ae-4a81-923e-fe3f4a0df0dc 28652 0 2020-03-27 12:46:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-z2c0 bootstrap-e2e-minion-group-z2c0   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-z2c0} 2020-03-27 12:46:43 +0000 UTC 2020-03-27 12:46:43 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [sig-storage] HostPath should give a volume the correct mode [LinuxOnly] [NodeConformance] [Conformance] 11s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\sHostPath\sshould\sgive\sa\svolume\sthe\scorrect\smode\s\[LinuxOnly\]\s\[NodeConformance\]\s\[Conformance\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:698
Mar 27 05:40:44.158: Unexpected error:
    <*errors.errorString | 0xc004d4f8b0>: {
        s: "expected \"mode of file \\\"/test-volume\\\": dtrwxrwx\" in container output: Expected\n    <string>: mount type of \"/test-volume\": tmpfs\n    mode of file \"/test-volume\": dgtrwxrwxrwx\n    \nto contain substring\n    <string>: mode of file \"/test-volume\": dtrwxrwx",
    }
    expected "mode of file \"/test-volume\": dtrwxrwx" in container output: Expected
        <string>: mount type of "/test-volume": tmpfs
        mode of file "/test-volume": dgtrwxrwxrwx
        
    to contain substring
        <string>: mode of file "/test-volume": dtrwxrwx
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:1667
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 5m46s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Mar 26 23:34:51.290: Unexpected error:
    <*errors.errorString | 0xc0029e6ac0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.20+12ff0e52eca73e]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-g277\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-xnw0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-z2c0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.20+12ff0e52eca73e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m     v1.17.5-beta.0.20+12ff0e52eca73e\\nbootstrap-e2e-minion-group-g277   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-xnw0   Ready                      <none>   9m59s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-z2c0   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.20+12ff0e52eca73e\\\"\\nname: \\\"bootstrap-e2e-minion-group-g277\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-xnw0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-z2c0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\n\", stderr \"Project: gce-cvm-upg-lat-1-4-ctl-skew\\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-g277 bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-z2c0\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 34.82.173.66; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.173.66  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: gce-cvm-upg-lat-1-4-ctl-skew\\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.20+12ff0e52eca73e]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-g277\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-xnw0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-z2c0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.20+12ff0e52eca73e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m     v1.17.5-beta.0.20+12ff0e52eca73e\nbootstrap-e2e-minion-group-g277   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-xnw0   Ready                      <none>   9m59s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-z2c0   Ready                      <none>   10m     v1.16.9-beta.0.17+bb616cba1cc983\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.20+12ff0e52eca73e\"\nname: \"bootstrap-e2e-minion-group-g277\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-xnw0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-z2c0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\n", stderr "Project: gce-cvm-upg-lat-1-4-ctl-skew\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-g277 bootstrap-e2e-minion-group-xnw0 bootstrap-e2e-minion-group-z2c0\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 34.82.173.66; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/gce-cvm-upg-lat-1-4-ctl-skew/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   34.82.173.66  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: gce-cvm-upg-lat-1-4-ctl-skew\nNetwork Project: gce-cvm-upg-lat-1-4-ctl-skew\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106 +0x12c
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc00338d200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc0005dadc0, 0x7757ca0, 0xc, 0xc, 0xc000724d50, 0xc000cf84b0, 0x0, 0xc001e351a0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:479 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:109 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc002a8a100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:110 +0x30a
k8s.io/kubernetes/test/e2e.TestE2E(0xc002a8a100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:112 +0x2b
testing.tRunner(0xc002a8a100, 0x49f96a0)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Test 14h35m

error during ./hack/ginkgo-e2e.sh --kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 15h0m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


UpgradeTest 18m30s

error during kubetest --test --test_args=--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/k8s-stable1 --report-dir=/workspace/_artifacts --disable-log-dump=true --report-prefix=upgrade --check-version-skew=false: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 940 Passed Tests