This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 6 failed / 894 succeeded
Started2020-03-23 19:11
Elapsed15h11m
Revision
Builderf0038dfe-6d39-11ea-9aaa-b24a98fe9858
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/44785430-b1bd-41f0-a055-c993791e6577/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/44785430-b1bd-41f0-a055-c993791e6577/targets/test
infra-commitaf68b02cf
job-versionv1.16.9-beta.0.7+5116ee4b159565
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.7+5116ee4b159565

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 18m23s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 23 19:27:41.430: Unexpected error:
    <*errors.errorString | 0xc00263eb90>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-18sc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-3pj0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-65f9\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m54s   v1.17.5-beta.0.11+4dfcd1cc87879f\\nbootstrap-e2e-minion-group-18sc   Ready                      <none>   9m59s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-3pj0   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-65f9   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.11+4dfcd1cc87879f\\\"\\nname: \\\"bootstrap-e2e-minion-group-18sc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-3pj0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-65f9\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\n\", stderr \"Project: k8s-jkns-gce-alpha-1-6\\nNetwork Project: k8s-jkns-gce-alpha-1-6\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-18sc bootstrap-e2e-minion-group-3pj0 bootstrap-e2e-minion-group-65f9\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.233.232.84; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.233.232.84  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-gce-alpha-1-6\\nNetwork Project: k8s-jkns-gce-alpha-1-6\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-18sc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-3pj0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-65f9\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m54s   v1.17.5-beta.0.11+4dfcd1cc87879f\nbootstrap-e2e-minion-group-18sc   Ready                      <none>   9m59s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-3pj0   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-65f9   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.11+4dfcd1cc87879f\"\nname: \"bootstrap-e2e-minion-group-18sc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-3pj0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-65f9\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\n", stderr "Project: k8s-jkns-gce-alpha-1-6\nNetwork Project: k8s-jkns-gce-alpha-1-6\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-18sc bootstrap-e2e-minion-group-3pj0 bootstrap-e2e-minion-group-65f9\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.233.232.84; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.233.232.84  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-gce-alpha-1-6\nNetwork Project: k8s-jkns-gce-alpha-1-6\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m16s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 24 09:05:29.830: Timed out after 60.002s.
Expected success, but got an error:
    <*errors.errorString | 0xc0055ae9c0>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-gggg.15ff2a1a5fbb46a7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2a1a5fbb46a7 87b406fd-19d9-42af-87ac-0b77344511cb 27554 0 2020-03-24 06:38:06 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 06:38:06 +0000 UTC 2020-03-24 08:14:49 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2cc90374baca  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2cc90374baca 36ff3e87-4b13-4515-8ea3-51c35ad013c1 27553 0 2020-03-24 07:27:15 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg 81d575c5-2e22-41bc-9b10-0431ea9f8ea2   } NodeNotReady Node bootstrap-e2e-minion-group-gggg status is now: NodeNotReady {node-controller } 2020-03-24 07:27:15 +0000 UTC 2020-03-24 08:14:46 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f619148d9f5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f619148d9f5 66af9b76-067b-47fc-b52e-a099ca19f8e8 27555 0 2020-03-24 08:14:49 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:49 +0000 UTC 2020-03-24 08:14:49 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a32ae9f7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a32ae9f7 0519d5b8-b829-4955-896d-b9291d5cb493 27559 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-gggg status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a32f5abb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a32f5abb 63b4161c-6506-4a88-a89b-61a12c18d070 27560 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-gggg status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a333c0d0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a333c0d0 341a4a0e-21ee-4e44-a948-d8a73ad98999 27561 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-gggg status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a5d6907a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a5d6907a e426f920-7f47-4c67-9b7a-5a4bfa455884 27562 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeNotReady Node bootstrap-e2e-minion-group-gggg status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61b7111f3a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61b7111f3a 601eaf88-75fc-4457-8b0f-4e7c6f8d3b51 27563 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61b8da9734  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61b8da9734 1501b8cc-9924-478d-8a50-6586049f9831 27564 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeReady Node bootstrap-e2e-minion-group-gggg status is now: NodeReady {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff32162bc5bbb8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff32162bc5bbb8 88bf39eb-a5cf-4be7-b92e-7821956ec214 28864 0 2020-03-24 09:04:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 09:04:24 +0000 UTC 2020-03-24 09:04:24 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff32162bc5da49  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff32162bc5da49 e5fdf50b-230f-46e1-b94c-93adcd22b3b2 28863 0 2020-03-24 09:04:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 09:04:24 +0000 UTC 2020-03-24 09:04:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m2tq.15ff3216c43329b7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m2tq.15ff3216c43329b7 32b6c486-60e9-48b7-b01c-c1404244d19a 28865 0 2020-03-24 09:04:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m2tq bootstrap-e2e-minion-group-m2tq   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-m2tq} 2020-03-24 09:04:27 +0000 UTC 2020-03-24 09:04:27 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m2tq.15ff3216c43386d9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m2tq.15ff3216c43386d9 e7a7cbdf-3adc-4a76-9f3c-a9ba699f77d8 28866 0 2020-03-24 09:04:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m2tq bootstrap-e2e-minion-group-m2tq   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-m2tq} 2020-03-24 09:04:27 +0000 UTC 2020-03-24 09:04:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mhfw.15ff321753194124  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mhfw.15ff321753194124 ada17d3a-d491-43d2-9338-4058daff685d 28868 0 2020-03-24 09:04:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mhfw bootstrap-e2e-minion-group-mhfw   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mhfw} 2020-03-24 09:04:29 +0000 UTC 2020-03-24 09:04:29 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mhfw.15ff32175319631c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mhfw.15ff32175319631c 3ac1153c-bda7-48de-a5c1-3dde292994b3 28867 0 2020-03-24 09:04:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mhfw bootstrap-e2e-minion-group-mhfw   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mhfw} 2020-03-24 09:04:29 +0000 UTC 2020-03-24 09:04:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-gggg.15ff2a1a5fbb46a7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2a1a5fbb46a7 87b406fd-19d9-42af-87ac-0b77344511cb 27554 0 2020-03-24 06:38:06 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 06:38:06 +0000 UTC 2020-03-24 08:14:49 +0000 UTC 5 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2cc90374baca  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2cc90374baca 36ff3e87-4b13-4515-8ea3-51c35ad013c1 27553 0 2020-03-24 07:27:15 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg 81d575c5-2e22-41bc-9b10-0431ea9f8ea2   } NodeNotReady Node bootstrap-e2e-minion-group-gggg status is now: NodeNotReady {node-controller } 2020-03-24 07:27:15 +0000 UTC 2020-03-24 08:14:46 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f619148d9f5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f619148d9f5 66af9b76-067b-47fc-b52e-a099ca19f8e8 27555 0 2020-03-24 08:14:49 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:49 +0000 UTC 2020-03-24 08:14:49 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a32ae9f7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a32ae9f7 0519d5b8-b829-4955-896d-b9291d5cb493 27559 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-gggg status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a32f5abb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a32f5abb 63b4161c-6506-4a88-a89b-61a12c18d070 27560 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-gggg status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a333c0d0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a333c0d0 341a4a0e-21ee-4e44-a948-d8a73ad98999 27561 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-gggg status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61a5d6907a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61a5d6907a e426f920-7f47-4c67-9b7a-5a4bfa455884 27562 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeNotReady Node bootstrap-e2e-minion-group-gggg status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61b7111f3a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61b7111f3a 601eaf88-75fc-4457-8b0f-4e7c6f8d3b51 27563 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff2f61b8da9734  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff2f61b8da9734 1501b8cc-9924-478d-8a50-6586049f9831 27564 0 2020-03-24 08:14:50 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } NodeReady Node bootstrap-e2e-minion-group-gggg status is now: NodeReady {kubelet bootstrap-e2e-minion-group-gggg} 2020-03-24 08:14:50 +0000 UTC 2020-03-24 08:14:50 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff32162bc5bbb8  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff32162bc5bbb8 88bf39eb-a5cf-4be7-b92e-7821956ec214 28864 0 2020-03-24 09:04:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 09:04:24 +0000 UTC 2020-03-24 09:04:24 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gggg.15ff32162bc5da49  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gggg.15ff32162bc5da49 e5fdf50b-230f-46e1-b94c-93adcd22b3b2 28863 0 2020-03-24 09:04:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gggg bootstrap-e2e-minion-group-gggg   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-gggg} 2020-03-24 09:04:24 +0000 UTC 2020-03-24 09:04:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m2tq.15ff3216c43329b7  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m2tq.15ff3216c43329b7 32b6c486-60e9-48b7-b01c-c1404244d19a 28865 0 2020-03-24 09:04:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m2tq bootstrap-e2e-minion-group-m2tq   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-m2tq} 2020-03-24 09:04:27 +0000 UTC 2020-03-24 09:04:27 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-m2tq.15ff3216c43386d9  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-m2tq.15ff3216c43386d9 e7a7cbdf-3adc-4a76-9f3c-a9ba699f77d8 28866 0 2020-03-24 09:04:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-m2tq bootstrap-e2e-minion-group-m2tq   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-m2tq} 2020-03-24 09:04:27 +0000 UTC 2020-03-24 09:04:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mhfw.15ff321753194124  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mhfw.15ff321753194124 ada17d3a-d491-43d2-9338-4058daff685d 28868 0 2020-03-24 09:04:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mhfw bootstrap-e2e-minion-group-mhfw   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mhfw} 2020-03-24 09:04:29 +0000 UTC 2020-03-24 09:04:29 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mhfw.15ff32175319631c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mhfw.15ff32175319631c 3ac1153c-bda7-48de-a5c1-3dde292994b3 28867 0 2020-03-24 09:04:29 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mhfw bootstrap-e2e-minion-group-mhfw   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mhfw} 2020-03-24 09:04:29 +0000 UTC 2020-03-24 09:04:29 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 6m6s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Mar 23 19:27:41.430: Unexpected error:
    <*errors.errorString | 0xc00263eb90>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-18sc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-3pj0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-65f9\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m54s   v1.17.5-beta.0.11+4dfcd1cc87879f\\nbootstrap-e2e-minion-group-18sc   Ready                      <none>   9m59s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-3pj0   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-65f9   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.11+4dfcd1cc87879f\\\"\\nname: \\\"bootstrap-e2e-minion-group-18sc\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-3pj0\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-65f9\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\n\", stderr \"Project: k8s-jkns-gce-alpha-1-6\\nNetwork Project: k8s-jkns-gce-alpha-1-6\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-18sc bootstrap-e2e-minion-group-3pj0 bootstrap-e2e-minion-group-65f9\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.233.232.84; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.233.232.84  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-gce-alpha-1-6\\nNetwork Project: k8s-jkns-gce-alpha-1-6\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-18sc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-3pj0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-65f9\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m54s   v1.17.5-beta.0.11+4dfcd1cc87879f\nbootstrap-e2e-minion-group-18sc   Ready                      <none>   9m59s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-3pj0   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-65f9   Ready                      <none>   9m58s   v1.16.9-beta.0.7+5116ee4b159565\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.11+4dfcd1cc87879f\"\nname: \"bootstrap-e2e-minion-group-18sc\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-3pj0\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-65f9\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\n", stderr "Project: k8s-jkns-gce-alpha-1-6\nNetwork Project: k8s-jkns-gce-alpha-1-6\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-18sc bootstrap-e2e-minion-group-3pj0 bootstrap-e2e-minion-group-65f9\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.233.232.84; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-gce-alpha-1-6/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.233.232.84  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-gce-alpha-1-6\nNetwork Project: k8s-jkns-gce-alpha-1-6\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106 +0x12c
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc002bef200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc000b7eb40, 0x7757ca0, 0xc, 0xc, 0xc000a43740, 0xc0011c33b0, 0x0, 0xc002351da0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:479 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:109 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc000e82100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:110 +0x30a
k8s.io/kubernetes/test/e2e.TestE2E(0xc000e82100)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:112 +0x2b
testing.tRunner(0xc000e82100, 0x49f9638)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Test 14h34m

error during ./hack/ginkgo-e2e.sh --kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 15h0m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


UpgradeTest 18m40s

error during kubetest --test --test_args=--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/k8s-stable1 --report-dir=/workspace/_artifacts --disable-log-dump=true --report-prefix=upgrade --check-version-skew=false: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 894 Passed Tests