This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 7 failed / 918 succeeded
Started2020-03-24 10:24
Elapsed15h11m
Revision
Builder9f60934b-6db9-11ea-9aaa-b24a98fe9858
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/1d3895ce-05ce-484a-a969-82951ecd0e26/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/1d3895ce-05ce-484a-a969-82951ecd0e26/targets/test
infra-commit4d97550ca
job-versionv1.16.9-beta.0.7+5116ee4b159565
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.7+5116ee4b159565

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 17m39s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 24 10:39:03.814: Unexpected error:
    <*errors.errorString | 0xc0007594a0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-2s11\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mzgr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-xr6f\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m3s    v1.17.5-beta.0.11+4dfcd1cc87879f\\nbootstrap-e2e-minion-group-2s11   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-mzgr   Ready                      <none>   9m5s    v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-xr6f   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.11+4dfcd1cc87879f\\\"\\nname: \\\"bootstrap-e2e-minion-group-2s11\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mzgr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-xr6f\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-ubuntu-serial\\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-2s11 bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-xr6f\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.58.133; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.58.133  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-ubuntu-serial\\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-2s11\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mzgr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-xr6f\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m3s    v1.17.5-beta.0.11+4dfcd1cc87879f\nbootstrap-e2e-minion-group-2s11   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-mzgr   Ready                      <none>   9m5s    v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-xr6f   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \netcd-1               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.11+4dfcd1cc87879f\"\nname: \"bootstrap-e2e-minion-group-2s11\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mzgr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-xr6f\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\n", stderr "Project: k8s-jkns-e2e-gce-ubuntu-serial\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-2s11 bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-xr6f\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.58.133; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.58.133  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-ubuntu-serial\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m17s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 24 16:59:26.130: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc00336cb40>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-mzgr.15ff4bf309beaa7c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mzgr.15ff4bf309beaa7c 19870562-8aba-41f3-8874-97d4c4a9a175 16268 0 2020-03-24 16:58:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-mzgr   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mzgr} 2020-03-24 16:58:21 +0000 UTC 2020-03-24 16:58:21 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mzgr.15ff4bf309bed102  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mzgr.15ff4bf309bed102 10646055-db29-48d9-ae63-d445d5debefb 16267 0 2020-03-24 16:58:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-mzgr   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mzgr} 2020-03-24 16:58:21 +0000 UTC 2020-03-24 16:58:21 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff40189f8ed359  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff40189f8ed359 8113042d-5e00-4e94-b631-17f6937deee9 13911 0 2020-03-24 16:08:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 13:21:08 +0000 UTC 2020-03-24 16:08:27 +0000 UTC 3 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4939654c84a0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4939654c84a0 2220def9-cc9a-4618-a8b1-b7ee6aeb9ffd 13910 0 2020-03-24 16:08:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w 35122393-851d-4c85-9189-db6005564d88   } NodeNotReady Node bootstrap-e2e-minion-group-n49w status is now: NodeNotReady {node-controller } 2020-03-24 16:08:24 +0000 UTC 2020-03-24 16:08:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a1ed6c160  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a1ed6c160 63a82e63-14d7-42a0-b54e-dd77e9a71fcd 13912 0 2020-03-24 16:08:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bee240  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bee240 c0b5a557-54d2-4a34-9a2c-1895e498910a 13916 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-n49w status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bf375a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bf375a 8632f72e-dd26-4fce-a3b9-9de63115f5aa 13917 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-n49w status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bf6c85  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bf6c85 d985f571-6b2d-407a-9557-a7a5f4b59555 13918 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-n49w status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a2b4e6e03  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a2b4e6e03 3108c38a-03e6-4105-bee5-6d6928dca449 13919 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeNotReady Node bootstrap-e2e-minion-group-n49w status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a35e8e5c2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a35e8e5c2 6b08ad35-6c60-4baf-ae16-0538ebcbd295 13920 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a37d0be1a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a37d0be1a fa30d083-7d28-4fbe-bc1d-755d6ab3f688 13921 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeReady Node bootstrap-e2e-minion-group-n49w status is now: NodeReady {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4bf3a08e613e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4bf3a08e613e 70526dae-3d39-42d1-a9cc-574721cb9ade 16270 0 2020-03-24 16:58:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 16:58:23 +0000 UTC 2020-03-24 16:58:23 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4bf3a08e8016  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4bf3a08e8016 f2ee4b6c-96dc-4381-9d2b-c870998326a7 16269 0 2020-03-24 16:58:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 16:58:23 +0000 UTC 2020-03-24 16:58:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xr6f.15ff4bf431a7dfb1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xr6f.15ff4bf431a7dfb1 fa90d0eb-47c1-44db-a0a1-343ffcb4c5a3 16272 0 2020-03-24 16:58:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xr6f bootstrap-e2e-minion-group-xr6f   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xr6f} 2020-03-24 16:58:26 +0000 UTC 2020-03-24 16:58:26 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xr6f.15ff4bf431a8380b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xr6f.15ff4bf431a8380b 69454c69-e794-42f1-a12b-7de6fff35803 16271 0 2020-03-24 16:58:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xr6f bootstrap-e2e-minion-group-xr6f   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xr6f} 2020-03-24 16:58:26 +0000 UTC 2020-03-24 16:58:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-mzgr.15ff4bf309beaa7c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mzgr.15ff4bf309beaa7c 19870562-8aba-41f3-8874-97d4c4a9a175 16268 0 2020-03-24 16:58:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-mzgr   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-mzgr} 2020-03-24 16:58:21 +0000 UTC 2020-03-24 16:58:21 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-mzgr.15ff4bf309bed102  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-mzgr.15ff4bf309bed102 10646055-db29-48d9-ae63-d445d5debefb 16267 0 2020-03-24 16:58:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-mzgr   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-mzgr} 2020-03-24 16:58:21 +0000 UTC 2020-03-24 16:58:21 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff40189f8ed359  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff40189f8ed359 8113042d-5e00-4e94-b631-17f6937deee9 13911 0 2020-03-24 16:08:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 13:21:08 +0000 UTC 2020-03-24 16:08:27 +0000 UTC 3 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4939654c84a0  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4939654c84a0 2220def9-cc9a-4618-a8b1-b7ee6aeb9ffd 13910 0 2020-03-24 16:08:24 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w 35122393-851d-4c85-9189-db6005564d88   } NodeNotReady Node bootstrap-e2e-minion-group-n49w status is now: NodeNotReady {node-controller } 2020-03-24 16:08:24 +0000 UTC 2020-03-24 16:08:24 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a1ed6c160  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a1ed6c160 63a82e63-14d7-42a0-b54e-dd77e9a71fcd 13912 0 2020-03-24 16:08:27 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:27 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bee240  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bee240 c0b5a557-54d2-4a34-9a2c-1895e498910a 13916 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-n49w status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bf375a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bf375a 8632f72e-dd26-4fce-a3b9-9de63115f5aa 13917 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-n49w status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a28bf6c85  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a28bf6c85 d985f571-6b2d-407a-9557-a7a5f4b59555 13918 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-n49w status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:27 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a2b4e6e03  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a2b4e6e03 3108c38a-03e6-4105-bee5-6d6928dca449 13919 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeNotReady Node bootstrap-e2e-minion-group-n49w status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a35e8e5c2  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a35e8e5c2 6b08ad35-6c60-4baf-ae16-0538ebcbd295 13920 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff493a37d0be1a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff493a37d0be1a fa30d083-7d28-4fbe-bc1d-755d6ab3f688 13921 0 2020-03-24 16:08:28 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } NodeReady Node bootstrap-e2e-minion-group-n49w status is now: NodeReady {kubelet bootstrap-e2e-minion-group-n49w} 2020-03-24 16:08:28 +0000 UTC 2020-03-24 16:08:28 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4bf3a08e613e  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4bf3a08e613e 70526dae-3d39-42d1-a9cc-574721cb9ade 16270 0 2020-03-24 16:58:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 16:58:23 +0000 UTC 2020-03-24 16:58:23 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-n49w.15ff4bf3a08e8016  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-n49w.15ff4bf3a08e8016 f2ee4b6c-96dc-4381-9d2b-c870998326a7 16269 0 2020-03-24 16:58:23 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-n49w bootstrap-e2e-minion-group-n49w   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-n49w} 2020-03-24 16:58:23 +0000 UTC 2020-03-24 16:58:23 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xr6f.15ff4bf431a7dfb1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xr6f.15ff4bf431a7dfb1 fa90d0eb-47c1-44db-a0a1-343ffcb4c5a3 16272 0 2020-03-24 16:58:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xr6f bootstrap-e2e-minion-group-xr6f   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-xr6f} 2020-03-24 16:58:26 +0000 UTC 2020-03-24 16:58:26 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-xr6f.15ff4bf431a8380b  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-xr6f.15ff4bf431a8380b 69454c69-e794-42f1-a12b-7de6fff35803 16271 0 2020-03-24 16:58:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-xr6f bootstrap-e2e-minion-group-xr6f   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-xr6f} 2020-03-24 16:58:26 +0000 UTC 2020-03-24 16:58:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [sig-scheduling] SchedulerPredicates [Serial] validates that NodeAffinity is respected if not matching 9.83s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-scheduling\]\sSchedulerPredicates\s\[Serial\]\svalidates\sthat\sNodeAffinity\sis\srespected\sif\snot\smatching$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/scheduling/predicates.go:411
Mar 24 18:24:55.101: Expected
    <bool>: false
to equal
    <bool>: true
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/pod/resource.go:648
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 4m54s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Mar 24 10:39:03.814: Unexpected error:
    <*errors.errorString | 0xc0007594a0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-2s11\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mzgr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-xr6f\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n........................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m3s    v1.17.5-beta.0.11+4dfcd1cc87879f\\nbootstrap-e2e-minion-group-2s11   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-mzgr   Ready                      <none>   9m5s    v1.16.9-beta.0.7+5116ee4b159565\\nbootstrap-e2e-minion-group-xr6f   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\ncontroller-manager   Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.11+4dfcd1cc87879f\\\"\\nname: \\\"bootstrap-e2e-minion-group-2s11\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-mzgr\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\nname: \\\"bootstrap-e2e-minion-group-xr6f\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.7+5116ee4b159565\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-ubuntu-serial\\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-2s11 bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-xr6f\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.58.133; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.58.133  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-ubuntu-serial\\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.11+4dfcd1cc87879f]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-2s11\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mzgr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-xr6f\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.11+4dfcd1cc87879f/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n........................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m3s    v1.17.5-beta.0.11+4dfcd1cc87879f\nbootstrap-e2e-minion-group-2s11   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-mzgr   Ready                      <none>   9m5s    v1.16.9-beta.0.7+5116ee4b159565\nbootstrap-e2e-minion-group-xr6f   Ready                      <none>   8m57s   v1.16.9-beta.0.7+5116ee4b159565\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\ncontroller-manager   Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \netcd-1               Healthy   {\"health\":\"true\"}   \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.11+4dfcd1cc87879f\"\nname: \"bootstrap-e2e-minion-group-2s11\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-mzgr\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\nname: \"bootstrap-e2e-minion-group-xr6f\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.7+5116ee4b159565\"\n", stderr "Project: k8s-jkns-e2e-gce-ubuntu-serial\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-2s11 bootstrap-e2e-minion-group-mzgr bootstrap-e2e-minion-group-xr6f\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.58.133; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-ubuntu-serial/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.58.133  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-ubuntu-serial\nNetwork Project: k8s-jkns-e2e-gce-ubuntu-serial\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106 +0x12c
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc00326b200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc0007e3cc0, 0x7757ca0, 0xc, 0xc, 0xc00079f410, 0xc000c11ef0, 0x0, 0xc00249c4e0)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:479 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:109 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001a6600)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:110 +0x30a
k8s.io/kubernetes/test/e2e.TestE2E(0xc0001a6600)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:112 +0x2b
testing.tRunner(0xc0001a6600, 0x49f9638)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Test 14h36m

error during ./hack/ginkgo-e2e.sh --kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 15h0m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


UpgradeTest 17m51s

error during kubetest --test --test_args=--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/k8s-stable1 --report-dir=/workspace/_artifacts --disable-log-dump=true --report-prefix=upgrade --check-version-skew=false: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 918 Passed Tests