This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 7 failed / 894 succeeded
Started2020-04-01 13:10
Elapsed15h11m
Revision
Builder0f51485b-741a-11ea-901a-3a7046a79cdf
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/5cf13294-cc53-4187-9b0e-dcba62dfcb2d/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/5cf13294-cc53-4187-9b0e-dcba62dfcb2d/targets/test
infra-commitc3269f485
job-versionv1.16.9-beta.0.17+bb616cba1cc983
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.17+bb616cba1cc983

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 17m24s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Apr  1 13:25:12.142: Unexpected error:
    <*errors.errorString | 0xc0007ecbf0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-2np8\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-441x\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-x7sp\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m47s   v1.17.5-beta.0.22+ea3eeb494a528e\\nbootstrap-e2e-minion-group-2np8   Ready                      <none>   9m45s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-441x   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-x7sp   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.22+ea3eeb494a528e\\\"\\nname: \\\"bootstrap-e2e-minion-group-2np8\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-441x\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-x7sp\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-2np8 bootstrap-e2e-minion-group-441x bootstrap-e2e-minion-group-x7sp\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.40.177; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.40.177  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gci-gce-alpha1-4\\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-2np8\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-441x\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-x7sp\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m47s   v1.17.5-beta.0.22+ea3eeb494a528e\nbootstrap-e2e-minion-group-2np8   Ready                      <none>   9m45s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-441x   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-x7sp   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.22+ea3eeb494a528e\"\nname: \"bootstrap-e2e-minion-group-2np8\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-441x\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-x7sp\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\n", stderr "Project: k8s-jkns-e2e-gci-gce-alpha1-4\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-2np8 bootstrap-e2e-minion-group-441x bootstrap-e2e-minion-group-x7sp\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.40.177; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.40.177  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gci-gce-alpha1-4\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m16s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Apr  1 20:23:21.260: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc001882780>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-d84v.1601c23e59b5982f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601c23e59b5982f ba883ae5-b319-4c0d-9a85-cad22320cb80 11897 0 2020-04-01 19:08:42 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v 67194f2e-69c3-4ecc-99b5-393dfc9f689e   } NodeNotReady Node bootstrap-e2e-minion-group-d84v status is now: NodeNotReady {node-controller } 2020-04-01 17:28:36 +0000 UTC 2020-04-01 20:01:33 +0000 UTC 3 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601c7dd0ee279df  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601c7dd0ee279df d0d6e963-cee8-45c8-b34e-1ff7e5d6d028 11898 0 2020-04-01 19:11:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 19:11:36 +0000 UTC 2020-04-01 20:01:34 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca975b282329  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca975b282329 d71ba081-816c-4168-a4ac-c933853ea86a 11899 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9763918ab5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9763918ab5 99b55039-9a04-4160-99c0-2a59c2c611b8 11903 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-d84v status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca976391c58d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca976391c58d 45ef9fba-1c05-4300-a16b-78cef6a7a987 11904 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-d84v status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca976391f0ac  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca976391f0ac 096e8fbf-0ad0-4b2b-b47b-e72119bec79d 11905 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-d84v status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca97661b9a22  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca97661b9a22 f36d316e-c8a8-4585-9d19-4001b8d74a15 11906 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeNotReady Node bootstrap-e2e-minion-group-d84v status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9774698d11  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9774698d11 994b62d5-c8e2-4cbd-879c-152f79fd8f16 11907 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9777beb03f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9777beb03f 01173090-a7d1-482b-b4bb-b5191c1c6ecb 11908 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeReady Node bootstrap-e2e-minion-group-d84v status is now: NodeReady {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601cbb844987e72  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601cbb844987e72 19d5b41d-b713-403f-bb38-44aa9e891c17 13077 0 2020-04-01 20:22:16 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 20:22:16 +0000 UTC 2020-04-01 20:22:16 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601cbb84498a0c1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601cbb84498a0c1 028ba802-c125-44af-9895-96a6e86804ca 13076 0 2020-04-01 20:22:16 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 20:22:16 +0000 UTC 2020-04-01 20:22:16 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fc80.1601cbb8d58ab7d6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fc80.1601cbb8d58ab7d6 772e6561-5445-4131-87e7-cf0b667fa96e 13079 0 2020-04-01 20:22:18 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-fc80 bootstrap-e2e-minion-group-fc80   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-fc80} 2020-04-01 20:22:18 +0000 UTC 2020-04-01 20:22:18 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fc80.1601cbb8d58ada16  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fc80.1601cbb8d58ada16 eb58af18-70a0-4e43-87c9-927d8835079a 13078 0 2020-04-01 20:22:18 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-fc80 bootstrap-e2e-minion-group-fc80   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-fc80} 2020-04-01 20:22:18 +0000 UTC 2020-04-01 20:22:18 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-x7sp.1601cbb96b836c79  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-x7sp.1601cbb96b836c79 78ab57b3-1f48-4367-bd48-b0a41a7c6b61 13081 0 2020-04-01 20:22:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-x7sp bootstrap-e2e-minion-group-x7sp   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-x7sp} 2020-04-01 20:22:21 +0000 UTC 2020-04-01 20:22:21 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-x7sp.1601cbb96b838a53  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-x7sp.1601cbb96b838a53 edef3026-0ac4-4966-bdf7-2765c643a08c 13080 0 2020-04-01 20:22:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-x7sp bootstrap-e2e-minion-group-x7sp   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-x7sp} 2020-04-01 20:22:21 +0000 UTC 2020-04-01 20:22:21 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-d84v.1601c23e59b5982f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601c23e59b5982f ba883ae5-b319-4c0d-9a85-cad22320cb80 11897 0 2020-04-01 19:08:42 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v 67194f2e-69c3-4ecc-99b5-393dfc9f689e   } NodeNotReady Node bootstrap-e2e-minion-group-d84v status is now: NodeNotReady {node-controller } 2020-04-01 17:28:36 +0000 UTC 2020-04-01 20:01:33 +0000 UTC 3 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601c7dd0ee279df  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601c7dd0ee279df d0d6e963-cee8-45c8-b34e-1ff7e5d6d028 11898 0 2020-04-01 19:11:36 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 19:11:36 +0000 UTC 2020-04-01 20:01:34 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca975b282329  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca975b282329 d71ba081-816c-4168-a4ac-c933853ea86a 11899 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9763918ab5  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9763918ab5 99b55039-9a04-4160-99c0-2a59c2c611b8 11903 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-d84v status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca976391c58d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca976391c58d 45ef9fba-1c05-4300-a16b-78cef6a7a987 11904 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-d84v status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca976391f0ac  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca976391f0ac 096e8fbf-0ad0-4b2b-b47b-e72119bec79d 11905 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-d84v status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca97661b9a22  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca97661b9a22 f36d316e-c8a8-4585-9d19-4001b8d74a15 11906 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeNotReady Node bootstrap-e2e-minion-group-d84v status is now: NodeNotReady {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9774698d11  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9774698d11 994b62d5-c8e2-4cbd-879c-152f79fd8f16 11907 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601ca9777beb03f  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601ca9777beb03f 01173090-a7d1-482b-b4bb-b5191c1c6ecb 11908 0 2020-04-01 20:01:35 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } NodeReady Node bootstrap-e2e-minion-group-d84v status is now: NodeReady {kubelet bootstrap-e2e-minion-group-d84v} 2020-04-01 20:01:35 +0000 UTC 2020-04-01 20:01:35 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601cbb844987e72  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601cbb844987e72 19d5b41d-b713-403f-bb38-44aa9e891c17 13077 0 2020-04-01 20:22:16 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 20:22:16 +0000 UTC 2020-04-01 20:22:16 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-d84v.1601cbb84498a0c1  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-d84v.1601cbb84498a0c1 028ba802-c125-44af-9895-96a6e86804ca 13076 0 2020-04-01 20:22:16 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-d84v bootstrap-e2e-minion-group-d84v   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-d84v} 2020-04-01 20:22:16 +0000 UTC 2020-04-01 20:22:16 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fc80.1601cbb8d58ab7d6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fc80.1601cbb8d58ab7d6 772e6561-5445-4131-87e7-cf0b667fa96e 13079 0 2020-04-01 20:22:18 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-fc80 bootstrap-e2e-minion-group-fc80   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-fc80} 2020-04-01 20:22:18 +0000 UTC 2020-04-01 20:22:18 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-fc80.1601cbb8d58ada16  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-fc80.1601cbb8d58ada16 eb58af18-70a0-4e43-87c9-927d8835079a 13078 0 2020-04-01 20:22:18 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-fc80 bootstrap-e2e-minion-group-fc80   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-fc80} 2020-04-01 20:22:18 +0000 UTC 2020-04-01 20:22:18 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-x7sp.1601cbb96b836c79  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-x7sp.1601cbb96b836c79 78ab57b3-1f48-4367-bd48-b0a41a7c6b61 13081 0 2020-04-01 20:22:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-x7sp bootstrap-e2e-minion-group-x7sp   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-x7sp} 2020-04-01 20:22:21 +0000 UTC 2020-04-01 20:22:21 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-x7sp.1601cbb96b838a53  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-x7sp.1601cbb96b838a53 edef3026-0ac4-4966-bdf7-2765c643a08c 13080 0 2020-04-01 20:22:21 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-x7sp bootstrap-e2e-minion-group-x7sp   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-x7sp} 2020-04-01 20:22:21 +0000 UTC 2020-04-01 20:22:21 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [sig-storage] [Serial] Volume metrics should create prometheus metrics for volume provisioning and attach/detach 45s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[sig\-storage\]\s\[Serial\]\sVolume\smetrics\sshould\screate\sprometheus\smetrics\sfor\svolume\sprovisioning\sand\sattach\/detach$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_metrics.go:99
Apr  2 01:21:37.726: New latency count 1 should be more than old count 1 for action volume_provision
Expected
    <int64>: 1
to be >
    <int64>: 1
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/volume_metrics.go:599
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 5m23s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Apr  1 13:25:12.142: Unexpected error:
    <*errors.errorString | 0xc0007ecbf0>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-2np8\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-441x\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-x7sp\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE     VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m47s   v1.17.5-beta.0.22+ea3eeb494a528e\\nbootstrap-e2e-minion-group-2np8   Ready                      <none>   9m45s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-441x   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\\nbootstrap-e2e-minion-group-x7sp   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\nscheduler            Healthy   ok                  \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.22+ea3eeb494a528e\\\"\\nname: \\\"bootstrap-e2e-minion-group-2np8\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-441x\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\nname: \\\"bootstrap-e2e-minion-group-x7sp\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.17+bb616cba1cc983\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-2np8 bootstrap-e2e-minion-group-441x bootstrap-e2e-minion-group-x7sp\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.247.40.177; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.40.177  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gci-gce-alpha1-4\\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.22+ea3eeb494a528e]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-2np8\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-441x\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-x7sp\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.22+ea3eeb494a528e/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE     VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   9m47s   v1.17.5-beta.0.22+ea3eeb494a528e\nbootstrap-e2e-minion-group-2np8   Ready                      <none>   9m45s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-441x   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\nbootstrap-e2e-minion-group-x7sp   Ready                      <none>   9m44s   v1.16.9-beta.0.17+bb616cba1cc983\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \nscheduler            Healthy   ok                  \netcd-0               Healthy   {\"health\":\"true\"}   \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.22+ea3eeb494a528e\"\nname: \"bootstrap-e2e-minion-group-2np8\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-441x\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\nname: \"bootstrap-e2e-minion-group-x7sp\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.17+bb616cba1cc983\"\n", stderr "Project: k8s-jkns-e2e-gci-gce-alpha1-4\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-2np8 bootstrap-e2e-minion-group-441x bootstrap-e2e-minion-group-x7sp\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.247.40.177; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gci-gce-alpha1-4/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP    STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.247.40.177  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gci-gce-alpha1-4\nNetwork Project: k8s-jkns-e2e-gci-gce-alpha1-4\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106 +0x12c
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc002b3b200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc000c81e00, 0x7757ca0, 0xc, 0xc, 0xc000859f80, 0xc001c5e000, 0x0, 0xc001464060)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:479 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:109 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0002ff500)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:110 +0x30a
k8s.io/kubernetes/test/e2e.TestE2E(0xc0002ff500)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:112 +0x2b
testing.tRunner(0xc0002ff500, 0x49f96a0)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Test 14h37m

error during ./hack/ginkgo-e2e.sh --kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 15h0m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


UpgradeTest 17m52s

error during kubetest --test --test_args=--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/k8s-stable1 --report-dir=/workspace/_artifacts --disable-log-dump=true --report-prefix=upgrade --check-version-skew=false: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 894 Passed Tests