This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 6 failed / 922 succeeded
Started2020-03-13 18:26
Elapsed15h12m
Revision
Builder0af7150e-6558-11ea-a8b0-8ac1937716fb
links{u'resultstore': {u'url': u'https://source.cloud.google.com/results/invocations/4d049d9f-0e0c-4e71-872a-75c48ad8d55c/targets/test'}}
resultstorehttps://source.cloud.google.com/results/invocations/4d049d9f-0e0c-4e71-872a-75c48ad8d55c/targets/test
infra-commit61481eb91
job-versionv1.16.9-beta.0.1+92e71139aa1639
master_os_imagecos-77-12371-175-0
node_os_imagecos-73-11647-163-0
revisionv1.16.9-beta.0.1+92e71139aa1639

Test Failures


Kubernetes e2e suite [k8s.io] [sig-cloud-provider-gcp] Upgrade [Feature:Upgrade] master upgrade should maintain a functioning cluster [Feature:MasterUpgrade] 18m21s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-cloud\-provider\-gcp\]\sUpgrade\s\[Feature\:Upgrade\]\smaster\supgrade\sshould\smaintain\sa\sfunctioning\scluster\s\[Feature\:MasterUpgrade\]$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:91
Mar 13 18:43:49.865: Unexpected error:
    <*errors.errorString | 0xc002c9c320>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.1+106c255ad7ab80]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-4hfn\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-d2sw\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-k0f2\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.1+106c255ad7ab80/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE   VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m   v1.17.5-beta.0.1+106c255ad7ab80\\nbootstrap-e2e-minion-group-4hfn   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nbootstrap-e2e-minion-group-d2sw   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nbootstrap-e2e-minion-group-k0f2   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.1+106c255ad7ab80\\\"\\nname: \\\"bootstrap-e2e-minion-group-4hfn\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-d2sw\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-k0f2\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-slow-1-2\\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-4hfn bootstrap-e2e-minion-group-d2sw bootstrap-e2e-minion-group-k0f2\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.230.57.24; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.230.57.24  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-slow-1-2\\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.1+106c255ad7ab80]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-4hfn\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-d2sw\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-k0f2\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.1+106c255ad7ab80/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE   VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m   v1.17.5-beta.0.1+106c255ad7ab80\nbootstrap-e2e-minion-group-4hfn   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nbootstrap-e2e-minion-group-d2sw   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nbootstrap-e2e-minion-group-k0f2   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.1+106c255ad7ab80\"\nname: \"bootstrap-e2e-minion-group-4hfn\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-d2sw\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-k0f2\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\n", stderr "Project: k8s-jkns-e2e-gce-slow-1-2\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-4hfn bootstrap-e2e-minion-group-d2sw bootstrap-e2e-minion-group-k0f2\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.230.57.24; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.230.57.24  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-slow-1-2\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106
				
				Click to see stdout/stderrfrom junit_upgrade01.xml

Filter through log files | View test history on testgrid


Kubernetes e2e suite [k8s.io] [sig-node] NodeProblemDetector [DisabledForLargeClusters] should run without error 2m17s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Kubernetes\se2e\ssuite\s\[k8s\.io\]\s\[sig\-node\]\sNodeProblemDetector\s\[DisabledForLargeClusters\]\sshould\srun\swithout\serror$'
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:58
Mar 14 09:02:47.009: Timed out after 60.001s.
Expected success, but got an error:
    <*errors.errorString | 0xc006208660>: {
        s: "Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-gvqt.15fc1ec9f9d1a944  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1ec9f9d1a944 40a927ce-bf2c-4e1c-9a94-ebda545f64c8 31688 0 2020-03-14 08:37:01 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } NodeNotReady Node bootstrap-e2e-minion-group-gvqt status is now: NodeNotReady {node-controller } 2020-03-14 08:37:01 +0000 UTC 2020-03-14 08:37:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gvqt.15fc1edda7725742  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1edda7725742 82f83041-9def-4cec-9018-529c6e3f5cf6 31691 0 2020-03-14 08:38:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } Deleting node bootstrap-e2e-minion-group-gvqt because it does not exist in the cloud provider Node bootstrap-e2e-minion-group-gvqt event: DeletingNode {cloud-node-lifecycle-controller } 2020-03-14 08:38:26 +0000 UTC 2020-03-14 08:38:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gvqt.15fc1eddc8e322ca  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1eddc8e322ca fd5f59fe-5cc0-4f19-a728-8c59b4931c12 31692 0 2020-03-14 08:38:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } RemovingNode Node bootstrap-e2e-minion-group-gvqt event: Removing Node bootstrap-e2e-minion-group-gvqt from Controller {node-controller } 2020-03-14 08:38:26 +0000 UTC 2020-03-14 08:38:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-k0f2.15fc20229a59d164  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-k0f2.15fc20229a59d164 67e6fa5d-39e0-40cb-a5fc-632bf46ddf05 32529 0 2020-03-14 09:01:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-k0f2 bootstrap-e2e-minion-group-k0f2   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-k0f2} 2020-03-14 09:01:41 +0000 UTC 2020-03-14 09:01:41 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-k0f2.15fc20229a59f1fb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-k0f2.15fc20229a59f1fb 53cf1d69-0a37-4f43-a696-bbd2fd528a50 32528 0 2020-03-14 09:01:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-k0f2 bootstrap-e2e-minion-group-k0f2   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-k0f2} 2020-03-14 09:01:41 +0000 UTC 2020-03-14 09:01:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qc0d.15fc202333586d84  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qc0d.15fc202333586d84 56a52abe-8192-4e1f-b37f-6d2ef5bd393e 32531 0 2020-03-14 09:01:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qc0d bootstrap-e2e-minion-group-qc0d   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qc0d} 2020-03-14 09:01:44 +0000 UTC 2020-03-14 09:01:44 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qc0d.15fc20233358982a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qc0d.15fc20233358982a c431a47c-c5c2-45bd-8ee1-1784034cc135 32530 0 2020-03-14 09:01:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qc0d bootstrap-e2e-minion-group-qc0d   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qc0d} 2020-03-14 09:01:44 +0000 UTC 2020-03-14 09:01:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb73f846481  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb73f846481 4c122505-1b67-4bc6-bb34-76a364095616 31645 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74930c515  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74930c515 c4af1fb7-65af-4ae8-950d-2519c97f1dd6 31652 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb749310838  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb749310838 3a90fbfe-f7ab-4185-88c2-b37ed7ddde66 31655 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb749313268  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb749313268 ac57ed1c-b5e6-4799-ade3-b0b4797010d7 31654 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74e9b586d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74e9b586d 515fcf69-57df-4765-97b3-d53a1757f417 31656 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74ee5ec49  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74ee5ec49 eadb302e-6613-4f38-ba1e-f00c43c0b754 31657 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeReady Node bootstrap-e2e-minion-group-qxz1 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb7586a08db  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb7586a08db a376a29f-fec1-44bd-bde5-915e3782f0a1 31658 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 f006903c-24a5-47f1-9c7c-e571287c9de3   } RegisteredNode Node bootstrap-e2e-minion-group-qxz1 event: Registered Node bootstrap-e2e-minion-group-qxz1 in Controller {node-controller } 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb7d090916c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb7d090916c c2bf0c72-4bcd-4ec7-b705-808bc172113d 31663 0 2020-03-14 08:35:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } Starting Starting kube-proxy. {kube-proxy bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:43 +0000 UTC 2020-03-14 08:35:43 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c22b6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c22b6 fd802d54-25ba-40b6-a23a-7cd494330478 31667 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } ContainerdStart Starting containerd container runtime... {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c6b47  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c6b47 26751095-49dd-4a74-ad70-be76b5fa0a07 31669 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } DockerStart Starting Docker Application Container Engine... {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83e2542a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83e2542a3 fe0f6543-591e-4cea-a34c-599c0c634506 31668 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc2023c1994073  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc2023c1994073 b4863565-f8c9-497e-b81a-bf82c3f3d425 32533 0 2020-03-14 09:01:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 09:01:46 +0000 UTC 2020-03-14 09:01:46 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc2023c1995f14  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc2023c1995f14 795ffe92-34ca-4c98-bcd5-2187b0618b3e 32532 0 2020-03-14 09:01:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 09:01:46 +0000 UTC 2020-03-14 09:01:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]",
    }
    Event KubeletStart does not exist: [{{ } {bootstrap-e2e-minion-group-gvqt.15fc1ec9f9d1a944  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1ec9f9d1a944 40a927ce-bf2c-4e1c-9a94-ebda545f64c8 31688 0 2020-03-14 08:37:01 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } NodeNotReady Node bootstrap-e2e-minion-group-gvqt status is now: NodeNotReady {node-controller } 2020-03-14 08:37:01 +0000 UTC 2020-03-14 08:37:01 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gvqt.15fc1edda7725742  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1edda7725742 82f83041-9def-4cec-9018-529c6e3f5cf6 31691 0 2020-03-14 08:38:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } Deleting node bootstrap-e2e-minion-group-gvqt because it does not exist in the cloud provider Node bootstrap-e2e-minion-group-gvqt event: DeletingNode {cloud-node-lifecycle-controller } 2020-03-14 08:38:26 +0000 UTC 2020-03-14 08:38:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-gvqt.15fc1eddc8e322ca  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-gvqt.15fc1eddc8e322ca fd5f59fe-5cc0-4f19-a728-8c59b4931c12 31692 0 2020-03-14 08:38:26 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-gvqt 79a52851-3e02-4485-8cdd-20cdb67022b9   } RemovingNode Node bootstrap-e2e-minion-group-gvqt event: Removing Node bootstrap-e2e-minion-group-gvqt from Controller {node-controller } 2020-03-14 08:38:26 +0000 UTC 2020-03-14 08:38:26 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-k0f2.15fc20229a59d164  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-k0f2.15fc20229a59d164 67e6fa5d-39e0-40cb-a5fc-632bf46ddf05 32529 0 2020-03-14 09:01:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-k0f2 bootstrap-e2e-minion-group-k0f2   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-k0f2} 2020-03-14 09:01:41 +0000 UTC 2020-03-14 09:01:41 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-k0f2.15fc20229a59f1fb  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-k0f2.15fc20229a59f1fb 53cf1d69-0a37-4f43-a696-bbd2fd528a50 32528 0 2020-03-14 09:01:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-k0f2 bootstrap-e2e-minion-group-k0f2   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-k0f2} 2020-03-14 09:01:41 +0000 UTC 2020-03-14 09:01:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qc0d.15fc202333586d84  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qc0d.15fc202333586d84 56a52abe-8192-4e1f-b37f-6d2ef5bd393e 32531 0 2020-03-14 09:01:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qc0d bootstrap-e2e-minion-group-qc0d   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qc0d} 2020-03-14 09:01:44 +0000 UTC 2020-03-14 09:01:44 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qc0d.15fc20233358982a  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qc0d.15fc20233358982a c431a47c-c5c2-45bd-8ee1-1784034cc135 32530 0 2020-03-14 09:01:44 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qc0d bootstrap-e2e-minion-group-qc0d   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qc0d} 2020-03-14 09:01:44 +0000 UTC 2020-03-14 09:01:44 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb73f846481  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb73f846481 4c122505-1b67-4bc6-bb34-76a364095616 31645 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } Starting Starting kubelet. {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74930c515  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74930c515 c4af1fb7-65af-4ae8-950d-2519c97f1dd6 31652 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasSufficientMemory Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasSufficientMemory {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb749310838  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb749310838 3a90fbfe-f7ab-4185-88c2-b37ed7ddde66 31655 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasNoDiskPressure Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasNoDiskPressure {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb749313268  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb749313268 ac57ed1c-b5e6-4799-ade3-b0b4797010d7 31654 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeHasSufficientPID Node bootstrap-e2e-minion-group-qxz1 status is now: NodeHasSufficientPID {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 2 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74e9b586d  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74e9b586d 515fcf69-57df-4765-97b3-d53a1757f417 31656 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeAllocatableEnforced Updated Node Allocatable limit across pods {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb74ee5ec49  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb74ee5ec49 eadb302e-6613-4f38-ba1e-f00c43c0b754 31657 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } NodeReady Node bootstrap-e2e-minion-group-qxz1 status is now: NodeReady {kubelet bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb7586a08db  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb7586a08db a376a29f-fec1-44bd-bde5-915e3782f0a1 31658 0 2020-03-14 08:35:41 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 f006903c-24a5-47f1-9c7c-e571287c9de3   } RegisteredNode Node bootstrap-e2e-minion-group-qxz1 event: Registered Node bootstrap-e2e-minion-group-qxz1 in Controller {node-controller } 2020-03-14 08:35:41 +0000 UTC 2020-03-14 08:35:41 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb7d090916c  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb7d090916c c2bf0c72-4bcd-4ec7-b705-808bc172113d 31663 0 2020-03-14 08:35:43 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } Starting Starting kube-proxy. {kube-proxy bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:43 +0000 UTC 2020-03-14 08:35:43 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c22b6  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c22b6 fd802d54-25ba-40b6-a23a-7cd494330478 31667 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } ContainerdStart Starting containerd container runtime... {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c6b47  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83c9c6b47 26751095-49dd-4a74-ad70-be76b5fa0a07 31669 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } DockerStart Starting Docker Application Container Engine... {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 2 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc1eb83e2542a3  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc1eb83e2542a3 fe0f6543-591e-4cea-a34c-599c0c634506 31668 0 2020-03-14 08:35:45 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } KubeletStart Started Kubernetes kubelet. {systemd-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 08:35:45 +0000 UTC 2020-03-14 08:35:45 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc2023c1994073  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc2023c1994073 b4863565-f8c9-497e-b81a-bf82c3f3d425 32533 0 2020-03-14 09:01:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } TaskHung kernel: INFO: task umount.aufs:21568 blocked for more than 120 seconds. {kernel-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 09:01:46 +0000 UTC 2020-03-14 09:01:46 +0000 UTC 1 Warning 0001-01-01 00:00:00 +0000 UTC nil  nil  } {{ } {bootstrap-e2e-minion-group-qxz1.15fc2023c1995f14  default /api/v1/namespaces/default/events/bootstrap-e2e-minion-group-qxz1.15fc2023c1995f14 795ffe92-34ca-4c98-bcd5-2187b0618b3e 32532 0 2020-03-14 09:01:46 +0000 UTC <nil> <nil> map[] map[] [] []  []} {Node  bootstrap-e2e-minion-group-qxz1 bootstrap-e2e-minion-group-qxz1   } AUFSUmountHung Node condition KernelDeadlock is now: True, reason: AUFSUmountHung {kernel-monitor bootstrap-e2e-minion-group-qxz1} 2020-03-14 09:01:46 +0000 UTC 2020-03-14 09:01:46 +0000 UTC 1 Normal 0001-01-01 00:00:00 +0000 UTC nil  nil  }]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/node/node_problem_detector.go:141
				
				Click to see stdout/stderrfrom junit_01.xml

Filter through log files | View test history on testgrid


Master upgrade [sig-cloud-provider-gcp] master-upgrade 5m30s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=Master\supgrade\s\[sig\-cloud\-provider\-gcp\]\smaster\-upgrade$'
Mar 13 18:43:49.865: Unexpected error:
    <*errors.errorString | 0xc002c9c320>: {
        s: "error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.1+106c255ad7ab80]; got error exit status 1, stdout \"Fetching the previously installed CoreDNS version\\n\\n***WARNING***\\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\\nbefore running this script:\\n\\n# example: pin to etcd v3.0.17\\nexport ETCD_IMAGE=3.0.17\\nexport ETCD_VERSION=3.0.17\\n\\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\\n\\n== Pre-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-4hfn\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-d2sw\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-k0f2\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.1+106c255ad7ab80/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\\n== Upgrading master environment variables. ==\\n== Waiting for new master to respond to API requests ==\\n.......................== Done ==\\nWaiting for CoreDNS to update\\nFetching the latest installed CoreDNS version\\n== Downloading the CoreDNS migration tool ==\\n== Upgrading the CoreDNS ConfigMap ==\\nconfigmap/coredns configured\\n== The CoreDNS Config has been updated ==\\n== Validating cluster post-upgrade ==\\nValidating gce cluster, MULTIZONE=\\nFound 4 node(s).\\nNAME                              STATUS                     ROLES    AGE   VERSION\\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m   v1.17.5-beta.0.1+106c255ad7ab80\\nbootstrap-e2e-minion-group-4hfn   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nbootstrap-e2e-minion-group-d2sw   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nbootstrap-e2e-minion-group-k0f2   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\\nValidate output:\\nNAME                 STATUS    MESSAGE             ERROR\\netcd-1               Healthy   {\\\"health\\\":\\\"true\\\"}   \\netcd-0               Healthy   {\\\"health\\\":\\\"true\\\"}   \\ncontroller-manager   Healthy   ok                  \\nscheduler            Healthy   ok                  \\n\\x1b[0;32mCluster validation succeeded\\x1b[0m\\n== Post-Upgrade Node OS and Kubelet Versions ==\\nname: \\\"bootstrap-e2e-master\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.17.5-beta.0.1+106c255ad7ab80\\\"\\nname: \\\"bootstrap-e2e-minion-group-4hfn\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-d2sw\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\nname: \\\"bootstrap-e2e-minion-group-k0f2\\\", osImage: \\\"Container-Optimized OS from Google\\\", kubeletVersion: \\\"v1.16.9-beta.0.1+92e71139aa1639\\\"\\n\", stderr \"Project: k8s-jkns-e2e-gce-slow-1-2\\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\\nZone: us-west1-b\\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\\nNODE_NAMES=bootstrap-e2e-minion-group-4hfn bootstrap-e2e-minion-group-d2sw bootstrap-e2e-minion-group-k0f2\\nTrying to find master named 'bootstrap-e2e-master'\\nLooking for address 'bootstrap-e2e-master-ip'\\nUsing master: bootstrap-e2e-master (external IP: 35.230.57.24; internal IP: (not set))\\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\\nWARNING: Some requests generated warnings:\\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\\n\\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.230.57.24  RUNNING\\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\\nProject: k8s-jkns-e2e-gce-slow-1-2\\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\\nZone: us-west1-b\\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\\n\"",
    }
    error running /workspace/kubernetes_skew/cluster/gce/upgrade.sh [-M v1.17.5-beta.0.1+106c255ad7ab80]; got error exit status 1, stdout "Fetching the previously installed CoreDNS version\n\n***WARNING***\nUpgrading Kubernetes with this script might result in an upgrade to a new etcd version.\nSome etcd version upgrades, such as 3.0.x to 3.1.x, DO NOT offer a downgrade path.\nTo pin the etcd version to your current one (e.g. v3.0.17), set the following variables\nbefore running this script:\n\n# example: pin to etcd v3.0.17\nexport ETCD_IMAGE=3.0.17\nexport ETCD_VERSION=3.0.17\n\nAlternatively, if you choose to allow an etcd upgrade that doesn't support downgrade,\nyou might still be able to downgrade Kubernetes by pinning to the newer etcd version.\nIn all cases, it is strongly recommended to have an etcd backup before upgrading.\n\n== Pre-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-4hfn\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-d2sw\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-k0f2\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nFound subnet for region us-west1 in network bootstrap-e2e: bootstrap-e2e\n== Upgrading master to 'https://storage.googleapis.com/kubernetes-release-dev/ci/v1.17.5-beta.0.1+106c255ad7ab80/kubernetes-server-linux-amd64.tar.gz'. Do not interrupt, deleting master instance. ==\n== Upgrading master environment variables. ==\n== Waiting for new master to respond to API requests ==\n.......................== Done ==\nWaiting for CoreDNS to update\nFetching the latest installed CoreDNS version\n== Downloading the CoreDNS migration tool ==\n== Upgrading the CoreDNS ConfigMap ==\nconfigmap/coredns configured\n== The CoreDNS Config has been updated ==\n== Validating cluster post-upgrade ==\nValidating gce cluster, MULTIZONE=\nFound 4 node(s).\nNAME                              STATUS                     ROLES    AGE   VERSION\nbootstrap-e2e-master              Ready,SchedulingDisabled   <none>   10m   v1.17.5-beta.0.1+106c255ad7ab80\nbootstrap-e2e-minion-group-4hfn   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nbootstrap-e2e-minion-group-d2sw   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nbootstrap-e2e-minion-group-k0f2   Ready                      <none>   10m   v1.16.9-beta.0.1+92e71139aa1639\nValidate output:\nNAME                 STATUS    MESSAGE             ERROR\netcd-1               Healthy   {\"health\":\"true\"}   \netcd-0               Healthy   {\"health\":\"true\"}   \ncontroller-manager   Healthy   ok                  \nscheduler            Healthy   ok                  \n\x1b[0;32mCluster validation succeeded\x1b[0m\n== Post-Upgrade Node OS and Kubelet Versions ==\nname: \"bootstrap-e2e-master\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.17.5-beta.0.1+106c255ad7ab80\"\nname: \"bootstrap-e2e-minion-group-4hfn\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-d2sw\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\nname: \"bootstrap-e2e-minion-group-k0f2\", osImage: \"Container-Optimized OS from Google\", kubeletVersion: \"v1.16.9-beta.0.1+92e71139aa1639\"\n", stderr "Project: k8s-jkns-e2e-gce-slow-1-2\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\nZone: us-west1-b\nINSTANCE_GROUPS=bootstrap-e2e-minion-group\nNODE_NAMES=bootstrap-e2e-minion-group-4hfn bootstrap-e2e-minion-group-d2sw bootstrap-e2e-minion-group-k0f2\nTrying to find master named 'bootstrap-e2e-master'\nLooking for address 'bootstrap-e2e-master-ip'\nUsing master: bootstrap-e2e-master (external IP: 35.230.57.24; internal IP: (not set))\nDeleted [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: You have selected a disk size of under [200GB]. This may result in poor I/O performance. For more information, see: https://developers.google.com/compute/docs/disks#performance.\nCreated [https://www.googleapis.com/compute/v1/projects/k8s-jkns-e2e-gce-slow-1-2/zones/us-west1-b/instances/bootstrap-e2e-master].\nWARNING: Some requests generated warnings:\n - Disk size: '20 GB' is larger than image size: '10 GB'. You might need to resize the root repartition manually if the operating system does not support automatic resizing. See https://cloud.google.com/compute/docs/disks/add-persistent-disk#resize_pd for details.\n - The resource 'projects/cos-cloud/global/images/cos-77-12371-175-0' is deprecated. A suggested replacement is 'projects/cos-cloud/global/images/cos-77-12371-183-0'.\n\nNAME                  ZONE        MACHINE_TYPE   PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP   STATUS\nbootstrap-e2e-master  us-west1-b  n1-standard-1               10.138.0.6   35.230.57.24  RUNNING\nWarning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply\nProject: k8s-jkns-e2e-gce-slow-1-2\nNetwork Project: k8s-jkns-e2e-gce-slow-1-2\nZone: us-west1-b\n/workspace/kubernetes_skew/cluster/gce/upgrade.sh: line 452: download_dir: unbound variable\n"
occurred

k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:106 +0x12c
k8s.io/kubernetes/test/e2e/chaosmonkey.(*Chaosmonkey).Do(0xc0034bb200)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/chaosmonkey/chaosmonkey.go:111 +0x38a
k8s.io/kubernetes/test/e2e/cloud/gcp.runUpgradeSuite(0xc000977540, 0x7757ca0, 0xc, 0xc, 0xc0006afc50, 0xc00220e780, 0x0, 0xc00163b160)
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:479 +0x47a
k8s.io/kubernetes/test/e2e/cloud/gcp.glob..func2.1.1()
	/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/cloud/gcp/cluster_upgrade.go:109 +0x222
k8s.io/kubernetes/test/e2e.RunE2ETests(0xc0001a8400)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:110 +0x30a
k8s.io/kubernetes/test/e2e.TestE2E(0xc0001a8400)
	_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e_test.go:112 +0x2b
testing.tRunner(0xc0001a8400, 0x49f9638)
	/usr/local/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:960 +0x350
				from junit_upgradeupgrades.xml

Filter through log files | View test history on testgrid


Test 14h33m

error during ./hack/ginkgo-e2e.sh --kubectl-path=../../../../kubernetes_skew/cluster/kubectl.sh --minStartupPods=8 --report-dir=/workspace/_artifacts --disable-log-dump=true (interrupted): exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 15h0m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


UpgradeTest 18m38s

error during kubetest --test --test_args=--ginkgo.focus=\[Feature:MasterUpgrade\] --upgrade-target=ci/k8s-stable1 --report-dir=/workspace/_artifacts --disable-log-dump=true --report-prefix=upgrade --check-version-skew=false: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 922 Passed Tests