This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 8 failed / 41 succeeded
Started2021-03-27 17:02
Elapsed5h45m
Revisionmaster
job-versionv1.22.0-alpha.0.3+770d3f181c5d7e
kubetest-version
masterInstanceIDs6905846522231053062
revisionv1.22.0-alpha.0.3+770d3f181c5d7e

Test Failures


ClusterLoaderV2 4h54m

error during /home/prow/go/src/k8s.io/perf-tests/run-e2e.sh cluster-loader2 --experimental-gcp-snapshot-prometheus-disk=true --experimental-prometheus-disk-snapshot-name=ci-kubernetes-e2e-gce-scale-performance-1375855502717620224 --nodes=5000 --prometheus-scrape-node-exporter --provider=gce --report-dir=/logs/artifacts --testconfig=testing/load/config.yaml --testconfig=testing/access-tokens/config.yaml --testoverrides=./testing/experiments/enable_restart_count_check.yaml --testoverrides=./testing/experiments/ignore_known_gce_container_restarts.yaml --testoverrides=./testing/overrides/5000_nodes.yaml: exit status 1
				from junit_runner.xml

Filter through log files | View test history on testgrid


ClusterLoaderV2 load overall (testing/load/config.yaml) 4h40m

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\soverall\s\(testing\/load\/config\.yaml\)$'
:0
[measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0]
measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0
measurement call WaitForControlledPodsRunning - WaitForRunningStatefulSets error: 2 objects timed out: StatefulSets: test-52h5qo-26/medium-statefulset-0, test-52h5qo-34/medium-statefulset-0
measurement call WaitForControlledPodsRunning - WaitForRunningJobs error: 11 objects timed out: Jobs: test-52h5qo-39/big-job-0, test-52h5qo-6/big-job-0, test-52h5qo-50/big-job-0, test-52h5qo-10/medium-job-0, test-52h5qo-8/big-job-0, test-52h5qo-38/big-job-0, test-52h5qo-18/big-job-0, test-52h5qo-23/big-job-0, test-52h5qo-41/big-job-0, test-52h5qo-5/big-job-0, test-52h5qo-9/big-job-0
measurement call WaitForControlledPodsRunning - WaitForRunningDeployments error: 110 objects timed out: Deployments: test-52h5qo-45/big-deployment-0, test-52h5qo-44/big-deployment-1, test-52h5qo-15/medium-deployment-9, test-52h5qo-23/small-deployment-33, test-52h5qo-29/small-deployment-29, test-52h5qo-7/small-deployment-162, test-52h5qo-9/small-deployment-204, test-52h5qo-17/big-deployment-1, test-52h5qo-50/medium-deployment-14, test-52h5qo-37/small-deployment-17, test-52h5qo-44/medium-deployment-11, test-52h5qo-48/small-deployment-97, test-52h5qo-20/medium-deployment-3, test-52h5qo-5/small-deployment-59, test-52h5qo-2/medium-deployment-1, test-52h5qo-29/small-deployment-212, test-52h5qo-17/small-deployment-134, test-52h5qo-12/big-deployment-0, test-52h5qo-34/small-deployment-6, test-52h5qo-49/big-deployment-0, test-52h5qo-13/small-deployment-86, test-52h5qo-5/small-deployment-249, test-52h5qo-47/medium-deployment-5, test-52h5qo-49/big-deployment-1, test-52h5qo-37/medium-deployment-16, test-52h5qo-44/small-deployment-276, test-52h5qo-31/small-deployment-249, test-52h5qo-1/big-deployment-1, test-52h5qo-40/small-deployment-166, test-52h5qo-33/small-deployment-116, test-52h5qo-49/small-deployment-237, test-52h5qo-3/small-deployment-13, test-52h5qo-29/small-deployment-138, test-52h5qo-42/small-deployment-244, test-52h5qo-8/big-deployment-0, test-52h5qo-29/small-deployment-261, test-52h5qo-36/big-deployment-1, test-52h5qo-24/big-deployment-1, test-52h5qo-22/medium-deployment-3, test-52h5qo-2/medium-deployment-18, test-52h5qo-18/big-deployment-0, test-52h5qo-31/big-deployment-0, test-52h5qo-23/small-deployment-270, test-52h5qo-13/big-deployment-1, test-52h5qo-21/big-deployment-0, test-52h5qo-6/medium-deployment-22, test-52h5qo-24/small-deployment-260, test-52h5qo-36/medium-deployment-22, test-52h5qo-45/small-deployment-218, test-52h5qo-35/small-deployment-223, test-52h5qo-10/small-deployment-64, test-52h5qo-50/big-deployment-0, test-52h5qo-9/small-deployment-91, test-52h5qo-40/big-deployment-0, test-52h5qo-44/small-deployment-115, test-52h5qo-11/small-deployment-130, test-52h5qo-38/medium-deployment-21, test-52h5qo-47/medium-deployment-0, test-52h5qo-2/small-deployment-98, test-52h5qo-9/medium-deployment-7, test-52h5qo-17/medium-deployment-8, test-52h5qo-13/small-deployment-57, test-52h5qo-31/medium-deployment-15, test-52h5qo-50/small-deployment-249, test-52h5qo-50/small-deployment-52, test-52h5qo-2/medium-deployment-10, test-52h5qo-49/medium-deployment-8, test-52h5qo-6/small-deployment-240, test-52h5qo-36/medium-deployment-14, test-52h5qo-3/medium-deployment-6, test-52h5qo-47/big-deployment-1, test-52h5qo-33/medium-deployment-18, test-52h5qo-4/big-deployment-1, test-52h5qo-2/small-deployment-4, test-52h5qo-3/small-deployment-246, test-52h5qo-12/medium-deployment-21, test-52h5qo-30/big-deployment-0, test-52h5qo-1/small-deployment-85, test-52h5qo-30/big-deployment-1, test-52h5qo-28/small-deployment-155, test-52h5qo-3/big-deployment-1, test-52h5qo-31/small-deployment-42, test-52h5qo-10/medium-deployment-19, test-52h5qo-11/medium-deployment-15, test-52h5qo-3/big-deployment-0, test-52h5qo-5/small-deployment-75, test-52h5qo-11/medium-deployment-10, test-52h5qo-1/small-deployment-79, test-52h5qo-8/small-deployment-27, test-52h5qo-48/small-deployment-274, test-52h5qo-23/small-deployment-275, test-52h5qo-48/small-deployment-215, test-52h5qo-7/medium-deployment-21, test-52h5qo-28/small-deployment-39, test-52h5qo-20/medium-deployment-17, test-52h5qo-32/small-deployment-125, test-52h5qo-1/medium-deployment-3, test-52h5qo-34/medium-deployment-2, test-52h5qo-43/small-deployment-227, test-52h5qo-17/medium-deployment-9, test-52h5qo-45/big-deployment-1, test-52h5qo-36/small-deployment-244, test-52h5qo-17/medium-deployment-1, test-52h5qo-10/medium-deployment-17, test-52h5qo-15/medium-deployment-13, test-52h5qo-46/big-deployment-1, test-52h5qo-31/small-deployment-276, test-52h5qo-14/small-deployment-255, test-52h5qo-19/medium-deployment-21, test-52h5qo-24/small-deployment-70]
measurement call WaitForBoundPVCs - WaitForPVCsToBeDeleted error: timeout while waiting for 0 PVCs to be running in namespace '' with labels 'group = load' and fields '' - only 1 found bound
measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0
measurement call WaitForControlledPodsRunning - WaitForRunningStatefulSets error: 2 objects timed out: StatefulSets: test-52h5qo-26/medium-statefulset-0, test-52h5qo-43/medium-statefulset-0
measurement call WaitForControlledPodsRunning - WaitForRunningJobs error: 10 objects timed out: Jobs: test-52h5qo-43/big-job-0, test-52h5qo-17/big-job-0, test-52h5qo-10/medium-job-0, test-52h5qo-8/big-job-0, test-52h5qo-38/big-job-0, test-52h5qo-31/big-job-0, test-52h5qo-23/big-job-0, test-52h5qo-9/big-job-0, test-52h5qo-39/big-job-0, test-52h5qo-6/big-job-0
measurement call WaitForControlledPodsRunning - WaitForRunningDeployments error: 163 objects timed out: Deployments: test-52h5qo-35/small-deployment-231, test-52h5qo-37/medium-deployment-11, test-52h5qo-41/small-deployment-63, test-52h5qo-1/medium-deployment-3, test-52h5qo-43/small-deployment-227, test-52h5qo-13/small-deployment-41, test-52h5qo-17/medium-deployment-9, test-52h5qo-45/big-deployment-1, test-52h5qo-20/medium-deployment-7, test-52h5qo-17/medium-deployment-1, test-52h5qo-23/small-deployment-237, test-52h5qo-8/medium-deployment-10, test-52h5qo-12/small-deployment-238, test-52h5qo-10/medium-deployment-17, test-52h5qo-15/medium-deployment-13, test-52h5qo-23/big-deployment-0, test-52h5qo-41/medium-deployment-13, test-52h5qo-46/big-deployment-1, test-52h5qo-36/small-deployment-200, test-52h5qo-31/small-deployment-276, test-52h5qo-14/small-deployment-255, test-52h5qo-49/medium-deployment-12, test-52h5qo-22/small-deployment-138, test-52h5qo-3/small-deployment-265, test-52h5qo-45/big-deployment-0, test-52h5qo-44/big-deployment-1, test-52h5qo-19/small-deployment-283, test-52h5qo-18/small-deployment-99, test-52h5qo-44/small-deployment-157, test-52h5qo-23/small-deployment-33, test-52h5qo-4/medium-deployment-1, test-52h5qo-29/small-deployment-29, test-52h5qo-5/medium-deployment-17, test-52h5qo-25/small-deployment-69, test-52h5qo-28/small-deployment-149, test-52h5qo-9/small-deployment-204, test-52h5qo-11/medium-deployment-17, test-52h5qo-14/medium-deployment-4, test-52h5qo-28/small-deployment-94, test-52h5qo-27/small-deployment-33, test-52h5qo-17/big-deployment-1, test-52h5qo-44/medium-deployment-11, test-52h5qo-48/small-deployment-97, test-52h5qo-15/small-deployment-267, test-52h5qo-5/small-deployment-59, test-52h5qo-9/big-deployment-0, test-52h5qo-10/big-deployment-1, test-52h5qo-18/medium-deployment-3, test-52h5qo-15/big-deployment-0, test-52h5qo-2/medium-deployment-1, test-52h5qo-17/small-deployment-134, test-52h5qo-12/big-deployment-0, test-52h5qo-2/small-deployment-194, test-52h5qo-32/small-deployment-130, test-52h5qo-49/big-deployment-0, test-52h5qo-8/small-deployment-62, test-52h5qo-46/small-deployment-44, test-52h5qo-38/medium-deployment-22, test-52h5qo-20/small-deployment-280, test-52h5qo-39/medium-deployment-2, test-52h5qo-1/small-deployment-16, test-52h5qo-47/medium-deployment-5, test-52h5qo-49/big-deployment-1, test-52h5qo-26/small-deployment-55, test-52h5qo-41/small-deployment-144, test-52h5qo-37/medium-deployment-16, test-52h5qo-1/big-deployment-1, test-52h5qo-44/small-deployment-276, test-52h5qo-40/small-deployment-166, test-52h5qo-12/big-deployment-1, test-52h5qo-15/big-deployment-1, test-52h5qo-24/small-deployment-12, test-52h5qo-49/small-deployment-237, test-52h5qo-3/small-deployment-13, test-52h5qo-21/big-deployment-1, test-52h5qo-40/medium-deployment-6, test-52h5qo-9/medium-deployment-17, test-52h5qo-33/medium-deployment-6, test-52h5qo-42/small-deployment-244, test-52h5qo-50/small-deployment-120, test-52h5qo-8/big-deployment-0, test-52h5qo-30/small-deployment-221, test-52h5qo-19/medium-deployment-6, test-52h5qo-33/small-deployment-236, test-52h5qo-29/small-deployment-261, test-52h5qo-36/big-deployment-1, test-52h5qo-24/big-deployment-1, test-52h5qo-22/medium-deployment-3, test-52h5qo-31/small-deployment-232, test-52h5qo-2/medium-deployment-18, test-52h5qo-18/big-deployment-0, test-52h5qo-45/small-deployment-107, test-52h5qo-10/medium-deployment-5, test-52h5qo-23/small-deployment-270, test-52h5qo-23/small-deployment-10, test-52h5qo-26/small-deployment-235, test-52h5qo-19/small-deployment-51, test-52h5qo-6/medium-deployment-22, test-52h5qo-49/small-deployment-94, test-52h5qo-41/small-deployment-151, test-52h5qo-19/small-deployment-32, test-52h5qo-45/small-deployment-218, test-52h5qo-18/small-deployment-131, test-52h5qo-27/small-deployment-236, test-52h5qo-35/small-deployment-223, test-52h5qo-1/small-deployment-134, test-52h5qo-29/small-deployment-264, test-52h5qo-28/small-deployment-272, test-52h5qo-9/small-deployment-91, test-52h5qo-5/medium-deployment-7, test-52h5qo-44/small-deployment-115, test-52h5qo-11/small-deployment-130, test-52h5qo-13/small-deployment-182, test-52h5qo-46/small-deployment-23, test-52h5qo-37/small-deployment-60, test-52h5qo-47/medium-deployment-0, test-52h5qo-2/small-deployment-98, test-52h5qo-9/medium-deployment-7, test-52h5qo-17/medium-deployment-8, test-52h5qo-23/small-deployment-261, test-52h5qo-13/small-deployment-57, test-52h5qo-34/small-deployment-230, test-52h5qo-32/small-deployment-275, test-52h5qo-37/small-deployment-43, test-52h5qo-6/small-deployment-287, test-52h5qo-12/medium-deployment-8, test-52h5qo-50/small-deployment-52, test-52h5qo-2/medium-deployment-10, test-52h5qo-49/medium-deployment-8, test-52h5qo-6/small-deployment-240, test-52h5qo-36/medium-deployment-14, test-52h5qo-46/small-deployment-10, test-52h5qo-3/medium-deployment-6, test-52h5qo-13/medium-deployment-15, test-52h5qo-47/big-deployment-1, test-52h5qo-33/medium-deployment-18, test-52h5qo-4/big-deployment-1, test-52h5qo-9/small-deployment-272, test-52h5qo-2/small-deployment-4, test-52h5qo-3/small-deployment-246, test-52h5qo-12/medium-deployment-21, test-52h5qo-11/medium-deployment-7, test-52h5qo-30/big-deployment-0, test-52h5qo-1/small-deployment-85, test-52h5qo-30/big-deployment-1, test-52h5qo-28/small-deployment-155, test-52h5qo-15/small-deployment-135, test-52h5qo-3/big-deployment-1, test-52h5qo-31/small-deployment-42, test-52h5qo-10/medium-deployment-19, test-52h5qo-11/medium-deployment-15, test-52h5qo-24/small-deployment-196, test-52h5qo-11/small-deployment-74, test-52h5qo-3/big-deployment-0, test-52h5qo-35/big-deployment-0, test-52h5qo-5/small-deployment-75, test-52h5qo-11/medium-deployment-10, test-52h5qo-35/medium-deployment-5, test-52h5qo-1/small-deployment-79, test-52h5qo-23/small-deployment-275, test-52h5qo-48/small-deployment-215, test-52h5qo-7/medium-deployment-21, test-52h5qo-28/small-deployment-39]
measurement call APIResponsivenessPrometheus - APIResponsivenessPrometheusSimple error: top latency metric: there should be no high-latency requests, but: [got: &{Resource:pods Subresource: Verb:LIST Scope:cluster Latency:perc50: 12.083333333s, perc90: 23.499999999s, perc99: 1m0s Count:39 SlowCount:2}; expected perc99 <= 30s]
measurement call TestMetrics - TestMetrics error: [action gather failed for ClusterOOMsTracker measurement: OOMs recorded: [{Node:gce-scale-cluster-minion-group-07qg Process:google_network_ ProcessMemory:35308kB ProcessID:577 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_clock_sk ProcessMemory:35184kB ProcessID:583 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:920840kB ProcessID:1945 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:398 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_network_ ProcessMemory:35308kB ProcessID:577 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_clock_sk ProcessMemory:35184kB ProcessID:583 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:920840kB ProcessID:1945 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:398 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:847108kB ProcessID:19210 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:19166 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:847108kB ProcessID:19369 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:device_policy_m ProcessMemory:635216kB ProcessID:432 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:19325 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:log-counter ProcessMemory:572388kB ProcessID:19404 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:log-counter ProcessMemory:498656kB ProcessID:19406 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:crictl ProcessMemory:732704kB ProcessID:19402 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:systemd-journal ProcessMemory:45632kB ProcessID:114 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:systemd-network ProcessMemory:23296kB ProcessID:341 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-pk32 Process:pause ProcessMemory:1068kB ProcessID:22922 Time:2021-03-27 19:11:04 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19883 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19894 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19864 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:116968kB ProcessID:19902 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:crash_sender ProcessMemory:20144kB ProcessID:19856 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:agetty ProcessMemory:8352kB ProcessID:19841 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:curl ProcessMemory:8368kB ProcessID:19895 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_oslogin_ ProcessMemory:5152kB ProcessID:19903 Time:2021-03-27 19:42:35 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:metrics-server ProcessMemory:4583852kB ProcessID:2522 Time:2021-03-27 19:42:36 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:57858 Time:2021-03-27 19:43:03 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:24730 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22798 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22413 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22154 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:21507 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:20274 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:19997 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:19700 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:17527 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-4gnj Process:metrics-server ProcessMemory:4073508kB ProcessID:52290 Time:2021-03-27 19:49:35 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:google_network_ ProcessMemory:35320kB ProcessID:482 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-mh33 Process:metrics-server ProcessMemory:3249416kB ProcessID:22712 Time:2021-03-27 20:19:40 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:98855 Time:2021-03-27 20:28:56 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:93029 Time:2021-03-27 20:28:57 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:92995 Time:2021-03-27 20:28:57 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:containerd ProcessMemory:49784kB ProcessID:67244 Time:2021-03-27 20:40:18 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:google_oslogin_ ProcessMemory:9396kB ProcessID:67230 Time:2021-03-27 20:40:19 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:metrics-server ProcessMemory:3516016kB ProcessID:66658 Time:2021-03-27 20:40:19 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5jqf Process:metrics-server ProcessMemory:3367612kB ProcessID:29503 Time:2021-03-27 20:43:51 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:100985 Time:2021-03-27 20:47:05 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:device_policy_m ProcessMemory:636624kB ProcessID:59585 Time:2021-03-27 20:57:21 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:node-problem-de ProcessMemory:937224kB ProcessID:59588 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:device_policy_m ProcessMemory:561484kB ProcessID:59773 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:google_osconfig ProcessMemory:119016kB ProcessID:59584 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-network ProcessMemory:24088kB ProcessID:59622 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-resolve ProcessMemory:18516kB ProcessID:59677 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-journal ProcessMemory:20804kB ProcessID:59779 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-logind ProcessMemory:12668kB ProcessID:59582 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:bash ProcessMemory:9392kB ProcessID:59590 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:bash ProcessMemory:9392kB ProcessID:59589 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:chronyd ProcessMemory:85908kB ProcessID:337 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:kubelet ProcessMemory:825092kB ProcessID:60088 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:sleep ProcessMemory:11452kB ProcessID:59861 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:sleep ProcessMemory:11452kB ProcessID:59885 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:agetty ProcessMemory:8352kB ProcessID:391 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:iptables-restor ProcessMemory:293692kB ProcessID:59947 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:google_network_ ProcessMemory:35308kB ProcessID:528 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:google_clock_sk ProcessMemory:35308kB ProcessID:532 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:device_policy_m ProcessMemory:561484kB ProcessID:417 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-zg6l Process:node-problem-de ProcessMemory:789760kB ProcessID:46904 Time:2021-03-27 21:21:28 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-jt7p Process:metrics-server ProcessMemory:3309056kB ProcessID:75516 Time:2021-03-27 21:21:33 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:96925 Time:2021-03-27 21:03:24 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:93527 Time:2021-03-27 21:47:16 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:40338 Time:2021-03-27 21:47:17 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:37290 Time:2021-03-27 21:47:18 +0000 UTC}]]]
:0
				from junit.xml

Filter through log files | View test history on testgrid


ClusterLoaderV2 load: [step: 06] Waiting for pods to be running 4m25s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\:\s\[step\:\s06\]\sWaiting\sfor\spods\sto\sbe\srunning$'
:0
[measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0]]
:0
				from junit.xml

Filter through log files | View test history on testgrid


ClusterLoaderV2 load: [step: 23] Waiting for objects to become scaled 29m59s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\:\s\[step\:\s23\]\sWaiting\sfor\sobjects\sto\sbecome\sscaled$'
:0
[measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0
measurement call WaitForControlledPodsRunning - WaitForRunningStatefulSets error: 2 objects timed out: StatefulSets: test-52h5qo-26/medium-statefulset-0, test-52h5qo-34/medium-statefulset-0
measurement call WaitForControlledPodsRunning - WaitForRunningJobs error: 11 objects timed out: Jobs: test-52h5qo-39/big-job-0, test-52h5qo-6/big-job-0, test-52h5qo-50/big-job-0, test-52h5qo-10/medium-job-0, test-52h5qo-8/big-job-0, test-52h5qo-38/big-job-0, test-52h5qo-18/big-job-0, test-52h5qo-23/big-job-0, test-52h5qo-41/big-job-0, test-52h5qo-5/big-job-0, test-52h5qo-9/big-job-0
measurement call WaitForControlledPodsRunning - WaitForRunningDeployments error: 110 objects timed out: Deployments: test-52h5qo-45/big-deployment-0, test-52h5qo-44/big-deployment-1, test-52h5qo-15/medium-deployment-9, test-52h5qo-23/small-deployment-33, test-52h5qo-29/small-deployment-29, test-52h5qo-7/small-deployment-162, test-52h5qo-9/small-deployment-204, test-52h5qo-17/big-deployment-1, test-52h5qo-50/medium-deployment-14, test-52h5qo-37/small-deployment-17, test-52h5qo-44/medium-deployment-11, test-52h5qo-48/small-deployment-97, test-52h5qo-20/medium-deployment-3, test-52h5qo-5/small-deployment-59, test-52h5qo-2/medium-deployment-1, test-52h5qo-29/small-deployment-212, test-52h5qo-17/small-deployment-134, test-52h5qo-12/big-deployment-0, test-52h5qo-34/small-deployment-6, test-52h5qo-49/big-deployment-0, test-52h5qo-13/small-deployment-86, test-52h5qo-5/small-deployment-249, test-52h5qo-47/medium-deployment-5, test-52h5qo-49/big-deployment-1, test-52h5qo-37/medium-deployment-16, test-52h5qo-44/small-deployment-276, test-52h5qo-31/small-deployment-249, test-52h5qo-1/big-deployment-1, test-52h5qo-40/small-deployment-166, test-52h5qo-33/small-deployment-116, test-52h5qo-49/small-deployment-237, test-52h5qo-3/small-deployment-13, test-52h5qo-29/small-deployment-138, test-52h5qo-42/small-deployment-244, test-52h5qo-8/big-deployment-0, test-52h5qo-29/small-deployment-261, test-52h5qo-36/big-deployment-1, test-52h5qo-24/big-deployment-1, test-52h5qo-22/medium-deployment-3, test-52h5qo-2/medium-deployment-18, test-52h5qo-18/big-deployment-0, test-52h5qo-31/big-deployment-0, test-52h5qo-23/small-deployment-270, test-52h5qo-13/big-deployment-1, test-52h5qo-21/big-deployment-0, test-52h5qo-6/medium-deployment-22, test-52h5qo-24/small-deployment-260, test-52h5qo-36/medium-deployment-22, test-52h5qo-45/small-deployment-218, test-52h5qo-35/small-deployment-223, test-52h5qo-10/small-deployment-64, test-52h5qo-50/big-deployment-0, test-52h5qo-9/small-deployment-91, test-52h5qo-40/big-deployment-0, test-52h5qo-44/small-deployment-115, test-52h5qo-11/small-deployment-130, test-52h5qo-38/medium-deployment-21, test-52h5qo-47/medium-deployment-0, test-52h5qo-2/small-deployment-98, test-52h5qo-9/medium-deployment-7, test-52h5qo-17/medium-deployment-8, test-52h5qo-13/small-deployment-57, test-52h5qo-31/medium-deployment-15, test-52h5qo-50/small-deployment-249, test-52h5qo-50/small-deployment-52, test-52h5qo-2/medium-deployment-10, test-52h5qo-49/medium-deployment-8, test-52h5qo-6/small-deployment-240, test-52h5qo-36/medium-deployment-14, test-52h5qo-3/medium-deployment-6, test-52h5qo-47/big-deployment-1, test-52h5qo-33/medium-deployment-18, test-52h5qo-4/big-deployment-1, test-52h5qo-2/small-deployment-4, test-52h5qo-3/small-deployment-246, test-52h5qo-12/medium-deployment-21, test-52h5qo-30/big-deployment-0, test-52h5qo-1/small-deployment-85, test-52h5qo-30/big-deployment-1, test-52h5qo-28/small-deployment-155, test-52h5qo-3/big-deployment-1, test-52h5qo-31/small-deployment-42, test-52h5qo-10/medium-deployment-19, test-52h5qo-11/medium-deployment-15, test-52h5qo-3/big-deployment-0, test-52h5qo-5/small-deployment-75, test-52h5qo-11/medium-deployment-10, test-52h5qo-1/small-deployment-79, test-52h5qo-8/small-deployment-27, test-52h5qo-48/small-deployment-274, test-52h5qo-23/small-deployment-275, test-52h5qo-48/small-deployment-215, test-52h5qo-7/medium-deployment-21, test-52h5qo-28/small-deployment-39, test-52h5qo-20/medium-deployment-17, test-52h5qo-32/small-deployment-125, test-52h5qo-1/medium-deployment-3, test-52h5qo-34/medium-deployment-2, test-52h5qo-43/small-deployment-227, test-52h5qo-17/medium-deployment-9, test-52h5qo-45/big-deployment-1, test-52h5qo-36/small-deployment-244, test-52h5qo-17/medium-deployment-1, test-52h5qo-10/medium-deployment-17, test-52h5qo-15/medium-deployment-13, test-52h5qo-46/big-deployment-1, test-52h5qo-31/small-deployment-276, test-52h5qo-14/small-deployment-255, test-52h5qo-19/medium-deployment-21, test-52h5qo-24/small-deployment-70]]
:0
				from junit.xml

Filter through log files | View test history on testgrid


ClusterLoaderV2 load: [step: 25] Waiting for pods to be deleted 28m43s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\:\s\[step\:\s25\]\sWaiting\sfor\spods\sto\sbe\sdeleted$'
:0
[measurement call WaitForBoundPVCs - WaitForPVCsToBeDeleted error: timeout while waiting for 0 PVCs to be running in namespace '' with labels 'group = load' and fields '' - only 1 found bound
measurement call WaitForControlledPodsRunning -  error: [measurement call WaitForControlledPodsRunning - WaitForRunningDaemonSets error: 1 objects timed out: DaemonSets: test-52h5qo-1/daemonset-0
measurement call WaitForControlledPodsRunning - WaitForRunningStatefulSets error: 2 objects timed out: StatefulSets: test-52h5qo-26/medium-statefulset-0, test-52h5qo-43/medium-statefulset-0
measurement call WaitForControlledPodsRunning - WaitForRunningJobs error: 10 objects timed out: Jobs: test-52h5qo-43/big-job-0, test-52h5qo-17/big-job-0, test-52h5qo-10/medium-job-0, test-52h5qo-8/big-job-0, test-52h5qo-38/big-job-0, test-52h5qo-31/big-job-0, test-52h5qo-23/big-job-0, test-52h5qo-9/big-job-0, test-52h5qo-39/big-job-0, test-52h5qo-6/big-job-0
measurement call WaitForControlledPodsRunning - WaitForRunningDeployments error: 163 objects timed out: Deployments: test-52h5qo-35/small-deployment-231, test-52h5qo-37/medium-deployment-11, test-52h5qo-41/small-deployment-63, test-52h5qo-1/medium-deployment-3, test-52h5qo-43/small-deployment-227, test-52h5qo-13/small-deployment-41, test-52h5qo-17/medium-deployment-9, test-52h5qo-45/big-deployment-1, test-52h5qo-20/medium-deployment-7, test-52h5qo-17/medium-deployment-1, test-52h5qo-23/small-deployment-237, test-52h5qo-8/medium-deployment-10, test-52h5qo-12/small-deployment-238, test-52h5qo-10/medium-deployment-17, test-52h5qo-15/medium-deployment-13, test-52h5qo-23/big-deployment-0, test-52h5qo-41/medium-deployment-13, test-52h5qo-46/big-deployment-1, test-52h5qo-36/small-deployment-200, test-52h5qo-31/small-deployment-276, test-52h5qo-14/small-deployment-255, test-52h5qo-49/medium-deployment-12, test-52h5qo-22/small-deployment-138, test-52h5qo-3/small-deployment-265, test-52h5qo-45/big-deployment-0, test-52h5qo-44/big-deployment-1, test-52h5qo-19/small-deployment-283, test-52h5qo-18/small-deployment-99, test-52h5qo-44/small-deployment-157, test-52h5qo-23/small-deployment-33, test-52h5qo-4/medium-deployment-1, test-52h5qo-29/small-deployment-29, test-52h5qo-5/medium-deployment-17, test-52h5qo-25/small-deployment-69, test-52h5qo-28/small-deployment-149, test-52h5qo-9/small-deployment-204, test-52h5qo-11/medium-deployment-17, test-52h5qo-14/medium-deployment-4, test-52h5qo-28/small-deployment-94, test-52h5qo-27/small-deployment-33, test-52h5qo-17/big-deployment-1, test-52h5qo-44/medium-deployment-11, test-52h5qo-48/small-deployment-97, test-52h5qo-15/small-deployment-267, test-52h5qo-5/small-deployment-59, test-52h5qo-9/big-deployment-0, test-52h5qo-10/big-deployment-1, test-52h5qo-18/medium-deployment-3, test-52h5qo-15/big-deployment-0, test-52h5qo-2/medium-deployment-1, test-52h5qo-17/small-deployment-134, test-52h5qo-12/big-deployment-0, test-52h5qo-2/small-deployment-194, test-52h5qo-32/small-deployment-130, test-52h5qo-49/big-deployment-0, test-52h5qo-8/small-deployment-62, test-52h5qo-46/small-deployment-44, test-52h5qo-38/medium-deployment-22, test-52h5qo-20/small-deployment-280, test-52h5qo-39/medium-deployment-2, test-52h5qo-1/small-deployment-16, test-52h5qo-47/medium-deployment-5, test-52h5qo-49/big-deployment-1, test-52h5qo-26/small-deployment-55, test-52h5qo-41/small-deployment-144, test-52h5qo-37/medium-deployment-16, test-52h5qo-1/big-deployment-1, test-52h5qo-44/small-deployment-276, test-52h5qo-40/small-deployment-166, test-52h5qo-12/big-deployment-1, test-52h5qo-15/big-deployment-1, test-52h5qo-24/small-deployment-12, test-52h5qo-49/small-deployment-237, test-52h5qo-3/small-deployment-13, test-52h5qo-21/big-deployment-1, test-52h5qo-40/medium-deployment-6, test-52h5qo-9/medium-deployment-17, test-52h5qo-33/medium-deployment-6, test-52h5qo-42/small-deployment-244, test-52h5qo-50/small-deployment-120, test-52h5qo-8/big-deployment-0, test-52h5qo-30/small-deployment-221, test-52h5qo-19/medium-deployment-6, test-52h5qo-33/small-deployment-236, test-52h5qo-29/small-deployment-261, test-52h5qo-36/big-deployment-1, test-52h5qo-24/big-deployment-1, test-52h5qo-22/medium-deployment-3, test-52h5qo-31/small-deployment-232, test-52h5qo-2/medium-deployment-18, test-52h5qo-18/big-deployment-0, test-52h5qo-45/small-deployment-107, test-52h5qo-10/medium-deployment-5, test-52h5qo-23/small-deployment-270, test-52h5qo-23/small-deployment-10, test-52h5qo-26/small-deployment-235, test-52h5qo-19/small-deployment-51, test-52h5qo-6/medium-deployment-22, test-52h5qo-49/small-deployment-94, test-52h5qo-41/small-deployment-151, test-52h5qo-19/small-deployment-32, test-52h5qo-45/small-deployment-218, test-52h5qo-18/small-deployment-131, test-52h5qo-27/small-deployment-236, test-52h5qo-35/small-deployment-223, test-52h5qo-1/small-deployment-134, test-52h5qo-29/small-deployment-264, test-52h5qo-28/small-deployment-272, test-52h5qo-9/small-deployment-91, test-52h5qo-5/medium-deployment-7, test-52h5qo-44/small-deployment-115, test-52h5qo-11/small-deployment-130, test-52h5qo-13/small-deployment-182, test-52h5qo-46/small-deployment-23, test-52h5qo-37/small-deployment-60, test-52h5qo-47/medium-deployment-0, test-52h5qo-2/small-deployment-98, test-52h5qo-9/medium-deployment-7, test-52h5qo-17/medium-deployment-8, test-52h5qo-23/small-deployment-261, test-52h5qo-13/small-deployment-57, test-52h5qo-34/small-deployment-230, test-52h5qo-32/small-deployment-275, test-52h5qo-37/small-deployment-43, test-52h5qo-6/small-deployment-287, test-52h5qo-12/medium-deployment-8, test-52h5qo-50/small-deployment-52, test-52h5qo-2/medium-deployment-10, test-52h5qo-49/medium-deployment-8, test-52h5qo-6/small-deployment-240, test-52h5qo-36/medium-deployment-14, test-52h5qo-46/small-deployment-10, test-52h5qo-3/medium-deployment-6, test-52h5qo-13/medium-deployment-15, test-52h5qo-47/big-deployment-1, test-52h5qo-33/medium-deployment-18, test-52h5qo-4/big-deployment-1, test-52h5qo-9/small-deployment-272, test-52h5qo-2/small-deployment-4, test-52h5qo-3/small-deployment-246, test-52h5qo-12/medium-deployment-21, test-52h5qo-11/medium-deployment-7, test-52h5qo-30/big-deployment-0, test-52h5qo-1/small-deployment-85, test-52h5qo-30/big-deployment-1, test-52h5qo-28/small-deployment-155, test-52h5qo-15/small-deployment-135, test-52h5qo-3/big-deployment-1, test-52h5qo-31/small-deployment-42, test-52h5qo-10/medium-deployment-19, test-52h5qo-11/medium-deployment-15, test-52h5qo-24/small-deployment-196, test-52h5qo-11/small-deployment-74, test-52h5qo-3/big-deployment-0, test-52h5qo-35/big-deployment-0, test-52h5qo-5/small-deployment-75, test-52h5qo-11/medium-deployment-10, test-52h5qo-35/medium-deployment-5, test-52h5qo-1/small-deployment-79, test-52h5qo-23/small-deployment-275, test-52h5qo-48/small-deployment-215, test-52h5qo-7/medium-deployment-21, test-52h5qo-28/small-deployment-39]]
:0
				from junit.xml

Filter through log files | View test history on testgrid


ClusterLoaderV2 load: [step: 28] Collecting measurements 10s

go run hack/e2e.go -v --test --test_args='--ginkgo.focus=ClusterLoaderV2\sload\:\s\[step\:\s28\]\sCollecting\smeasurements$'
:0
[measurement call APIResponsivenessPrometheus - APIResponsivenessPrometheusSimple error: top latency metric: there should be no high-latency requests, but: [got: &{Resource:pods Subresource: Verb:LIST Scope:cluster Latency:perc50: 12.083333333s, perc90: 23.499999999s, perc99: 1m0s Count:39 SlowCount:2}; expected perc99 <= 30s]
measurement call TestMetrics - TestMetrics error: [action gather failed for ClusterOOMsTracker measurement: OOMs recorded: [{Node:gce-scale-cluster-minion-group-07qg Process:google_network_ ProcessMemory:35308kB ProcessID:577 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_clock_sk ProcessMemory:35184kB ProcessID:583 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:920840kB ProcessID:1945 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:398 Time:2021-03-27 18:05:23 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_network_ ProcessMemory:35308kB ProcessID:577 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_clock_sk ProcessMemory:35184kB ProcessID:583 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:920840kB ProcessID:1945 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:398 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:847108kB ProcessID:19210 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:19166 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:node-problem-de ProcessMemory:847108kB ProcessID:19369 Time:2021-03-27 18:06:45 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:device_policy_m ProcessMemory:635216kB ProcessID:432 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:119016kB ProcessID:19325 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:log-counter ProcessMemory:572388kB ProcessID:19404 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:log-counter ProcessMemory:498656kB ProcessID:19406 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:crictl ProcessMemory:732704kB ProcessID:19402 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:systemd-journal ProcessMemory:45632kB ProcessID:114 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:systemd-network ProcessMemory:23296kB ProcessID:341 Time:2021-03-27 18:06:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-pk32 Process:pause ProcessMemory:1068kB ProcessID:22922 Time:2021-03-27 19:11:04 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19883 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19894 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:bash ProcessMemory:9392kB ProcessID:19864 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_osconfig ProcessMemory:116968kB ProcessID:19902 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:crash_sender ProcessMemory:20144kB ProcessID:19856 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:agetty ProcessMemory:8352kB ProcessID:19841 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:curl ProcessMemory:8368kB ProcessID:19895 Time:2021-03-27 19:42:34 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:google_oslogin_ ProcessMemory:5152kB ProcessID:19903 Time:2021-03-27 19:42:35 +0000 UTC} {Node:gce-scale-cluster-minion-group-07qg Process:metrics-server ProcessMemory:4583852kB ProcessID:2522 Time:2021-03-27 19:42:36 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:57858 Time:2021-03-27 19:43:03 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:24730 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22798 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22413 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:22154 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:21507 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:20274 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:19997 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:19700 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:pause ProcessMemory:1068kB ProcessID:17527 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-4gnj Process:metrics-server ProcessMemory:4073508kB ProcessID:52290 Time:2021-03-27 19:49:35 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-vxkt Process:google_network_ ProcessMemory:35320kB ProcessID:482 Time:2021-03-27 19:47:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-mh33 Process:metrics-server ProcessMemory:3249416kB ProcessID:22712 Time:2021-03-27 20:19:40 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:98855 Time:2021-03-27 20:28:56 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:93029 Time:2021-03-27 20:28:57 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-nfs0 Process:pause ProcessMemory:1068kB ProcessID:92995 Time:2021-03-27 20:28:57 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:containerd ProcessMemory:49784kB ProcessID:67244 Time:2021-03-27 20:40:18 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:google_oslogin_ ProcessMemory:9396kB ProcessID:67230 Time:2021-03-27 20:40:19 +0000 UTC} {Node:gce-scale-cluster-minion-group-2-rz32 Process:metrics-server ProcessMemory:3516016kB ProcessID:66658 Time:2021-03-27 20:40:19 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5jqf Process:metrics-server ProcessMemory:3367612kB ProcessID:29503 Time:2021-03-27 20:43:51 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:100985 Time:2021-03-27 20:47:05 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:device_policy_m ProcessMemory:636624kB ProcessID:59585 Time:2021-03-27 20:57:21 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:node-problem-de ProcessMemory:937224kB ProcessID:59588 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:device_policy_m ProcessMemory:561484kB ProcessID:59773 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:google_osconfig ProcessMemory:119016kB ProcessID:59584 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-network ProcessMemory:24088kB ProcessID:59622 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-resolve ProcessMemory:18516kB ProcessID:59677 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-journal ProcessMemory:20804kB ProcessID:59779 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:systemd-logind ProcessMemory:12668kB ProcessID:59582 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:bash ProcessMemory:9392kB ProcessID:59590 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:bash ProcessMemory:9392kB ProcessID:59589 Time:2021-03-27 21:01:13 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:chronyd ProcessMemory:85908kB ProcessID:337 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:kubelet ProcessMemory:825092kB ProcessID:60088 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:sleep ProcessMemory:11452kB ProcessID:59861 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:sleep ProcessMemory:11452kB ProcessID:59885 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:agetty ProcessMemory:8352kB ProcessID:391 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-rptd Process:iptables-restor ProcessMemory:293692kB ProcessID:59947 Time:2021-03-27 21:01:14 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:google_network_ ProcessMemory:35308kB ProcessID:528 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:google_clock_sk ProcessMemory:35308kB ProcessID:532 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-lpbl Process:device_policy_m ProcessMemory:561484kB ProcessID:417 Time:2021-03-27 21:19:46 +0000 UTC} {Node:gce-scale-cluster-minion-group-1-zg6l Process:node-problem-de ProcessMemory:789760kB ProcessID:46904 Time:2021-03-27 21:21:28 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-jt7p Process:metrics-server ProcessMemory:3309056kB ProcessID:75516 Time:2021-03-27 21:21:33 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:96925 Time:2021-03-27 21:03:24 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:93527 Time:2021-03-27 21:47:16 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:40338 Time:2021-03-27 21:47:17 +0000 UTC} {Node:gce-scale-cluster-minion-group-3-5cnq Process:pause ProcessMemory:1068kB ProcessID:37290 Time:2021-03-27 21:47:18 +0000 UTC}]]]
:0
				from junit.xml

Filter through log files | View test history on testgrid


DumpClusterLogs 24m9s

error during /workspace/log-dump.sh /logs/artifacts gs://sig-scalability-logs/ci-kubernetes-e2e-gce-scale-performance/1375855502717620224 (interrupted): signal: interrupt
				from junit_runner.xml

Filter through log files | View test history on testgrid


Timeout 5h30m

kubetest --timeout triggered
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 41 Passed Tests

Show 2 Skipped Tests