kubernetes-sigs/kueue PR #433 pull-kueue-test-integration-main #1596125988603826176
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | mimowo: Design for all-or-nothing semantics for workload resource assignment |
| Result | ABORTED |
| Tests | 0 failed / 36 succeeded |
| Started | |
| Elapsed | 2m53s |
| Revision | a10513f2ffc835e73a0a7b62de6bbac997e2c687 |
| Refs |
433 |
Core Controllers Suite [AfterSuite]
Core Controllers Suite [AfterSuite]
Core Controllers Suite [BeforeSuite]
Core Controllers Suite [BeforeSuite]
Core Controllers Suite [It] ClusterQueue controller when Deleting clusterQueues Should be stuck in termination until admitted workloads finished running
Core Controllers Suite [It] ClusterQueue controller when Deleting clusterQueues Should be stuck in termination until admitted workloads finished running
Core Controllers Suite [It] ClusterQueue controller when Deleting clusterQueues Should delete clusterQueues successfully when no admitted workloads are running
Core Controllers Suite [It] ClusterQueue controller when Deleting clusterQueues Should delete clusterQueues successfully when no admitted workloads are running
Core Controllers Suite [It] ClusterQueue controller when Reconciling clusterQueue status Should update status when workloads are assigned and finish
Core Controllers Suite [It] ClusterQueue controller when Reconciling clusterQueue status Should update status when workloads are assigned and finish
Core Controllers Suite [It] Queue controller Should update status when workloads are created
Core Controllers Suite [It] Queue controller Should update status when workloads are created
Core Controllers Suite [It] ResourceFlavor controller when one clusterQueue references resourceFlavors Should delete the resourceFlavor when the corresponding clusterQueue is deleted
Core Controllers Suite [It] ResourceFlavor controller when one clusterQueue references resourceFlavors Should delete the resourceFlavor when the corresponding clusterQueue is deleted
Core Controllers Suite [It] ResourceFlavor controller when one clusterQueue references resourceFlavors Should delete the resourceFlavor when the corresponding clusterQueue no longer uses the resourceFlavor
Core Controllers Suite [It] ResourceFlavor controller when one clusterQueue references resourceFlavors Should delete the resourceFlavor when the corresponding clusterQueue no longer uses the resourceFlavor
Core Controllers Suite [It] Workload controller when Workload with RuntimeClass defined Should accumulate RuntimeClass's overhead
Core Controllers Suite [It] Workload controller when Workload with RuntimeClass defined Should accumulate RuntimeClass's overhead
Core Controllers Suite [It] Workload controller when Workload with non-existent RuntimeClass defined Should not accumulate RuntimeClass's overhead
Core Controllers Suite [It] Workload controller when Workload with non-existent RuntimeClass defined Should not accumulate RuntimeClass's overhead
Core Controllers Suite [It] Workload controller when the clusterqueue doesn't exist Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the clusterqueue doesn't exist Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the queue doesn't exist Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the queue doesn't exist Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the queue is not defined in the workload Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the queue is not defined in the workload Should update status when workloads are created
Core Controllers Suite [It] Workload controller when the workload is admitted Should update the workload's condition
Core Controllers Suite [It] Workload controller when the workload is admitted Should update the workload's condition
Job Controller Suite [It] Job Webhook when With manageJobsWithoutQueueName enabled Should not update unsuspend Job successfully when adding queue name
Job Controller Suite [It] Job Webhook when With manageJobsWithoutQueueName enabled Should suspend a Job even no queue name specified
Job Controller Suite [It] Job Webhook when with manageJobsWithoutQueueName disabled should not suspend a Job when no queue name specified
Job Controller Suite [It] Job Webhook when with manageJobsWithoutQueueName disabled should not update unsuspend Job successfully when changing queue name
Job Controller Suite [It] Job Webhook when with manageJobsWithoutQueueName disabled should suspend a Job when created in unsuspend state
Job Controller Suite [It] Job controller Should reconcile workload and job for all jobs
Job Controller Suite [It] Job controller for workloads with no queue set Should reconcile jobs only when queue is set
Job Controller Suite [It] Job controller interacting with scheduler Should schedule jobs as they fit in their ClusterQueue
... skipping 645 lines ... 2022-11-25T12:59:14.293499476Z LEVEL(-2) core/resourceflavor_controller.go:69 Reconciling ResourceFlavor {"controller": "resourceflavor", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ResourceFlavor", "resourceFlavor": {"name":"on-demand"}, "namespace": "", "name": "on-demand", "reconcileID": "012233db-dd57-4634-80d9-d8d7ab6dfb5a", "resourceFlavor": {"name":"on-demand"}} 2022-11-25T12:59:14.295612122Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-resourceflavor", "UID": "57987954-f210-435b-876c-2f800878c7f0", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ResourceFlavor", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"resourceflavors"}} 2022-11-25T12:59:14.295755604Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-resourceflavor", "code": 200, "reason": "", "UID": "57987954-f210-435b-876c-2f800878c7f0", "allowed": true} 2022-11-25T12:59:14.297785858Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "fc204faf-bfeb-433f-8a38-a7e2e33edea9", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:14.298552515Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "fc204faf-bfeb-433f-8a38-a7e2e33edea9", "allowed": true} 2022-11-25T12:59:14.29980665Z LEVEL(-2) resourceflavor-reconciler core/resourceflavor_controller.go:128 ResourceFlavor delete event {"resourceFlavor": {"name":"on-demand"}} 2022/11/25 12:59:14 http: TLS handshake error from 127.0.0.1:39086: EOF 2022-11-25T12:59:14.302411085Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "363f9db4-c08b-48d8-8c13-7f4fbfb6775d", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:14.302773673Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "363f9db4-c08b-48d8-8c13-7f4fbfb6775d", "allowed": true} 2022-11-25T12:59:14.305138617Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:145 ClusterQueue create event {"clusterQueue": {"name":"clusterqueue"}} 2022-11-25T12:59:14.305349815Z LEVEL(-2) core/clusterqueue_controller.go:81 Reconciling ClusterQueue {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue", "clusterQueue": {"name":"clusterqueue"}, "namespace": "", "name": "clusterqueue", "reconcileID": "0e1be036-3b3c-4328-973d-95b1774401d2", "clusterQueue": {"name":"clusterqueue"}} 2022-11-25T12:59:14.306964092Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-localqueue", "UID": "d7b80612-4830-4b59-8781-8b6c26763609", "kind": "kueue.x-k8s.io/v1alpha2, Kind=LocalQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"localqueues"}} 2022-11-25T12:59:14.307137185Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-localqueue", "code": 200, "reason": "", "UID": "d7b80612-4830-4b59-8781-8b6c26763609", "allowed": true} ... skipping 115 lines ... 2022-11-25T12:59:15.901785317Z LEVEL(-2) core/localqueue_controller.go:75 Reconciling LocalQueue {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue", "localQueue": {"name":"queue","namespace":"core-workload-6sqkv"}, "namespace": "core-workload-6sqkv", "name": "queue", "reconcileID": "0fe717a9-2b21-4ef8-9fb7-eb9a20404e42", "localQueue": {"name":"queue","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.90422918Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "UID": "5a12baae-5e9e-44d7-bcc8-5e29a790fc0d", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:15.904627902Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "5a12baae-5e9e-44d7-bcc8-5e29a790fc0d", "allowed": true} 2022-11-25T12:59:15.906359856Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-workload", "UID": "4eaf69ea-90c4-4a45-8987-67c71b6164aa", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:15.906552057Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "4eaf69ea-90c4-4a45-8987-67c71b6164aa", "allowed": true} 2022-11-25T12:59:15.908852047Z LEVEL(-2) workload-reconciler core/workload_controller.go:119 Workload create event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}, "queue": "queue", "status": "admitted"} 2022-11-25T12:59:15.908924064Z ERROR workload-reconciler core/workload_controller.go:287 Could not get RuntimeClass {"error": "RuntimeClass.node.k8s.io \"kata\" not found"} sigs.k8s.io/kueue/pkg/controller/core.handlePodOverhead /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:287 sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Create /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:126 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnAdd /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:57 ... skipping 13 lines ... /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:15.909030255Z LEVEL(-2) core/workload_controller.go:82 Reconciling Workload {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "workload": {"name":"one","namespace":"core-workload-6sqkv"}, "namespace": "core-workload-6sqkv", "name": "one", "reconcileID": "f766024e-236c-47e0-a67d-7abdbee6ec7f", "workload": {"name":"one","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.9092792Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.909317012Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}} [1mSTEP:[0m Got ClusterQueueStatus [38;5;243m11/25/22 12:59:15.909[0m 2022-11-25T12:59:15.915742842Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}, "queue": "queue", "status": "admitted", "clusterQueue": "clusterqueue"} 2022-11-25T12:59:15.915817089Z ERROR workload-reconciler core/workload_controller.go:287 Could not get RuntimeClass {"error": "RuntimeClass.node.k8s.io \"kata\" not found"} sigs.k8s.io/kueue/pkg/controller/core.handlePodOverhead /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:287 sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:202 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 ... skipping 54 lines ... 2022-11-25T12:59:15.901785317Z LEVEL(-2) core/localqueue_controller.go:75 Reconciling LocalQueue {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue", "localQueue": {"name":"queue","namespace":"core-workload-6sqkv"}, "namespace": "core-workload-6sqkv", "name": "queue", "reconcileID": "0fe717a9-2b21-4ef8-9fb7-eb9a20404e42", "localQueue": {"name":"queue","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.90422918Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "UID": "5a12baae-5e9e-44d7-bcc8-5e29a790fc0d", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:15.904627902Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "5a12baae-5e9e-44d7-bcc8-5e29a790fc0d", "allowed": true} 2022-11-25T12:59:15.906359856Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-workload", "UID": "4eaf69ea-90c4-4a45-8987-67c71b6164aa", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:15.906552057Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "4eaf69ea-90c4-4a45-8987-67c71b6164aa", "allowed": true} 2022-11-25T12:59:15.908852047Z LEVEL(-2) workload-reconciler core/workload_controller.go:119 Workload create event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}, "queue": "queue", "status": "admitted"} 2022-11-25T12:59:15.908924064Z ERROR workload-reconciler core/workload_controller.go:287 Could not get RuntimeClass {"error": "RuntimeClass.node.k8s.io \"kata\" not found"} sigs.k8s.io/kueue/pkg/controller/core.handlePodOverhead /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:287 sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Create /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:126 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnAdd /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:57 ... skipping 13 lines ... /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:15.909030255Z LEVEL(-2) core/workload_controller.go:82 Reconciling Workload {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "workload": {"name":"one","namespace":"core-workload-6sqkv"}, "namespace": "core-workload-6sqkv", "name": "one", "reconcileID": "f766024e-236c-47e0-a67d-7abdbee6ec7f", "workload": {"name":"one","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.9092792Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}} 2022-11-25T12:59:15.909317012Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}} [1mSTEP:[0m Got ClusterQueueStatus [38;5;243m11/25/22 12:59:15.909[0m 2022-11-25T12:59:15.915742842Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"one","namespace":"core-workload-6sqkv"}, "queue": "queue", "status": "admitted", "clusterQueue": "clusterqueue"} 2022-11-25T12:59:15.915817089Z ERROR workload-reconciler core/workload_controller.go:287 Could not get RuntimeClass {"error": "RuntimeClass.node.k8s.io \"kata\" not found"} sigs.k8s.io/kueue/pkg/controller/core.handlePodOverhead /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:287 sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:202 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 ... skipping 215 lines ... 2022-11-25T12:59:18.108273623Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.108375206Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.108265238Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.113656435Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "UID": "db46e8a3-d821-4e77-aa2d-4730ca322622", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:18.114246194Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "db46e8a3-d821-4e77-aa2d-4730ca322622", "allowed": true} 2022-11-25T12:59:18.114717129Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "admitted", "clusterQueue": "other"} 2022-11-25T12:59:18.114807103Z ERROR workload-reconciler core/workload_controller.go:240 Updating workload in cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "admitted", "clusterQueue": "other", "error": "old ClusterQueue doesn't exist"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:240 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:816 ... skipping 45 lines ... 2022-11-25T12:59:19.173909212Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173953472Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173974778Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173988209Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.17393518Z LEVEL(-2) core/workload_controller.go:82 Reconciling Workload {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "namespace": "core-clusterqueue-sx74m", "name": "four", "reconcileID": "260eecb1-fc1e-43e1-9487-1811b15ec54c", "workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.182580734Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "prevStatus": "admitted", "clusterQueue": "other"} 2022-11-25T12:59:19.182653129Z ERROR workload-reconciler core/workload_controller.go:207 Failed to delete workload from cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "prevStatus": "admitted", "clusterQueue": "other", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:207 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:816 ... skipping 31 lines ... 2022-11-25T12:59:20.221005304Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "b4ff3ba6-52b9-4d68-ab03-db126c6d3ce5", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:20.221101744Z LEVEL(-2) localqueue-reconciler core/localqueue_controller.go:119 LocalQueue delete event {"localQueue": {"name":"queue","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.221511159Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "b4ff3ba6-52b9-4d68-ab03-db126c6d3ce5", "allowed": true} 2022-11-25T12:59:20.225868629Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:165 ClusterQueue delete event {"clusterQueue": {"name":"cluster-queue"}} 2022-11-25T12:59:20.226138366Z LEVEL(-2) resourceflavor-reconciler core/resourceflavor_controller.go:156 Got generic event {"obj": {"name":"cluster-queue"}, "kind": "/, Kind="} 2022-11-25T12:59:20.228516024Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished"} 2022-11-25T12:59:20.22857333Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.228685784Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.22871646Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.230177607Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.230220697Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.230282901Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.230323558Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.232766354Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.232813635Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 12 lines ... 2022-11-25T12:59:20.232981246Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.23304536Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.23467726Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.234770511Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.234752441Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.237022952Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.237064311Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.237183872Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.237233498Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.238644289Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"two","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.238681595Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"two","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 202 lines ... 2022-11-25T12:59:18.108273623Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.108375206Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.108265238Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:18.113656435Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "UID": "db46e8a3-d821-4e77-aa2d-4730ca322622", "kind": "kueue.x-k8s.io/v1alpha2, Kind=Workload", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"workloads"}} 2022-11-25T12:59:18.114246194Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/mutate-kueue-x-k8s-io-v1alpha2-workload", "code": 200, "reason": "", "UID": "db46e8a3-d821-4e77-aa2d-4730ca322622", "allowed": true} 2022-11-25T12:59:18.114717129Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "admitted", "clusterQueue": "other"} 2022-11-25T12:59:18.114807103Z ERROR workload-reconciler core/workload_controller.go:240 Updating workload in cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "admitted", "clusterQueue": "other", "error": "old ClusterQueue doesn't exist"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:240 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:816 ... skipping 45 lines ... 2022-11-25T12:59:19.173909212Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173953472Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173974778Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.173988209Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.17393518Z LEVEL(-2) core/workload_controller.go:82 Reconciling Workload {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload", "workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "namespace": "core-clusterqueue-sx74m", "name": "four", "reconcileID": "260eecb1-fc1e-43e1-9487-1811b15ec54c", "workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:19.182580734Z LEVEL(-2) workload-reconciler core/workload_controller.go:198 Workload update event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "prevStatus": "admitted", "clusterQueue": "other"} 2022-11-25T12:59:19.182653129Z ERROR workload-reconciler core/workload_controller.go:207 Failed to delete workload from cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "prevStatus": "admitted", "clusterQueue": "other", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Update /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:207 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnUpdate /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:88 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:816 ... skipping 31 lines ... 2022-11-25T12:59:20.221005304Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "b4ff3ba6-52b9-4d68-ab03-db126c6d3ce5", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:20.221101744Z LEVEL(-2) localqueue-reconciler core/localqueue_controller.go:119 LocalQueue delete event {"localQueue": {"name":"queue","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.221511159Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "b4ff3ba6-52b9-4d68-ab03-db126c6d3ce5", "allowed": true} 2022-11-25T12:59:20.225868629Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:165 ClusterQueue delete event {"clusterQueue": {"name":"cluster-queue"}} 2022-11-25T12:59:20.226138366Z LEVEL(-2) resourceflavor-reconciler core/resourceflavor_controller.go:156 Got generic event {"obj": {"name":"cluster-queue"}, "kind": "/, Kind="} 2022-11-25T12:59:20.228516024Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished"} 2022-11-25T12:59:20.22857333Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}, "queue": "other", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.228685784Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.22871646Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"five","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.230177607Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.230220697Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.230282901Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.230323558Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"four","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.232766354Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.232813635Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 12 lines ... 2022-11-25T12:59:20.232981246Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.23304536Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"one","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.23467726Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.234770511Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.234752441Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"six","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.237022952Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.237064311Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 9 lines ... /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:812 k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1 /home/prow/go/pkg/mod/k8s.io/apimachinery@v0.24.3/pkg/util/wait/wait.go:73 2022-11-25T12:59:20.237183872Z LEVEL(-3) localqueue-reconciler core/localqueue_controller.go:144 Got Workload event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.237233498Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:201 Got Workload event {"workload": {"name":"three","namespace":"core-clusterqueue-sx74m"}} 2022-11-25T12:59:20.238644289Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"two","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:20.238681595Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"two","namespace":"core-clusterqueue-sx74m"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 106 lines ... 2022-11-25T12:59:21.809162732Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "0fd7f6ba-4f9d-4014-9e22-e3ccce4aafd8", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:21.809362873Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "0fd7f6ba-4f9d-4014-9e22-e3ccce4aafd8", "allowed": true} 2022-11-25T12:59:21.81264038Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:165 ClusterQueue delete event {"clusterQueue": {"name":"foo-cq"}} 2022-11-25T12:59:21.812855242Z LEVEL(-2) resourceflavor-reconciler core/resourceflavor_controller.go:156 Got generic event {"obj": {"name":"foo-cq"}, "kind": "/, Kind="} 2022-11-25T12:59:21.841264785Z LEVEL(-2) localqueue-reconciler core/localqueue_controller.go:119 LocalQueue delete event {"localQueue": {"name":"queue","namespace":"core-clusterqueue-s4cb4"}} 2022-11-25T12:59:21.846596563Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"workload","namespace":"core-clusterqueue-s4cb4"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:21.846651961Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"workload","namespace":"core-clusterqueue-s4cb4"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 62 lines ... 2022-11-25T12:59:21.809162732Z DEBUG controller-runtime.webhook.webhooks admission/http.go:96 received request {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "UID": "0fd7f6ba-4f9d-4014-9e22-e3ccce4aafd8", "kind": "kueue.x-k8s.io/v1alpha2, Kind=ClusterQueue", "resource": {"group":"kueue.x-k8s.io","version":"v1alpha2","resource":"clusterqueues"}} 2022-11-25T12:59:21.809362873Z DEBUG controller-runtime.webhook.webhooks admission/http.go:143 wrote response {"webhook": "/validate-kueue-x-k8s-io-v1alpha2-clusterqueue", "code": 200, "reason": "", "UID": "0fd7f6ba-4f9d-4014-9e22-e3ccce4aafd8", "allowed": true} 2022-11-25T12:59:21.81264038Z LEVEL(-2) cluster-queue-reconciler core/clusterqueue_controller.go:165 ClusterQueue delete event {"clusterQueue": {"name":"foo-cq"}} 2022-11-25T12:59:21.812855242Z LEVEL(-2) resourceflavor-reconciler core/resourceflavor_controller.go:156 Got generic event {"obj": {"name":"foo-cq"}, "kind": "/, Kind="} 2022-11-25T12:59:21.841264785Z LEVEL(-2) localqueue-reconciler core/localqueue_controller.go:119 LocalQueue delete event {"localQueue": {"name":"queue","namespace":"core-clusterqueue-s4cb4"}} 2022-11-25T12:59:21.846596563Z LEVEL(-2) workload-reconciler core/workload_controller.go:149 Workload delete event {"workload": {"name":"workload","namespace":"core-clusterqueue-s4cb4"}, "queue": "queue", "status": "finished"} 2022-11-25T12:59:21.846651961Z ERROR workload-reconciler core/workload_controller.go:158 Failed to delete workload from cache {"workload": {"name":"workload","namespace":"core-clusterqueue-s4cb4"}, "queue": "queue", "status": "finished", "error": "cluster queue not found"} sigs.k8s.io/kueue/pkg/controller/core.(*WorkloadReconciler).Delete /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/core/workload_controller.go:158 sigs.k8s.io/controller-runtime/pkg/source/internal.EventHandler.OnDelete /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/source/internal/eventsource.go:131 k8s.io/client-go/tools/cache.(*processorListener).run.func1 /home/prow/go/pkg/mod/k8s.io/client-go@v0.24.3/tools/cache/shared_informer.go:820 ... skipping 317 lines ... 2022-11-25T12:59:25.009226936Z LEVEL(-2) core/clusterqueue_controller.go:81 Reconciling ClusterQueue {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue", "clusterQueue": {"name":"cluster-queue.queue-controller"}, "namespace": "", "name": "cluster-queue.queue-controller", "reconcileID": "7f770119-e10e-4602-b8e2-4c9fbace3065", "clusterQueue": {"name":"cluster-queue.queue-controller"}} 2022-11-25T12:59:25.009201621Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for non leader election runnables 2022-11-25T12:59:25.009434926Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for leader election runnables 2022-11-25T12:59:25.009417658Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-395016376/tls.key\": REMOVE"} 2022-11-25T12:59:25.009490071Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "resourceflavor", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ResourceFlavor"} 2022-11-25T12:59:25.009536054Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload"} 2022-11-25T12:59:25.009534541Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009548349Z ERROR controller/controller.go:326 Reconciler error {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue", "clusterQueue": {"name":"cluster-queue.queue-controller"}, "namespace": "", "name": "cluster-queue.queue-controller", "reconcileID": "7f770119-e10e-4602-b8e2-4c9fbace3065", "error": "Put \"https://127.0.0.1:44317/apis/kueue.x-k8s.io/v1alpha2/clusterqueues/cluster-queue.queue-controller\": context canceled"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:25.009572711Z INFO controller/controller.go:249 All workers finished {"controller": "resourceflavor", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ResourceFlavor"} 2022-11-25T12:59:25.009583151Z INFO controller/controller.go:249 All workers finished {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload"} 2022-11-25T12:59:25.009518944Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue"} 2022-11-25T12:59:25.009589076Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-395016376/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009608949Z INFO controller/controller.go:249 All workers finished {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue"} 2022-11-25T12:59:25.009609832Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-395016376/tls.crt\": REMOVE"} 2022-11-25T12:59:25.009628731Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009547942Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue"} 2022-11-25T12:59:25.009648558Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-395016376/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009649132Z INFO controller/controller.go:249 All workers finished {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue"} ... skipping 12 lines ... 2022-11-25T12:59:25.009226936Z LEVEL(-2) core/clusterqueue_controller.go:81 Reconciling ClusterQueue {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue", "clusterQueue": {"name":"cluster-queue.queue-controller"}, "namespace": "", "name": "cluster-queue.queue-controller", "reconcileID": "7f770119-e10e-4602-b8e2-4c9fbace3065", "clusterQueue": {"name":"cluster-queue.queue-controller"}} 2022-11-25T12:59:25.009201621Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for non leader election runnables 2022-11-25T12:59:25.009434926Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for leader election runnables 2022-11-25T12:59:25.009417658Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-395016376/tls.key\": REMOVE"} 2022-11-25T12:59:25.009490071Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "resourceflavor", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ResourceFlavor"} 2022-11-25T12:59:25.009536054Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload"} 2022-11-25T12:59:25.009534541Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009548349Z ERROR controller/controller.go:326 Reconciler error {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue", "clusterQueue": {"name":"cluster-queue.queue-controller"}, "namespace": "", "name": "cluster-queue.queue-controller", "reconcileID": "7f770119-e10e-4602-b8e2-4c9fbace3065", "error": "Put \"https://127.0.0.1:44317/apis/kueue.x-k8s.io/v1alpha2/clusterqueues/cluster-queue.queue-controller\": context canceled"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:25.009572711Z INFO controller/controller.go:249 All workers finished {"controller": "resourceflavor", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ResourceFlavor"} 2022-11-25T12:59:25.009583151Z INFO controller/controller.go:249 All workers finished {"controller": "workload", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "Workload"} 2022-11-25T12:59:25.009518944Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue"} 2022-11-25T12:59:25.009589076Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-395016376/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009608949Z INFO controller/controller.go:249 All workers finished {"controller": "localqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "LocalQueue"} 2022-11-25T12:59:25.009609832Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-395016376/tls.crt\": REMOVE"} 2022-11-25T12:59:25.009628731Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009547942Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue"} 2022-11-25T12:59:25.009648558Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-395016376/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:25.009649132Z INFO controller/controller.go:249 All workers finished {"controller": "clusterqueue", "controllerGroup": "kueue.x-k8s.io", "controllerKind": "ClusterQueue"} ... skipping 9 lines ... [38;5;10m[ReportAfterSuite] PASSED [0.004 seconds][0m [ReportAfterSuite] Autogenerated ReportAfterSuite for --junit-report [38;5;243mautogenerated by Ginkgo[0m [38;5;243m------------------------------[0m [38;5;10m[1mRan 12 of 12 Specs in 23.189 seconds[0m [38;5;10m[1mSUCCESS![0m -- [38;5;10m[1m12 Passed[0m | [38;5;9m[1m0 Failed[0m | [38;5;11m[1m0 Pending[0m | [38;5;14m[1m0 Skipped[0m PASS Running Suite: Job Controller Suite - /home/prow/go/src/sigs.k8s.io/kueue/test/integration/controller/job ========================================================================================================= Random Seed: [1m1669381133[0m Will run [1m8[0m of [1m8[0m specs ... skipping 41 lines ... 2022-11-25T12:59:34.407197612Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "76d2c3f7-7098-4b30-a8c5-3fabd9c64b6b", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.407306855Z LEVEL(-2) job/job_controller.go:200 Job changed queues, updating workload {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "76d2c3f7-7098-4b30-a8c5-3fabd9c64b6b", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.413989468Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "375e91f2-ea9e-468d-9f05-09195979638c", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.414097219Z LEVEL(-3) job/job_controller.go:208 Job is suspended and workload not yet admitted by a clusterQueue, nothing to do {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "375e91f2-ea9e-468d-9f05-09195979638c", "job": {"name":"test-job","namespace":"default"}} [1mSTEP:[0m checking a second non-matching workload is deleted [38;5;243m11/25/22 12:59:34.664[0m 2022-11-25T12:59:34.67384398Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.679557942Z ERROR job/job_controller.go:153 Getting existing workloads {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "job": {"name":"test-job","namespace":"default"}, "error": "only one workload should exist, found 2"} sigs.k8s.io/kueue/pkg/controller/workload/job.(*JobReconciler).Reconcile /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/workload/job/job_controller.go:153 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:121 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:320 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:34.679670527Z ERROR controller/controller.go:326 Reconciler error {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "error": "only one workload should exist, found 2"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 ... skipping 9 lines ... 2022-11-25T12:59:38.092835668Z DEBUG events recorder/recorder.go:103 Normal {"object": {"kind":"Job","namespace":"default","name":"test-job","uid":"aa04ec8e-bab7-4dd8-a615-dcad19f9338a","apiVersion":"batch/v1","resourceVersion":"227"}, "reason": "Started", "message": "Admitted by clusterQueue cluster-queue"} 2022-11-25T12:59:38.092949161Z LEVEL(-3) job/job_controller.go:223 Job running with admitted workload, nothing to do {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "e601a184-8a21-4de5-8041-6c0e4ba3fcaf", "job": {"name":"test-job","namespace":"default"}} [1mSTEP:[0m checking the job gets suspended when parallelism changes and the added node selectors are removed [38;5;243m11/25/22 12:59:41.246[0m 2022-11-25T12:59:41.252833455Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:41.252943831Z LEVEL(-2) job/job_controller.go:362 job with no matching workload, suspending {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:41.256781665Z DEBUG events recorder/recorder.go:103 Normal {"object": {"kind":"Job","namespace":"default","name":"test-job","uid":"aa04ec8e-bab7-4dd8-a615-dcad19f9338a","apiVersion":"batch/v1","resourceVersion":"231"}, "reason": "Stopped", "message": "No matching Workload"} 2022-11-25T12:59:41.265015687Z ERROR job/job_controller.go:153 Getting existing workloads {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}, "error": "no matching workload was found, tried deleting 1 existing workload(s)"} sigs.k8s.io/kueue/pkg/controller/workload/job.(*JobReconciler).Reconcile /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/workload/job/job_controller.go:153 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:121 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:320 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:41.26509569Z ERROR controller/controller.go:326 Reconciler error {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "error": "no matching workload was found, tried deleting 1 existing workload(s)"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 ... skipping 17 lines ... 2022-11-25T12:59:45.037812481Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for leader election runnables 2022-11-25T12:59:45.037839632Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job"} 2022-11-25T12:59:45.037890756Z INFO controller/controller.go:249 All workers finished {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job"} 2022-11-25T12:59:45.037905573Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for caches 2022-11-25T12:59:45.038107772Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-2730287444/tls.key\": REMOVE"} 2022-11-25T12:59:45.03817182Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for webhooks 2022-11-25T12:59:45.038205039Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:45.038248862Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-2730287444/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:45.038276783Z INFO controller-runtime.webhook logr@v1.2.3/logr.go:261 shutting down webhook server ... skipping 46 lines ... 2022-11-25T12:59:34.407197612Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "76d2c3f7-7098-4b30-a8c5-3fabd9c64b6b", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.407306855Z LEVEL(-2) job/job_controller.go:200 Job changed queues, updating workload {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "76d2c3f7-7098-4b30-a8c5-3fabd9c64b6b", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.413989468Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "375e91f2-ea9e-468d-9f05-09195979638c", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.414097219Z LEVEL(-3) job/job_controller.go:208 Job is suspended and workload not yet admitted by a clusterQueue, nothing to do {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "375e91f2-ea9e-468d-9f05-09195979638c", "job": {"name":"test-job","namespace":"default"}} [1mSTEP:[0m checking a second non-matching workload is deleted [38;5;243m11/25/22 12:59:34.664[0m 2022-11-25T12:59:34.67384398Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:34.679557942Z ERROR job/job_controller.go:153 Getting existing workloads {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "job": {"name":"test-job","namespace":"default"}, "error": "only one workload should exist, found 2"} sigs.k8s.io/kueue/pkg/controller/workload/job.(*JobReconciler).Reconcile /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/workload/job/job_controller.go:153 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:121 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:320 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:34.679670527Z ERROR controller/controller.go:326 Reconciler error {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "37748b64-8be4-42f6-a26f-6fb8ac9f4a58", "error": "only one workload should exist, found 2"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 ... skipping 9 lines ... 2022-11-25T12:59:38.092835668Z DEBUG events recorder/recorder.go:103 Normal {"object": {"kind":"Job","namespace":"default","name":"test-job","uid":"aa04ec8e-bab7-4dd8-a615-dcad19f9338a","apiVersion":"batch/v1","resourceVersion":"227"}, "reason": "Started", "message": "Admitted by clusterQueue cluster-queue"} 2022-11-25T12:59:38.092949161Z LEVEL(-3) job/job_controller.go:223 Job running with admitted workload, nothing to do {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "e601a184-8a21-4de5-8041-6c0e4ba3fcaf", "job": {"name":"test-job","namespace":"default"}} [1mSTEP:[0m checking the job gets suspended when parallelism changes and the added node selectors are removed [38;5;243m11/25/22 12:59:41.246[0m 2022-11-25T12:59:41.252833455Z LEVEL(-2) job/job_controller.go:141 Reconciling Job {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:41.252943831Z LEVEL(-2) job/job_controller.go:362 job with no matching workload, suspending {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}} 2022-11-25T12:59:41.256781665Z DEBUG events recorder/recorder.go:103 Normal {"object": {"kind":"Job","namespace":"default","name":"test-job","uid":"aa04ec8e-bab7-4dd8-a615-dcad19f9338a","apiVersion":"batch/v1","resourceVersion":"231"}, "reason": "Stopped", "message": "No matching Workload"} 2022-11-25T12:59:41.265015687Z ERROR job/job_controller.go:153 Getting existing workloads {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "job": {"name":"test-job","namespace":"default"}, "error": "no matching workload was found, tried deleting 1 existing workload(s)"} sigs.k8s.io/kueue/pkg/controller/workload/job.(*JobReconciler).Reconcile /home/prow/go/src/sigs.k8s.io/kueue/pkg/controller/workload/job/job_controller.go:153 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:121 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:320 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 2022-11-25T12:59:41.26509569Z ERROR controller/controller.go:326 Reconciler error {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job", "job": {"name":"test-job","namespace":"default"}, "namespace": "default", "name": "test-job", "reconcileID": "699cba3e-49f2-4afe-9fb2-f1d6efb1af9e", "error": "no matching workload was found, tried deleting 1 existing workload(s)"} sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:326 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:273 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2 /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/internal/controller/controller.go:234 ... skipping 17 lines ... 2022-11-25T12:59:45.037812481Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for leader election runnables 2022-11-25T12:59:45.037839632Z INFO controller/controller.go:247 Shutdown signal received, waiting for all workers to finish {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job"} 2022-11-25T12:59:45.037890756Z INFO controller/controller.go:249 All workers finished {"controller": "job", "controllerGroup": "batch", "controllerKind": "Job"} 2022-11-25T12:59:45.037905573Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for caches 2022-11-25T12:59:45.038107772Z DEBUG controller-runtime.certwatcher logr@v1.2.3/logr.go:261 certificate event {"event": "\"/tmp/envtest-serving-certs-2730287444/tls.key\": REMOVE"} 2022-11-25T12:59:45.03817182Z INFO logr@v1.2.3/logr.go:261 Stopping and waiting for webhooks 2022-11-25T12:59:45.038205039Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-watching file {"error": "no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:147 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:45.038248862Z ERROR controller-runtime.certwatcher logr@v1.2.3/logr.go:279 error re-reading certificate {"error": "open /tmp/envtest-serving-certs-2730287444/tls.crt: no such file or directory"} github.com/go-logr/logr.Logger.Error /home/prow/go/pkg/mod/github.com/go-logr/logr@v1.2.3/logr.go:279 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).handleEvent /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:152 sigs.k8s.io/controller-runtime/pkg/certwatcher.(*CertWatcher).Watch /home/prow/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.12.3/pkg/certwatcher/certwatcher.go:103 2022-11-25T12:59:45.038276783Z INFO controller-runtime.webhook logr@v1.2.3/logr.go:261 shutting down webhook server ... skipping 2 lines ... [38;5;243m------------------------------[0m [0mJob Webhook [38;5;243mwhen With manageJobsWithoutQueueName enabled[0m [1mShould suspend a Job even no queue name specified[0m [38;5;243m/home/prow/go/src/sigs.k8s.io/kueue/test/integration/controller/job/job_webhook_test.go:59[0m [1mSTEP:[0m bootstrapping test environment [38;5;243m11/25/22 12:59:46.189[0m 2022-11-25T12:59:46.189864729Z DEBUG controller-runtime.test-env logr@v1.2.3/logr.go:261 starting control plane {"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:168","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Entrypoint received interrupt: terminated","severity":"error","time":"2022-11-25T12:59:47Z"}