This job view page is being replaced by Spyglass soon. Check out the new job view.
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2022-08-10 03:03
Elapsed5m20s
Revisionrelease-0.3

No Test Failures!


Error lines from build-log.txt

... skipping 773 lines ...
I0810 03:06:18.318137    8362 server.go:121] controller-runtime/webhook "msg"="registering webhook"  "path"="/validate-addons-cluster-x-k8s-io-v1alpha3-clusterresourceset"
I0810 03:06:18.319741    8362 server.go:141] controller-runtime/webhook/webhooks "msg"="starting webhook server"  
I0810 03:06:18.320810    8362 certwatcher.go:127] controller-runtime/certwatcher "msg"="Updated current TLS certificate"  
I0810 03:06:18.320983    8362 server.go:199] controller-runtime/webhook "msg"="serving webhook server"  "host"="" "port"=38187
I0810 03:06:18.321148    8362 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
I0810 03:06:18.891653    8362 kubeadmconfig_controller.go:223]  "msg"="Cluster infrastructure is not ready, waiting" "kind"="Machine" "kubeadmconfig"={"Namespace":"default","Name":"my-machine-config"} "name"="my-machine" "version"="109" 
•E0810 03:06:18.901206    8362 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0810 03:06:18.901260    8362 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-782674015/tls.crt: no such file or directory"  
E0810 03:06:18.901287    8362 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0810 03:06:18.901314    8362 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-782674015/tls.crt: no such file or directory"  
I0810 03:06:18.901407    8362 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 1 of 1 Specs in 14.279 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (14.28s)
PASS
ok  	sigs.k8s.io/cluster-api/bootstrap/kubeadm/controllers	51.974s
=== RUN   TestNewInitControlPlaneAdditionalFileEncodings
--- PASS: TestNewInitControlPlaneAdditionalFileEncodings (0.00s)
=== RUN   TestNewInitControlPlaneCommands
... skipping 316 lines ...
=== RUN   Test_clusterctlClient_GetProvidersConfig/Returns_default_providers_and_custom_providers_if_defined
--- PASS: Test_clusterctlClient_GetProvidersConfig (0.00s)
    --- PASS: Test_clusterctlClient_GetProvidersConfig/Returns_default_providers (0.00s)
    --- PASS: Test_clusterctlClient_GetProvidersConfig/Returns_default_providers_and_custom_providers_if_defined (0.00s)
=== RUN   Test_clusterctlClient_GetProviderComponents
=== RUN   Test_clusterctlClient_GetProviderComponents/Pass
=== RUN   Test_clusterctlClient_GetProviderComponents/Fail
--- PASS: Test_clusterctlClient_GetProviderComponents (0.00s)
    --- PASS: Test_clusterctlClient_GetProviderComponents/Pass (0.00s)
    --- PASS: Test_clusterctlClient_GetProviderComponents/Fail (0.00s)
=== RUN   Test_getComponentsByName_withEmptyVariables
--- PASS: Test_getComponentsByName_withEmptyVariables (0.00s)
=== RUN   Test_clusterctlClient_templateOptionsToVariables
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_KubernetesVersion_from_template_options)
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_KubernetesVersion_from_env_variables)
=== RUN   Test_clusterctlClient_templateOptionsToVariables/pass_(using_defaults_for_machine_counts)
... skipping 1403 lines ...
I0810 03:05:59.707354   10479 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="machinedeployment" "worker count"=1
=== RUN   TestClusterReconcilePhases
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_no_error_if_infrastructure_ref_is_nil
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_error_if_unable_to_reconcile_infrastructure_ref
I0810 03:05:59.906311   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test-cluster" "namespace"="test-namespace" "err"="cannot create a metadata client without a rest config"
2022/08/10 03:05:59 http: TLS handshake error from 127.0.0.1:44172: EOF
I0810 03:05:59.910406   10479 cluster_controller_phases.go:76]  "msg"="Could not find external object for cluster, requeuing" "cluster"="test-cluster" "namespace"="test-namespace" "refGroupVersionKind"={"Group":"infrastructure.cluster.x-k8s.io","Version":"v1alpha3","Kind":"InfrastructureMachine"} "refName"="test"
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_no_error_if_infra_config_is_marked_for_deletion
I0810 03:05:59.910987   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test-cluster" "namespace"="test-namespace" "err"="cannot create a metadata client without a rest config"
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_no_error_if_infrastructure_is_marked_ready_on_cluster
I0810 03:05:59.912768   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test-cluster" "namespace"="test-namespace" "err"="cannot create a metadata client without a rest config"
=== RUN   TestClusterReconcilePhases/reconcile_infrastructure/returns_error_if_infrastructure_has_the_paused_annotation
... skipping 100 lines ...
    --- PASS: TestReconcileBootstrap/existing_machine,_machineset_owner_and_version_v1alpha2,_and_ownerref_updated (0.00s)
=== RUN   TestReconcileInfrastructure
=== RUN   TestReconcileInfrastructure/new_machine,_infrastructure_config_ready
I0810 03:05:59.947465   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
=== RUN   TestReconcileInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machine_is_running,_infra_object_is_deleted,_expect_failed
I0810 03:05:59.950548   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
E0810 03:05:59.951135   10479 machine_controller_phases.go:246]  "msg"="Machine infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-test\" in namespace \"default\", requeuing: requeue in 1s"  
=== RUN   TestReconcileInfrastructure/infrastructure_ref_is_paused
I0810 03:05:59.951926   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
--- PASS: TestReconcileInfrastructure (0.01s)
    --- PASS: TestReconcileInfrastructure/new_machine,_infrastructure_config_ready (0.00s)
    --- PASS: TestReconcileInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machine_is_running,_infra_object_is_deleted,_expect_failed (0.00s)
    --- PASS: TestReconcileInfrastructure/infrastructure_ref_is_paused (0.00s)
=== RUN   TestWatches
I0810 03:06:00.159172   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={}
E0810 03:06:00.164160   10479 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
I0810 03:06:00.172148   10479 tracker.go:55] controllers/Machine "msg"="Adding watcher on external object" "machine"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr" "GroupVersionKind"="bootstrap.cluster.x-k8s.io/v1alpha3, Kind=BootstrapMachine"
I0810 03:06:00.172234   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={"Type":{"apiVersion":"bootstrap.cluster.x-k8s.io/v1alpha3","kind":"BootstrapMachine"}}
I0810 03:06:00.281201   10479 tracker.go:55] controllers/Machine "msg"="Adding watcher on external object" "machine"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr" "GroupVersionKind"="infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine"
I0810 03:06:00.281285   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machine" "source"={"Type":{"apiVersion":"infrastructure.cluster.x-k8s.io/v1alpha3","kind":"InfrastructureMachine"}}
I0810 03:06:00.381808   10479 machine_controller_noderef.go:86] controllers/Machine "msg"="Set Machine's NodeRef" "machine"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr" "noderef"="node-1"
E0810 03:06:00.391092   10479 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
E0810 03:06:00.391145   10479 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="node-1"
E0810 03:06:00.523781   10479 machine_controller_phases.go:246] controllers/Machine "msg"="Machine infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-created-6dfm2\" in namespace \"test-machine-watches-j2vxr\", requeuing: requeue in 1s"  
E0810 03:06:00.537171   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureMachine \"infra-config1\" for Machine \"machine-created-6dfm2\" in namespace \"test-machine-watches-j2vxr\", requeuing: requeue in 1s, no matching Node for Machine \"machine-created-6dfm2\" in namespace \"test-machine-watches-j2vxr\": cannot find node with matching ProviderID]" "controller"="machine" "name"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr"
--- PASS: TestWatches (0.61s)
=== RUN   TestIndexMachineByNodeName
=== RUN   TestIndexMachineByNodeName/when_the_machine_has_no_NodeRef
=== RUN   TestIndexMachineByNodeName/when_the_machine_has_valid_a_NodeRef
--- PASS: TestIndexMachineByNodeName (0.00s)
    --- PASS: TestIndexMachineByNodeName/when_the_machine_has_no_NodeRef (0.00s)
... skipping 2 lines ...
=== RUN   TestMachine_Reconcile/reconcile_create
I0810 03:06:00.754111   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="machine-reconcile-442ff" "namespace"="test-machine-watches-j2vxr" "count"=1
I0810 03:06:00.754189   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="machine-reconcile-442ff" "namespace"="test-machine-watches-j2vxr" "descendants"="Worker machines: machine-created-6dfm2" "indirect descendants count"=0
I0810 03:06:00.771746   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="machine-reconcile-442ff" "namespace"="test-machine-watches-j2vxr" "count"=1
I0810 03:06:00.771809   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="machine-reconcile-442ff" "namespace"="test-machine-watches-j2vxr" "descendants"="Worker machines: machine-created-6dfm2" "indirect descendants count"=0
I0810 03:06:01.537723   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-442ff" "machine"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr" "cause"="cluster is being deleted" "node"={"kind":"Node","name":"node-1","uid":"01551030-57ce-4be0-a0a7-25ad20efdfc2","apiVersion":"v1"}
E0810 03:06:01.589190   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"machine-created-6dfm2\" not found" "controller"="machine" "name"="machine-created-6dfm2" "namespace"="test-machine-watches-j2vxr"
E0810 03:06:02.645475   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[retrieved empty dataSecretName from bootstrap provider for Machine \"machine-created-jf89v\" in namespace \"default\", error creating client and cache for remote cluster: error fetching REST client config for remote cluster \"default/machine-reconcile-c7mss\": failed to retrieve kubeconfig secret for Cluster default/machine-reconcile-c7mss: secrets \"machine-reconcile-c7mss-kubeconfig\" not found]" "controller"="machine" "name"="machine-created-jf89v" "namespace"="default"
I0810 03:06:03.646033   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-c7mss" "machine"="machine-created-jf89v" "namespace"="default" "cause"="noderef is nil" "node"=null
I0810 03:06:03.690091   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-c7mss" "machine"="machine-created-jf89v" "namespace"="default" "cause"="noderef is nil" "node"=null
I0810 03:06:03.713316   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="machine-reconcile-c7mss" "machine"="machine-created-jf89v" "namespace"="default" "cause"="noderef is nil" "node"=null
E0810 03:06:03.751176   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"machine-created-jf89v\" not found" "controller"="machine" "name"="machine-created-jf89v" "namespace"="default"
--- PASS: TestMachine_Reconcile (3.20s)
    --- PASS: TestMachine_Reconcile/reconcile_create (3.20s)
=== RUN   TestMachineFinalizer
=== RUN   TestMachineFinalizer/should_add_a_machine_finalizer_to_the_machine_if_it_doesn't_have_one
=== RUN   TestMachineFinalizer/should_append_the_machine_finalizer_to_the_machine_if_it_already_has_a_finalizer
--- PASS: TestMachineFinalizer (0.00s)
... skipping 4 lines ...
I0810 03:06:03.772477   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine2" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0810 03:06:03.772610   10479 machine_controller_noderef.go:45]  "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="machine2" "namespace"="default" 
=== RUN   TestMachineOwnerReference/should_not_add_cluster_owner_reference_if_machine_is_owned_by_a_machine_set
I0810 03:06:03.781080   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine3" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0810 03:06:03.781221   10479 machine_controller_noderef.go:45]  "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="machine3" "namespace"="default" 
=== RUN   TestMachineOwnerReference/should_not_add_cluster_owner_reference_if_machine_has_a_controller_owner
E0810 03:06:03.784378   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"machine-reconcile-c7mss\" not found" "controller"="cluster" "name"="machine-reconcile-c7mss" "namespace"="default"
I0810 03:06:03.787723   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine4" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0810 03:06:03.787851   10479 machine_controller_noderef.go:45]  "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="machine4" "namespace"="default" 
--- PASS: TestMachineOwnerReference (0.02s)
    --- PASS: TestMachineOwnerReference/should_add_owner_reference_to_machine_referencing_a_cluster_with_correct_type_meta (0.00s)
    --- PASS: TestMachineOwnerReference/should_not_add_cluster_owner_reference_if_machine_is_owned_by_a_machine_set (0.01s)
    --- PASS: TestMachineOwnerReference/should_not_add_cluster_owner_reference_if_machine_has_a_controller_owner (0.01s)
... skipping 123 lines ...
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist
I0810 03:06:03.976701   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6mz9z" "namespace"="test-mhc-n8f77" 
I0810 03:06:04.024831   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist
error cannot retrieve mhc in ctx: MachineHealthCheck.cluster.x-k8s.io "test-mhc-jfblq" not foundE0810 03:06:04.126434   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-n8f77/test-cluster-ghs5g"
E0810 03:06:04.130073   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[MachineHealthCheck.cluster.x-k8s.io \"test-mhc-6mz9z\" not found, machinehealthchecks.cluster.x-k8s.io \"test-mhc-6mz9z\" not found]" "controller"="machinehealthcheck" "name"="test-mhc-6mz9z" "namespace"="test-mhc-n8f77"
I0810 03:06:05.130363   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qrp58" "namespace"="test-mhc-s2dsp" 
I0810 03:06:05.130481   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-hgdpx" "namespace"="test-mhc-2g9dk" 
I0810 03:06:05.130519   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jfblq" "namespace"="test-mhc-mlkvc" 
I0810 03:06:05.161268   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:05.284636   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6mz9z" "namespace"="test-mhc-n8f77" 
I0810 03:06:05.284710   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jfblq" "namespace"="test-mhc-mlkvc" 
I0810 03:06:05.313074   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jfblq" "namespace"="test-mhc-mlkvc" 
=== RUN   TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_modifying_existing_ones
I0810 03:06:05.366808   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jfblq" "namespace"="test-mhc-mlkvc" 
E0810 03:06:05.386050   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test-cluster-s7nrw\" not found" "controller"="cluster" "name"="test-cluster-s7nrw" "namespace"="test-mhc-mlkvc"
E0810 03:06:05.386569   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-mlkvc/test-cluster-s7nrw"
I0810 03:06:05.493371   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qlmhm" "namespace"="test-mhc-g9hn8" 
I0810 03:06:05.513205   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:05.632996   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qlmhm" "namespace"="test-mhc-g9hn8" 
E0810 03:06:05.698844   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-g9hn8/test-cluster-kzx9p"
=== RUN   TestMachineHealthCheck_Reconcile/it_doesn't_mark_anything_unhealthy_when_all_Machines_are_healthy
I0810 03:06:05.706930   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qlmhm" "namespace"="test-mhc-g9hn8" 
I0810 03:06:05.728963   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
inframachine created: test-mhc-machine-infra-5stzs
I0810 03:06:05.775571   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
machine created: test-mhc-machine-qq47z
I0810 03:06:05.881028   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-qq47z/"
E0810 03:06:05.901068   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-bzv5p, got []"  "node"="test-mhc-node-bzv5p"
node created: test-mhc-node-bzv5p
E0810 03:06:05.901430   10479 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="test-mhc-node-bzv5p"
I0810 03:06:05.925250   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
inframachine created: test-mhc-machine-infra-tspmz
I0810 03:06:05.942470   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
machine created: test-mhc-machine-jr8s9
I0810 03:06:05.947018   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:05.955480   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
... skipping 47 lines ...
I0810 03:06:06.097468   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.100160   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.103442   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.105893   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.108221   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.111475   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.115130   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
I0810 03:06:06.127395   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.127868   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
I0810 03:06:06.131055   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.131467   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
I0810 03:06:06.144313   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.145727   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
E0810 03:06:06.150566   10479 machine_controller.go:685] controllers/Machine "msg"="Unable to retrieve machine from node" "error"="no matching Machine"  "node"="test-mhc-node-7q6h4"
node created: test-mhc-node-7q6h4
E0810 03:06:06.153863   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-7q6h4, got []"  "node"="test-mhc-node-7q6h4"
I0810 03:06:06.156467   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.156961   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
I0810 03:06:06.162445   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.312659   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.315415   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
Cleaning up nodes, machines and infra machines.
I0810 03:06:06.323266   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.323716   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-qq47z/"
I0810 03:06:06.348227   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.348800   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-qq47z/"
I0810 03:06:06.349480   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
I0810 03:06:06.379175   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
I0810 03:06:06.379759   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-qq47z/"
I0810 03:06:06.380391   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-t4l87/test-mhc-6zzlg/test-mhc-machine-jr8s9/"
E0810 03:06:06.383684   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-t4l87/test-cluster-ghx65"
I0810 03:06:06.390698   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-6zzlg" "namespace"="test-mhc-t4l87" 
=== RUN   TestMachineHealthCheck_Reconcile/it_marks_unhealthy_machines_for_remediation_when_there_is_one_unhealthy_Machine
E0810 03:06:06.401046   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-machine-watches-j2vxr/machine-reconcile-442ff"
I0810 03:06:06.508409   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
inframachine created: test-mhc-machine-infra-m22lg
machine created: test-mhc-machine-475tw
E0810 03:06:06.551599   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": cannot find node with matching ProviderID" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:06.552802   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:06.597184   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:06.602837   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:06.604270   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:06.606299   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:06.608070   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 436 lines ...
I0810 03:06:07.541895   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.543581   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.545241   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.546893   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.549050   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.550809   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:07.551963   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:07.552563   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.565264   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.568501   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.571464   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.573402   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:07.576491   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 434 lines ...
I0810 03:06:08.557307   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.559174   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.561193   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.561906   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.563124   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.571221   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.572044   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-475tw/"
E0810 03:06:08.576219   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:08.586165   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.586555   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-475tw/"
node created: test-mhc-node-ngqch
E0810 03:06:08.655181   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-ngqch, got []"  "node"="test-mhc-node-ngqch"
I0810 03:06:08.662167   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
inframachine created: test-mhc-machine-infra-8dj7p
I0810 03:06:08.719723   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:08.723436   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
machine created: test-mhc-machine-v5zsn
I0810 03:06:08.754944   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 296 lines ...
I0810 03:06:09.563142   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.565629   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.567957   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.570433   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.572855   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.575308   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:09.576679   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:09.577498   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.580007   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.582200   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.584450   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.586832   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:09.589007   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 308 lines ...
I0810 03:06:10.601383   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.603464   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.605914   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.609526   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.611043   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.612499   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:10.614108   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:10.614885   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.616320   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.618635   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.619908   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.621296   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:10.622374   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 332 lines ...
I0810 03:06:11.619389   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.621585   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.623398   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.624481   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.625608   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.626692   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:11.627470   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:11.627779   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.628117   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-v5zsn/"
I0810 03:06:11.637731   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.638193   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-v5zsn/"
E0810 03:06:11.657225   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-nvml9, got []"  "node"="test-mhc-node-nvml9"
node created: test-mhc-node-nvml9
I0810 03:06:11.684023   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
inframachine created: test-mhc-machine-infra-sjtp6
machine created: test-mhc-machine-spknh
I0810 03:06:11.708327   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:11.715315   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 236 lines ...
I0810 03:06:12.627864   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.630767   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.633383   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.635876   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.638368   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.646482   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:12.648533   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:12.648615   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.658557   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.660899   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.662991   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.664966   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:12.666570   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 290 lines ...
I0810 03:06:13.671322   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.673835   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.675498   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.677573   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.687834   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.690013   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
E0810 03:06:13.692575   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:13.708372   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.710785   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.713017   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.715184   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.717508   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:13.719684   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
... skipping 261 lines ...
I0810 03:06:14.706651   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.709316   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.712742   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.716610   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.718220   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.720502   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.720969   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-spknh/"
E0810 03:06:14.732069   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:14.733465   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.734214   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-spknh/"
node created: test-mhc-node-tk4br
E0810 03:06:14.750720   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-tk4br, got []"  "node"="test-mhc-node-tk4br"
I0810 03:06:14.763583   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.764165   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-spknh/test-mhc-node-tk4br"
Cleaning up nodes, machines and infra machines.
Cleaning up nodes, machines and infra machines.
I0810 03:06:14.800447   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:14.803808   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-475tw/"
I0810 03:06:14.836491   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-nd9qs" "namespace"="test-mhc-25rxc" "count"=2
I0810 03:06:14.836565   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-nd9qs" "namespace"="test-mhc-25rxc" "descendants"="Worker machines: test-mhc-machine-475tw,test-mhc-machine-v5zsn,test-mhc-machine-spknh" "indirect descendants count"=1
I0810 03:06:14.840389   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-nd9qs" "namespace"="test-mhc-25rxc" "count"=2
I0810 03:06:14.840454   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-nd9qs" "namespace"="test-mhc-25rxc" "descendants"="Worker machines: test-mhc-machine-475tw,test-mhc-machine-v5zsn,test-mhc-machine-spknh" "indirect descendants count"=1
=== RUN   TestMachineHealthCheck_Reconcile/it_marks_unhealthy_machines_for_remediation_when_the_unhealthy_Machines_exceed_MaxUnhealthy
inframachine created: test-mhc-machine-infra-smnhg
machine created: test-mhc-machine-8wmvp
I0810 03:06:14.941694   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-v5zsn/"
I0810 03:06:14.953356   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-25rxc/test-mhc-nd7vk/test-mhc-machine-spknh/"
E0810 03:06:14.970472   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinehealthchecks.cluster.x-k8s.io \"test-mhc-nd7vk\" not found" "controller"="machinehealthcheck" "name"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc"
I0810 03:06:15.734150   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-nd9qs" "machine"="test-mhc-machine-v5zsn" "namespace"="test-mhc-25rxc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-nvml9"}
E0810 03:06:15.785498   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-v5zsn\" not found" "controller"="machine" "name"="test-mhc-machine-v5zsn" "namespace"="test-mhc-25rxc"
I0810 03:06:15.970761   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:15.987011   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:16.113098   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-nd7vk" "namespace"="test-mhc-25rxc" 
I0810 03:06:16.113177   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.119427   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.140394   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.181425   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.262443   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.423461   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:16.744625   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:16.785921   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:17.385680   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:17.786498   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-nd9qs" "machine"="test-mhc-machine-spknh" "namespace"="test-mhc-25rxc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-tk4br"}
E0810 03:06:17.838880   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-spknh\" not found" "controller"="machine" "name"="test-mhc-machine-spknh" "namespace"="test-mhc-25rxc"
I0810 03:06:18.666698   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:18.842572   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-nd9qs" "machine"="test-mhc-machine-475tw" "namespace"="test-mhc-25rxc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-ngqch"}
E0810 03:06:18.921061   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-475tw\" not found" "controller"="machine" "name"="test-mhc-machine-475tw" "namespace"="test-mhc-25rxc"
E0810 03:06:19.854429   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-25rxc/test-cluster-nd9qs"
E0810 03:06:19.921467   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:20.935473   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:20.941840   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:20.942008   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:20.966799   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:21.035693   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-zc9bc, got []"  "node"="test-mhc-node-zc9bc"
node created: test-mhc-node-zc9bc
I0810 03:06:21.044182   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
inframachine created: test-mhc-machine-infra-zvvb6
machine created: test-mhc-machine-2426x
I0810 03:06:21.068927   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.194087   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.200662   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.210836   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.252473   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.334342   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.495684   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:21.821305   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:21.942451   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:22.463681   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:22.956405   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:22.971103   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:22.982497   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:22.989835   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:22.997604   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:23.003664   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:23.003822   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:23.029578   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:23.071345   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-292dh, got []"  "node"="test-mhc-node-292dh"
node created: test-mhc-node-292dh
I0810 03:06:23.080167   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
inframachine created: test-mhc-machine-infra-txw24
I0810 03:06:23.094628   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
machine created: test-mhc-machine-jnmxs
I0810 03:06:23.097353   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:23.108892   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:23.745195   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:24.004203   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:25.019993   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.052476   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.063152   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.089183   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.102925   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.128509   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.134407   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
E0810 03:06:25.136164   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:25.154371   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
node created: test-mhc-node-fqfgv
E0810 03:06:25.218437   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-fqfgv, got []"  "node"="test-mhc-node-fqfgv"
Cleaning up nodes, machines and infra machines.
I0810 03:06:25.225332   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.238280   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
Cleaning up nodes, machines and infra machines.
I0810 03:06:25.282933   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-8xzzd" "namespace"="test-mhc-q8khs" "count"=2
I0810 03:06:25.283016   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-8xzzd" "namespace"="test-mhc-q8khs" "descendants"="Worker machines: test-mhc-machine-8wmvp,test-mhc-machine-2426x,test-mhc-machine-jnmxs" "indirect descendants count"=1
... skipping 3 lines ...
inframachine created: test-mhc-machine-infra-hsnkg
machine created: test-mhc-machine-gjtsb
I0810 03:06:25.432364   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:25.461655   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:25.583509   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-jh9b6" "namespace"="test-mhc-q8khs" 
I0810 03:06:25.583580   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
E0810 03:06:26.142132   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:27.142719   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-8xzzd" "machine"="test-mhc-machine-8wmvp" "namespace"="test-mhc-q8khs" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-zc9bc"}
E0810 03:06:27.196938   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-8wmvp\" not found" "controller"="machine" "name"="test-mhc-machine-8wmvp" "namespace"="test-mhc-q8khs"
I0810 03:06:28.197492   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-8xzzd" "machine"="test-mhc-machine-2426x" "namespace"="test-mhc-q8khs" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-292dh"}
E0810 03:06:28.243390   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-2426x\" not found" "controller"="machine" "name"="test-mhc-machine-2426x" "namespace"="test-mhc-q8khs"
I0810 03:06:29.243899   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-8xzzd" "machine"="test-mhc-machine-jnmxs" "namespace"="test-mhc-q8khs" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-fqfgv"}
E0810 03:06:29.289271   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-jnmxs\" not found" "controller"="machine" "name"="test-mhc-machine-jnmxs" "namespace"="test-mhc-q8khs"
I0810 03:06:30.296651   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:30.308328   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
E0810 03:06:30.308668   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
E0810 03:06:30.310934   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-q8khs/test-cluster-8xzzd"
E0810 03:06:30.347509   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-ndp88, got []"  "node"="test-mhc-node-ndp88"
node created: test-mhc-node-ndp88
I0810 03:06:30.354847   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
inframachine created: test-mhc-machine-infra-cphm5
I0810 03:06:30.371050   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
machine created: test-mhc-machine-w2hmr
I0810 03:06:30.373230   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:30.380319   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
E0810 03:06:31.309042   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:32.323638   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.343639   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.357620   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.367216   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.376565   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.380931   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
E0810 03:06:32.381377   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
node created: test-mhc-node-d769h
E0810 03:06:32.386368   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-d769h, got []"  "node"="test-mhc-node-d769h"
I0810 03:06:32.399271   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
inframachine created: test-mhc-machine-infra-8nvqp
machine created: test-mhc-machine-zn6sx
I0810 03:06:32.426586   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.431999   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:32.433405   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
... skipping 11 lines ...
I0810 03:06:33.535663   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:33.546239   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:33.546454   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-n9n4r" "machine"="test-mhc-machine-zn6sx" "namespace"="test-mhc-gl2jc" "cause"="noderef is nil" "node"=null
Cleaning up nodes, machines and infra machines.
I0810 03:06:33.553679   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:33.570979   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:33.571922   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-gl2jc/test-mhc-px66g/test-mhc-machine-gjtsb/"
I0810 03:06:33.594403   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:33.595112   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-gl2jc/test-mhc-px66g/test-mhc-machine-gjtsb/"
I0810 03:06:33.595572   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-gl2jc/test-mhc-px66g/test-mhc-machine-w2hmr/"
E0810 03:06:33.611663   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-zn6sx\" not found" "controller"="machine" "name"="test-mhc-machine-zn6sx" "namespace"="test-mhc-gl2jc"
I0810 03:06:33.616696   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-n9n4r" "namespace"="test-mhc-gl2jc" "count"=2
I0810 03:06:33.616793   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-n9n4r" "namespace"="test-mhc-gl2jc" "descendants"="Worker machines: test-mhc-machine-gjtsb,test-mhc-machine-w2hmr" "indirect descendants count"=0
I0810 03:06:33.623098   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-n9n4r" "namespace"="test-mhc-gl2jc" "count"=2
I0810 03:06:33.623179   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-n9n4r" "namespace"="test-mhc-gl2jc" "descendants"="Worker machines: test-mhc-machine-gjtsb,test-mhc-machine-w2hmr" "indirect descendants count"=0
=== RUN   TestMachineHealthCheck_Reconcile/when_a_Machine_has_no_Node_ref_for_longer_than_the_NodeStartupTimeout
=== RUN   TestMachineHealthCheck_Reconcile/when_a_Machine's_Node_has_gone_away
=== RUN   TestMachineHealthCheck_Reconcile/should_react_when_a_Node_transitions_to_unhealthy
inframachine created: test-mhc-machine-infra-pxfp9
machine created: test-mhc-machine-p8xv7
E0810 03:06:33.738600   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinehealthchecks.cluster.x-k8s.io \"test-mhc-px66g\" not found" "controller"="machinehealthcheck" "name"="test-mhc-px66g" "namespace"="test-mhc-gl2jc"
I0810 03:06:34.612223   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-n9n4r" "machine"="test-mhc-machine-gjtsb" "namespace"="test-mhc-gl2jc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-ndp88"}
E0810 03:06:34.653321   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-gjtsb\" not found" "controller"="machine" "name"="test-mhc-machine-gjtsb" "namespace"="test-mhc-gl2jc"
I0810 03:06:34.738845   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:34.754611   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:34.780661   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-px66g" "namespace"="test-mhc-gl2jc" 
I0810 03:06:34.780726   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:34.781854   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:34.783617   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
... skipping 414 lines ...
I0810 03:06:35.715799   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.717750   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.720673   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.723010   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.724850   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.726649   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
E0810 03:06:35.728312   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-w2hmr\" not found" "controller"="machine" "name"="test-mhc-machine-w2hmr" "namespace"="test-mhc-gl2jc"
I0810 03:06:35.732476   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.734515   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.736452   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.738465   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.741041   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:35.742793   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
... skipping 481 lines ...
I0810 03:06:36.733066   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.737606   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.739530   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.740110   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.741219   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.743122   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
E0810 03:06:36.744643   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:36.745394   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.745624   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/"
I0810 03:06:36.756625   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.756932   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/"
E0810 03:06:36.774114   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-8d2jx, got []"  "node"="test-mhc-node-8d2jx"
E0810 03:06:36.774381   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-8d2jx, got []"  "node"="test-mhc-node-8d2jx"
node created: test-mhc-node-8d2jx
I0810 03:06:36.782989   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.797682   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.807303   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.808889   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.809190   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/test-mhc-node-8d2jx"
I0810 03:06:36.825920   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.826347   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/test-mhc-node-8d2jx"
Cleaning up nodes, machines and infra machines.
I0810 03:06:36.838913   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.839270   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/test-mhc-node-8d2jx"
I0810 03:06:36.840405   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.840672   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/"
I0810 03:06:36.862197   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-wtlsc" "namespace"="test-mhc-bt6wp" "descendants"="Worker machines: test-mhc-machine-p8xv7" "indirect descendants count"=1
I0810 03:06:36.864209   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
I0810 03:06:36.864557   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="NodeNotFound" "target"="test-mhc-bt6wp/test-mhc-zvttv/test-mhc-machine-p8xv7/"
I0810 03:06:36.871557   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-wtlsc" "namespace"="test-mhc-bt6wp" "descendants"="Worker machines: test-mhc-machine-p8xv7" "indirect descendants count"=1
I0810 03:06:36.873944   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-zvttv" "namespace"="test-mhc-bt6wp" 
=== RUN   TestMachineHealthCheck_Reconcile/when_in_a_MachineSet,_unhealthy_machines_should_be_deleted
I0810 03:06:36.984114   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" "creating"=1 "need"=1
I0810 03:06:36.984161   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:36.996615   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"mhc-ms-dq784-cb8z7\"" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:37.084658   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:37.108680   10479 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinehealthcheck" "source"={}
I0810 03:06:37.258934   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
E0810 03:06:37.745095   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:38.259161   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
E0810 03:06:38.637323   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-gl2jc/test-cluster-n9n4r"
I0810 03:06:38.745639   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-wtlsc" "machine"="test-mhc-machine-p8xv7" "namespace"="test-mhc-bt6wp" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-8d2jx"}
E0810 03:06:38.804521   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-p8xv7\" not found" "controller"="machine" "name"="test-mhc-machine-p8xv7" "namespace"="test-mhc-bt6wp"
I0810 03:06:39.260202   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.814125   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.814630   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.838300   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.839061   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.848988   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
... skipping 2 lines ...
I0810 03:06:39.856031   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.861873   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.861881   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.866765   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:39.866802   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.261270   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.261881   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-gt9pb/test-mhc-sdwzl/mhc-ms-dq784-cb8z7/"
I0810 03:06:40.270034   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.275661   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.275722   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.275770   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.276027   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-gt9pb/test-mhc-sdwzl/mhc-ms-dq784-cb8z7/"
I0810 03:06:40.283436   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-sdwzl" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.283807   10479 machinehealthcheck_controller.go:387] controllers/MachineHealthCheck "msg"="Target has failed health check, marking for remediation"  "message"="Node failed to report startup in 1s" "reason"="NodeStartupTimeout" "target"="test-mhc-gt9pb/test-mhc-sdwzl/mhc-ms-dq784-cb8z7/"
I0810 03:06:40.284209   10479 machineset_controller.go:139] controllers/MachineSet "msg"="Reconciliation is paused for this object" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.296219   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.296268   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" 
I0810 03:06:40.368472   10479 cluster_controller.go:223] controllers/Cluster "msg"="Cluster still has children - deleting them first" "cluster"="test-cluster-fk67p" "namespace"="test-mhc-gt9pb" "count"=1
I0810 03:06:40.368543   10479 cluster_controller.go:241] controllers/Cluster "msg"="Deleting child" "cluster"="test-cluster-fk67p" "namespace"="test-mhc-gt9pb" "gvk"="cluster.x-k8s.io/v1alpha3, Kind=MachineSet" "name"="mhc-ms-dq784"
I0810 03:06:40.371574   10479 machineset_controller.go:237] controllers/MachineSet "msg"="Deleting unhealthy machine" "machineset"="mhc-ms-dq784" "namespace"="test-mhc-gt9pb" "machine"="mhc-ms-dq784-cb8z7"
... skipping 35 lines ...
I0810 03:06:40.585472   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.587339   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.590342   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.591147   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.593566   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-fk67p" "machine"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb" "cause"="cluster is being deleted" "node"=null
I0810 03:06:40.595771   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
E0810 03:06:40.616544   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-vnsm9, got []"  "node"="test-mhc-node-vnsm9"
node created: test-mhc-node-vnsm9
E0810 03:06:40.616751   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-vnsm9, got []"  "node"="test-mhc-node-vnsm9"
E0810 03:06:40.616794   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-vnsm9, got []"  "node"="test-mhc-node-vnsm9"
E0810 03:06:40.667350   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"mhc-ms-dq784-cb8z7\" not found" "controller"="machine" "name"="mhc-ms-dq784-cb8z7" "namespace"="test-mhc-gt9pb"
I0810 03:06:40.668107   10479 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "remediation request name"="test-mhc-machine-gnjgg" "target"="test-mhc-wxjsq/test-mhc-j7qmt/test-mhc-machine-gnjgg/"
I0810 03:06:40.680491   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.707283   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.711834   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.714949   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.720670   10479 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "remediation request name"="test-mhc-machine-gnjgg" "target"="test-mhc-wxjsq/test-mhc-j7qmt/test-mhc-machine-gnjgg/test-mhc-node-vnsm9"
I0810 03:06:40.744026   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
Cleaning up nodes, machines and infra machines.
I0810 03:06:40.758548   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.764984   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.770663   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-j7qmt" "namespace"="test-mhc-wxjsq" 
I0810 03:06:40.788756   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-ttrfq" "namespace"="test-mhc-wxjsq" "descendants"="Worker machines: test-mhc-machine-gnjgg" "indirect descendants count"=1
... skipping 349 lines ...
I0810 03:06:41.698367   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.703026   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.704846   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.706770   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.708725   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.712828   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
E0810 03:06:41.713341   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-gnjgg\" not found" "controller"="machine" "name"="test-mhc-machine-gnjgg" "namespace"="test-mhc-wxjsq"
I0810 03:06:41.714683   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.716506   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.718360   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.720160   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.722155   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.724463   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
... skipping 66 lines ...
I0810 03:06:41.870156   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.873302   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.876357   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.879033   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.880943   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.882747   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
E0810 03:06:41.883772   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-bt6wp/test-cluster-wtlsc"
I0810 03:06:41.884520   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.886460   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.888363   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.890215   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.892068   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:41.893964   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
... skipping 387 lines ...
I0810 03:06:42.717857   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.719627   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.720752   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.721418   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.723024   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.724998   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
E0810 03:06:42.725720   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
I0810 03:06:42.733651   10479 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Node failed to report startup in 1ms" "reason"="NodeStartupTimeout" "remediation request name"="test-mhc-machine-g7dnt" "target"="test-mhc-hfzhc/test-mhc-qsrls/test-mhc-machine-g7dnt/"
I0810 03:06:42.746624   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
E0810 03:06:42.763355   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-99wv6, got []"  "node"="test-mhc-node-99wv6"
E0810 03:06:42.763638   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-99wv6, got []"  "node"="test-mhc-node-99wv6"
node created: test-mhc-node-99wv6
E0810 03:06:42.764463   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node test-mhc-node-99wv6, got []"  "node"="test-mhc-node-99wv6"
I0810 03:06:42.771884   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.812298   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.817317   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.820638   10479 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="Condition Ready on node is reporting status Unknown for more than 5m0s" "reason"="UnhealthyNode" "remediation request name"="test-mhc-machine-g7dnt" "target"="test-mhc-hfzhc/test-mhc-qsrls/test-mhc-machine-g7dnt/test-mhc-node-99wv6"
I0810 03:06:42.837051   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.854391   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.877298   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
Cleaning up nodes, machines and infra machines.
I0810 03:06:42.890907   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
I0810 03:06:42.898957   10479 machinehealthcheck_controller.go:379] controllers/MachineHealthCheck "msg"="Target has failed health check, creating an external remediation request"  "message"="" "reason"="NodeNotFound" "remediation request name"="test-mhc-machine-g7dnt" "target"="test-mhc-hfzhc/test-mhc-qsrls/test-mhc-machine-g7dnt/"
I0810 03:06:42.918529   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-545bv" "namespace"="test-mhc-hfzhc" "descendants"="Worker machines: test-mhc-machine-g7dnt" "indirect descendants count"=1
E0810 03:06:42.933406   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machinehealthchecks.cluster.x-k8s.io \"test-mhc-qsrls\" not found" "controller"="machinehealthcheck" "name"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc"
I0810 03:06:42.933598   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster-545bv" "namespace"="test-mhc-hfzhc" "descendants"="Worker machines: test-mhc-machine-g7dnt" "indirect descendants count"=1
--- PASS: TestMachineHealthCheck_Reconcile (38.99s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_no_existing_labels_exist (0.08s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_the_label_has_the_wrong_value (0.05s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_the_correct_cluster-name_label_when_other_labels_are_present (0.03s)
    --- PASS: TestMachineHealthCheck_Reconcile/it_should_ensure_an_owner_reference_is_present_when_no_existing_ones_exist (1.26s)
... skipping 11 lines ...
    --- SKIP: TestMachineHealthCheck_Reconcile/when_a_machine_is_paused (0.00s)
        machinehealthcheck_controller_test.go:933: skipping until made stable
    --- PASS: TestMachineHealthCheck_Reconcile/When_remediationTemplate_is_set_and_node_transitions_to_unhealthy,_new_Remediation_Request_should_be_created (0.41s)
    --- PASS: TestMachineHealthCheck_Reconcile/When_remediationTemplate_is_set_and_node_transitions_back_to_healthy,_new_Remediation_Request_should_be_deleted (2.14s)
=== RUN   TestClusterToMachineHealthCheck
=== RUN   TestClusterToMachineHealthCheck/when_the_object_passed_isn't_a_cluster
E0810 03:06:42.939240   10479 machinehealthcheck_controller.go:416]  "msg"="expected a Cluster" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Cluster_in_the_same_namespace
=== RUN   TestClusterToMachineHealthCheck/when_2_MachineHealthChecks_exists_for_the_Cluster_in_the_same_namespace
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_the_same_namespace
=== RUN   TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_another_namespace
--- PASS: TestClusterToMachineHealthCheck (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_the_object_passed_isn't_a_cluster (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_2_MachineHealthChecks_exists_for_the_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_the_same_namespace (0.00s)
    --- PASS: TestClusterToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_another_Cluster_in_another_namespace (0.00s)
=== RUN   TestMachineToMachineHealthCheck
=== RUN   TestMachineToMachineHealthCheck/when_the_object_passed_isn't_a_machine
E0810 03:06:42.940996   10479 machinehealthcheck_controller.go:445]  "msg"="expected a Machine" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_2_MachineHealthChecks_match_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_does_not_match_labels_for_the_Machine_in_the_same_namespace
=== RUN   TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_another_namespace
--- PASS: TestMachineToMachineHealthCheck (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_the_object_passed_isn't_a_machine (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_2_MachineHealthChecks_match_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_does_not_match_labels_for_the_Machine_in_the_same_namespace (0.00s)
    --- PASS: TestMachineToMachineHealthCheck/when_a_MachineHealthCheck_matches_labels_for_the_Machine_in_another_namespace (0.00s)
=== RUN   TestNodeToMachineHealthCheck
=== RUN   TestNodeToMachineHealthCheck/when_the_object_passed_isn't_a_Node
E0810 03:06:42.943052   10479 machinehealthcheck_controller.go:474]  "msg"="expected a Node" "error"="incorrect type"  "type"="handler.MapObject"
=== RUN   TestNodeToMachineHealthCheck/when_no_Machine_exists_for_the_Node
E0810 03:06:42.943385   10479 machinehealthcheck_controller.go:480]  "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node node1, got []"  "node"="node1"
=== RUN   TestNodeToMachineHealthCheck/when_two_Machines_exist_for_the_Node
E0810 03:06:42.943914   10479 machinehealthcheck_controller.go:480]  "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node node1, got [machine1 machine2]"  "node"="node1"
=== RUN   TestNodeToMachineHealthCheck/when_no_MachineHealthCheck_exists_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_two_MachineHealthChecks_exist_for_the_Node_in_the_Machine's_namespace
=== RUN   TestNodeToMachineHealthCheck/when_a_MachineHealthCheck_exists_for_the_Node,_but_not_in_the_Machine's_cluster
--- PASS: TestNodeToMachineHealthCheck (0.00s)
    --- PASS: TestNodeToMachineHealthCheck/when_the_object_passed_isn't_a_Node (0.00s)
... skipping 31 lines ...
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_nil (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_not_an_int_or_percentage (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_an_int (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_a_40%_(of_5) (0.00s)
    --- PASS: TestGetMaxUnhealthy/when_maxUnhealthy_is_a_60%_(of_7) (0.00s)
=== RUN   TestPatchTargets
I0810 03:06:42.947899   10479 machinehealthcheck_controller.go:387]  "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="" "target"="default/mhc/machine1/"
I0810 03:06:42.948591   10479 machinehealthcheck_controller.go:387]  "msg"="Target has failed health check, marking for remediation"  "message"="" "reason"="" "target"="default/mhc/machine2/"
E0810 03:06:42.950753   10479 machinehealthcheck_controller.go:320]  "msg"="failed to patch healthy machine status for machine" "error"="machines.cluster.x-k8s.io \"machine1\" not found"  "machine"="machine1"
--- PASS: TestPatchTargets (0.01s)
=== RUN   TestGetTargetsFromMHC
=== RUN   TestGetTargetsFromMHC/with_no_matching_machines
=== RUN   TestGetTargetsFromMHC/when_a_machine's_node_is_missing
=== RUN   TestGetTargetsFromMHC/when_a_machine's_labels_do_not_match_the_selector
=== RUN   TestGetTargetsFromMHC/with_multiple_machines,_should_match_correct_nodes
... skipping 26 lines ...
--- PASS: TestMachineSetOwnerReference (0.00s)
    --- PASS: TestMachineSetOwnerReference/should_add_cluster_owner_reference_to_machine_set (0.00s)
    --- PASS: TestMachineSetOwnerReference/should_not_add_cluster_owner_reference_if_machine_is_owned_by_a_machine_deployment (0.00s)
=== RUN   TestMachineSetReconcile
=== RUN   TestMachineSetReconcile/ignore_machine_sets_marked_for_deletion
=== RUN   TestMachineSetReconcile/records_event_if_reconcile_fails
E0810 03:06:42.966109   10479 machineset_controller.go:151]  "msg"="Failed to reconcile MachineSet" "error"="failed to calculate MachineSet's Status: failed to calculate status for MachineSet default/machineset1: invalid label key \"--$-invalid\": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName',  or 'my.name',  or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')" "machineset"="machineset1" "namespace"="default" 
--- PASS: TestMachineSetReconcile (0.00s)
    --- PASS: TestMachineSetReconcile/ignore_machine_sets_marked_for_deletion (0.00s)
    --- PASS: TestMachineSetReconcile/records_event_if_reconcile_fails (0.00s)
=== RUN   TestMachineSetToMachines
=== RUN   TestMachineSetToMachines/should_return_empty_request_when_controller_is_set
=== RUN   TestMachineSetToMachines/should_return_nil_if_machine_has_no_owner_reference
... skipping 8 lines ...
--- PASS: TestAdoptOrphan (0.00s)
=== RUN   TestHasMatchingLabels
=== RUN   TestHasMatchingLabels/machine_set_and_machine_have_matching_labels
=== RUN   TestHasMatchingLabels/machine_set_and_machine_do_not_have_matching_labels
=== RUN   TestHasMatchingLabels/machine_set_has_empty_selector
=== RUN   TestHasMatchingLabels/machine_set_has_bad_selector
E0810 03:06:42.968047   10479 machineset_controller.go:569]  "msg"="Unable to convert selector" "error"="\"bad-operator\" is not a valid pod selector operator" "machine"="match" "machineset"="" "namespace"="" 
--- PASS: TestHasMatchingLabels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_and_machine_have_matching_labels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_and_machine_do_not_have_matching_labels (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_has_empty_selector (0.00s)
    --- PASS: TestHasMatchingLabels/machine_set_has_bad_selector (0.00s)
=== RUN   TestMachineToDelete
... skipping 73 lines ...
I0810 03:06:43.361588   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "creating"=1 "need"=3
I0810 03:06:43.361638   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" 
I0810 03:06:43.374779   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-6657c7fddb-jwjj9\"" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" 
I0810 03:06:43.472339   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=1
I0810 03:06:43.472429   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
I0810 03:06:43.485025   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-cdfc6fd6c-mc929\"" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
E0810 03:06:43.562114   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.562114   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.562163   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.666979   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.667033   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.667110   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.667164   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.667220   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.667255   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-mc929-lmm7c, got []"  "node"="md-v6bv9-cdfc6fd6c-mc929-lmm7c"
E0810 03:06:43.712160   10479 machineset_controller.go:625] controllers/MachineSet "msg"="Unable to retrieve Node status" "error"="error retrieving node md-v6bv9-cdfc6fd6c-mc929-lmm7c for machine md-test/md-v6bv9-cdfc6fd6c-mc929: the cache is not started, can not read objects" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
I0810 03:06:43.726151   10479 machine_controller.go:305] controllers/Machine "msg"="Deleting Kubernetes Node associated with Machine is not allowed" "cluster"="test-cluster-545bv" "machine"="test-mhc-machine-g7dnt" "namespace"="test-mhc-hfzhc" "cause"="cluster is being deleted" "node"={"name":"test-mhc-node-99wv6"}
I0810 03:06:43.742470   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=2
I0810 03:06:43.742574   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:43.753868   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "machine"="md-v6bv9-6657c7fddb-8k674"
E0810 03:06:43.804282   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="machines.cluster.x-k8s.io \"test-mhc-machine-g7dnt\" not found" "controller"="machine" "name"="test-mhc-machine-g7dnt" "namespace"="test-mhc-hfzhc"
I0810 03:06:43.865387   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=2
I0810 03:06:43.865438   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
I0810 03:06:43.879200   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-cdfc6fd6c-2j884\"" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
E0810 03:06:43.921087   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:43.921254   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:43.921467   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
I0810 03:06:43.933934   10479 machinehealthcheck_controller.go:109] controllers/MachineHealthCheck "msg"="Reconciling" "machinehealthcheck"="test-mhc-qsrls" "namespace"="test-mhc-hfzhc" 
E0810 03:06:44.024522   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:44.024584   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:44.024651   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:44.024690   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:44.024717   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
E0810 03:06:44.024749   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-2j884-558dk, got []"  "node"="md-v6bv9-cdfc6fd6c-2j884-558dk"
I0810 03:06:44.072884   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=1
I0810 03:06:44.073063   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:44.078430   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "machine"="md-v6bv9-6657c7fddb-bg9b5"
I0810 03:06:44.115269   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "creating"=1 "need"=3
I0810 03:06:44.115375   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
I0810 03:06:44.129735   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-cdfc6fd6c-8vd5j\"" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" 
E0810 03:06:44.195794   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.195828   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.195992   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.198851   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.198902   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.198851   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.198939   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.199115   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
E0810 03:06:44.199156   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-cdfc6fd6c-8vd5j-fmp49, got []"  "node"="md-v6bv9-cdfc6fd6c-8vd5j-fmp49"
I0810 03:06:44.248859   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "deleting"=1 "need"=0
I0810 03:06:44.248908   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:44.258711   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-6657c7fddb" "namespace"="md-test" "machine"="md-v6bv9-6657c7fddb-jwjj9"
I0810 03:06:44.522898   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=1
I0810 03:06:44.522967   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(1) \u003e currentMachineCount(0) )" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
I0810 03:06:44.534636   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-74d45c49c5-rcs6t\"" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
E0810 03:06:44.613841   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.613902   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.613904   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.716739   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.716793   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.716954   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.716992   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.716988   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
E0810 03:06:44.717094   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-rcs6t-zcj29, got []"  "node"="md-v6bv9-74d45c49c5-rcs6t-zcj29"
I0810 03:06:44.763006   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=2
I0810 03:06:44.763044   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:44.769836   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "machine"="md-v6bv9-cdfc6fd6c-2j884"
E0810 03:06:44.804594   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:44.881877   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=2
I0810 03:06:44.881931   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
I0810 03:06:44.892496   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-74d45c49c5-9znhj\"" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
E0810 03:06:44.963300   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:44.963399   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:44.963633   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.068270   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.068330   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.068333   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.068392   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.069055   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
E0810 03:06:45.069239   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-9znhj-2qrjw, got []"  "node"="md-v6bv9-74d45c49c5-9znhj-2qrjw"
I0810 03:06:45.111213   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=1
I0810 03:06:45.111257   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:45.117859   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "machine"="md-v6bv9-cdfc6fd6c-8vd5j"
I0810 03:06:45.130578   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" "creating"=1 "need"=3
I0810 03:06:45.130625   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(3) \u003e currentMachineCount(2) )" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
I0810 03:06:45.142811   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"md-v6bv9-74d45c49c5-jvj7p\"" "machineset"="md-v6bv9-74d45c49c5" "namespace"="md-test" 
E0810 03:06:45.206698   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.206801   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.206856   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209142   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209177   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209444   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209493   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209667   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
E0810 03:06:45.209698   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node md-v6bv9-74d45c49c5-jvj7p-ldvp9, got []"  "node"="md-v6bv9-74d45c49c5-jvj7p-ldvp9"
I0810 03:06:45.297868   10479 machineset_controller.go:388] controllers/MachineSet "msg"="Too many replicas" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "deleting"=1 "need"=0
I0810 03:06:45.297911   10479 machineset_controller.go:394] controllers/MachineSet "msg"="Found delete policy" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "delete-policy"="Random"
I0810 03:06:45.303258   10479 machineset_controller.go:405] controllers/MachineSet "msg"="Deleted machine" "machineset"="md-v6bv9-cdfc6fd6c" "namespace"="md-test" "machine"="md-v6bv9-cdfc6fd6c-mc929"
I0810 03:06:45.370445   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-v6bv9-74d45c49c5,md-v6bv9-cdfc6fd6c;Worker machines: md-v6bv9-6657c7fddb-99pdn,md-v6bv9-74d45c49c5-9znhj,md-v6bv9-74d45c49c5-jvj7p,md-v6bv9-74d45c49c5-rcs6t,md-v6bv9-6657c7fddb-cdnqp" "indirect descendants count"=7
•I0810 03:06:45.374800   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-v6bv9-74d45c49c5,md-v6bv9-cdfc6fd6c;Worker machines: md-v6bv9-74d45c49c5-9znhj,md-v6bv9-74d45c49c5-jvj7p,md-v6bv9-74d45c49c5-rcs6t,md-v6bv9-6657c7fddb-cdnqp,md-v6bv9-6657c7fddb-99pdn" "indirect descendants count"=7
E0810 03:06:45.390248   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-gt9pb/test-cluster-fk67p"
I0810 03:06:45.445754   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="ms-62thx" "namespace"="ms-test" "creating"=2 "need"=2
I0810 03:06:45.445838   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 2, ( spec.replicas(2) \u003e currentMachineCount(0) )" "machineset"="ms-62thx" "namespace"="ms-test" 
I0810 03:06:45.464248   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 2 with name \"ms-62thx-t6rj4\"" "machineset"="ms-62thx" "namespace"="ms-test" 
I0810 03:06:45.464313   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 2 of 2, ( spec.replicas(2) \u003e currentMachineCount(0) )" "machineset"="ms-62thx" "namespace"="ms-test" 
I0810 03:06:45.487343   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 2 of 2 with name \"ms-62thx-9mnl5\"" "machineset"="ms-62thx" "namespace"="ms-test" 
I0810 03:06:45.579076   10479 machineset_controller.go:316] controllers/MachineSet "msg"="Too few replicas" "machineset"="ms-62thx" "namespace"="ms-test" "creating"=1 "need"=2
I0810 03:06:45.579127   10479 machineset_controller.go:324] controllers/MachineSet "msg"="Creating machine 1 of 1, ( spec.replicas(2) \u003e currentMachineCount(1) )" "machineset"="ms-62thx" "namespace"="ms-test" 
I0810 03:06:45.607048   10479 machineset_controller.go:378] controllers/MachineSet "msg"="Created machine 1 of 1 with name \"ms-62thx-kp894\"" "machineset"="ms-62thx" "namespace"="ms-test" 
E0810 03:06:45.698203   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.698411   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.814315   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.814372   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.814782   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.814829   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-kp894-q62sr, got []"  "node"="ms-62thx-kp894-q62sr"
E0810 03:06:45.823982   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-wxjsq/test-cluster-ttrfq"
E0810 03:06:45.857405   10479 machineset_controller.go:625] controllers/MachineSet "msg"="Unable to retrieve Node status" "error"="error retrieving node ms-62thx-kp894-q62sr for machine ms-test/ms-62thx-kp894: the cache is not started, can not read objects" "machineset"="ms-62thx" "namespace"="ms-test" 
E0810 03:06:45.866802   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-9mnl5-9fnnr, got []"  "node"="ms-62thx-9mnl5-9fnnr"
I0810 03:06:45.928874   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:45.928923   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:45.948285   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:06:45.948329   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:06:45.971714   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-9mnl5-9fnnr, got []"  "node"="ms-62thx-9mnl5-9fnnr"
E0810 03:06:45.971766   10479 machinehealthcheck_controller.go:480] controllers/MachineHealthCheck "msg"="Unable to retrieve machine from node" "error"="expecting one machine for node ms-62thx-9mnl5-9fnnr, got []"  "node"="ms-62thx-9mnl5-9fnnr"
E0810 03:06:45.996845   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
I0810 03:06:46.093427   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="ms-test" "descendants"="Worker machines: ms-62thx-kp894,ms-62thx-9mnl5" "indirect descendants count"=2
•I0810 03:06:46.097824   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="ms-test" "descendants"="Worker machines: ms-62thx-kp894,ms-62thx-9mnl5" "indirect descendants count"=2
•E0810 03:06:46.216373   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test1-ftn54\" not found" "controller"="cluster" "name"="test1-ftn54" "namespace"="default"
E0810 03:06:47.019159   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
•I0810 03:06:47.340863   10479 conversion.go:64] controllers/Cluster "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test2-7h227" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
I0810 03:06:47.465697   10479 conversion.go:64] controllers/Cluster "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "cluster"="test2-7h227" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
E0810 03:06:47.470239   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\"" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:06:48.042527   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
E0810 03:06:48.483233   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
•I0810 03:06:49.109944   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:49.109985   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:49.113131   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:06:49.113163   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:06:49.116880   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
E0810 03:06:49.493244   10479 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="test-mhc-hfzhc/test-cluster-545bv"
E0810 03:06:49.504863   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusters.cluster.x-k8s.io \"test3-d9xh2\" not found" "controller"="cluster" "name"="test3-d9xh2" "namespace"="default"
E0810 03:06:50.121681   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:06:50.505728   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:06:51.126915   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
I0810 03:06:51.514432   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-v6bv9-cdfc6fd6c,md-v6bv9-74d45c49c5;Worker machines: md-v6bv9-74d45c49c5-rcs6t,md-v6bv9-6657c7fddb-cdnqp,md-v6bv9-6657c7fddb-99pdn,md-v6bv9-74d45c49c5-9znhj,md-v6bv9-74d45c49c5-jvj7p" "indirect descendants count"=7
E0810 03:06:51.514846   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
•E0810 03:06:52.145747   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
I0810 03:06:52.515474   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="ms-test" "descendants"="Worker machines: ms-62thx-kp894,ms-62thx-9mnl5" "indirect descendants count"=2
E0810 03:06:52.519574   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
I0810 03:06:53.150274   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:53.150321   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:53.154392   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:06:53.154433   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:06:53.158547   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:06:53.520278   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:06:54.163833   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
E0810 03:06:54.529283   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
•E0810 03:06:55.175038   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
E0810 03:06:55.530106   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:06:56.175469   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-qq47z\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-qq47z" "namespace"="test-mhc-t4l87"
E0810 03:06:56.530874   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:06:57.180568   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:06:57.573062   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
I0810 03:06:58.185170   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:58.185215   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:06:58.188943   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:06:58.189033   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:06:58.193012   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
I0810 03:06:58.573993   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-v6bv9-74d45c49c5,md-v6bv9-cdfc6fd6c;Worker machines: md-v6bv9-6657c7fddb-cdnqp,md-v6bv9-6657c7fddb-99pdn,md-v6bv9-74d45c49c5-9znhj,md-v6bv9-74d45c49c5-jvj7p,md-v6bv9-74d45c49c5-rcs6t" "indirect descendants count"=7
E0810 03:06:58.574540   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:06:59.193431   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get cluster \"test-cluster-ghx65\" for machine \"test-mhc-machine-jr8s9\" in namespace \"test-mhc-t4l87\": Cluster.cluster.x-k8s.io \"test-cluster-ghx65\" not found" "controller"="machine" "name"="test-mhc-machine-jr8s9" "namespace"="test-mhc-t4l87"
I0810 03:06:59.575202   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="ms-test" "descendants"="Worker machines: ms-62thx-9mnl5,ms-62thx-kp894" "indirect descendants count"=2
E0810 03:06:59.575938   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:07:00.198718   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
E0810 03:07:00.576757   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:07:01.205475   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:07:01.577579   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
E0810 03:07:02.218468   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
E0810 03:07:02.578469   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
I0810 03:07:03.223454   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:07:03.223492   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:07:03.227310   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:07:03.227347   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:07:03.231341   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
E0810 03:07:03.579226   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:07:04.236411   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:07:04.580054   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
I0810 03:07:05.252437   10479 conversion.go:64] controllers/Machine "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="test6-68zzg" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
E0810 03:07:05.262038   10479 machine_controller_noderef.go:73] controllers/Machine "msg"="Failed to retrieve Node by ProviderID" "error"="the cache is not started, can not read objects" "machine"="test6-68zzg" "namespace"="default" 
E0810 03:07:05.276753   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="[failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\", the cache is not started, can not read objects]" "controller"="machine" "name"="test6-68zzg" "namespace"="default"
I0810 03:07:05.580818   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="md-test" "descendants"="Machine sets: md-v6bv9-cdfc6fd6c,md-v6bv9-74d45c49c5;Worker machines: md-v6bv9-6657c7fddb-cdnqp,md-v6bv9-6657c7fddb-99pdn,md-v6bv9-74d45c49c5-9znhj,md-v6bv9-74d45c49c5-jvj7p,md-v6bv9-74d45c49c5-rcs6t" "indirect descendants count"=7
E0810 03:07:05.581374   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:07:06.281460   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-jvj7p\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-jvj7p" "namespace"="md-test"
I0810 03:07:06.581928   10479 cluster_controller.go:256] controllers/Cluster "msg"="Cluster still has descendants - need to requeue" "cluster"="test-cluster" "namespace"="ms-test" "descendants"="Worker machines: ms-62thx-kp894,ms-62thx-9mnl5" "indirect descendants count"=2
E0810 03:07:06.582458   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
E0810 03:07:07.286477   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-rcs6t\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-rcs6t" "namespace"="md-test"
E0810 03:07:07.583286   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test-too\" for Cluster default/test2-7h227: failed to retrieve  external object \"default\"/\"test-too\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test2-7h227" "namespace"="default"
I0810 03:07:08.290815   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:07:08.290865   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-cdnqp" "namespace"="md-test" 
I0810 03:07:08.294724   10479 machine_controller_phases.go:278] controllers/Machine "msg"="Infrastructure provider is not ready, requeuing" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
I0810 03:07:08.294758   10479 machine_controller_noderef.go:45] controllers/Machine "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="md-v6bv9-6657c7fddb-99pdn" "namespace"="md-test" 
E0810 03:07:08.298521   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="no matching Node for Machine \"md-v6bv9-74d45c49c5-9znhj\" in namespace \"md-test\": cannot find node with matching ProviderID" "controller"="machine" "name"="md-v6bv9-74d45c49c5-9znhj" "namespace"="md-test"
E0810 03:07:08.584182   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to get  \"test\" for Cluster default/test4-m28vj: failed to retrieve  external object \"default\"/\"test\": Object 'Kind' is missing in 'unstructured object has no kind'" "controller"="cluster" "name"="test4-m28vj" "namespace"="default"
I0810 03:07:09.312694   10479 conversion.go:64] controllers/Machine "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="test6-68zzg" "namespace"="default" "err"="you requested PartialObjectMetadata, but the requested object is a list (*apiextensions.CustomResourceDefinitionList)"
I0810 03:07:09.312949   10479 machine_controller_noderef.go:86] controllers/Machine "msg"="Set Machine's NodeRef" "machine"="test6-68zzg" "namespace"="default" "noderef"="id-node-1"
E0810 03:07:09.325221   10479 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to find a CustomResourceDefinition for /, Kind= with contract \"cluster.x-k8s.io/v1alpha3\"" "controller"="machine" "name"="test6-68zzg" "namespace"="default"

------------------------------
• [SLOW TEST:15.061 seconds]
Cluster Reconciler
/home/prow/go/src/sigs.k8s.io/cluster-api/controllers/cluster_controller_test.go:45
  Should successfully set Status.ControlPlaneInitialized on the cluster object if controlplane is ready
... skipping 25 lines ...
I0810 03:07:09.659717   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
•I0810 03:07:09.661103   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0810 03:07:09.661728   10479 conversion.go:64]  "msg"="Cannot retrieve CRD with metadata only client, falling back to slower listing" "machine"="machine-test" "namespace"="default" "err"="cannot create a metadata client without a rest config"
I0810 03:07:09.662322   10479 machine_controller_phases.go:278]  "msg"="Infrastructure provider is not ready, requeuing" "machine"="machine-test" "namespace"="default" 
I0810 03:07:09.662349   10479 machine_controller_noderef.go:45]  "msg"="Cannot reconcile Machine's Node, no valid ProviderID yet" "machine"="machine-test" "namespace"="default" 
•I0810 03:07:09.662825   10479 machine_controller.go:327]  "msg"="Draining node" "cluster"="test-cluster" "machine"="machine-test" "namespace"="default" "node"="machine-test-node"
E0810 03:07:09.664034   10479 machine_controller.go:494]  "msg"="Error creating a remote client while deleting Machine, won't retry" "error"="failed to create REST configuration for Cluster default/test-cluster: invalid configuration: no configuration has been provided" "cluster"="test-cluster" "machine"="machine-test" "namespace"="default" "node"="machine-test-node" 
•

Ran 16 of 16 Specs in 26.695 seconds
SUCCESS! -- 16 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestGinkgoSuite (26.70s)
PASS
Tearing down test suite
I0810 03:07:09.665447   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinedeployment" 
I0810 03:07:09.665527   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machineset" 
I0810 03:07:09.665566   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
I0810 03:07:09.665593   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
I0810 03:07:09.665611   10479 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
I0810 03:07:09.665620   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machine" 
I0810 03:07:09.665666   10479 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinehealthcheck" 
E0810 03:07:09.665870   10479 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="inotify instance already closed"  
E0810 03:07:09.665939   10479 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-033255248/tls.crt: no such file or directory"  
E0810 03:07:09.695007   10479 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1.Node: Get http://127.0.0.1:42913/api/v1/nodes?allowWatchBookmarks=true&resourceVersion=1291&timeout=10s&timeoutSeconds=428&watch=true: dial tcp 127.0.0.1:42913: connect: connection refused
E0810 03:07:09.695087   10479 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1.Node: Get http://127.0.0.1:42913/api/v1/nodes?allowWatchBookmarks=true&resourceVersion=1291&timeout=10s&timeoutSeconds=504&watch=true: dial tcp 127.0.0.1:42913: connect: connection refused
E0810 03:07:09.695097   10479 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1.Node: Get http://127.0.0.1:42913/api/v1/nodes?allowWatchBookmarks=true&resourceVersion=1291&timeout=10s&timeoutSeconds=569&watch=true: dial tcp 127.0.0.1:42913: connect: connection refused
ok  	sigs.k8s.io/cluster-api/controllers	82.452s
=== RUN   TestGetResourceFound
--- PASS: TestGetResourceFound (0.00s)
=== RUN   TestGetResourceNotFound
--- PASS: TestGetResourceNotFound (0.00s)
=== RUN   TestCloneTemplateResourceNotFound
... skipping 260 lines ...
I0810 03:06:00.312909   10830 server.go:199] controller-runtime/webhook "msg"="serving webhook server"  "host"="" "port"=45201
I0810 03:06:00.313591   10830 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
I0810 03:06:00.357056   10830 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="cluster" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"controlPlaneEndpoint":{"host":"","port":0}},"status":{"infrastructureReady":false,"controlPlaneInitialized":false}}}
I0810 03:06:00.559669   10830 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="cluster" 
I0810 03:06:00.559747   10830 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="cluster" "worker count"=1
•I0810 03:06:00.842654   10830 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="cluster" 
E0810 03:06:02.884002   10830 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="timed out waiting for the condition"  "cluster"="cluster-cache-test-skjqq/test-cluster"
•E0810 03:06:03.380284   10830 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="the server could not find the requested resource"  "cluster"="cluster-cache-test-lr4qs/test-cluster"
•E0810 03:06:03.980921   10830 cluster_cache.go:366]  "msg"="Error health checking cluster" "error"="Get http://127.0.0.1:33179/?timeout=50ms: dial tcp 127.0.0.1:33179: connect: connection refused"  "cluster"="cluster-cache-test-nhj97/test-cluster"
•I0810 03:06:04.202123   10830 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinedeployment" "source"={"Type":{"metadata":{"creationTimestamp":null},"spec":{"clusterName":"","selector":{},"template":{"metadata":{},"spec":{"clusterName":"","bootstrap":{},"infrastructureRef":{}}}},"status":{}}}
I0810 03:06:04.402504   10830 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="machinedeployment" 
I0810 03:06:04.402581   10830 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="machinedeployment" "worker count"=1
I0810 03:06:04.402655   10830 controller.go:132] controller-runtime/controller "msg"="Starting EventSource" "controller"="machinedeployment" "source"={}
•E0810 03:06:04.927677   10830 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0810 03:06:04.927735   10830 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-006762601/tls.crt: no such file or directory"  
I0810 03:06:04.927769   10830 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
I0810 03:06:04.928007   10830 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="machinedeployment" 
E0810 03:06:04.972378   10830 reflector.go:309] pkg/mod/k8s.io/client-go@v0.17.9/tools/cache/reflector.go:105: Failed to watch *v1alpha3.Cluster: Get http://127.0.0.1:38779/apis/cluster.x-k8s.io/v1alpha3/clusters?allowWatchBookmarks=true&resourceVersion=146&timeout=10s&timeoutSeconds=478&watch=true: dial tcp 127.0.0.1:38779: connect: connection refused


Ran 5 of 5 Specs in 16.030 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestGinkgoSuite (16.03s)
PASS
ok  	sigs.k8s.io/cluster-api/controllers/remote	16.182s
?   	sigs.k8s.io/cluster-api/controllers/remote/fake	[no test files]
?   	sigs.k8s.io/cluster-api/controlplane/kubeadm	[no test files]
=== RUN   TestKubeadmControlPlaneDefault
... skipping 208 lines ...
=== RUN   TestReconcileNoClusterOwnerRef
I0810 03:06:13.850556   11622 controller.go:130]  "msg"="Cluster Controller has not yet set OwnerRef" "kubeadmControlPlane"="foo" "namespace"="test" 
--- PASS: TestReconcileNoClusterOwnerRef (0.00s)
=== RUN   TestReconcileNoKCP
--- PASS: TestReconcileNoKCP (0.00s)
=== RUN   TestReconcileNoCluster
E0810 03:06:13.851978   11622 controller.go:126]  "msg"="Failed to retrieve owner Cluster from the API Server" "error"="clusters.cluster.x-k8s.io \"foo\" not found" "kubeadmControlPlane"="foo" "namespace"="test" 
--- PASS: TestReconcileNoCluster (0.00s)
=== RUN   TestReconcilePaused
I0810 03:06:13.852719   11622 controller.go:136]  "msg"="Reconciliation is paused for this object" "cluster"="foo" "kubeadmControlPlane"="foo" "namespace"="test" 
I0810 03:06:13.852925   11622 controller.go:136]  "msg"="Reconciliation is paused for this object" "cluster"="foo" "kubeadmControlPlane"="foo" "namespace"="test" 
--- PASS: TestReconcilePaused (0.00s)
=== RUN   TestReconcileClusterNoEndpoints
... skipping 192 lines ...
    --- PASS: TestSelectMachineForScaleDown/when_there_is_an_up_to_date_machine_with_delete_annotation,_while_there_are_any_outdated_machines_without_annotatio_that_still_exist,_it_returns_oldest_marked_machine_first (0.00s)
=== RUN   TestPreflightChecks
=== RUN   TestPreflightChecks/control_plane_without_machines_(not_initialized)_should_pass
=== RUN   TestPreflightChecks/control_plane_with_a_deleting_machine_should_requeue
I0810 03:06:29.966035   11622 scale.go:173]  "msg"="Waiting for machines to be deleted" "cluster"="" "kubeadmControlPlane"="" "namespace"="" "Machines"=""
=== RUN   TestPreflightChecks/control_plane_with_an_unhealthy_machine_condition_should_requeue
I0810 03:06:29.966184   11622 scale.go:212]  "msg"="Waiting for control plane to pass preflight checks" "cluster"="" "kubeadmControlPlane"="" "namespace"="" "failures"="machine  reports APIServerPodHealthy condition is false (Error, )"
=== RUN   TestPreflightChecks/control_plane_with_an_healthy_machine_and_an_healthy_kcp_condition_should_pass
--- PASS: TestPreflightChecks (0.00s)
    --- PASS: TestPreflightChecks/control_plane_without_machines_(not_initialized)_should_pass (0.00s)
    --- PASS: TestPreflightChecks/control_plane_with_a_deleting_machine_should_requeue (0.00s)
    --- PASS: TestPreflightChecks/control_plane_with_an_unhealthy_machine_condition_should_requeue (0.00s)
    --- PASS: TestPreflightChecks/control_plane_with_an_healthy_machine_and_an_healthy_kcp_condition_should_pass (0.00s)
... skipping 18 lines ...
=== RUN   TestAPIs
Running Suite: Controller Suite
===============================
Random Seed: 1660100760
Will run 1 of 1 specs

E0810 03:06:29.989616   11622 controller.go:126]  "msg"="Failed to retrieve owner Cluster from the API Server" "error"="Cluster.cluster.x-k8s.io \"kcp-foo-ir520d\" not found" "kubeadmControlPlane"="kcp-foo-ir520d" "namespace"="test" 
•

Ran 1 of 1 Specs in 0.019 seconds
SUCCESS! -- 1 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (0.02s)
=== RUN   TestKubeadmControlPlaneReconciler_upgradeControlPlane
I0810 03:06:29.991120   11622 controller.go:246]  "msg"="Reconcile KubeadmControlPlane" "cluster"="kcp-foo-ol8ikt" "kubeadmControlPlane"="kcp-foo-ol8ikt" "namespace"="test" 
I0810 03:06:31.608729   11622 controller.go:331]  "msg"="Rolling out Control Plane machines" "cluster"="kcp-foo-ol8ikt" "kubeadmControlPlane"="kcp-foo-ol8ikt" "namespace"="test" "needRollout"=["kcp-foo-ol8ikt-zb265"]
I0810 03:06:31.608896   11622 scale.go:212]  "msg"="Waiting for control plane to pass preflight checks" "cluster"="kcp-foo-ol8ikt" "kubeadmControlPlane"="kcp-foo-ol8ikt" "namespace"="test" "failures"="[machine kcp-foo-ol8ikt-c7wfp does not have APIServerPodHealthy condition, machine kcp-foo-ol8ikt-c7wfp does not have ControllerManagerPodHealthy condition, machine kcp-foo-ol8ikt-c7wfp does not have SchedulerPodHealthy condition, machine kcp-foo-ol8ikt-c7wfp does not have EtcdPodHealthy condition, machine kcp-foo-ol8ikt-c7wfp does not have EtcdMemberHealthy condition]"
--- PASS: TestKubeadmControlPlaneReconciler_upgradeControlPlane (1.62s)
... skipping 60 lines ...
==================================
Random Seed: 1660100759
Will run 7 of 7 specs

•••••••
Ran 7 of 7 Specs in 0.001 seconds
SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestControlPlane (0.00s)
=== RUN   TestHasUnhealthyMachine
--- PASS: TestHasUnhealthyMachine (0.00s)
=== RUN   TestNewFailureDomainPicker
=== RUN   TestNewFailureDomainPicker/simple
=== RUN   TestNewFailureDomainPicker/no_machines
... skipping 111 lines ...
=======================================
Random Seed: 1660100759
Will run 7 of 7 specs

•••••••
Ran 7 of 7 Specs in 0.000 seconds
SUCCESS! -- 7 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestMachineCollection (0.00s)
=== RUN   TestUpdateEtcdConditions
=== RUN   TestUpdateEtcdConditions/if_list_nodes_return_an_error_should_report_all_the_conditions_Unknown
=== RUN   TestUpdateEtcdConditions/node_without_machine_should_be_ignored_if_there_are_provisioning_machines
=== RUN   TestUpdateEtcdConditions/node_without_machine_should_report_a_problem_at_KCP_level_if_there_are_no_provisioning_machines
=== RUN   TestUpdateEtcdConditions/failure_creating_the_etcd_client_should_report_unknown_condition
... skipping 297 lines ...
    --- PASS: TestUpdateKubeProxyImageInfo/returns_error_if_image_repository_is_invalid (0.00s)
    --- PASS: TestUpdateKubeProxyImageInfo/does_not_update_image_repository_when_no_kube-proxy_update_is_requested (0.00s)
=== RUN   TestRemoveMachineFromKubeadmConfigMap
=== RUN   TestRemoveMachineFromKubeadmConfigMap/does_not_panic_if_machine_is_nil
=== RUN   TestRemoveMachineFromKubeadmConfigMap/does_not_panic_if_machine_noderef_is_nil
=== RUN   TestRemoveMachineFromKubeadmConfigMap/returns_error_if_unable_to_find_kubeadm-config_for_Kubernetes_version_<_1.22.0
E0810 03:06:14.932640   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:06:23.255709   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:06:32.241718   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:06:43.372035   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:06:59.845287   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:07:12.853920   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:07:30.501383   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:07:51.424705   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:08:22.441177   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
E0810 03:09:07.598396   11569 workload_cluster.go:354]  "msg"="unable to get kubeadmConfigMap" "error"="error getting kube-system/kubeadm-config configmap from target cluster: configmaps \"kubeadm-config\" not found"  
=== RUN   TestRemoveMachineFromKubeadmConfigMap/returns_error_if_unable_to_remove_api_endpoint_for_Kubernetes_version_<_1.22.0
=== RUN   TestRemoveMachineFromKubeadmConfigMap/removes_the_machine_node_ref_from_kubeadm_config_for_Kubernetes_version_<_1.22.0
=== RUN   TestRemoveMachineFromKubeadmConfigMap/no_op_for_Kubernetes_version_>=_1.22.0
--- PASS: TestRemoveMachineFromKubeadmConfigMap (172.67s)
    --- PASS: TestRemoveMachineFromKubeadmConfigMap/does_not_panic_if_machine_is_nil (0.00s)
    --- PASS: TestRemoveMachineFromKubeadmConfigMap/does_not_panic_if_machine_noderef_is_nil (0.00s)
... skipping 83 lines ...
    --- PASS: TestIsKubernetesVersionSupported/workload_cluster_and_version_<_v1.22.0 (0.00s)
    --- PASS: TestIsKubernetesVersionSupported/workload_cluster_and_version_>=_v1.22.0 (0.00s)
    --- PASS: TestIsKubernetesVersionSupported/management_cluster_and_version_<_v1.22.0 (0.00s)
    --- PASS: TestIsKubernetesVersionSupported/management_cluster_and_version_>=_v1.22.0 (0.00s)
    --- PASS: TestIsKubernetesVersionSupported/management_cluster_and_version_>=_v1.23.0 (0.00s)
PASS
E0810 03:09:07.607630   11569 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0810 03:09:07.607669   11569 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-017362981/tls.crt: no such file or directory"  
E0810 03:09:07.607691   11569 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="no such file or directory"  
E0810 03:09:07.607713   11569 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-017362981/tls.crt: no such file or directory"  
I0810 03:09:07.608191   11569 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
ok  	sigs.k8s.io/cluster-api/controlplane/kubeadm/internal	188.632s
=== RUN   TestEtcdMembers_WithErrors
--- PASS: TestEtcdMembers_WithErrors (0.00s)
=== RUN   TestEtcdMembers_WithSuccess
--- PASS: TestEtcdMembers_WithSuccess (0.00s)
... skipping 288 lines ...
I0810 03:06:20.834012   11907 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="clusterresourcesetbinding" "worker count"=1
I0810 03:06:20.834245   11907 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="clusterresourceset" "source"={"Type":{"kind":"ConfigMap","apiVersion":"v1","metadata":{"creationTimestamp":null}}}
I0810 03:06:20.934648   11907 controller.go:155] controller-runtime/controller "msg"="Starting EventSource" "controller"="clusterresourceset" "source"={"Type":{"kind":"Secret","apiVersion":"v1","metadata":{"creationTimestamp":null}}}
I0810 03:06:21.035095   11907 controller.go:162] controller-runtime/controller "msg"="Starting Controller" "controller"="clusterresourceset" 
I0810 03:06:21.035177   11907 controller.go:189] controller-runtime/controller "msg"="Starting workers" "controller"="clusterresourceset" "worker count"=1
I0810 03:06:21.187885   11907 clusterresourcesetbinding_controller.go:84]  "msg"="deleting ClusterResourceSetBinding because the owner Cluster no longer exists" "clusterresourcesetbinding"={"Namespace":"default","Name":"cluster-cvszxy"} 
E0810 03:06:21.227926   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•I0810 03:06:22.284019   11907 clusterresourcesetbinding_controller.go:84]  "msg"="deleting ClusterResourceSetBinding because the owner Cluster no longer exists" "clusterresourcesetbinding"={"Namespace":"default","Name":"cluster-uhphis"} 
E0810 03:06:22.319426   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•E0810 03:06:33.399989   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"

------------------------------
• Failure [11.078 seconds]
ClusterResourceSet Reconciler
/home/prow/go/src/sigs.k8s.io/cluster-api/exp/addons/controllers/clusterresourceset_controller_test.go:40
  Should reconcile a ClusterResourceSet when a resource is created that is part of ClusterResourceSet resources [It]
... skipping 8 lines ...
------------------------------
STEP: Creating the Cluster
STEP: Creating the remote Cluster kubeconfig
STEP: Creating a Secret and a ConfigMap with ConfigMap in their data field
STEP: Verifying ClusterResourceSetBinding is created with cluster owner reference
STEP: Deleting the Kubeconfigsecret
E0810 03:06:34.559822   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•E0810 03:06:35.596447   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset2\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset2" "namespace"="default"
E0810 03:06:36.611519   11907 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="clusterresourcesets.addons.cluster.x-k8s.io \"test-clusterresourceset\" not found" "controller"="clusterresourceset" "name"="test-clusterresourceset" "namespace"="default"
•I0810 03:06:36.618510   11907 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="clusterresourceset" 
I0810 03:06:36.618586   11907 controller.go:203] controller-runtime/controller "msg"="Stopping workers" "controller"="clusterresourcesetbinding" 
I0810 03:06:36.618653   11907 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  
E0810 03:06:36.619244   11907 certwatcher.go:143] controller-runtime/certwatcher "msg"="error re-watching file" "error"="inotify instance already closed"  
E0810 03:06:36.619307   11907 certwatcher.go:148] controller-runtime/certwatcher "msg"="error re-reading certificate" "error"="open /tmp/envtest-serving-certs-602042460/tls.crt: no such file or directory"  



Summarizing 1 Failure:

[Fail] ClusterResourceSet Reconciler [It] Should reconcile a ClusterResourceSet when a resource is created that is part of ClusterResourceSet resources 
/home/prow/go/src/sigs.k8s.io/cluster-api/exp/addons/controllers/clusterresourceset_controller_test.go:307

Ran 5 of 5 Specs in 28.769 seconds
FAIL! -- 4 Passed | 1 Failed | 0 Pending | 0 Skipped
--- FAIL: TestAPIs (28.77s)
FAIL
FAIL	sigs.k8s.io/cluster-api/exp/addons/controllers	28.869s
?   	sigs.k8s.io/cluster-api/exp/addons/controllers/predicates	[no test files]
=== RUN   TestMachinePoolDefault
=== RUN   TestMachinePoolDefault/for_MachinePool
=== RUN   TestMachinePoolDefault/for_MachinePool/validate-on-create
=== RUN   TestMachinePoolDefault/for_MachinePool/validate-on-update
=== RUN   TestMachinePoolDefault/for_MachinePool/validate-on-delete
... skipping 58 lines ...
    --- PASS: TestReconcileMachinePoolBootstrap/new_machinepool,_no_bootstrap_config_or_data (0.00s)
    --- PASS: TestReconcileMachinePoolBootstrap/existing_machinepool,_bootstrap_data_should_not_change (0.00s)
    --- PASS: TestReconcileMachinePoolBootstrap/existing_machinepool,_bootstrap_provider_is_to_not_ready (0.00s)
=== RUN   TestReconcileMachinePoolInfrastructure
=== RUN   TestReconcileMachinePoolInfrastructure/new_machinepool,_infrastructure_config_ready
=== RUN   TestReconcileMachinePoolInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machinepool_is_running,_infra_object_is_deleted,_expect_failed
E0810 03:06:13.013778   12062 machinepool_controller_phases.go:237]  "msg"="MachinePool infrastructure reference has been deleted after being ready, setting failure state" "error"="could not find infrastructure.cluster.x-k8s.io/v1alpha3, Kind=InfrastructureConfig \"infra-config1\" for MachinePool \"machinepool-test\" in namespace \"default\", requeuing: failed to retrieve InfrastructureConfig external object \"default\"/\"infra-config1\": infrastructureconfigs.infrastructure.cluster.x-k8s.io \"infra-config1\" not found" "cluster"="test-cluster" "machinepool"="machinepool-test" "namespace"="default" 
=== RUN   TestReconcileMachinePoolInfrastructure/infrastructure_ref_is_paused
--- PASS: TestReconcileMachinePoolInfrastructure (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/new_machinepool,_infrastructure_config_ready (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/ready_bootstrap,_infra,_and_nodeRef,_machinepool_is_running,_infra_object_is_deleted,_expect_failed (0.00s)
    --- PASS: TestReconcileMachinePoolInfrastructure/infrastructure_ref_is_paused (0.00s)
=== RUN   TestMachinePoolFinalizer
... skipping 95 lines ...
•I0810 03:06:26.089745   12062 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
•I0810 03:06:26.117700   12062 machinepool_controller_noderef.go:90]  "msg"="Cannot assign NodeRefs to MachinePool, no matching Nodes"  
••I0810 03:06:26.120397   12062 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 9 of 9 Specs in 13.046 seconds
SUCCESS! -- 9 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestAPIs (13.05s)
PASS
ok  	sigs.k8s.io/cluster-api/exp/controllers	13.394s
?   	sigs.k8s.io/cluster-api/exp/util	[no test files]
?   	sigs.k8s.io/cluster-api/feature	[no test files]
?   	sigs.k8s.io/cluster-api/hack/boilerplate/test	[no test files]
... skipping 266 lines ...
--- PASS: TestGetFirstReasonAndMessage (0.00s)
=== RUN   TestNewConditionsGroup
--- PASS: TestNewConditionsGroup (0.00s)
=== RUN   TestMergeRespectPriority
=== RUN   TestMergeRespectPriority/aggregate_nil_list_return_nil
=== RUN   TestMergeRespectPriority/aggregate_empty_list_return_nil
=== RUN   TestMergeRespectPriority/When_there_is_false/error_it_returns_false/error
=== RUN   TestMergeRespectPriority/When_there_is_false/warning_and_no_false/error,_it_returns_false/warning
=== RUN   TestMergeRespectPriority/When_there_is_false/info_and_no_false/error_or_false/warning,_it_returns_false/info
=== RUN   TestMergeRespectPriority/When_there_is_true_and_no_false/*,_it_returns_info
=== RUN   TestMergeRespectPriority/When_there_is_unknown_and_no_true_or_false/*,_it_returns_unknown
=== RUN   TestMergeRespectPriority/nil_conditions_are_ignored
--- PASS: TestMergeRespectPriority (0.00s)
    --- PASS: TestMergeRespectPriority/aggregate_nil_list_return_nil (0.00s)
    --- PASS: TestMergeRespectPriority/aggregate_empty_list_return_nil (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/error_it_returns_false/error (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/warning_and_no_false/error,_it_returns_false/warning (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_false/info_and_no_false/error_or_false/warning,_it_returns_false/info (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_true_and_no_false/*,_it_returns_info (0.00s)
    --- PASS: TestMergeRespectPriority/When_there_is_unknown_and_no_true_or_false/*,_it_returns_unknown (0.00s)
    --- PASS: TestMergeRespectPriority/nil_conditions_are_ignored (0.00s)
=== RUN   TestNewPatch
=== RUN   TestNewPatch/No_changes_return_empty_patch
... skipping 204 lines ...
I0810 03:06:50.568696   15145 server.go:199] controller-runtime/webhook "msg"="serving webhook server"  "host"="" "port"=43547
I0810 03:06:50.568871   15145 certwatcher.go:83] controller-runtime/certwatcher "msg"="Starting certificate watcher"  
••••••••••••••I0810 03:06:51.317628   15145 server.go:208] controller-runtime/webhook "msg"="shutting down webhook server"  


Ran 14 of 14 Specs in 6.964 seconds
SUCCESS! -- 14 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestPatch (6.96s)
=== RUN   TestToUnstructured
=== RUN   TestToUnstructured/with_a_typed_object
=== RUN   TestToUnstructured/with_an_unstructured_object
--- PASS: TestToUnstructured (0.00s)
    --- PASS: TestToUnstructured/with_a_typed_object (0.00s)
... skipping 105 lines ...
    --- PASS: TestToUnstructured/returns_error_for_invalid_yaml#01 (0.00s)
    --- PASS: TestToUnstructured/returns_error_for_invalid_yaml#02 (0.00s)
=== RUN   TestFromUnstructured
--- PASS: TestFromUnstructured (0.00s)
PASS
ok  	sigs.k8s.io/cluster-api/util/yaml	0.042s
FAIL
make: *** [Makefile:116: test] Error 1