Recent runs || View in Spyglass
PR | MartinForReal: [release-1.24] fix: should update dns label if target pip already have one and it needs update |
Result | FAILURE |
Tests | 0 failed / 0 succeeded |
Started | |
Elapsed | 6m15s |
Revision | 0e82aa6797f56cd39fd72012d4c89ba9b81034bb |
Refs |
2701 |
... skipping 1225 lines ... --- PASS: TestNodeUpdateRetryTimeout/count_1 (0.00s) --- PASS: TestNodeUpdateRetryTimeout/count_2 (0.00s) --- PASS: TestNodeUpdateRetryTimeout/count_3 (0.00s) --- PASS: TestNodeUpdateRetryTimeout/count_50 (0.00s) === RUN TestNewCloudCIDRAllocator I1104 08:42:47.688799 24392 cloud_cidr_allocator.go:100] Sending events to api server. E1104 08:42:47.689832 24392 range_allocator.go:325] Error filtering out service cidr out cluster cidr:10.10.0.0/24 (index:0) 10.10.0.0/25: CIDR: 0.0.0.0/0 is out of the range of CIDR allocator I1104 08:42:47.689985 24392 cloud_cidr_allocator.go:200] Using cloud CIDR allocator (provider: azure) --- PASS: TestNewCloudCIDRAllocator (0.00s) === RUN TestUpdateMaxSubnetSizes === RUN TestUpdateMaxSubnetSizes/updateMaxSubnetSizes_should_calculate_the_correct_max_sizes === RUN TestUpdateMaxSubnetSizes/updateMaxSubnetSizes_should_work_with_the_dual_stack_cidrs --- PASS: TestUpdateMaxSubnetSizes (0.00s) ... skipping 24 lines ... I1104 08:42:47.696262 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.696300 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. === RUN TestOccupyPreExistingCIDR/success,_dual_stack_both_allocated_correctly I1104 08:42:47.696893 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.697233 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.697283 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. === RUN TestOccupyPreExistingCIDR/fail,_single_stack_incorrect_node_allocation I1104 08:42:47.698086 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.698527 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.698573 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. === RUN TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_from_non_existing_cidr I1104 08:42:47.699826 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.700199 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.700243 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. === RUN TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_bad_v4 I1104 08:42:47.701490 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.701985 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.702037 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. === RUN TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_bad_v6 I1104 08:42:47.702676 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.703118 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.703170 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. --- PASS: TestOccupyPreExistingCIDR (0.01s) --- PASS: TestOccupyPreExistingCIDR/success,_single_stack_no_node_allocation (0.00s) --- PASS: TestOccupyPreExistingCIDR/success,_dual_stack_no_node_allocation (0.00s) --- PASS: TestOccupyPreExistingCIDR/success,_single_stack_correct_node_allocation (0.00s) --- PASS: TestOccupyPreExistingCIDR/success,_dual_stack_both_allocated_correctly (0.00s) --- PASS: TestOccupyPreExistingCIDR/fail,_single_stack_incorrect_node_allocation (0.00s) --- PASS: TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_from_non_existing_cidr (0.00s) --- PASS: TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_bad_v4 (0.00s) --- PASS: TestOccupyPreExistingCIDR/fail,_dualstack_node_allocating_bad_v6 (0.00s) === RUN TestAllocateOrOccupyCIDRSuccess I1104 08:42:47.704114 24392 range_allocator.go:82] Sending events to api server. I1104 08:42:47.704615 24392 range_allocator.go:110] No Service CIDR provided. Skipping filtering out service addresses. I1104 08:42:47.704683 24392 range_allocator.go:116] No Secondary Service CIDR provided. Skipping filtering out secondary service addresses. I1104 08:42:47.704991 24392 range_allocator.go:172] Starting range CIDR allocator I1104 08:42:47.705072 24392 shared_informer.go:255] Waiting for caches to sync for cidrallocator ... skipping 224 lines ... --- PASS: TestGetPrivateIPsForMachine (0.00s) === RUN TestGetIPForMachineWithRetry W1104 08:43:16.508857 25195 azure_wrap.go:77] Unable to find node vm: instance not found E1104 08:43:16.508913 25195 azure_backoff.go:141] GetIPForMachineWithRetry(vm): backoff failure, will retry,err=instance not found --- PASS: TestGetIPForMachineWithRetry (0.00s) === RUN TestCreateOrUpdateSecurityGroupCanceled W1104 08:43:16.509524 25195 azure_backoff.go:164] CreateOrUpdateSecurityGroup(sg) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: canceledandsupersededduetoanotheroperation, NSG request: {} --- PASS: TestCreateOrUpdateSecurityGroupCanceled (0.00s) === RUN TestCreateOrUpdateLB W1104 08:43:16.510283 25195 azure_backoff.go:225] LoadBalancerClient.CreateOrUpdate(lb) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 412, RawError: %!w(<nil>), LoadBalancer request: {} W1104 08:43:16.510774 25195 azure_backoff.go:225] LoadBalancerClient.CreateOrUpdate(lb) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: canceledandsupersededduetoanotheroperation, LoadBalancer request: {} W1104 08:43:16.511148 25195 azure_backoff.go:225] LoadBalancerClient.CreateOrUpdate(lb) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: Code="ReferencedResourceNotProvisioned" Message="Cannot proceed with operation because resource /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/publicIPAddresses/pip used by resource /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb is not in Succeeded state. Resource is in Failed state and the last operation that updated/is updating the resource is PutPublicIpAddressOperation.", LoadBalancer request: {} --- PASS: TestCreateOrUpdateLB (0.00s) === RUN TestListAgentPoolLBs E1104 08:43:16.512529 25195 azure_backoff.go:381] LoadBalancerClient.List(rg) failure with err=&{false 500 0001-01-01 00:00:00 +0000 UTC <nil>} W1104 08:43:16.513115 25195 azure_backoff.go:333] ListManagedLBs: no LBs found --- PASS: TestListAgentPoolLBs (0.00s) === RUN TestListPIP E1104 08:43:16.513541 25195 azure_backoff.go:399] PublicIPAddressesClient.List(rg) failure with err=&{false 500 0001-01-01 00:00:00 +0000 UTC <nil>} --- PASS: TestListPIP (0.00s) === RUN TestCreateOrUpdatePIP W1104 08:43:16.514021 25195 azure_backoff.go:421] PublicIPAddressesClient.CreateOrUpdate(rg, nic) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 412, RawError: %!w(<nil>), PublicIP request: {} W1104 08:43:16.514350 25195 azure_backoff.go:421] PublicIPAddressesClient.CreateOrUpdate(rg, nic) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: canceledandsupersededduetoanotheroperation, PublicIP request: {} W1104 08:43:16.514598 25195 azure_backoff.go:421] PublicIPAddressesClient.CreateOrUpdate(rg, nic) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: %!w(<nil>), PublicIP request: {} --- PASS: TestCreateOrUpdatePIP (0.00s) === RUN TestCreateOrUpdateInterface E1104 08:43:16.514918 25195 azure_backoff.go:448] InterfacesClient.CreateOrUpdate(nic) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: %!w(<nil>) --- PASS: TestCreateOrUpdateInterface (0.00s) === RUN TestDeletePublicIP E1104 08:43:16.515186 25195 azure_backoff.go:463] PublicIPAddressesClient.Delete(pip) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: %!w(<nil>) --- PASS: TestDeletePublicIP (0.00s) === RUN TestDeleteLB E1104 08:43:16.515429 25195 azure_backoff.go:491] LoadBalancerClient.Delete(lb) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: %!w(<nil>) --- PASS: TestDeleteLB (0.00s) === RUN TestCreateOrUpdateRouteTable W1104 08:43:16.515734 25195 azure_backoff.go:509] RouteTablesClient.CreateOrUpdate(rt) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 412, RawError: %!w(<nil>), RouteTable request: {} E1104 08:43:16.515792 25195 azure_backoff.go:521] RouteTablesClient.CreateOrUpdate(rt) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 412, RawError: %!w(<nil>) W1104 08:43:16.516022 25195 azure_backoff.go:509] RouteTablesClient.CreateOrUpdate(rt) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: canceledandsupersededduetoanotheroperation, RouteTable request: {} E1104 08:43:16.516102 25195 azure_backoff.go:521] RouteTablesClient.CreateOrUpdate(rt) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: canceledandsupersededduetoanotheroperation --- PASS: TestCreateOrUpdateRouteTable (0.00s) === RUN TestCreateOrUpdateRoute --- PASS: TestCreateOrUpdateRoute (0.00s) === RUN TestDeleteRouteWithName E1104 08:43:16.517087 25195 azure_backoff.go:560] RoutesClient.Delete(rt, rt) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: %!w(<nil>) --- PASS: TestDeleteRouteWithName (0.00s) === RUN TestCreateOrUpdateVMSS E1104 08:43:16.517421 25195 azure_backoff.go:574] CreateOrUpdateVMSS: error getting vmss(vmss): &{false 500 0001-01-01 00:00:00 +0000 UTC <nil>} E1104 08:43:16.517552 25195 azure_backoff.go:574] CreateOrUpdateVMSS: error getting vmss(vmss): &{false 429 0001-01-01 00:00:00 +0000 UTC <nil>} E1104 08:43:16.517661 25195 azure_backoff.go:574] CreateOrUpdateVMSS: error getting vmss(vmss): &{false 0 0001-01-01 00:00:00 +0000 UTC azure cloud provider rate limited(write) for operation CreateOrUpdate} --- PASS: TestCreateOrUpdateVMSS (0.00s) === RUN TestRequestBackoff --- PASS: TestRequestBackoff (0.00s) === RUN TestCreateOrUpdateLBBackendPool --- PASS: TestCreateOrUpdateLBBackendPool (0.00s) === RUN TestDeleteLBBackendPool ... skipping 12 lines ... --- PASS: TestGetConfigFromSecret/Error_should_be_reported_when_secret_doesn't_exists (0.00s) --- PASS: TestGetConfigFromSecret/Error_should_be_reported_when_secret_exists_but_cloud-config_data_is_not_provided (0.00s) --- PASS: TestGetConfigFromSecret/Error_should_be_reported_when_it_failed_to_parse_Azure_cloud-config (0.00s) === RUN TestInitializeCloudFromSecret === RUN TestInitializeCloudFromSecret/Azure_config_shouldn't_be_override_when_cloud_config_type_is_file === RUN TestInitializeCloudFromSecret/Azure_config_shouldn't_be_override_when_cloud_config_type_is_unknown E1104 08:43:16.530648 25195 azure_config.go:55] Failed to initialize Azure cloud provider: cloudConfigType UnknownConfigType is not supported, supported values are [file merge secret] === RUN TestInitializeCloudFromSecret/Azure_config_should_be_override_when_cloud_config_type_is_secret === RUN TestInitializeCloudFromSecret/Azure_config_should_be_override_when_cloud_config_type_is_merge === RUN TestInitializeCloudFromSecret/Error_should_be_reported_when_secret_doesn't_exists E1104 08:43:16.538672 25195 azure_config.go:45] Failed to get cloud-config from secret: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found === RUN TestInitializeCloudFromSecret/Error_should_be_reported_when_secret_exists_but_cloud-config_data_is_not_provided E1104 08:43:16.539196 25195 azure_config.go:45] Failed to get cloud-config from secret: cloud-config is not set in the secret (kube-system/azure-cloud-provider) --- PASS: TestInitializeCloudFromSecret (0.01s) --- PASS: TestInitializeCloudFromSecret/Azure_config_shouldn't_be_override_when_cloud_config_type_is_file (0.00s) --- PASS: TestInitializeCloudFromSecret/Azure_config_shouldn't_be_override_when_cloud_config_type_is_unknown (0.00s) --- PASS: TestInitializeCloudFromSecret/Azure_config_should_be_override_when_cloud_config_type_is_secret (0.00s) --- PASS: TestInitializeCloudFromSecret/Azure_config_should_be_override_when_cloud_config_type_is_merge (0.00s) --- PASS: TestInitializeCloudFromSecret/Error_should_be_reported_when_secret_doesn't_exists (0.00s) ... skipping 3 lines ... === RUN TestConfigSecretMetadata/configSecretMetadata_should_set_the_secret_metadata_from_the_default_values --- PASS: TestConfigSecretMetadata (0.00s) --- PASS: TestConfigSecretMetadata/configSecretMetadata_should_set_the_secret_metadata_from_the_given_parameters (0.00s) --- PASS: TestConfigSecretMetadata/configSecretMetadata_should_set_the_secret_metadata_from_the_default_values (0.00s) === RUN TestCommonAttachDisk W1104 08:43:16.541359 25195 azure_wrap.go:77] Unable to find node vm1: instance not found E1104 08:43:16.541415 25195 azure_controller_common.go:466] error of getting data disks for node vm1: instance not found W1104 08:43:16.543055 25195 azure_controller_common.go:183] size of disk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name) is 4096GB which is bigger than limit(4096GB), set cacheMode as None W1104 08:43:16.543684 25195 azure_controller_common.go:183] size of disk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name) is 4096GB which is bigger than limit(4096GB), set cacheMode as None W1104 08:43:16.544050 25195 azure_controller_common.go:162] volume /subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name is actually attached to current node vm1, invalidate vm cache and return error --- PASS: TestCommonAttachDisk (0.00s) === RUN TestCommonAttachDiskWithVMSS E1104 08:43:16.545221 25195 azure_controller_common.go:466] error of getting data disks for node vm1: error of converting vmSet (&{%!q(*provider.Cloud=&{{{ tenant false subscription } {{false 0 0 0 0} <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil> <nil>} rg westus vnet rg subnet nsg rg rt rg as vmss vmss map[] false false false false false 0 0 <nil> <nil> 250 0 0 0 0 0 60 120 120 120 120 600 120 0 nodeIPConfiguration 0 rg} { } { { }} 0xc0005adb00 0xc0005adb30 0xc0005adad0 0xc0005adb10 0xc0005adae0 0xc0005adaf0 0xc0005adb20 0xc0005adb60 <nil> 0xc0005adab0 0xc0005adac0 <nil> 0xc0005adb40 0xc0005adb50 <nil> <nil> <nil> <nil> <nil> <nil> <nil> 0xc0005adb70 {0 0 0 0 0} <nil> 0xc0005adb80 0xc0005adc00 false false {{0 0} 0 0 0 0} map[] map[] map[] map[] map[] map[] 0x228f400 {0 0} map[] map[westus:[1 2 3]] {{0 0} 0 0 0 0} <nil> <nil> 0xc0005adaa0 <nil> 0xc0003ddb30 0xc0003ddc20 0xc0003ddd40 0xc0003dde30 0xc0003ddf20 0xc0005ae0f0 <nil> {0 0} 0xc000588648 0xc0005720e0}) %!q(*cache.TimedCache=&{0xc00051d290 {0 0} 0x2309a40 600000000000})}) to ScaleSet with vmType "vmss" E1104 08:43:16.545372 25195 azure_controller_common.go:466] error of getting data disks for node vmss1: not a vmss instance --- PASS: TestCommonAttachDiskWithVMSS (0.00s) === RUN TestCommonDetachDisk W1104 08:43:16.545711 25195 azure_wrap.go:77] Unable to find node vm1: instance not found W1104 08:43:16.545783 25195 azure_standard.go:520] Unable to find node vm1: instance not found W1104 08:43:16.545803 25195 azure_controller_common.go:315] azureDisk - failed to get azure instance id(vm1), DetachDisk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name) will assume disk is already detached --- PASS: TestCommonDetachDisk (0.00s) === RUN TestCommonUpdateVM --- PASS: TestCommonUpdateVM (0.00s) === RUN TestGetDiskLun --- PASS: TestGetDiskLun (0.00s) === RUN TestSetDiskLun ... skipping 11 lines ... === RUN TestFilterNonExistingDisks E1104 08:43:16.553059 25195 azure_controller_common.go:588] disk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/DiskName2) does not exist, removed from data disk list E1104 08:43:16.553216 25195 azure_controller_common.go:588] disk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/DiskName3) does not exist, removed from data disk list E1104 08:43:16.553310 25195 azure_controller_common.go:588] disk(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/DiskName4) does not exist, removed from data disk list --- PASS: TestFilterNonExistingDisks (0.00s) === RUN TestFilterNonExistingDisksWithSpecialHTTPStatusCode E1104 08:43:16.553666 25195 azure_controller_common.go:583] checkDiskExists(/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/specialdisk) failed with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: instance not found --- PASS: TestFilterNonExistingDisksWithSpecialHTTPStatusCode (0.00s) === RUN TestIsInstanceNotFoundError --- PASS: TestIsInstanceNotFoundError (0.00s) === RUN TestAttachDiskRequestFuncs --- PASS: TestAttachDiskRequestFuncs (0.00s) === RUN TestDetachDiskRequestFuncs --- PASS: TestDetachDiskRequestFuncs (0.00s) === RUN TestStandardAttachDisk W1104 08:43:16.555388 25195 azure_wrap.go:77] Unable to find node vm2: instance not found E1104 08:43:16.556237 25195 azure_controller_standard.go:105] azureDisk - attach disk list(map[uri:%!s(*provider.AttachDiskOptions=&{ReadOnly DiskEncryptionSetID false 0})]) on rg(rg) vm(vm1) failed, err: &{false 404 0001-01-01 00:00:00 +0000 UTC instance not found} E1104 08:43:16.556333 25195 azure_controller_standard.go:107] azureDisk - begin to filterNonExistingDisks(map[uri:0xc000032f40]) on rg(rg) vm(vm1) E1104 08:43:16.556371 25195 azure_controller_common.go:583] checkDiskExists(uri) failed with error: invalid disk URI: uri --- PASS: TestStandardAttachDisk (0.00s) === RUN TestStandardDetachDisk W1104 08:43:16.556884 25195 azure_wrap.go:77] Unable to find node vm2: instance not found W1104 08:43:16.556925 25195 azure_controller_standard.go:139] azureDisk - cannot find node vm2, skip detaching disk list(map[]) E1104 08:43:16.557294 25195 azure_controller_standard.go:168] detach azure disk on node(vm1): disk list(map[/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/diskx:diskx]) not found E1104 08:43:16.558825 25195 azure_controller_standard.go:168] detach azure disk on node(vm1): disk list(map[]) not found E1104 08:43:16.558895 25195 azure_controller_standard.go:197] azureDisk - detach disk list(map[]) on rg(rg) vm(vm1) failed, err: &{false 404 0001-01-01 00:00:00 +0000 UTC instance not found} E1104 08:43:16.558942 25195 azure_controller_standard.go:199] azureDisk - begin to filterNonExistingDisks(map[]) on rg(rg) vm(vm1) --- PASS: TestStandardDetachDisk (0.00s) === RUN TestStandardUpdateVM --- PASS: TestStandardUpdateVM (0.00s) === RUN TestGetDataDisks W1104 08:43:16.560492 25195 azure_wrap.go:77] Unable to find node vm2: instance not found --- PASS: TestGetDataDisks (2.00s) === RUN TestAttachDiskWithVMSS --- PASS: TestAttachDiskWithVMSS (0.00s) === RUN TestDetachDiskWithVMSS E1104 08:43:18.575996 25195 azure_controller_vmss.go:201] azureDisk - detach disk list(map[/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name:disk-name]) on rg(rg) vm(vmss00-vm-000000) failed, err: &{false 404 0001-01-01 00:00:00 +0000 UTC instance not found} E1104 08:43:18.576270 25195 azure_controller_vmss.go:203] azureDisk - begin to filterNonExistingDisks(map[/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name:disk-name]) on rg(rg) vm(vmss00-vm-000000) E1104 08:43:18.578150 25195 azure_controller_vmss.go:171] detach azure disk on node(vmss00-vm-000000): disk list(map[/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Compute/disks/disk-name-err:disk-name-err]) not found --- PASS: TestDetachDiskWithVMSS (0.01s) === RUN TestUpdateVMWithVMSS --- PASS: TestUpdateVMWithVMSS (0.01s) === RUN TestGetDataDisksWithVMSS ... skipping 14 lines ... E1104 08:43:18.640374 25195 azure_backoff.go:141] GetIPForMachineWithRetry(vm2): backoff failure, will retry,err=instance not found --- PASS: TestNodeAddresses (0.03s) === RUN TestInstanceExistsByProviderID W1104 08:43:18.652329 25195 azure_wrap.go:77] Unable to find node vm3: instance not found W1104 08:43:18.652385 25195 azure_standard.go:520] Unable to find node vm3: instance not found E1104 08:43:18.655668 25195 azure_vmss.go:415] Unable to find node by providerID azure:///subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssee6c2/virtualMachines/0: instance not found E1104 08:43:18.656664 25195 azure_vmss.go:756] VirtualMachineScaleSetVMsClient.List(rg, missing-vmss) failed: &{false 404 0001-01-01 00:00:00 +0000 UTC <nil>} E1104 08:43:18.656748 25195 azure_vmss.go:415] Unable to find node by providerID azure:///subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/missing-vmss/virtualMachines/0: instance not found --- PASS: TestInstanceExistsByProviderID (0.01s) === RUN TestNodeAddressesByProviderID --- PASS: TestNodeAddressesByProviderID (0.01s) === RUN TestCurrentNodeName --- PASS: TestCurrentNodeName (0.00s) ... skipping 19 lines ... --- PASS: TestCleanupVMSetFromBackendPoolForInstanceNotFound (0.00s) === RUN TestReconcileBackendPoolsNodeIPConfig --- PASS: TestReconcileBackendPoolsNodeIPConfig (0.00s) === RUN TestReconcileBackendPoolsNodeIPConfigPreConfigured --- PASS: TestReconcileBackendPoolsNodeIPConfigPreConfigured (0.00s) === RUN TestReconcileBackendPoolsNodeIPToIPConfig E1104 08:43:18.679138 25195 azure_loadbalancer_backendpool.go:172] bc.ReconcileBackendPools for service (default/test): failed to cleanup IP based backend pool testCluster: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: create or update LB backend pool error --- PASS: TestReconcileBackendPoolsNodeIPToIPConfig (0.00s) === RUN TestReconcileBackendPoolsNodeIP --- PASS: TestReconcileBackendPoolsNodeIP (0.00s) === RUN TestReconcileBackendPoolsNodeIPPreConfigured --- PASS: TestReconcileBackendPoolsNodeIPPreConfigured (0.00s) === RUN TestReconcileBackendPoolsNodeIPConfigToIP E1104 08:43:18.680592 25195 azure_loadbalancer_backendpool.go:463] bi.ReconcileBackendPools for service (default/test): failed to EnsureBackendPoolDeleted: delete LB backend pool error --- PASS: TestReconcileBackendPoolsNodeIPConfigToIP (0.00s) === RUN TestRemoveNodeIPAddressFromBackendPool --- PASS: TestRemoveNodeIPAddressFromBackendPool (0.00s) === RUN TestGetBackendPrivateIPsNodeIPConfig W1104 08:43:18.681406 25195 azure_loadbalancer_backendpool.go:256] bc.GetBackendPrivateIPs for service (default/svc1): failed to get private IPs of node node2 --- PASS: TestGetBackendPrivateIPsNodeIPConfig (0.00s) === RUN TestGetLoadBalancer --- PASS: TestGetLoadBalancer (0.00s) === RUN TestFindProbe --- PASS: TestFindProbe (0.00s) === RUN TestFindRule --- PASS: TestFindRule (0.00s) === RUN TestSubnet --- PASS: TestSubnet (0.00s) === RUN TestEnsureLoadBalancerDeleted E1104 08:43:18.699774 25195 azure_loadbalancer.go:139] reconcileLoadBalancer(default/service6) failed: findMatchedPIPByLoadBalancerIP: cannot find public IP with IP address 1.2.3.4 in resource group random-rg E1104 08:43:18.701079 25195 azure_loadbalancer.go:139] reconcileLoadBalancer(default/service7) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: instance not found W1104 08:43:18.701693 25195 azure_backoff.go:333] ListManagedLBs: no LBs found W1104 08:43:18.701929 25195 azure_backoff.go:333] ListManagedLBs: no LBs found --- PASS: TestEnsureLoadBalancerDeleted (0.02s) === RUN TestServiceOwnsPublicIP === RUN TestServiceOwnsPublicIP/false_should_be_returned_when_pip_is_nil W1104 08:43:18.702726 25195 azure_loadbalancer.go:3373] serviceOwnsPublicIP: nil service or public IP ... skipping 119 lines ... W1104 08:43:18.760057 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress === RUN TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv6 W1104 08:43:18.760814 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress === RUN TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv6#01 W1104 08:43:18.761602 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress azure_loadbalancer_test.go:4417: Error Trace: /home/prow/go/src/sigs.k8s.io/cloud-provider-azure/pkg/provider/azure_loadbalancer_test.go:4417 Error: Not equal: expected: &network.PublicIPAddress{Response:autorest.Response{Response:(*http.Response)(nil)}, ExtendedLocation:(*network.ExtendedLocation)(nil), Sku:(*network.PublicIPAddressSku)(nil), PublicIPAddressPropertiesFormat:(*network.PublicIPAddressPropertiesFormat)(0xc00016a8f0), Etag:(*string)(nil), Zones:(*[]string)(nil), ID:(*string)(0xc00078db10), Name:(*string)(0xc00078db00), Type:(*string)(nil), Location:(*string)(nil), Tags:map[string]*string(nil)} actual : &network.PublicIPAddress{Response:autorest.Response{Response:(*http.Response)(nil)}, ExtendedLocation:(*network.ExtendedLocation)(nil), Sku:(*network.PublicIPAddressSku)(nil), PublicIPAddressPropertiesFormat:(*network.PublicIPAddressPropertiesFormat)(0xc000462000), Etag:(*string)(nil), Zones:(*[]string)(nil), ID:(*string)(0xc000438ad0), Name:(*string)(0xc0004388c0), Type:(*string)(nil), Location:(*string)(nil), Tags:map[string]*string{"k8s-azure-dns-label-service":(*string)(0xc000438950)}} Diff: --- Expected +++ Actual ... skipping 5 lines ... + } }) Test: TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv6#01 === RUN TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv4 W1104 08:43:18.763411 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress azure_loadbalancer_test.go:4417: Error Trace: /home/prow/go/src/sigs.k8s.io/cloud-provider-azure/pkg/provider/azure_loadbalancer_test.go:4417 Error: Not equal: expected: &network.PublicIPAddress{Response:autorest.Response{Response:(*http.Response)(nil)}, ExtendedLocation:(*network.ExtendedLocation)(nil), Sku:(*network.PublicIPAddressSku)(nil), PublicIPAddressPropertiesFormat:(*network.PublicIPAddressPropertiesFormat)(0xc00016aa50), Etag:(*string)(nil), Zones:(*[]string)(nil), ID:(*string)(0xc00078db60), Name:(*string)(0xc00078db50), Type:(*string)(nil), Location:(*string)(nil), Tags:map[string]*string(nil)} actual : &network.PublicIPAddress{Response:autorest.Response{Response:(*http.Response)(nil)}, ExtendedLocation:(*network.ExtendedLocation)(nil), Sku:(*network.PublicIPAddressSku)(nil), PublicIPAddressPropertiesFormat:(*network.PublicIPAddressPropertiesFormat)(0xc0004622c0), Etag:(*string)(nil), Zones:(*[]string)(nil), ID:(*string)(0xc000439180), Name:(*string)(0xc000438f70), Type:(*string)(nil), Location:(*string)(nil), Tags:map[string]*string{"k8s-azure-dns-label-service":(*string)(0xc000439000)}} Diff: --- Expected +++ Actual ... skipping 12 lines ... === RUN TestEnsurePublicIPExists/shall_tag_the_service_name_to_the_pip_correctly W1104 08:43:18.766203 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress === RUN TestEnsurePublicIPExists/shall_not_call_the_PUT_API_for_IPV6_pip_if_it_is_not_necessary W1104 08:43:18.766739 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress === RUN TestEnsurePublicIPExists/shall_update_pip_tags_if_there_is_any_change W1104 08:43:18.767168 25195 azure_loadbalancer.go:3378] serviceOwnsPublicIP: empty pip.IPAddress --- FAIL: TestEnsurePublicIPExists (0.01s) --- PASS: TestEnsurePublicIPExists/shall_return_existed_PIP_if_there_is_any (0.00s) --- PASS: TestEnsurePublicIPExists/shall_create_a_new_pip_if_there_is_no_existed_pip (0.00s) --- PASS: TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label (0.00s) --- PASS: TestEnsurePublicIPExists/shall_delete_DNS_from_PIP_if_DNS_label_is_set_empty (0.00s) --- PASS: TestEnsurePublicIPExists/shall_not_delete_DNS_from_PIP_if_DNS_label_annotation_is_not_set (0.00s) --- PASS: TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv6 (0.00s) --- FAIL: TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv6#01 (0.00s) --- FAIL: TestEnsurePublicIPExists/shall_update_existed_PIP's_dns_label_for_IPv4 (0.00s) --- PASS: TestEnsurePublicIPExists/shall_report_an_conflict_error_if_the_DNS_label_is_conflicted (0.00s) --- PASS: TestEnsurePublicIPExists/shall_return_the_pip_without_calling_PUT_API_if_the_tags_are_good (0.00s) --- PASS: TestEnsurePublicIPExists/shall_tag_the_service_name_to_the_pip_correctly (0.00s) --- PASS: TestEnsurePublicIPExists/shall_not_call_the_PUT_API_for_IPV6_pip_if_it_is_not_necessary (0.00s) --- PASS: TestEnsurePublicIPExists/shall_update_pip_tags_if_there_is_any_change (0.00s) === RUN TestEnsurePublicIPExistsWithExtendedLocation ... skipping 20 lines ... --- PASS: TestIsFrontendIPConfigIsUnsafeToDelete (0.00s) === RUN TestCheckLoadBalancerResourcesConflicted --- PASS: TestCheckLoadBalancerResourcesConflicted (0.00s) === RUN TestEnsurePIPTagged === RUN TestEnsurePIPTagged/ensurePIPTagged_should_ensure_the_pip_is_tagged_as_configured W1104 08:43:18.772457 25195 azure_utils.go:115] parseTags: empty key, ignoring this key-value pair W1104 08:43:18.772488 25195 azure_utils.go:110] parseTags: error when parsing key-value pair ghi, would ignore this one === RUN TestEnsurePIPTagged/ensurePIPTagged_should_delete_the_old_tags_if_the_SystemTags_is_set W1104 08:43:18.772667 25195 azure_utils.go:115] parseTags: empty key, ignoring this key-value pair W1104 08:43:18.772696 25195 azure_utils.go:110] parseTags: error when parsing key-value pair ghi, would ignore this one === RUN TestEnsurePIPTagged/ensurePIPTagged_should_support_TagsMap W1104 08:43:18.772872 25195 azure_utils.go:115] parseTags: empty key, ignoring this key-value pair W1104 08:43:18.772899 25195 azure_utils.go:110] parseTags: error when parsing key-value pair ghi, would ignore this one --- PASS: TestEnsurePIPTagged (0.00s) --- PASS: TestEnsurePIPTagged/ensurePIPTagged_should_ensure_the_pip_is_tagged_as_configured (0.00s) --- PASS: TestEnsurePIPTagged/ensurePIPTagged_should_delete_the_old_tags_if_the_SystemTags_is_set (0.00s) --- PASS: TestEnsurePIPTagged/ensurePIPTagged_should_support_TagsMap (0.00s) === RUN TestEnsureLoadBalancerTagged === RUN TestEnsureLoadBalancerTagged/ensureLoadBalancerTagged_should_not_delete_the_old_tags_if_SystemTags_is_not_specified ... skipping 16 lines ... === RUN TestRemoveFrontendIPConfigurationFromLoadBalancerUpdate === RUN TestRemoveFrontendIPConfigurationFromLoadBalancerUpdate/removeFrontendIPConfigurationFromLoadBalancer_should_remove_the_unwanted_frontend_IP_configuration_and_update_the_LB_if_there_are_remaining_frontend_IP_configurations --- PASS: TestRemoveFrontendIPConfigurationFromLoadBalancerUpdate (0.00s) --- PASS: TestRemoveFrontendIPConfigurationFromLoadBalancerUpdate/removeFrontendIPConfigurationFromLoadBalancer_should_remove_the_unwanted_frontend_IP_configuration_and_update_the_LB_if_there_are_remaining_frontend_IP_configurations (0.00s) === RUN TestCleanOrphanedLoadBalancerLBInUseByVMSS === RUN TestCleanOrphanedLoadBalancerLBInUseByVMSS/cleanOrphanedLoadBalancer_should_retry_deleting_lb_when_meeting_LoadBalancerInUseByVirtualMachineScaleSet E1104 08:43:18.775149 25195 azure_backoff.go:491] LoadBalancerClient.Delete(test) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: { "error": { "code": "LoadBalancerInUseByVirtualMachineScaleSet", "message": "Cannot delete load balancer /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb since its child resources lb are in use by virtual machine scale set /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmss.", "details": [] } } W1104 08:43:18.775211 25195 azure_loadbalancer.go:483] cleanOrphanedLoadBalancer(test, default/test, test): failed to DeleteLB: &{false 0 0001-01-01 00:00:00 +0000 UTC { "error": { "code": "LoadBalancerInUseByVirtualMachineScaleSet", "message": "Cannot delete load balancer /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb since its child resources lb are in use by virtual machine scale set /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmss.", "details": [] } }} === RUN TestCleanOrphanedLoadBalancerLBInUseByVMSS/cleanupOrphanedLoadBalancer_should_not_call_delete_api_if_the_lb_does_not_exist --- PASS: TestCleanOrphanedLoadBalancerLBInUseByVMSS (0.00s) --- PASS: TestCleanOrphanedLoadBalancerLBInUseByVMSS/cleanOrphanedLoadBalancer_should_retry_deleting_lb_when_meeting_LoadBalancerInUseByVirtualMachineScaleSet (0.00s) --- PASS: TestCleanOrphanedLoadBalancerLBInUseByVMSS/cleanupOrphanedLoadBalancer_should_not_call_delete_api_if_the_lb_does_not_exist (0.00s) === RUN TestReconcileZonesForFrontendIPConfigs === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_reconcile_the_zones_for_the_new_fip_config === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_reconcile_the_zones_for_the_new_internal_fip_config === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_report_an_error_if_failed_to_get_zones E1104 08:43:18.777872 25195 azure_zones.go:100] getRegionZonesMapWrapper: failed to fetch zones information: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: get zone failed E1104 08:43:18.777923 25195 azure_loadbalancer.go:1912] reconcileLoadBalancer for service (default/test)(true): failed to getFrontendZones: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: get zone failed === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_use_the_nil_zones_of_the_existing_frontend === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_use_the_non-nil_zones_of_the_existing_frontend === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_reuse_the_existing_private_IP_for_internal_services_when_subnet_does_not_change === RUN TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_not_reuse_the_existing_private_IP_for_internal_services_when_subnet_changes --- PASS: TestReconcileZonesForFrontendIPConfigs (0.00s) --- PASS: TestReconcileZonesForFrontendIPConfigs/reconcileFrontendIPConfigs_should_reconcile_the_zones_for_the_new_fip_config (0.00s) ... skipping 8 lines ... === RUN TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_the_basic_load_balancer_is_used W1104 08:43:18.780305 25195 azure_backoff.go:333] ListManagedLBs: no LBs found === RUN TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_`nodes`_is_nil W1104 08:43:18.780666 25195 azure_backoff.go:333] ListManagedLBs: no LBs found === RUN TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_the_vmSet_is_not_sharing_the_primary_slb === RUN TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_report_an_error_if_failed_to_list_managed_load_balancers E1104 08:43:18.781358 25195 azure_backoff.go:381] LoadBalancerClient.List(rg) failure with err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} --- PASS: TestReconcileSharedLoadBalancer (0.00s) --- PASS: TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_decouple_the_vmSet_from_its_dedicated_lb_if_the_vmSet_is_sharing_the_primary_slb (0.00s) --- PASS: TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_the_basic_load_balancer_is_used (0.00s) --- PASS: TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_`nodes`_is_nil (0.00s) --- PASS: TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_do_nothing_if_the_vmSet_is_not_sharing_the_primary_slb (0.00s) --- PASS: TestReconcileSharedLoadBalancer/reconcileSharedLoadBalancer_should_report_an_error_if_failed_to_list_managed_load_balancers (0.00s) ... skipping 11 lines ... --- PASS: TestDeleteManagedDisk (0.00s) === RUN TestGetDisk --- PASS: TestGetDisk (0.00s) === RUN TestResizeDisk --- PASS: TestResizeDisk (0.00s) === RUN TestGetLabelsForVolume E1104 08:43:18.787890 25195 azure_managedDiskController.go:415] Failed to get information for AzureDisk fakeGetDiskFailed: &{false 0 0001-01-01 00:00:00 +0000 UTC Get Disk failed} E1104 08:43:18.788063 25195 azure_managedDiskController.go:402] Failed to get resource group for AzureDisk invalidDiskURI: invalid disk URI: invalidDiskURI --- PASS: TestGetLabelsForVolume (0.00s) === RUN TestReconcilePrivateLinkService --- PASS: TestReconcilePrivateLinkService (0.00s) === RUN TestDisablePLSNetworkPolicy --- PASS: TestDisablePLSNetworkPolicy (0.00s) === RUN TestSafeDeletePLS ... skipping 62 lines ... --- PASS: TestInitializeCloudProviderRateLimitConfig (0.00s) === RUN TestDeleteRoute --- PASS: TestDeleteRoute (0.10s) === RUN TestDeleteRouteDualStack --- PASS: TestDeleteRouteDualStack (0.20s) === RUN TestCreateRoute E1104 08:43:19.205816 25195 azure_routes.go:130] getRouteTable() failed with error: Route table name is not configured E1104 08:43:19.205957 25195 azure_routes.go:438] CreateRoute failed for node "node" with error: Route table name is not configured W1104 08:43:19.406942 25195 azure_backoff.go:509] RouteTablesClient.CreateOrUpdate(rt2) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error, RouteTable request: {"location":"location","properties":{"routes":[{"name":"node","properties":{"addressPrefix":"1.2.3.4/24","nextHopIpAddress":"2.4.6.8","nextHopType":"VirtualAppliance"}}]}} E1104 08:43:19.407035 25195 azure_backoff.go:521] RouteTablesClient.CreateOrUpdate(rt2) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error E1104 08:43:19.407074 25195 azure_routes.go:207] CreateOrUpdateRouteTable() failed with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error E1104 08:43:19.407134 25195 azure_routes.go:438] CreateRoute failed for node "node" with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error W1104 08:43:19.605900 25195 azure_backoff.go:509] RouteTablesClient.CreateOrUpdate(rt4) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error, RouteTable request: {"location":"location","properties":{}} E1104 08:43:19.606150 25195 azure_backoff.go:521] RouteTablesClient.CreateOrUpdate(rt4) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error E1104 08:43:19.606228 25195 azure_routes.go:138] createRouteTable() failed with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error E1104 08:43:19.606313 25195 azure_routes.go:438] CreateRoute failed for node "node" with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: CreateOrUpdate error E1104 08:43:19.705649 25195 azure_routes.go:144] getRouteTable() failed with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: Get error E1104 08:43:19.705737 25195 azure_routes.go:438] CreateRoute failed for node "node" with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: Get error E1104 08:43:19.806422 25195 azure_routes.go:130] getRouteTable() failed with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: Get error E1104 08:43:19.806517 25195 azure_routes.go:438] CreateRoute failed for node "node" with error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 500, RawError: Get error E1104 08:43:19.806782 25195 azure_backoff.go:141] GetIPForMachineWithRetry(node): backoff failure, will retry,err=getIP error --- PASS: TestCreateRoute (0.80s) === RUN TestCreateRouteTable --- PASS: TestCreateRouteTable (0.00s) === RUN TestProcessRoutes --- PASS: TestProcessRoutes (0.00s) === RUN TestFindFirstIPByFamily ... skipping 54 lines ... --- PASS: TestGetIPConfigByIPFamily (0.00s) === RUN TestGetBackendPoolName --- PASS: TestGetBackendPoolName (0.00s) === RUN TestGetStandardInstanceIDByNodeName W1104 08:43:19.912582 25195 azure_wrap.go:77] Unable to find node vm2: instance not found W1104 08:43:19.912635 25195 azure_standard.go:520] Unable to find node vm2: instance not found E1104 08:43:19.913177 25195 azure_standard.go:539] convertResourceGroupNameToLower failed with error: "/subscriptions/subscription/resourceGroups/rg/Microsoft.Compute/virtualMachines/vm4" isn't in Azure resource ID format ".*/subscriptions/(?:.*)/resourceGroups/(.+)/providers/(?:.*)" --- PASS: TestGetStandardInstanceIDByNodeName (0.00s) === RUN TestGetStandardVMPowerStatusByNodeName W1104 08:43:19.913456 25195 azure_wrap.go:77] Unable to find node vm1: instance not found --- PASS: TestGetStandardVMPowerStatusByNodeName (0.00s) === RUN TestGetStandardVMProvisioningStateByNodeName W1104 08:43:19.914173 25195 azure_wrap.go:77] Unable to find node vm1: instance not found ... skipping 7 lines ... E1104 08:43:19.915750 25195 azure_standard.go:761] as.GetVMSetNames - No availability sets found for nodes in the cluster, node count(1) E1104 08:43:19.915839 25195 azure_standard.go:773] as.GetVMSetNames - Availability set (vm2) in service annotation not found --- PASS: TestGetStandardVMSetNames (0.00s) === RUN TestExtractResourceGroupByNicID --- PASS: TestExtractResourceGroupByNicID (0.00s) === RUN TestStandardEnsureHostInPool E1104 08:43:19.917153 25195 azure_standard.go:931] error: az.EnsureHostInPool(vm2), az.VMSet.GetPrimaryInterface.Get(vm2, availabilityset-1), err=resource name was missing from identifier W1104 08:43:19.917652 25195 azure_standard.go:936] EnsureHostInPool skips node vm3 because its primary nic nic3 is in Failed state --- PASS: TestStandardEnsureHostInPool (0.01s) === RUN TestStandardEnsureHostsInPool --- PASS: TestStandardEnsureHostsInPool (0.00s) === RUN TestServiceOwnsFrontendIP W1104 08:43:19.925542 25195 azure_standard.go:366] serviceOwnsFrontendIP: unexpected error when finding match public IP of the service with loadBalancerLP 1.2.3.4: findMatchedPIPByLoadBalancerIP: cannot find public IP with IP address 1.2.3.4 in resource group rg --- PASS: TestServiceOwnsFrontendIP (0.00s) === RUN TestStandardEnsureBackendPoolDeleted --- PASS: TestStandardEnsureBackendPoolDeleted (0.00s) === RUN TestStandardGetNodeNameByIPConfigurationID --- PASS: TestStandardGetNodeNameByIPConfigurationID (0.00s) === RUN TestGetAvailabilitySetByNodeName W1104 08:43:19.928445 25195 azure_standard.go:1280] Unable to find node vm-2: instance not found E1104 08:43:19.928566 25195 azure_standard.go:469] AvailabilitySetsClient.List failed: &{false 0 0001-01-01 00:00:00 +0000 UTC error during vmas list} --- PASS: TestGetAvailabilitySetByNodeName (0.00s) === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMasksByProviderID_should_report_an_error_if_the_providerID_is_not_valid === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_report_cloudprovider.InstanceNotFound_if_there_is_no_matching_vmas W1104 08:43:19.929873 25195 azure_standard.go:1280] Unable to find node vm-1: instance not found === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes_even_if_some_of_the_tags_are_not_specified === RUN TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_not_fail_even_if_some_of_the_tag_is_invalid E1104 08:43:19.930894 25195 azure_standard.go:1306] GetNodeCIDRMasksByProviderID: error when paring the value of the ipv4 mask size abc: strconv.Atoi: parsing "abc": invalid syntax --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMasksByProviderID_should_report_an_error_if_the_providerID_is_not_valid (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_report_cloudprovider.InstanceNotFound_if_there_is_no_matching_vmas (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes_even_if_some_of_the_tags_are_not_specified (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderIDAvailabilitySet/GetNodeCIDRMaksByProviderID_should_not_fail_even_if_some_of_the_tag_is_invalid (0.00s) ... skipping 4 lines ... --- PASS: TestGetAvailabilitySetNameByID (0.00s) --- PASS: TestGetAvailabilitySetNameByID/getAvailabilitySetNameByID_should_return_empty_string_if_the_given_ID_is_empty (0.00s) --- PASS: TestGetAvailabilitySetNameByID/getAvailabilitySetNameByID_should_report_an_error_if_the_format_of_the_given_ID_is_wrong (0.00s) --- PASS: TestGetAvailabilitySetNameByID/getAvailabilitySetNameByID_should_extract_the_VMAS_name_from_the_given_ID (0.00s) === RUN TestGetNodeVMSetName W1104 08:43:19.931704 25195 azure_standard.go:795] as.GetNodeVMSetName: cannot get host name from node E1104 08:43:19.931819 25195 azure_backoff.go:98] VirtualMachinesClient.List(rg) failure with err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:19.931882 25195 azure_standard.go:801] as.GetNodeVMSetName - ListVirtualMachines failed, err=Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error E1104 08:43:19.932017 25195 azure_standard.go:813] as.GetNodeVMSetName: failed to get last segment of ID /: resource name was missing from identifier --- PASS: TestGetNodeVMSetName (0.00s) === RUN TestCreateFileShare --- PASS: TestCreateFileShare (0.00s) === RUN TestDeleteFileShare --- PASS: TestDeleteFileShare (0.00s) === RUN TestResizeFileShare ... skipping 26 lines ... === RUN TestFlipServiceInternalAnnotation --- PASS: TestFlipServiceInternalAnnotation (0.00s) === RUN TestAddPort --- PASS: TestAddPort (0.00s) === RUN TestLoadBalancerInternalServiceModeSelection E1104 08:43:19.996521 25195 azure_loadbalancer.go:814] selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(true) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) E1104 08:43:19.996597 25195 azure_loadbalancer.go:1535] reconcileLoadBalancer: failed to get load balancer for service "default/service-2", error: selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(true) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) E1104 08:43:19.996627 25195 azure_loadbalancer.go:139] reconcileLoadBalancer(default/service-2) failed: selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(true) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) W1104 08:43:20.011780 25195 azure_loadbalancer.go:1434] isFrontendIPConfigUnsafeToDelete: frontend IP configuration with ID fip on LB testCluster-internal cannot be deleted because it is being referenced by load balancing rules of other services W1104 08:43:20.012944 25195 azure_loadbalancer.go:1434] isFrontendIPConfigUnsafeToDelete: frontend IP configuration with ID fip on LB as-3-internal cannot be deleted because it is being referenced by load balancing rules of other services W1104 08:43:20.014039 25195 azure_loadbalancer.go:1434] isFrontendIPConfigUnsafeToDelete: frontend IP configuration with ID fip on LB as-2-internal cannot be deleted because it is being referenced by load balancing rules of other services W1104 08:43:20.014957 25195 azure_loadbalancer.go:1434] isFrontendIPConfigUnsafeToDelete: frontend IP configuration with ID fip on LB as-1-internal cannot be deleted because it is being referenced by load balancing rules of other services W1104 08:43:20.015808 25195 azure_loadbalancer.go:1434] isFrontendIPConfigUnsafeToDelete: frontend IP configuration with ID fip on LB testCluster-internal cannot be deleted because it is being referenced by load balancing rules of other services --- PASS: TestLoadBalancerInternalServiceModeSelection (0.08s) === RUN TestLoadBalancerExternalServiceModeSelection E1104 08:43:20.123169 25195 azure_loadbalancer.go:814] selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(false) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) E1104 08:43:20.123380 25195 azure_loadbalancer.go:1535] reconcileLoadBalancer: failed to get load balancer for service "default/service-2", error: selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(false) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) E1104 08:43:20.123454 25195 azure_loadbalancer.go:139] reconcileLoadBalancer(default/service-2) failed: selectLoadBalancer: cluster(testCluster) service(default/service-2) isInternal(false) - all available load balancers have exceeded maximum rule limit 1, vmSetNames ([as]) --- PASS: TestLoadBalancerExternalServiceModeSelection (0.14s) === RUN TestReconcileLoadBalancerAddServiceOnInternalSubnet --- PASS: TestReconcileLoadBalancerAddServiceOnInternalSubnet (0.00s) === RUN TestReconcileSecurityGroupFromAnyDestinationAddressPrefixToLoadBalancerIP --- PASS: TestReconcileSecurityGroupFromAnyDestinationAddressPrefixToLoadBalancerIP (0.00s) === RUN TestReconcileSecurityGroupDynamicLoadBalancerIP ... skipping 28 lines ... --- PASS: TestReconcileSecurityGroupRemoveService (0.00s) === RUN TestReconcileSecurityGroupRemoveServiceRemovesPort --- PASS: TestReconcileSecurityGroupRemoveServiceRemovesPort (0.00s) === RUN TestReconcileSecurityWithSourceRanges --- PASS: TestReconcileSecurityWithSourceRanges (0.00s) === RUN TestReconcileSecurityGroupEtagMismatch W1104 08:43:20.196448 25195 azure_backoff.go:164] CreateOrUpdateSecurityGroup(nsg) failed: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 412, RawError: PreconditionFailedEtagMismatch, NSG request: {"properties":{"securityRules":[{"name":"aservice1-TCP-80-Internet","properties":{"access":"Allow","destinationAddressPrefix":"1.2.3.4","destinationPortRange":"80","direction":"Inbound","priority":500,"protocol":"Tcp","sourceAddressPrefix":"Internet","sourcePortRange":"*"}}]}} --- PASS: TestReconcileSecurityGroupEtagMismatch (0.00s) === RUN TestReconcilePublicIPWithNewService --- PASS: TestReconcilePublicIPWithNewService (0.00s) === RUN TestReconcilePublicIPRemoveService --- PASS: TestReconcilePublicIPRemoveService (0.00s) === RUN TestReconcilePublicIPWithInternalService ... skipping 107 lines ... === RUN TestVMSSVMCacheWithDeletingNodes W1104 08:43:22.427285 25195 azure_vmss.go:743] Unable to find node vmssee6c2000000: instance not found W1104 08:43:22.427461 25195 azure_vmss.go:743] Unable to find node vmssee6c2000001: instance not found W1104 08:43:22.427693 25195 azure_vmss.go:743] Unable to find node vmssee6c2000002: instance not found --- PASS: TestVMSSVMCacheWithDeletingNodes (0.00s) === RUN TestVMSSVMCacheClearedWhenRGDeleted W1104 08:43:22.429010 25195 azure_vmss_cache.go:70] Skip caching vmss for resource group rg due to error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: %!w(<nil>) W1104 08:43:22.429115 25195 azure_vmss_cache.go:70] Skip caching vmss for resource group rg due to error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: %!w(<nil>) W1104 08:43:22.429164 25195 azure_vmss.go:743] Unable to find node vmssee6c2000000: instance not found --- PASS: TestVMSSVMCacheClearedWhenRGDeleted (0.00s) === RUN TestGetScaleSetVMInstanceID --- PASS: TestGetScaleSetVMInstanceID (0.00s) === RUN TestGetNodeIdentityByNodeName W1104 08:43:22.430969 25195 azure_vmss.go:743] Unable to find node agente6c2000005: instance not found ... skipping 4 lines ... --- PASS: TestGetInstanceIDByNodeName (0.00s) === RUN TestGetZoneByNodeName W1104 08:43:22.436624 25195 azure_vmss.go:743] Unable to find node agente6c2000005: instance not found --- PASS: TestGetZoneByNodeName (0.00s) === RUN TestGetIPByNodeName W1104 08:43:22.439316 25195 azure_vmss.go:743] Unable to find node agente6c2000005: instance not found E1104 08:43:22.439437 25195 azure_vmss.go:872] error: ss.GetPrimaryInterface(agente6c2000005), ss.getVmssVM(agente6c2000005), err=instance not found E1104 08:43:22.439478 25195 azure_vmss.go:510] error: ss.GetIPByNodeName(agente6c2000005), GetPrimaryInterface("agente6c2000005"), err=instance not found --- PASS: TestGetIPByNodeName (0.00s) === RUN TestGetNodeNameByIPConfigurationID E1104 08:43:22.442098 25195 azure_vmss.go:1498] Unable to find node by ipConfigurationID /subscriptions/script/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/scaleset2/virtualMachines/3/networkInterfaces/scaleset1/ipConfigurations/ipconfig1: instance not found E1104 08:43:22.442361 25195 azure_vmss.go:1487] Unable to find node by IPConfigurationID invalid-configuration-id: invalid ip config ID invalid-configuration-id --- PASS: TestGetNodeNameByIPConfigurationID (0.00s) === RUN TestExtractResourceGroupByVMSSNicID --- PASS: TestExtractResourceGroupByVMSSNicID (0.00s) === RUN TestGetVMSS E1104 08:43:22.443993 25195 azure_vmss_cache.go:74] VirtualMachineScaleSetsClient.List failed: &{false 0 0001-01-01 00:00:00 +0000 UTC error during vmss list} --- PASS: TestGetVMSS (0.00s) === RUN TestGetVmssVM W1104 08:43:22.445779 25195 azure_vmss.go:216] Unable to find node vmss-vm-000001: instance not found --- PASS: TestGetVmssVM (0.00s) === RUN TestGetPowerStatusByNodeName --- PASS: TestGetPowerStatusByNodeName (0.00s) === RUN TestGetProvisioningStateByNodeName --- PASS: TestGetProvisioningStateByNodeName (0.00s) === RUN TestGetVmssVMByInstanceID --- PASS: TestGetVmssVMByInstanceID (0.00s) === RUN TestGetVmssVMByNodeIdentity === RUN TestGetVmssVMByNodeIdentity/getVmssVMByNodeIdentity_should_return_the_correct_VMSS_VM === RUN TestGetVmssVMByNodeIdentity/getVmssVMByNodeIdentity_should_not_panic_with_a_gone_VMSS_VM_but_cache_for_the_VMSS_VM_entry_still_exists W1104 08:43:22.450458 25195 azure_vmss.go:173] failed to get VM with vmssVirtualMachinesEntry on Node "vmss-vm-000001" W1104 08:43:22.450650 25195 azure_vmss.go:173] failed to get VM with vmssVirtualMachinesEntry on Node "vmss-vm-000001" W1104 08:43:22.450847 25195 azure_vmss.go:216] Unable to find node vmss-vm-000001: instance not found --- PASS: TestGetVmssVMByNodeIdentity (0.00s) --- PASS: TestGetVmssVMByNodeIdentity/getVmssVMByNodeIdentity_should_return_the_correct_VMSS_VM (0.00s) --- PASS: TestGetVmssVMByNodeIdentity/getVmssVMByNodeIdentity_should_not_panic_with_a_gone_VMSS_VM_but_cache_for_the_VMSS_VM_entry_still_exists (0.00s) === RUN TestGetInstanceTypeByNodeName E1104 08:43:22.452079 25195 azure_backoff.go:98] VirtualMachinesClient.List(rg) failure with err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:22.452224 25195 azure_vmss.go:431] Failed to check isNodeManagedByAvailabilitySet: newAvailabilitySetNodesCache: failed to list vms in the resource group rg: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error --- PASS: TestGetInstanceTypeByNodeName (0.00s) === RUN TestGetPrimaryInterfaceID --- PASS: TestGetPrimaryInterfaceID (0.00s) === RUN TestGetPrimaryInterface E1104 08:43:22.453753 25195 azure_backoff.go:98] VirtualMachinesClient.List(rg) failure with err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:22.453841 25195 azure_vmss.go:857] Failed to check isNodeManagedByAvailabilitySet: newAvailabilitySetNodesCache: failed to list vms in the resource group rg: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error E1104 08:43:22.454092 25195 azure_vmss_cache.go:74] VirtualMachineScaleSetsClient.List failed: &{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:22.454143 25195 azure_vmss.go:872] error: ss.GetPrimaryInterface(vmss-vm-000000), ss.getVmssVM(vmss-vm-000000), err=Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error E1104 08:43:22.455284 25195 azure_vmss.go:878] error: ss.GetPrimaryInterface(vmss-vm-000000), ss.getPrimaryInterfaceID(), err=failed to find a primary nic for the vm. vmname="vmss_0" E1104 08:43:22.456451 25195 azure_vmss.go:884] error: ss.GetPrimaryInterface(vmss-vm-000000), getLastSegment(invalid/id/), err=resource name was missing from identifier E1104 08:43:22.458062 25195 azure_vmss.go:900] error: ss.GetPrimaryInterface(vmss-vm-000000), ss.GetVirtualMachineScaleSetNetworkInterface.Get(rg, vmss, vmss-vm-000000), err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} --- PASS: TestGetPrimaryInterface (0.01s) === RUN TestGetVMSSPublicIPAddress --- PASS: TestGetVMSSPublicIPAddress (0.00s) === RUN TestGetPrivateIPsByNodeName E1104 08:43:22.462197 25195 azure_backoff.go:98] VirtualMachinesClient.List(rg) failure with err=&{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:22.462293 25195 azure_vmss.go:857] Failed to check isNodeManagedByAvailabilitySet: newAvailabilitySetNodesCache: failed to list vms in the resource group rg: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error E1104 08:43:22.462314 25195 azure_vmss.go:573] error: ss.GetIPByNodeName(vmss-vm-000000), GetPrimaryInterface("vmss-vm-000000"), err=newAvailabilitySetNodesCache: failed to list vms in the resource group rg: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error --- PASS: TestGetPrivateIPsByNodeName (0.00s) === RUN TestGetVmssMachineID --- PASS: TestGetVmssMachineID (0.00s) === RUN TestExtractScaleSetNameByProviderID --- PASS: TestExtractScaleSetNameByProviderID (0.00s) === RUN TestExtractResourceGroupByProviderID --- PASS: TestExtractResourceGroupByProviderID (0.00s) === RUN TestListScaleSets E1104 08:43:22.463397 25195 azure_vmss.go:664] VirtualMachineScaleSetsClient.List failed: &{false 0 0001-01-01 00:00:00 +0000 UTC error} --- PASS: TestListScaleSets (0.00s) === RUN TestListScaleSetVMs E1104 08:43:22.463723 25195 azure_vmss.go:756] VirtualMachineScaleSetVMsClient.List(rg, vmss) failed: &{false 0 0001-01-01 00:00:00 +0000 UTC error} --- PASS: TestListScaleSetVMs (0.00s) === RUN TestGetAgentPoolScaleSets --- PASS: TestGetAgentPoolScaleSets (0.00s) === RUN TestGetVMSetNames W1104 08:43:22.465234 25195 azure_loadbalancer.go:3329] single standard load balancer doesn't work with annotation "service.beta.kubernetes.io/azure-load-balancer-mode", would ignore it W1104 08:43:22.465578 25195 azure_vmss_cache.go:195] skip caching vmssVM vmss-vm-000003 since its network profile hasn't initialized yet (probably still under creating) W1104 08:43:22.466150 25195 azure_vmss_cache.go:195] skip caching vmssVM vmss-vm-000003 since its network profile hasn't initialized yet (probably still under creating) E1104 08:43:22.466375 25195 azure_vmss.go:834] ss.GetVMSetNames - scale set (vmss-1) in service annotation not found W1104 08:43:22.466778 25195 azure_vmss_cache.go:195] skip caching vmssVM vmss-vm-000003 since its network profile hasn't initialized yet (probably still under creating) W1104 08:43:22.467468 25195 azure_vmss_cache.go:195] skip caching vmssVM vmss-vm-000003 since its network profile hasn't initialized yet (probably still under creating) W1104 08:43:22.467817 25195 azure_vmss.go:216] Unable to find node vmss-vm-000003: instance not found E1104 08:43:22.467875 25195 azure_vmss.go:816] ss.GetVMSetNames - getAgentPoolScaleSets failed err=(instance not found) W1104 08:43:22.468417 25195 azure_vmss_cache.go:195] skip caching vmssVM vmss-vm-000003 since its network profile hasn't initialized yet (probably still under creating) --- PASS: TestGetVMSetNames (0.00s) === RUN TestGetPrimaryNetworkInterfaceConfigurationForScaleSet --- PASS: TestGetPrimaryNetworkInterfaceConfigurationForScaleSet (0.00s) === RUN TestGetPrimaryIPConfigFromVMSSNetworkConfig --- PASS: TestGetPrimaryIPConfigFromVMSSNetworkConfig (0.00s) === RUN TestGetConfigForScaleSetByIPFamily --- PASS: TestGetConfigForScaleSetByIPFamily (0.00s) === RUN TestEnsureHostInPool W1104 08:43:22.472576 25195 azure_vmss.go:173] failed to get VM with vmssVirtualMachinesEntry on Node "vmss-vm-000000" W1104 08:43:22.472672 25195 azure_vmss.go:173] failed to get VM with vmssVirtualMachinesEntry on Node "vmss-vm-000000" W1104 08:43:22.472787 25195 azure_vmss.go:173] failed to get VM with vmssVirtualMachinesEntry on Node "vmss-vm-000000" W1104 08:43:22.472831 25195 azure_vmss.go:216] Unable to find node vmss-vm-000000: instance not found I1104 08:43:22.472865 25195 azure_vmss.go:992] EnsureHostInPool: skipping node vmss-vm-000000 because it is not found --- PASS: TestEnsureHostInPool (0.00s) === RUN TestGetVmssAndResourceGroupNameByVMProviderID --- PASS: TestGetVmssAndResourceGroupNameByVMProviderID (0.00s) === RUN TestEnsureVMSSInPool ... skipping 6 lines ... W1104 08:43:22.488379 25195 azure_vmss.go:216] Unable to find node vmss-vm-000001: instance not found I1104 08:43:22.488452 25195 azure_vmss.go:1413] ensureBackendPoolDeletedFromNode: skipping node vmss-vm-000001 because it is not found --- PASS: TestEnsureBackendPoolDeletedFromNode (0.00s) === RUN TestGetScaleSetAndResourceGroupNameByIPConfigurationID --- PASS: TestGetScaleSetAndResourceGroupNameByIPConfigurationID (0.00s) === RUN TestEnsureBackendPoolDeletedFromVMSS E1104 08:43:22.494750 25195 azure_backoff.go:585] CreateOrUpdateVMSS: error CreateOrUpdate vmss(vmss): &{false 0 0001-01-01 00:00:00 +0000 UTC error} E1104 08:43:22.494865 25195 azure_vmss.go:1838] EnsureBackendPoolDeletedFromVMSets CreateOrUpdateVMSS(vmss) with new backendPoolID /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-0, err: &{false 0 0001-01-01 00:00:00 +0000 UTC error} --- PASS: TestEnsureBackendPoolDeletedFromVMSS (0.00s) === RUN TestEnsureBackendPoolDeleted E1104 08:43:22.498461 25195 azure_vmss.go:1487] Unable to find node by IPConfigurationID /invalid/id: invalid ip config ID /invalid/id E1104 08:43:22.503069 25195 azure_vmss.go:1716] "Failed to update VMs for VMSS" operation="EnsureBackendPoolDeleted UpdateVMSSVMs" vmssName="vmss" resourceGroup="rg" backendPoolID="/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-0" E1104 08:43:22.506609 25195 azure_vmss.go:1498] Unable to find node by ipConfigurationID /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmss/virtualMachines/6/networkInterfaces/nic: instance not found I1104 08:43:22.506683 25195 azure_vmss.go:1652] EnsureBackendPoolDeleted(/): skipping ip config /subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmss/virtualMachines/6/networkInterfaces/nic because the corresponding vmss vm is not found --- PASS: TestEnsureBackendPoolDeleted (0.01s) === RUN TestEnsureBackendPoolDeletedConcurrently E1104 08:43:22.508847 25195 azure_vmss.go:1774] ensureBackendPoolDeletedFromVMSS: failed to get VMSS vmss-2: instance not found --- PASS: TestEnsureBackendPoolDeletedConcurrently (0.00s) === RUN TestGetNodeCIDRMasksByProviderID === RUN TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMasksByProviderID_should_report_an_error_if_the_providerID_is_not_valid === RUN TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes === RUN TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes_even_if_some_of_the_tags_are_not_specified === RUN TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_not_fail_even_if_some_of_the_tag_is_invalid E1104 08:43:22.514092 25195 azure_vmss.go:1753] GetNodeCIDRMasksByProviderID: error when paring the value of the ipv4 mask size abc: strconv.Atoi: parsing "abc": invalid syntax --- PASS: TestGetNodeCIDRMasksByProviderID (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMasksByProviderID_should_report_an_error_if_the_providerID_is_not_valid (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_return_the_correct_mask_sizes_even_if_some_of_the_tags_are_not_specified (0.00s) --- PASS: TestGetNodeCIDRMasksByProviderID/GetNodeCIDRMaksByProviderID_should_not_fail_even_if_some_of_the_tag_is_invalid (0.00s) === RUN TestGetAgentPoolVMSetNamesMixedInstances --- PASS: TestGetAgentPoolVMSetNamesMixedInstances (0.00s) === RUN TestGetNodeVMSetNameVMSS W1104 08:43:22.516313 25195 azure_vmss.go:1899] ss.GetNodeVMSetName: the provider ID invalid of node does not match the format of a VMSS instance, assuming it is managed by an availability set --- PASS: TestGetNodeVMSetNameVMSS (0.00s) === RUN TestScaleSet_VMSSBatchSize === RUN TestScaleSet_VMSSBatchSize/failed_to_get_vmss E1104 08:43:22.517020 25195 azure_vmss_cache.go:74] VirtualMachineScaleSetsClient.List failed: &{false 0 0001-01-01 00:00:00 +0000 UTC list vmss error} === RUN TestScaleSet_VMSSBatchSize/vmss_contains_batch_operation_tag === RUN TestScaleSet_VMSSBatchSize/vmss_doesn't_contain_batch_operation_tag --- PASS: TestScaleSet_VMSSBatchSize (0.00s) --- PASS: TestScaleSet_VMSSBatchSize/failed_to_get_vmss (0.00s) --- PASS: TestScaleSet_VMSSBatchSize/vmss_contains_batch_operation_tag (0.00s) --- PASS: TestScaleSet_VMSSBatchSize/vmss_doesn't_contain_batch_operation_tag (0.00s) ... skipping 24 lines ... === RUN TestGetZoneByProviderID --- PASS: TestGetZoneByProviderID (0.00s) === RUN TestAvailabilitySetGetZoneByNodeName --- PASS: TestAvailabilitySetGetZoneByNodeName (0.00s) === RUN TestSyncRegionZonesMap === RUN TestSyncRegionZonesMap/syncRegionZonesMap_should_report_an_error_if_failed_to_get_zones W1104 08:43:22.541222 25195 azure_zones.go:51] syncRegionZonesMap: error when get zones: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error, will retry after 30m0s === RUN TestSyncRegionZonesMap/syncRegionZonesMap_should_not_report_an_error_if_the_given_zones_map_is_empty W1104 08:43:22.541384 25195 azure_zones.go:55] syncRegionZonesMap: empty zone list --- PASS: TestSyncRegionZonesMap (0.00s) --- PASS: TestSyncRegionZonesMap/syncRegionZonesMap_should_report_an_error_if_failed_to_get_zones (0.00s) --- PASS: TestSyncRegionZonesMap/syncRegionZonesMap_should_not_report_an_error_if_the_given_zones_map_is_empty (0.00s) === RUN TestGetRegionZonesBackoff === RUN TestGetRegionZonesBackoff/syncRegionZonesMap_should_report_an_error_if_failed_to_get_zones E1104 08:43:22.541664 25195 azure_zones.go:100] getRegionZonesMapWrapper: failed to fetch zones information: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error === RUN TestGetRegionZonesBackoff/syncRegionZonesMap_should_not_report_an_error_if_the_given_zones_map_is_empty --- PASS: TestGetRegionZonesBackoff (0.00s) --- PASS: TestGetRegionZonesBackoff/syncRegionZonesMap_should_report_an_error_if_failed_to_get_zones (0.00s) --- PASS: TestGetRegionZonesBackoff/syncRegionZonesMap_should_not_report_an_error_if_the_given_zones_map_is_empty (0.00s) FAIL FAIL sigs.k8s.io/cloud-provider-azure/pkg/provider 6.126s FAIL make: *** [Makefile:313: test-unit] Error 1 + EXIT_VALUE=2 + set +o xtrace