This job view page is being replaced by Spyglass soon. Check out the new job view.
PRzmyzheng: Draft PR to test VMSSFlex-CSI integration
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2022-09-14 06:13
Elapsed2h15m
Revision66bd2c4682a3e986d488777037b23bffd8b2219a
Refs 1512

No Test Failures!


Error lines from build-log.txt

... skipping 491 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 254 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 861 lines ...
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - "--leader-election-namespace=kube-system"
            - '-handle-volume-inuse-error=false'
            - '-feature-gates=RecoverVolumeExpansionFailure=true'
            - "-timeout=240s"
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
... skipping 209 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 06:19:58.424: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-9pdgp" in namespace "azuredisk-8081" to be "Succeeded or Failed"
Sep 14 06:19:58.483: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 59.182436ms
Sep 14 06:20:00.545: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.120728066s
Sep 14 06:20:02.605: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.180316435s
Sep 14 06:20:04.666: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.241924507s
Sep 14 06:20:06.738: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.313412024s
Sep 14 06:20:08.798: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.374170701s
... skipping 426 lines ...
Sep 14 06:34:48.408: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m49.984018664s
Sep 14 06:34:50.467: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.042419914s
Sep 14 06:34:52.527: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.10255846s
Sep 14 06:34:54.587: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.162849537s
Sep 14 06:34:56.648: INFO: Pod "azuredisk-volume-tester-9pdgp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.223421654s
Sep 14 06:34:58.648: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-9pdgp"
Sep 14 06:34:58.760: INFO: Error getting logs for pod azuredisk-volume-tester-9pdgp: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-9pdgp)
STEP: Deleting pod azuredisk-volume-tester-9pdgp in namespace azuredisk-8081
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 06:34:58.933: INFO: deleting PVC "azuredisk-8081"/"pvc-tbrck"
Sep 14 06:34:58.933: INFO: Deleting PersistentVolumeClaim "pvc-tbrck"
STEP: waiting for claim's PV "pvc-d3461844-06e9-4029-827d-dedf6fe05dbc" to be deleted
... skipping 10 lines ...
STEP: Found 7 events.
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:19:58 +0000 UTC - event for pvc-tbrck: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:19:58 +0000 UTC - event for pvc-tbrck: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:19:58 +0000 UTC - event for pvc-tbrck: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-8081/pvc-tbrck"
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:20:01 +0000 UTC - event for pvc-tbrck: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-d3461844-06e9-4029-827d-dedf6fe05dbc
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:20:02 +0000 UTC - event for azuredisk-volume-tester-9pdgp: {default-scheduler } Scheduled: Successfully assigned azuredisk-8081/azuredisk-volume-tester-9pdgp to k8s-agentpool-26343132-vmss000000
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:20:03 +0000 UTC - event for azuredisk-volume-tester-9pdgp: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-d3461844-06e9-4029-827d-dedf6fe05dbc" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 06:35:19.479: INFO: At 2022-09-14 06:22:05 +0000 UTC - event for azuredisk-volume-tester-9pdgp: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 06:35:19.537: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 06:35:19.537: INFO: 
Sep 14 06:35:19.652: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 06:35:19.711: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 4005 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 06:35:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 06:35:13 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 06:35:13 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 06:35:13 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 06:35:13 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a volume on demand with mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:101

    Unexpected error:
        <*errors.errorString | 0xc000700590>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-9pdgp\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-9pdgp" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should create a pod with volume mount subpath [disk.csi.azure.com] [Windows]
... skipping 18 lines ...
Sep 14 06:35:24.582: INFO: PersistentVolumeClaim pvc-fjw85 found but phase is Pending instead of Bound.
Sep 14 06:35:26.641: INFO: PersistentVolumeClaim pvc-fjw85 found and phase=Bound (4.178141565s)
STEP: checking the PVC
STEP: validating provisioned PV
STEP: checking the PV
STEP: deploying the pod
STEP: checking that the pods command exits with no error
Sep 14 06:35:26.818: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fnsdh" in namespace "azuredisk-2540" to be "Succeeded or Failed"
Sep 14 06:35:26.878: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 59.91673ms
Sep 14 06:35:28.938: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11991438s
Sep 14 06:35:30.998: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 4.180040346s
Sep 14 06:35:33.059: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 6.241065316s
Sep 14 06:35:35.119: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 8.301481055s
Sep 14 06:35:37.180: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 10.362058646s
... skipping 426 lines ...
Sep 14 06:50:16.859: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.041376665s
Sep 14 06:50:18.919: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.101876209s
Sep 14 06:50:20.979: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.161236466s
Sep 14 06:50:23.040: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.222459673s
Sep 14 06:50:25.100: INFO: Pod "azuredisk-volume-tester-fnsdh": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.282776422s
Sep 14 06:50:27.102: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-fnsdh"
Sep 14 06:50:27.201: INFO: Error getting logs for pod azuredisk-volume-tester-fnsdh: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-fnsdh)
STEP: Deleting pod azuredisk-volume-tester-fnsdh in namespace azuredisk-2540
Sep 14 06:50:27.262: INFO: deleting PVC "azuredisk-2540"/"pvc-fjw85"
Sep 14 06:50:27.262: INFO: Deleting PersistentVolumeClaim "pvc-fjw85"
STEP: waiting for claim's PV "pvc-75a98026-d146-455b-ab3c-d6c807596602" to be deleted
Sep 14 06:50:27.321: INFO: Waiting up to 10m0s for PersistentVolume pvc-75a98026-d146-455b-ab3c-d6c807596602 to get deleted
Sep 14 06:50:27.380: INFO: PersistentVolume pvc-75a98026-d146-455b-ab3c-d6c807596602 found and phase=Bound (58.169159ms)
... skipping 6 lines ...
STEP: Collecting events from namespace "azuredisk-2540".
STEP: Found 6 events.
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:35:22 +0000 UTC - event for pvc-fjw85: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:35:22 +0000 UTC - event for pvc-fjw85: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2540/pvc-fjw85"
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:35:25 +0000 UTC - event for pvc-fjw85: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-75a98026-d146-455b-ab3c-d6c807596602
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:35:26 +0000 UTC - event for azuredisk-volume-tester-fnsdh: {default-scheduler } Scheduled: Successfully assigned azuredisk-2540/azuredisk-volume-tester-fnsdh to k8s-agentpool-26343132-vmss000001
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:35:26 +0000 UTC - event for azuredisk-volume-tester-fnsdh: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-75a98026-d146-455b-ab3c-d6c807596602" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000001" (instance not found)
Sep 14 06:50:42.747: INFO: At 2022-09-14 06:37:29 +0000 UTC - event for azuredisk-volume-tester-fnsdh: {kubelet k8s-agentpool-26343132-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 06:50:42.804: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 06:50:42.804: INFO: 
Sep 14 06:50:42.918: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 06:50:42.982: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 6690 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 06:35:13 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 06:50:14 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 06:50:14 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 06:50:14 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 06:50:14 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a pod with volume mount subpath [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:150

    Unexpected error:
        <*errors.errorString | 0xc000da45a0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-fnsdh\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-fnsdh" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  Should create and attach a volume with basic perfProfile [enableBursting][disk.csi.azure.com] [Windows]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 06:50:45.877: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-m9tjv" in namespace "azuredisk-4728" to be "Succeeded or Failed"
Sep 14 06:50:45.934: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 57.19887ms
Sep 14 06:50:47.994: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11690521s
Sep 14 06:50:50.052: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 4.175263588s
Sep 14 06:50:52.112: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 6.234701316s
Sep 14 06:50:54.171: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 8.293967092s
Sep 14 06:50:56.232: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 10.355290706s
... skipping 426 lines ...
Sep 14 07:05:36.169: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.292079292s
Sep 14 07:05:38.229: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.351921426s
Sep 14 07:05:40.289: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.411623496s
Sep 14 07:05:42.348: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.471254652s
Sep 14 07:05:44.408: INFO: Pod "azuredisk-volume-tester-m9tjv": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.531224005s
Sep 14 07:05:46.410: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-m9tjv"
Sep 14 07:05:46.509: INFO: Error getting logs for pod azuredisk-volume-tester-m9tjv: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-m9tjv)
STEP: Deleting pod azuredisk-volume-tester-m9tjv in namespace azuredisk-4728
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 07:05:46.689: INFO: deleting PVC "azuredisk-4728"/"pvc-wmbbk"
Sep 14 07:05:46.689: INFO: Deleting PersistentVolumeClaim "pvc-wmbbk"
STEP: waiting for claim's PV "pvc-85812e6d-c76e-423e-acd7-11c0d08dfe45" to be deleted
... skipping 12 lines ...
STEP: Found 7 events.
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:45 +0000 UTC - event for pvc-wmbbk: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:45 +0000 UTC - event for pvc-wmbbk: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:45 +0000 UTC - event for pvc-wmbbk: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-4728/pvc-wmbbk"
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:48 +0000 UTC - event for azuredisk-volume-tester-m9tjv: {default-scheduler } Scheduled: Successfully assigned azuredisk-4728/azuredisk-volume-tester-m9tjv to k8s-agentpool-26343132-vmss000000
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:48 +0000 UTC - event for pvc-wmbbk: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-85812e6d-c76e-423e-acd7-11c0d08dfe45
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:50:49 +0000 UTC - event for azuredisk-volume-tester-m9tjv: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-85812e6d-c76e-423e-acd7-11c0d08dfe45" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 07:06:17.351: INFO: At 2022-09-14 06:52:51 +0000 UTC - event for azuredisk-volume-tester-m9tjv: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 07:06:17.409: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 07:06:17.409: INFO: 
Sep 14 07:06:17.522: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 07:06:17.580: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 9539 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 07:06:05 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 07:06:05 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 07:06:05 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 07:06:05 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 07:06:05 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    Should create and attach a volume with basic perfProfile [enableBursting][disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:185

    Unexpected error:
        <*errors.errorString | 0xc0004270e0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-m9tjv\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-m9tjv" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should receive FailedMount event with invalid mount options [kubernetes.io/azure-disk] [disk.csi.azure.com]
... skipping 13 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod has 'FailedMount' event
Sep 14 07:11:20.597: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-nl88l"
Sep 14 07:11:20.696: INFO: Error getting logs for pod azuredisk-volume-tester-nl88l: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-nl88l)
STEP: Deleting pod azuredisk-volume-tester-nl88l in namespace azuredisk-5466
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 07:11:20.874: INFO: deleting PVC "azuredisk-5466"/"pvc-b8qvq"
Sep 14 07:11:20.875: INFO: Deleting PersistentVolumeClaim "pvc-b8qvq"
STEP: waiting for claim's PV "pvc-6b2836a8-24ba-47e7-80fa-523c914c9ab1" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:20 +0000 UTC - event for pvc-b8qvq: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:20 +0000 UTC - event for pvc-b8qvq: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:20 +0000 UTC - event for pvc-b8qvq: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5466/pvc-b8qvq"
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:22 +0000 UTC - event for pvc-b8qvq: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-6b2836a8-24ba-47e7-80fa-523c914c9ab1
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:23 +0000 UTC - event for azuredisk-volume-tester-nl88l: {default-scheduler } Scheduled: Successfully assigned azuredisk-5466/azuredisk-volume-tester-nl88l to k8s-agentpool-26343132-vmss000000
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:06:23 +0000 UTC - event for azuredisk-volume-tester-nl88l: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-6b2836a8-24ba-47e7-80fa-523c914c9ab1" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 07:11:46.505: INFO: At 2022-09-14 07:08:26 +0000 UTC - event for azuredisk-volume-tester-nl88l: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 07:11:46.564: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 07:11:46.564: INFO: 
Sep 14 07:11:46.679: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 07:11:46.738: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 10575 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 07:11:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 07:11:35 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 07:11:35 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 07:11:35 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 07:11:35 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should receive FailedMount event with invalid mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:221

    Unexpected error:
        <*errors.errorString | 0xc0001f24a0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

... skipping 15 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 07:11:49.652: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-pxslb" in namespace "azuredisk-2790" to be "Succeeded or Failed"
Sep 14 07:11:49.712: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 59.942851ms
Sep 14 07:11:51.773: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.120534944s
Sep 14 07:11:53.833: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 4.180574459s
Sep 14 07:11:55.893: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 6.240867422s
Sep 14 07:11:57.954: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 8.301510132s
Sep 14 07:12:00.016: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 10.36422488s
... skipping 426 lines ...
Sep 14 07:26:39.438: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m49.785893536s
Sep 14 07:26:41.496: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.84379821s
Sep 14 07:26:43.554: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.902134502s
Sep 14 07:26:45.612: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.960126631s
Sep 14 07:26:47.671: INFO: Pod "azuredisk-volume-tester-pxslb": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.019213393s
Sep 14 07:26:49.673: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-pxslb"
Sep 14 07:26:49.771: INFO: Error getting logs for pod azuredisk-volume-tester-pxslb: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-pxslb)
STEP: Deleting pod azuredisk-volume-tester-pxslb in namespace azuredisk-2790
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 07:26:49.945: INFO: deleting PVC "azuredisk-2790"/"pvc-lmqnl"
Sep 14 07:26:49.945: INFO: Deleting PersistentVolumeClaim "pvc-lmqnl"
STEP: waiting for claim's PV "pvc-a9a1076a-23e0-4147-b07d-0f35d63365f3" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 07:27:15.537: INFO: At 2022-09-14 07:11:49 +0000 UTC - event for pvc-lmqnl: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 07:27:15.537: INFO: At 2022-09-14 07:11:49 +0000 UTC - event for pvc-lmqnl: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 07:27:15.537: INFO: At 2022-09-14 07:11:49 +0000 UTC - event for pvc-lmqnl: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2790/pvc-lmqnl"
Sep 14 07:27:15.537: INFO: At 2022-09-14 07:11:51 +0000 UTC - event for pvc-lmqnl: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-a9a1076a-23e0-4147-b07d-0f35d63365f3
Sep 14 07:27:15.537: INFO: At 2022-09-14 07:11:52 +0000 UTC - event for azuredisk-volume-tester-pxslb: {default-scheduler } Scheduled: Successfully assigned azuredisk-2790/azuredisk-volume-tester-pxslb to k8s-agentpool-26343132-vmss000000
Sep 14 07:27:15.538: INFO: At 2022-09-14 07:11:52 +0000 UTC - event for azuredisk-volume-tester-pxslb: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-a9a1076a-23e0-4147-b07d-0f35d63365f3" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 07:27:15.538: INFO: At 2022-09-14 07:13:55 +0000 UTC - event for azuredisk-volume-tester-pxslb: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 07:27:15.594: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 07:27:15.594: INFO: 
Sep 14 07:27:15.705: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 07:27:15.762: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 13358 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 07:27:06 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 07:27:06 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 07:27:06 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 07:27:06 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 07:27:06 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a raw block volume on demand [kubernetes.io/azure-disk] [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:258

    Unexpected error:
        <*errors.errorString | 0xc000becf80>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-pxslb\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-pxslb" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with an error
Sep 14 07:27:18.431: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-pvmnw" in namespace "azuredisk-5356" to be "Error status code"
Sep 14 07:27:18.489: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 57.12197ms
Sep 14 07:27:20.547: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.116018863s
Sep 14 07:27:22.605: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.173525188s
Sep 14 07:27:24.663: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.231503843s
Sep 14 07:27:26.721: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.290028994s
Sep 14 07:27:28.781: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.349162401s
... skipping 426 lines ...
Sep 14 07:42:08.576: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.144142316s
Sep 14 07:42:10.637: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.205268391s
Sep 14 07:42:12.698: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.266729386s
Sep 14 07:42:14.759: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.328032794s
Sep 14 07:42:16.820: INFO: Pod "azuredisk-volume-tester-pvmnw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.388567284s
Sep 14 07:42:18.822: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-pvmnw"
Sep 14 07:42:18.922: INFO: Error getting logs for pod azuredisk-volume-tester-pvmnw: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-pvmnw)
STEP: Deleting pod azuredisk-volume-tester-pvmnw in namespace azuredisk-5356
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 07:42:19.106: INFO: deleting PVC "azuredisk-5356"/"pvc-gzl55"
Sep 14 07:42:19.106: INFO: Deleting PersistentVolumeClaim "pvc-gzl55"
STEP: waiting for claim's PV "pvc-8f4976c9-fec1-404f-b869-534e019d2a66" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:18 +0000 UTC - event for pvc-gzl55: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:18 +0000 UTC - event for pvc-gzl55: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:18 +0000 UTC - event for pvc-gzl55: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5356/pvc-gzl55"
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:20 +0000 UTC - event for pvc-gzl55: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-8f4976c9-fec1-404f-b869-534e019d2a66
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:21 +0000 UTC - event for azuredisk-volume-tester-pvmnw: {default-scheduler } Scheduled: Successfully assigned azuredisk-5356/azuredisk-volume-tester-pvmnw to k8s-agentpool-26343132-vmss000000
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:27:21 +0000 UTC - event for azuredisk-volume-tester-pvmnw: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-8f4976c9-fec1-404f-b869-534e019d2a66" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 07:42:44.715: INFO: At 2022-09-14 07:29:24 +0000 UTC - event for azuredisk-volume-tester-pvmnw: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 07:42:44.774: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 07:42:44.774: INFO: 
Sep 14 07:42:44.889: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 07:42:44.948: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 16139 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 07:42:37 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 07:42:37 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 07:42:37 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 07:42:37 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 07:42:37 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:290

    Unexpected error:
        <*errors.errorString | 0xc0004dd2a0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-pvmnw\" to be \"Error status code\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-pvmnw" to be "Error status code"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:844
------------------------------
Dynamic Provisioning [multi-az] 
  should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 13 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod is running
Sep 14 07:57:47.998: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-whjdw"
Sep 14 07:57:48.098: INFO: Error getting logs for pod azuredisk-volume-tester-whjdw: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-whjdw)
STEP: Deleting pod azuredisk-volume-tester-whjdw in namespace azuredisk-5194
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 07:57:48.275: INFO: deleting PVC "azuredisk-5194"/"pvc-44zhs"
Sep 14 07:57:48.275: INFO: Deleting PersistentVolumeClaim "pvc-44zhs"
STEP: waiting for claim's PV "pvc-72b6e639-351d-4509-add8-9f441a9dfe1c" to be deleted
... skipping 8 lines ...
STEP: Collecting events from namespace "azuredisk-5194".
STEP: Found 7 events.
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:47 +0000 UTC - event for pvc-44zhs: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:47 +0000 UTC - event for pvc-44zhs: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:47 +0000 UTC - event for pvc-44zhs: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5194/pvc-44zhs"
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:50 +0000 UTC - event for azuredisk-volume-tester-whjdw: {default-scheduler } Scheduled: Successfully assigned azuredisk-5194/azuredisk-volume-tester-whjdw to k8s-agentpool-26343132-vmss000000
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:50 +0000 UTC - event for azuredisk-volume-tester-whjdw: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-72b6e639-351d-4509-add8-9f441a9dfe1c" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:42:50 +0000 UTC - event for pvc-44zhs: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-72b6e639-351d-4509-add8-9f441a9dfe1c
Sep 14 07:58:03.754: INFO: At 2022-09-14 07:44:53 +0000 UTC - event for azuredisk-volume-tester-whjdw: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 07:58:03.813: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 07:58:03.813: INFO: 
Sep 14 07:58:03.928: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
... skipping 94 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:324

    Unexpected error:
        <*errors.errorString | 0xc0001f24a0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

... skipping 193 lines ...
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:06 +0000 UTC - event for azuredisk-volume-tester-qbpjz-66d4f655c8: {replicaset-controller } SuccessfulCreate: Created pod: azuredisk-volume-tester-qbpjz-66d4f655c8-8kwlg
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:06 +0000 UTC - event for pvc-68nm5: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:06 +0000 UTC - event for pvc-68nm5: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:06 +0000 UTC - event for pvc-68nm5: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-1353/pvc-68nm5"
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:09 +0000 UTC - event for azuredisk-volume-tester-qbpjz-66d4f655c8-8kwlg: {default-scheduler } Scheduled: Successfully assigned azuredisk-1353/azuredisk-volume-tester-qbpjz-66d4f655c8-8kwlg to k8s-agentpool-26343132-vmss000000
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:09 +0000 UTC - event for pvc-68nm5: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-e7c052ab-fbb5-4d2e-92e3-0fda6bd9165c
Sep 14 08:03:37.906: INFO: At 2022-09-14 07:58:10 +0000 UTC - event for azuredisk-volume-tester-qbpjz-66d4f655c8-8kwlg: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-e7c052ab-fbb5-4d2e-92e3-0fda6bd9165c" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 08:03:37.906: INFO: At 2022-09-14 08:00:12 +0000 UTC - event for azuredisk-volume-tester-qbpjz-66d4f655c8-8kwlg: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 default-token-mcjjq]: timed out waiting for the condition
Sep 14 08:03:37.965: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 08:03:37.965: INFO: 
Sep 14 08:03:38.079: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 08:03:38.138: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 19936 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 08:03:28 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 08:03:28 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 08:03:28 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 08:03:28 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 08:03:28 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a deployment object, write and read to it, delete the pod and write and read to it again [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:388

    Unexpected error:
        <*errors.errorString | 0xc000623480>: {
            s: "error waiting for deployment \"azuredisk-volume-tester-qbpjz\" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:\"Available\", Status:\"False\", LastUpdateTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), Reason:\"MinimumReplicasUnavailable\", Message:\"Deployment does not have minimum availability.\"}, v1.DeploymentCondition{Type:\"Progressing\", Status:\"True\", LastUpdateTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), Reason:\"ReplicaSetUpdated\", Message:\"ReplicaSet \\\"azuredisk-volume-tester-qbpjz-66d4f655c8\\\" is progressing.\"}}, CollisionCount:(*int32)(nil)}",
        }
        error waiting for deployment "azuredisk-volume-tester-qbpjz" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 7, 58, 6, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"azuredisk-volume-tester-qbpjz-66d4f655c8\" is progressing."}}, CollisionCount:(*int32)(nil)}
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:503
------------------------------
Dynamic Provisioning [multi-az] 
  should delete PV with reclaimPolicy "Delete" [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 103 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 08:03:59.919: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-6vrlk" in namespace "azuredisk-4376" to be "Succeeded or Failed"
Sep 14 08:03:59.977: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 57.996989ms
Sep 14 08:04:02.037: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11749391s
Sep 14 08:04:04.097: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.177618717s
Sep 14 08:04:06.156: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.237250517s
Sep 14 08:04:08.216: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.296867783s
Sep 14 08:04:10.277: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 10.35756056s
... skipping 275 lines ...
Sep 14 08:13:38.989: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m39.06975729s
Sep 14 08:13:41.050: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m41.130438843s
Sep 14 08:13:43.110: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m43.191364401s
Sep 14 08:13:45.171: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m45.25165187s
Sep 14 08:13:47.231: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m47.312014226s
Sep 14 08:13:49.291: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 9m49.371517315s
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2022-09-14T08:13:50Z"}
++ early_exit_handler
++ '[' -n 170 ']'
++ kill -TERM 170
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 160 lines ...
Sep 14 08:18:50.162: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 14m50.242668483s
Sep 14 08:18:52.221: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 14m52.302125425s
Sep 14 08:18:54.281: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 14m54.361507519s
Sep 14 08:18:56.340: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 14m56.421299615s
Sep 14 08:18:58.401: INFO: Pod "azuredisk-volume-tester-6vrlk": Phase="Pending", Reason="", readiness=false. Elapsed: 14m58.481613554s
Sep 14 08:19:00.401: INFO: deleting Pod "azuredisk-4376"/"azuredisk-volume-tester-6vrlk"
Sep 14 08:19:00.493: INFO: Error getting logs for pod azuredisk-volume-tester-6vrlk: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-6vrlk)
STEP: Deleting pod azuredisk-volume-tester-6vrlk in namespace azuredisk-4376
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 08:19:00.671: INFO: deleting PVC "azuredisk-4376"/"pvc-md8n2"
Sep 14 08:19:00.671: INFO: Deleting PersistentVolumeClaim "pvc-md8n2"
STEP: waiting for claim's PV "pvc-dd8e52ce-913a-4c60-87e6-48318b85d06b" to be deleted
... skipping 9 lines ...
STEP: Found 7 events.
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:03:59 +0000 UTC - event for pvc-md8n2: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:03:59 +0000 UTC - event for pvc-md8n2: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:03:59 +0000 UTC - event for pvc-md8n2: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } Provisioning: External provisioner is provisioning volume for claim "azuredisk-4376/pvc-md8n2"
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:04:02 +0000 UTC - event for azuredisk-volume-tester-6vrlk: {default-scheduler } Scheduled: Successfully assigned azuredisk-4376/azuredisk-volume-tester-6vrlk to k8s-agentpool-26343132-vmss000000
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:04:02 +0000 UTC - event for pvc-md8n2: {disk.csi.azure.com_k8s-agentpool-26343132-vmss000000_2abf1ddc-d1d1-4110-83dc-6cf8ffe59cb5 } ProvisioningSucceeded: Successfully provisioned volume pvc-dd8e52ce-913a-4c60-87e6-48318b85d06b
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:04:03 +0000 UTC - event for azuredisk-volume-tester-6vrlk: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-dd8e52ce-913a-4c60-87e6-48318b85d06b" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-26343132-vmss000000" (instance not found)
Sep 14 08:19:16.165: INFO: At 2022-09-14 08:06:05 +0000 UTC - event for azuredisk-volume-tester-6vrlk: {kubelet k8s-agentpool-26343132-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 08:19:16.223: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 08:19:16.223: INFO: 
Sep 14 08:19:16.337: INFO: 
Logging node info for node k8s-agentpool-26343132-vmss000000
Sep 14 08:19:16.397: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-26343132-vmss000000    e8dcfbb0-c127-4e1d-9c79-a0c4cf559d9e 22810 0 2022-09-14 06:16:51 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-knozofsk kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-26343132-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-26343132-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 06:16:52 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 06:17:01 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 06:17:42 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 08:19:09 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-knozofsk/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-26343132-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344715264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558283264 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 08:19:09 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 08:19:09 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 08:19:09 +0000 UTC,LastTransitionTime:2022-09-14 06:16:49 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 08:19:09 +0000 UTC,LastTransitionTime:2022-09-14 06:17:01 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-26343132-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:8b4651c6922a4edb8026433f6f91272c,SystemUUID:02a0b629-a238-db44-b51a-724948fc9eab,BootID:300a1777-b4fe-4f45-bf6f-258ae1ad6ab8,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should clone a volume from an existing volume and read from it [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:464

    Unexpected error:
        <*errors.errorString | 0xc0004dc390>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-6vrlk\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-6vrlk" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 08:19:19.200: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-s24bh" in namespace "azuredisk-7996" to be "Succeeded or Failed"
Sep 14 08:19:19.259: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 58.292849ms
Sep 14 08:19:21.319: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 2.118819881s
Sep 14 08:19:23.401: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 4.200759647s
Sep 14 08:19:25.462: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 6.261293753s
Sep 14 08:19:27.521: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 8.320813527s
Sep 14 08:19:29.581: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 10.380311485s
... skipping 266 lines ...
Sep 14 08:28:39.574: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m20.373563895s
Sep 14 08:28:41.632: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m22.431928979s
Sep 14 08:28:43.691: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m24.490599458s
Sep 14 08:28:45.750: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m26.54985851s
Sep 14 08:28:47.810: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m28.609175685s
Sep 14 08:28:49.868: INFO: Pod "azuredisk-volume-tester-s24bh": Phase="Pending", Reason="", readiness=false. Elapsed: 9m30.667754352s
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2022-09-14T08:28:50Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:257","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2022-09-14T08:28:50Z"}