This job view page is being replaced by Spyglass soon. Check out the new job view.
PRzmyzheng: Draft PR to test VMSSFlex-CSI integration
ResultFAILURE
Tests 0 failed / 0 succeeded
Started2022-09-14 01:22
Elapsed2h15m
Revision66bd2c4682a3e986d488777037b23bffd8b2219a
Refs 1512

No Test Failures!


Error lines from build-log.txt

... skipping 492 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 60 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 254 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 108 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 861 lines ...
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - "--leader-election-namespace=kube-system"
            - '-handle-volume-inuse-error=false'
            - '-feature-gates=RecoverVolumeExpansionFailure=true'
            - "-timeout=240s"
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
... skipping 209 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 01:29:59.049: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fw8v9" in namespace "azuredisk-8081" to be "Succeeded or Failed"
Sep 14 01:29:59.105: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 55.925603ms
Sep 14 01:30:01.158: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.108581419s
Sep 14 01:30:03.211: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.161798613s
Sep 14 01:30:05.264: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.214969467s
Sep 14 01:30:07.321: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.27191169s
Sep 14 01:30:09.377: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.327862415s
... skipping 428 lines ...
Sep 14 01:44:50.476: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.426416181s
Sep 14 01:44:52.527: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.478134302s
Sep 14 01:44:54.582: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.532239627s
Sep 14 01:44:56.633: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.583997062s
Sep 14 01:44:58.685: INFO: Pod "azuredisk-volume-tester-fw8v9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.636030921s
Sep 14 01:45:00.687: INFO: deleting Pod "azuredisk-8081"/"azuredisk-volume-tester-fw8v9"
Sep 14 01:45:00.786: INFO: Error getting logs for pod azuredisk-volume-tester-fw8v9: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-fw8v9)
STEP: Deleting pod azuredisk-volume-tester-fw8v9 in namespace azuredisk-8081
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 01:45:00.943: INFO: deleting PVC "azuredisk-8081"/"pvc-tfkxp"
Sep 14 01:45:00.943: INFO: Deleting PersistentVolumeClaim "pvc-tfkxp"
STEP: waiting for claim's PV "pvc-fcb00087-b259-4e27-8a32-5e903942d8db" to be deleted
... skipping 9 lines ...
STEP: Found 7 events.
Sep 14 01:45:16.366: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-fw8v9: { } Scheduled: Successfully assigned azuredisk-8081/azuredisk-volume-tester-fw8v9 to k8s-agentpool-38427154-vmss000001
Sep 14 01:45:16.366: INFO: At 2022-09-14 01:29:58 +0000 UTC - event for pvc-tfkxp: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 01:45:16.366: INFO: At 2022-09-14 01:29:59 +0000 UTC - event for pvc-tfkxp: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 01:45:16.367: INFO: At 2022-09-14 01:29:59 +0000 UTC - event for pvc-tfkxp: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-8081/pvc-tfkxp"
Sep 14 01:45:16.367: INFO: At 2022-09-14 01:30:01 +0000 UTC - event for pvc-tfkxp: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-fcb00087-b259-4e27-8a32-5e903942d8db
Sep 14 01:45:16.367: INFO: At 2022-09-14 01:30:02 +0000 UTC - event for azuredisk-volume-tester-fw8v9: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-fcb00087-b259-4e27-8a32-5e903942d8db" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 01:45:16.367: INFO: At 2022-09-14 01:32:05 +0000 UTC - event for azuredisk-volume-tester-fw8v9: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 01:45:16.418: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 01:45:16.418: INFO: 
Sep 14 01:45:16.518: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 01:45:16.569: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 4183 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 01:45:02 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 01:45:02 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 01:45:02 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 01:45:02 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a volume on demand with mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:101

    Unexpected error:
        <*errors.errorString | 0xc000ad8ef0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-fw8v9\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-fw8v9" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should create a pod with volume mount subpath [disk.csi.azure.com] [Windows]
... skipping 18 lines ...
Sep 14 01:45:21.249: INFO: PersistentVolumeClaim pvc-qwc8w found but phase is Pending instead of Bound.
Sep 14 01:45:23.300: INFO: PersistentVolumeClaim pvc-qwc8w found and phase=Bound (4.156671865s)
STEP: checking the PVC
STEP: validating provisioned PV
STEP: checking the PV
STEP: deploying the pod
STEP: checking that the pods command exits with no error
Sep 14 01:45:23.456: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-crcwp" in namespace "azuredisk-2540" to be "Succeeded or Failed"
Sep 14 01:45:23.506: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 50.319598ms
Sep 14 01:45:25.558: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.101916075s
Sep 14 01:45:27.611: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.154881342s
Sep 14 01:45:29.663: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.207581345s
Sep 14 01:45:31.718: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.261871082s
Sep 14 01:45:33.771: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.315696149s
... skipping 428 lines ...
Sep 14 02:00:14.524: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.068022495s
Sep 14 02:00:16.577: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.121499433s
Sep 14 02:00:18.629: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.17347908s
Sep 14 02:00:20.681: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.225689253s
Sep 14 02:00:22.734: INFO: Pod "azuredisk-volume-tester-crcwp": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.278148439s
Sep 14 02:00:24.735: INFO: deleting Pod "azuredisk-2540"/"azuredisk-volume-tester-crcwp"
Sep 14 02:00:24.822: INFO: Error getting logs for pod azuredisk-volume-tester-crcwp: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-crcwp)
STEP: Deleting pod azuredisk-volume-tester-crcwp in namespace azuredisk-2540
Sep 14 02:00:24.876: INFO: deleting PVC "azuredisk-2540"/"pvc-qwc8w"
Sep 14 02:00:24.876: INFO: Deleting PersistentVolumeClaim "pvc-qwc8w"
STEP: waiting for claim's PV "pvc-71351667-8f30-4db9-aa46-b1e2c4bcc5e9" to be deleted
Sep 14 02:00:24.929: INFO: Waiting up to 10m0s for PersistentVolume pvc-71351667-8f30-4db9-aa46-b1e2c4bcc5e9 to get deleted
Sep 14 02:00:24.981: INFO: PersistentVolume pvc-71351667-8f30-4db9-aa46-b1e2c4bcc5e9 found and phase=Bound (51.164326ms)
... skipping 6 lines ...
STEP: Collecting events from namespace "azuredisk-2540".
STEP: Found 6 events.
Sep 14 02:00:40.302: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-crcwp: { } Scheduled: Successfully assigned azuredisk-2540/azuredisk-volume-tester-crcwp to k8s-agentpool-38427154-vmss000001
Sep 14 02:00:40.302: INFO: At 2022-09-14 01:45:19 +0000 UTC - event for pvc-qwc8w: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 02:00:40.302: INFO: At 2022-09-14 01:45:19 +0000 UTC - event for pvc-qwc8w: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2540/pvc-qwc8w"
Sep 14 02:00:40.302: INFO: At 2022-09-14 01:45:21 +0000 UTC - event for pvc-qwc8w: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-71351667-8f30-4db9-aa46-b1e2c4bcc5e9
Sep 14 02:00:40.302: INFO: At 2022-09-14 01:45:23 +0000 UTC - event for azuredisk-volume-tester-crcwp: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-71351667-8f30-4db9-aa46-b1e2c4bcc5e9" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 02:00:40.302: INFO: At 2022-09-14 01:47:26 +0000 UTC - event for azuredisk-volume-tester-crcwp: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 02:00:40.353: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 02:00:40.353: INFO: 
Sep 14 02:00:40.453: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 02:00:40.505: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 6877 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 02:00:03 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 02:00:03 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 02:00:03 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 02:00:03 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a pod with volume mount subpath [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:150

    Unexpected error:
        <*errors.errorString | 0xc000638af0>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-crcwp\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-crcwp" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  Should create and attach a volume with basic perfProfile [enableBursting][disk.csi.azure.com] [Windows]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 02:00:44.516: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-fkh76" in namespace "azuredisk-4728" to be "Succeeded or Failed"
Sep 14 02:00:44.569: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 52.956473ms
Sep 14 02:00:46.621: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 2.10470307s
Sep 14 02:00:48.672: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 4.156256148s
Sep 14 02:00:50.725: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 6.208948845s
Sep 14 02:00:52.776: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 8.260009994s
Sep 14 02:00:54.829: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 10.312422031s
... skipping 428 lines ...
Sep 14 02:15:35.632: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.115426568s
Sep 14 02:15:37.685: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.168638429s
Sep 14 02:15:39.737: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.221102208s
Sep 14 02:15:41.790: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.273758078s
Sep 14 02:15:43.843: INFO: Pod "azuredisk-volume-tester-fkh76": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.326614048s
Sep 14 02:15:45.844: INFO: deleting Pod "azuredisk-4728"/"azuredisk-volume-tester-fkh76"
Sep 14 02:15:45.932: INFO: Error getting logs for pod azuredisk-volume-tester-fkh76: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-fkh76)
STEP: Deleting pod azuredisk-volume-tester-fkh76 in namespace azuredisk-4728
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 02:15:46.092: INFO: deleting PVC "azuredisk-4728"/"pvc-p7ggt"
Sep 14 02:15:46.092: INFO: Deleting PersistentVolumeClaim "pvc-p7ggt"
STEP: waiting for claim's PV "pvc-05e1fe19-5f0e-4f92-99ab-4535fdefa7f4" to be deleted
... skipping 11 lines ...
STEP: Collecting events from namespace "azuredisk-4728".
STEP: Found 7 events.
Sep 14 02:16:16.687: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-fkh76: { } Scheduled: Successfully assigned azuredisk-4728/azuredisk-volume-tester-fkh76 to k8s-agentpool-38427154-vmss000001
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:00:44 +0000 UTC - event for pvc-p7ggt: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:00:44 +0000 UTC - event for pvc-p7ggt: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:00:44 +0000 UTC - event for pvc-p7ggt: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-4728/pvc-p7ggt"
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:00:48 +0000 UTC - event for azuredisk-volume-tester-fkh76: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-05e1fe19-5f0e-4f92-99ab-4535fdefa7f4" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:00:48 +0000 UTC - event for pvc-p7ggt: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-05e1fe19-5f0e-4f92-99ab-4535fdefa7f4
Sep 14 02:16:16.687: INFO: At 2022-09-14 02:02:51 +0000 UTC - event for azuredisk-volume-tester-fkh76: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 02:16:16.738: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 02:16:16.738: INFO: 
Sep 14 02:16:16.838: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
... skipping 94 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    Should create and attach a volume with basic perfProfile [enableBursting][disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:185

    Unexpected error:
        <*errors.errorString | 0xc0006b6520>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-fkh76\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-fkh76" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should receive FailedMount event with invalid mount options [kubernetes.io/azure-disk] [disk.csi.azure.com]
... skipping 13 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod has 'FailedMount' event
Sep 14 02:21:19.631: INFO: deleting Pod "azuredisk-5466"/"azuredisk-volume-tester-bvshg"
Sep 14 02:21:19.714: INFO: Error getting logs for pod azuredisk-volume-tester-bvshg: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-bvshg)
STEP: Deleting pod azuredisk-volume-tester-bvshg in namespace azuredisk-5466
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 02:21:19.872: INFO: deleting PVC "azuredisk-5466"/"pvc-hl9n7"
Sep 14 02:21:19.872: INFO: Deleting PersistentVolumeClaim "pvc-hl9n7"
STEP: waiting for claim's PV "pvc-00f91399-3270-4c0a-87a9-9fd7fab64e48" to be deleted
... skipping 10 lines ...
STEP: Collecting events from namespace "azuredisk-5466".
STEP: Found 7 events.
Sep 14 02:21:45.406: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-bvshg: { } Scheduled: Successfully assigned azuredisk-5466/azuredisk-volume-tester-bvshg to k8s-agentpool-38427154-vmss000001
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:16:19 +0000 UTC - event for pvc-hl9n7: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:16:19 +0000 UTC - event for pvc-hl9n7: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:16:19 +0000 UTC - event for pvc-hl9n7: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5466/pvc-hl9n7"
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:16:22 +0000 UTC - event for azuredisk-volume-tester-bvshg: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-00f91399-3270-4c0a-87a9-9fd7fab64e48" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:16:22 +0000 UTC - event for pvc-hl9n7: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-00f91399-3270-4c0a-87a9-9fd7fab64e48
Sep 14 02:21:45.406: INFO: At 2022-09-14 02:18:25 +0000 UTC - event for azuredisk-volume-tester-bvshg: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 02:21:45.457: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 02:21:45.457: INFO: 
Sep 14 02:21:45.558: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
... skipping 94 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should receive FailedMount event with invalid mount options [kubernetes.io/azure-disk] [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:221

    Unexpected error:
        <*errors.errorString | 0xc0001f24d0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

... skipping 15 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 02:21:48.243: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-2ghl9" in namespace "azuredisk-2790" to be "Succeeded or Failed"
Sep 14 02:21:48.295: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 51.470872ms
Sep 14 02:21:50.347: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.103455045s
Sep 14 02:21:52.400: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.156336166s
Sep 14 02:21:54.453: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.209215838s
Sep 14 02:21:56.505: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.26157579s
Sep 14 02:21:58.558: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.314935704s
... skipping 428 lines ...
Sep 14 02:36:39.356: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.112477338s
Sep 14 02:36:41.408: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.164049002s
Sep 14 02:36:43.462: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.218567999s
Sep 14 02:36:45.514: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.27042793s
Sep 14 02:36:47.568: INFO: Pod "azuredisk-volume-tester-2ghl9": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.324602829s
Sep 14 02:36:49.569: INFO: deleting Pod "azuredisk-2790"/"azuredisk-volume-tester-2ghl9"
Sep 14 02:36:49.658: INFO: Error getting logs for pod azuredisk-volume-tester-2ghl9: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-2ghl9)
STEP: Deleting pod azuredisk-volume-tester-2ghl9 in namespace azuredisk-2790
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 02:36:49.815: INFO: deleting PVC "azuredisk-2790"/"pvc-wp5gz"
Sep 14 02:36:49.815: INFO: Deleting PersistentVolumeClaim "pvc-wp5gz"
STEP: waiting for claim's PV "pvc-6fac7204-c693-478f-831a-96d7eaadde71" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 02:37:15.351: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-2ghl9: { } Scheduled: Successfully assigned azuredisk-2790/azuredisk-volume-tester-2ghl9 to k8s-agentpool-38427154-vmss000001
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:21:48 +0000 UTC - event for pvc-wp5gz: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:21:48 +0000 UTC - event for pvc-wp5gz: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:21:48 +0000 UTC - event for pvc-wp5gz: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-2790/pvc-wp5gz"
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:21:50 +0000 UTC - event for pvc-wp5gz: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-6fac7204-c693-478f-831a-96d7eaadde71
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:21:51 +0000 UTC - event for azuredisk-volume-tester-2ghl9: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-6fac7204-c693-478f-831a-96d7eaadde71" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 02:37:15.351: INFO: At 2022-09-14 02:23:54 +0000 UTC - event for azuredisk-volume-tester-2ghl9: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 02:37:15.402: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 02:37:15.402: INFO: 
Sep 14 02:37:15.503: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 02:37:15.555: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 13216 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 02:35:05 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 02:35:05 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 02:35:05 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 02:35:05 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a raw block volume on demand [kubernetes.io/azure-disk] [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:258

    Unexpected error:
        <*errors.errorString | 0xc000ad9040>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-2ghl9\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-2ghl9" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with an error
Sep 14 02:37:18.179: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-plcbx" in namespace "azuredisk-5356" to be "Error status code"
Sep 14 02:37:18.229: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 50.683552ms
Sep 14 02:37:20.286: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 2.106962913s
Sep 14 02:37:22.337: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 4.158361417s
Sep 14 02:37:24.390: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 6.210945749s
Sep 14 02:37:26.442: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 8.263291642s
Sep 14 02:37:28.494: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 10.315445476s
... skipping 428 lines ...
Sep 14 02:52:09.284: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.105229188s
Sep 14 02:52:11.335: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.156887054s
Sep 14 02:52:13.389: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.210663033s
Sep 14 02:52:15.442: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.263076288s
Sep 14 02:52:17.493: INFO: Pod "azuredisk-volume-tester-plcbx": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.314702505s
Sep 14 02:52:19.495: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-plcbx"
Sep 14 02:52:19.583: INFO: Error getting logs for pod azuredisk-volume-tester-plcbx: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-plcbx)
STEP: Deleting pod azuredisk-volume-tester-plcbx in namespace azuredisk-5356
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 02:52:19.744: INFO: deleting PVC "azuredisk-5356"/"pvc-2kt25"
Sep 14 02:52:19.744: INFO: Deleting PersistentVolumeClaim "pvc-2kt25"
STEP: waiting for claim's PV "pvc-41be1ab6-ac33-4092-bb0f-aef21a239548" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 02:52:45.269: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-plcbx: { } Scheduled: Successfully assigned azuredisk-5356/azuredisk-volume-tester-plcbx to k8s-agentpool-38427154-vmss000001
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:37:18 +0000 UTC - event for pvc-2kt25: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:37:18 +0000 UTC - event for pvc-2kt25: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:37:18 +0000 UTC - event for pvc-2kt25: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5356/pvc-2kt25"
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:37:20 +0000 UTC - event for pvc-2kt25: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-41be1ab6-ac33-4092-bb0f-aef21a239548
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:37:21 +0000 UTC - event for azuredisk-volume-tester-plcbx: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-41be1ab6-ac33-4092-bb0f-aef21a239548" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 02:52:45.269: INFO: At 2022-09-14 02:39:24 +0000 UTC - event for azuredisk-volume-tester-plcbx: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 02:52:45.320: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 02:52:45.320: INFO: 
Sep 14 02:52:45.419: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 02:52:45.470: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 15919 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 02:50:06 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 02:50:06 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 02:50:06 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 02:50:06 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a volume on demand and mount it as readOnly in a pod [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:290

    Unexpected error:
        <*errors.errorString | 0xc000638d70>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-plcbx\" to be \"Error status code\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-plcbx" to be "Error status code"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:844
------------------------------
Dynamic Provisioning [multi-az] 
  should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 13 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod is running
Sep 14 03:07:48.165: INFO: deleting Pod "azuredisk-5194"/"azuredisk-volume-tester-hjlk9"
Sep 14 03:07:48.249: INFO: Error getting logs for pod azuredisk-volume-tester-hjlk9: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-hjlk9)
STEP: Deleting pod azuredisk-volume-tester-hjlk9 in namespace azuredisk-5194
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 03:07:48.406: INFO: deleting PVC "azuredisk-5194"/"pvc-xd94b"
Sep 14 03:07:48.406: INFO: Deleting PersistentVolumeClaim "pvc-xd94b"
STEP: waiting for claim's PV "pvc-e12f6b2b-f18b-4721-9d98-4770bb18f336" to be deleted
... skipping 11 lines ...
STEP: Found 7 events.
Sep 14 03:08:13.949: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-hjlk9: { } Scheduled: Successfully assigned azuredisk-5194/azuredisk-volume-tester-hjlk9 to k8s-agentpool-38427154-vmss000001
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:52:47 +0000 UTC - event for pvc-xd94b: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:52:47 +0000 UTC - event for pvc-xd94b: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:52:47 +0000 UTC - event for pvc-xd94b: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-5194/pvc-xd94b"
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:52:50 +0000 UTC - event for pvc-xd94b: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-e12f6b2b-f18b-4721-9d98-4770bb18f336
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:52:51 +0000 UTC - event for azuredisk-volume-tester-hjlk9: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-e12f6b2b-f18b-4721-9d98-4770bb18f336" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 03:08:13.949: INFO: At 2022-09-14 02:54:54 +0000 UTC - event for azuredisk-volume-tester-hjlk9: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 03:08:14.000: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 03:08:14.000: INFO: 
Sep 14 03:08:14.100: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 03:08:14.152: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 18615 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 03:05:07 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 03:05:07 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 03:05:07 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 03:05:07 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create multiple PV objects, bind to PVCs and attach all to different pods on the same node [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:324

    Unexpected error:
        <*errors.errorString | 0xc0001f24d0>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

... skipping 190 lines ...
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:16 +0000 UTC - event for azuredisk-volume-tester-45j9r: {deployment-controller } ScalingReplicaSet: Scaled up replica set azuredisk-volume-tester-45j9r-6988b66cdb to 1
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:16 +0000 UTC - event for azuredisk-volume-tester-45j9r-6988b66cdb: {replicaset-controller } SuccessfulCreate: Created pod: azuredisk-volume-tester-45j9r-6988b66cdb-9w899
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:16 +0000 UTC - event for pvc-vwbk6: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:16 +0000 UTC - event for pvc-vwbk6: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:16 +0000 UTC - event for pvc-vwbk6: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-1353/pvc-vwbk6"
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:19 +0000 UTC - event for pvc-vwbk6: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-04301790-611a-431c-8391-66c3d50bbbea
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:08:20 +0000 UTC - event for azuredisk-volume-tester-45j9r-6988b66cdb-9w899: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-04301790-611a-431c-8391-66c3d50bbbea" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:10:22 +0000 UTC - event for azuredisk-volume-tester-45j9r-6988b66cdb-9w899: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1 default-token-pn827]: timed out waiting for the condition
Sep 14 03:13:32.518: INFO: At 2022-09-14 03:12:41 +0000 UTC - event for azuredisk-volume-tester-45j9r-6988b66cdb-9w899: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[default-token-pn827 test-volume-1]: timed out waiting for the condition
Sep 14 03:13:32.570: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 03:13:32.570: INFO: 
Sep 14 03:13:32.669: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
... skipping 94 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should create a deployment object, write and read to it, delete the pod and write and read to it again [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:388

    Unexpected error:
        <*errors.errorString | 0xc000ad90d0>: {
            s: "error waiting for deployment \"azuredisk-volume-tester-45j9r\" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:\"Available\", Status:\"False\", LastUpdateTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), Reason:\"MinimumReplicasUnavailable\", Message:\"Deployment does not have minimum availability.\"}, v1.DeploymentCondition{Type:\"Progressing\", Status:\"True\", LastUpdateTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), Reason:\"ReplicaSetUpdated\", Message:\"ReplicaSet \\\"azuredisk-volume-tester-45j9r-6988b66cdb\\\" is progressing.\"}}, CollisionCount:(*int32)(nil)}",
        }
        error waiting for deployment "azuredisk-volume-tester-45j9r" status to match expectation: deployment status: v1.DeploymentStatus{ObservedGeneration:1, Replicas:1, UpdatedReplicas:1, ReadyReplicas:0, AvailableReplicas:0, UnavailableReplicas:1, Conditions:[]v1.DeploymentCondition{v1.DeploymentCondition{Type:"Available", Status:"False", LastUpdateTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), Reason:"MinimumReplicasUnavailable", Message:"Deployment does not have minimum availability."}, v1.DeploymentCondition{Type:"Progressing", Status:"True", LastUpdateTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), LastTransitionTime:time.Date(2022, time.September, 14, 3, 8, 16, 0, time.Local), Reason:"ReplicaSetUpdated", Message:"ReplicaSet \"azuredisk-volume-tester-45j9r-6988b66cdb\" is progressing."}}, CollisionCount:(*int32)(nil)}
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:503
------------------------------
Dynamic Provisioning [multi-az] 
  should delete PV with reclaimPolicy "Delete" [kubernetes.io/azure-disk] [disk.csi.azure.com] [Windows]
... skipping 103 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 03:13:53.918: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-c84jw" in namespace "azuredisk-4376" to be "Succeeded or Failed"
Sep 14 03:13:53.969: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 50.871598ms
Sep 14 03:13:56.020: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 2.102596545s
Sep 14 03:13:58.072: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 4.154092923s
Sep 14 03:14:00.125: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 6.206772454s
Sep 14 03:14:02.177: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8.259119954s
Sep 14 03:14:04.229: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 10.311621689s
... skipping 227 lines ...
Sep 14 03:21:52.310: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 7m58.392545857s
Sep 14 03:21:54.363: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8m0.445471295s
Sep 14 03:21:56.415: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8m2.497284035s
Sep 14 03:21:58.468: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8m4.550613255s
Sep 14 03:22:00.522: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8m6.604180497s
Sep 14 03:22:02.574: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 8m8.656361151s
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:165","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2022-09-14T03:22:02Z"}
++ early_exit_handler
++ '[' -n 174 ']'
++ kill -TERM 174
++ cleanup_dind
++ [[ true == \t\r\u\e ]]
++ echo 'Cleaning up after docker'
... skipping 210 lines ...
Sep 14 03:28:44.989: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m51.071176758s
Sep 14 03:28:47.043: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m53.124768794s
Sep 14 03:28:49.095: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m55.177491026s
Sep 14 03:28:51.149: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m57.230792489s
Sep 14 03:28:53.201: INFO: Pod "azuredisk-volume-tester-c84jw": Phase="Pending", Reason="", readiness=false. Elapsed: 14m59.283212479s
Sep 14 03:28:55.202: INFO: deleting Pod "azuredisk-4376"/"azuredisk-volume-tester-c84jw"
Sep 14 03:28:55.300: INFO: Error getting logs for pod azuredisk-volume-tester-c84jw: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-c84jw)
STEP: Deleting pod azuredisk-volume-tester-c84jw in namespace azuredisk-4376
STEP: validating provisioned PV
STEP: checking the PV
Sep 14 03:28:55.457: INFO: deleting PVC "azuredisk-4376"/"pvc-dwmdc"
Sep 14 03:28:55.457: INFO: Deleting PersistentVolumeClaim "pvc-dwmdc"
STEP: waiting for claim's PV "pvc-c92671cb-ba25-423a-a509-e8207634d24f" to be deleted
... skipping 10 lines ...
STEP: Found 7 events.
Sep 14 03:29:15.932: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azuredisk-volume-tester-c84jw: { } Scheduled: Successfully assigned azuredisk-4376/azuredisk-volume-tester-c84jw to k8s-agentpool-38427154-vmss000001
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:13:53 +0000 UTC - event for pvc-dwmdc: {persistentvolume-controller } WaitForFirstConsumer: waiting for first consumer to be created before binding
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:13:53 +0000 UTC - event for pvc-dwmdc: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "disk.csi.azure.com" or manually created by system administrator
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:13:53 +0000 UTC - event for pvc-dwmdc: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } Provisioning: External provisioner is provisioning volume for claim "azuredisk-4376/pvc-dwmdc"
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:13:56 +0000 UTC - event for pvc-dwmdc: {disk.csi.azure.com_k8s-agentpool-38427154-vmss000001_b6c780cd-364d-471d-92c3-28e9aa269a4f } ProvisioningSucceeded: Successfully provisioned volume pvc-c92671cb-ba25-423a-a509-e8207634d24f
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:13:57 +0000 UTC - event for azuredisk-volume-tester-c84jw: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-c92671cb-ba25-423a-a509-e8207634d24f" : rpc error: code = NotFound desc = failed to get azure instance id for node "k8s-agentpool-38427154-vmss000001" (instance not found)
Sep 14 03:29:15.932: INFO: At 2022-09-14 03:15:59 +0000 UTC - event for azuredisk-volume-tester-c84jw: {kubelet k8s-agentpool-38427154-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[test-volume-1], unattached volumes=[test-volume-1]: timed out waiting for the condition
Sep 14 03:29:15.982: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 14 03:29:15.983: INFO: 
Sep 14 03:29:16.084: INFO: 
Logging node info for node k8s-agentpool-38427154-vmss000000
Sep 14 03:29:16.135: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool-38427154-vmss000000    70ebc5b5-ba15-40ef-87d5-ebab5225d3f9 22333 0 2022-09-14 01:24:50 +0000 UTC <nil> <nil> map[agentpool:agentpool beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D2s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westus2 failure-domain.beta.kubernetes.io/zone:westus2-1 kubernetes.azure.com/cluster:kubetest-er7tehvb kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool-38427154-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D2s_v3 storageprofile:managed storagetier:Premium_LRS topology.disk.csi.azure.com/zone:westus2-1 topology.kubernetes.io/region:westus2 topology.kubernetes.io/zone:westus2-1] map[csi.volume.kubernetes.io/nodeid:{"disk.csi.azure.com":"k8s-agentpool-38427154-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubectl-label Update v1 2022-09-14 01:24:53 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2022-09-14 01:25:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2022-09-14 01:25:51 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-09-14 01:29:47 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:csi.volume.kubernetes.io/nodeid":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{},"f:topology.disk.csi.azure.com/zone":{}}},"f:status":{"f:addresses":{".":{},"k:{\"type\":\"Hostname\"}":{".":{},"f:address":{},"f:type":{}},"k:{\"type\":\"InternalIP\"}":{".":{},"f:address":{},"f:type":{}}},"f:allocatable":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:capacity":{".":{},"f:cpu":{},"f:ephemeral-storage":{},"f:hugepages-1Gi":{},"f:hugepages-2Mi":{},"f:memory":{},"f:pods":{}},"f:conditions":{".":{},"k:{\"type\":\"DiskPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"MemoryPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"PIDPressure\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}},"f:daemonEndpoints":{"f:kubeletEndpoint":{"f:Port":{}}},"f:images":{},"f:nodeInfo":{"f:architecture":{},"f:bootID":{},"f:containerRuntimeVersion":{},"f:kernelVersion":{},"f:kubeProxyVersion":{},"f:kubeletVersion":{},"f:machineID":{},"f:operatingSystem":{},"f:osImage":{},"f:systemUUID":{}}}} }]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-er7tehvb/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool-38427154-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{8344711168 0} {<nil>} 8149132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7558279168 0} {<nil>} 7381132Ki BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-09-14 03:25:08 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-09-14 03:25:08 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-09-14 03:25:08 +0000 UTC,LastTransitionTime:2022-09-14 01:24:46 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-09-14 03:25:08 +0000 UTC,LastTransitionTime:2022-09-14 01:25:00 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.248.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool-38427154-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:abc85822e2934066b2cc06452ebd0ab5,SystemUUID:cee6d6a3-7da8-8c46-b31b-3b936b52da11,BootID:2c9ac668-039d-4eae-9721-57b488502936,KernelVersion:5.4.0-1089-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:containerd://1.5.13+azure-1,KubeletVersion:v1.20.15,KubeProxyVersion:v1.20.15,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:250750921,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:235451754,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.9],SizeBytes:136484824,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.23.6],SizeBytes:136480728,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.24.3],SizeBytes:131053523,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.12],SizeBytes:129576915,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.7],SizeBytes:129568725,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.14],SizeBytes:127000541,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.9],SizeBytes:126306115,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.23.6],SizeBytes:126302019,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.12],SizeBytes:123264833,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.7],SizeBytes:123257155,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.14],SizeBytes:121200451,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.24.3],SizeBytes:120702785,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.6],SizeBytes:114231361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.9],SizeBytes:114210871,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.24.3],SizeBytes:111851575,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.7],SizeBytes:105458380,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.12],SizeBytes:105430215,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.14],SizeBytes:105351884,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333634,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:09047853ce1b152490b583fe8ae3777638c7e90811cbc9227ad67c329ab85a73 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.15],SizeBytes:101469515,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:101308513,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:8ae621ec6b591ce5b4f9990206b84213744ed37e00672aefa194142cd2f0a55d k8sprow.azurecr.io/azuredisk-csi:v1.23.0-e6ae87d70ab5eba40431d3d3f3f6a61280f4f9e5],SizeBytes:95998450,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:65672784,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:2f9a214fe823b4b5099e7cff4f4d16e7e4298cd086478880845de8858d49a3b3 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.2.0],SizeBytes:60382210,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145360,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:544e74bd67c649fd49500e195ff4a4ee675cfd26768574262dc6fa0250373d59 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0],SizeBytes:57519578,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:7e5af2ed16e053822e58f6576423c0bb77e59050c3698986f319d257b4551023 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.5.0],SizeBytes:56936934,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55577643,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55474791,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55106610,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.9],SizeBytes:54823234,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.23.6],SizeBytes:54819138,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.12],SizeBytes:53931842,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.7],SizeBytes:53924160,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52838996,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:52722904,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:52391108,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.24.3],SizeBytes:52332864,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:52153533,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.14],SizeBytes:52130112,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:50416860,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:50121924,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 93 lines ...
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:40
  [multi-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:47
    should clone a volume from an existing volume and read from it [disk.csi.azure.com] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/dynamic_provisioning_test.go:464

    Unexpected error:
        <*errors.errorString | 0xc000516a10>: {
            s: "Gave up after waiting 15m0s for pod \"azuredisk-volume-tester-c84jw\" to be \"Succeeded or Failed\"",
        }
        Gave up after waiting 15m0s for pod "azuredisk-volume-tester-c84jw" to be "Succeeded or Failed"
    occurred

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/testsuites/testsuites.go:805
------------------------------
Dynamic Provisioning [multi-az] 
  should clone a volume of larger size than the source volume and make sure the filesystem is appropriately adjusted [disk.csi.azure.com]
... skipping 11 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Sep 14 03:29:18.666: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-wq4cp" in namespace "azuredisk-7996" to be "Succeeded or Failed"
Sep 14 03:29:18.717: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 51.126484ms
Sep 14 03:29:20.770: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 2.104366424s
Sep 14 03:29:22.825: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 4.159028491s
Sep 14 03:29:24.878: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 6.211497509s
Sep 14 03:29:26.930: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 8.263805397s
Sep 14 03:29:28.981: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 10.31515157s
... skipping 214 lines ...
Sep 14 03:36:50.456: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m31.790179974s
Sep 14 03:36:52.510: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m33.843717258s
Sep 14 03:36:54.563: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m35.896671921s
Sep 14 03:36:56.615: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m37.948898355s
Sep 14 03:36:58.667: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m40.00060968s
Sep 14 03:37:00.720: INFO: Pod "azuredisk-volume-tester-wq4cp": Phase="Pending", Reason="", readiness=false. Elapsed: 7m42.054032778s
{"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:255","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Process did not exit before 15m0s grace period","severity":"error","time":"2022-09-14T03:37:02Z"}
{"component":"entrypoint","error":"os: process already finished","file":"k8s.io/test-infra/prow/entrypoint/run.go:257","func":"k8s.io/test-infra/prow/entrypoint.gracefullyTerminate","level":"error","msg":"Could not kill process after grace period","severity":"error","time":"2022-09-14T03:37:02Z"}