This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: feat: enable fsGroup for nfs
ResultFAILURE
Tests 1 failed / 11 succeeded
Started2021-09-15 04:04
Elapsed31m10s
Revision11caf06e9f643bd89582d895ac7b8cc7a965ae81
Refs 759
job-versionv1.23.0-alpha.2.9+5449ce7c5c9c3e
kubetest-version
revisionv1.23.0-alpha.2.9+5449ce7c5c9c3e

Test Failures


kubetest Test 18m53s

error during make e2e-test: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 11 Passed Tests

Error lines from build-log.txt

... skipping 249 lines ...
helm install azurefile-csi-driver charts/latest/azurefile-csi-driver --namespace kube-system --wait --timeout=15m -v=5 --debug \
	--set image.azurefile.repository=k8sprow.azurecr.io/azurefile-csi --set image.azurefile.tag=e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 --set linux.dnsPolicy=ClusterFirstWithHostNet --set driver.userAgentSuffix="e2e-test"  --set feature.enableFSGroupPolicy=true --set image.csiProvisioner.tag=v3.0.0 --set snapshot.apiVersion=ga --set driver.name=test.csi.azure.com --set controller.name=csi-test-controller --set linux.dsName=csi-test-node --set windows.dsName=csi-test-node-win \
	--set snapshot.enabled=true
install.go:173: [debug] Original chart version: ""
install.go:190: [debug] CHART PATH: /home/prow/go/src/sigs.k8s.io/azurefile-csi-driver/charts/latest/azurefile-csi-driver

I0915 04:09:24.861408     924 request.go:1347] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:\"apiVersion,omitempty\""; Kind string "json:\"kind,omitempty\"" }
I0915 04:09:24.861520     924 cached_discovery.go:78] skipped caching discovery info due to the server is currently unable to handle the request
I0915 04:09:25.244956     924 request.go:1347] body was not decodable (unable to check for Status): couldn't get version/kind; json parse error: json: cannot unmarshal string into Go value of type struct { APIVersion string "json:\"apiVersion,omitempty\""; Kind string "json:\"kind,omitempty\"" }
I0915 04:09:25.244999     924 cached_discovery.go:78] skipped caching discovery info due to the server is currently unable to handle the request
client.go:128: [debug] creating 27 resource(s)
wait.go:47: [debug] beginning wait for 27 resources with timeout of 15m0s
ready.go:304: [debug] DaemonSet is not ready: kube-system/csi-test-node. 0 out of 2 expected pods are ready
ready.go:304: [debug] DaemonSet is not ready: kube-system/csi-test-node. 0 out of 2 expected pods are ready
ready.go:304: [debug] DaemonSet is not ready: kube-system/csi-test-node. 0 out of 2 expected pods are ready
... skipping 257 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 55 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 239 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 99 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 829 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=false'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: socket-dir
... skipping 471 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "test.csi.azure.com" does not support topology - skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 419 lines ...
Sep 15 04:10:56.990: INFO: PersistentVolumeClaim test.csi.azure.comqlj7l found but phase is Pending instead of Bound.
Sep 15 04:10:59.028: INFO: PersistentVolumeClaim test.csi.azure.comqlj7l found but phase is Pending instead of Bound.
Sep 15 04:11:01.066: INFO: PersistentVolumeClaim test.csi.azure.comqlj7l found but phase is Pending instead of Bound.
Sep 15 04:11:03.108: INFO: PersistentVolumeClaim test.csi.azure.comqlj7l found and phase=Bound (53.047302269s)
STEP: Creating pod pod-subpath-test-dynamicpv-5pqn
STEP: Creating a pod to test subpath
Sep 15 04:11:03.223: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-5pqn" in namespace "provisioning-8795" to be "Succeeded or Failed"
Sep 15 04:11:03.266: INFO: Pod "pod-subpath-test-dynamicpv-5pqn": Phase="Pending", Reason="", readiness=false. Elapsed: 42.926878ms
Sep 15 04:11:05.305: INFO: Pod "pod-subpath-test-dynamicpv-5pqn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.081306637s
Sep 15 04:11:07.343: INFO: Pod "pod-subpath-test-dynamicpv-5pqn": Phase="Failed", Reason="", readiness=false. Elapsed: 4.119819113s
Sep 15 04:11:07.846: INFO: Output of node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-5pqn" container "init-volume-dynamicpv-5pqn": mkdir: can't create directory '/test-volume/provisioning-8795': Permission denied

Sep 15 04:11:07.884: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-5pqn" container "test-init-subpath-dynamicpv-5pqn": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-5pqn)
Sep 15 04:11:07.923: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-5pqn" container "test-container-subpath-dynamicpv-5pqn": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-5pqn)
Sep 15 04:11:07.962: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-5pqn" container "test-container-volume-dynamicpv-5pqn": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-5pqn)
STEP: delete the pod
Sep 15 04:11:08.007: INFO: Waiting for pod pod-subpath-test-dynamicpv-5pqn to disappear
Sep 15 04:11:08.046: INFO: Pod pod-subpath-test-dynamicpv-5pqn no longer exists
Sep 15 04:11:08.046: FAIL: Unexpected error:
    <*errors.errorString | 0xc003c38440>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-5pqn\" success: pod \"pod-subpath-test-dynamicpv-5pqn\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-5pqn test-init-subpath-dynamicpv-5pqn]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.53 PodIPs:[{IP:10.240.0.53}] StartTime:2021-09-15 04:11:03 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-5pqn State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:05 +0000 UTC,FinishedAt:2021-09-15 04:11:05 +0000 UTC,ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483 Started:<nil>} {Name:test-init-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305b} {Name:test-container-volume-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305c}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-5pqn" success: pod "pod-subpath-test-dynamicpv-5pqn" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-5pqn test-init-subpath-dynamicpv-5pqn]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.53 PodIPs:[{IP:10.240.0.53}] StartTime:2021-09-15 04:11:03 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-5pqn State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:05 +0000 UTC,FinishedAt:2021-09-15 04:11:05 +0000 UTC,ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483 Started:<nil>} {Name:test-init-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305b} {Name:test-container-volume-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305c}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc0022d0dc0, 0x706a80a, 0x7, 0xc004313800, 0x1, 0xc000a470d0, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 27 lines ...
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:10:10 +0000 UTC - event for test.csi.azure.comqlj7l: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-8795/test.csi.azure.comqlj7l"
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:02 +0000 UTC - event for test.csi.azure.comqlj7l: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-f9739352-0b83-4d5d-b7cb-8aa7f11c6cd6
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:04 +0000 UTC - event for pod-subpath-test-dynamicpv-5pqn: {kubelet k8s-agentpool1-30741368-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:05 +0000 UTC - event for pod-subpath-test-dynamicpv-5pqn: {kubelet k8s-agentpool1-30741368-vmss000000} Created: Created container init-volume-dynamicpv-5pqn
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:05 +0000 UTC - event for pod-subpath-test-dynamicpv-5pqn: {kubelet k8s-agentpool1-30741368-vmss000000} Started: Started container init-volume-dynamicpv-5pqn
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:05 +0000 UTC - event for pod-subpath-test-dynamicpv-5pqn: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 726.581545ms
Sep 15 04:11:13.273: INFO: At 2021-09-15 04:11:07 +0000 UTC - event for pod-subpath-test-dynamicpv-5pqn: {kubelet k8s-agentpool1-30741368-vmss000000} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-sv7b4" : object "provisioning-8795"/"kube-root-ca.crt" not registered
Sep 15 04:11:13.310: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:13.310: INFO: 
Sep 15 04:11:13.383: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:13.421: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:13.421: INFO: 
... skipping 101 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directory [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:205

    Sep 15 04:11:08.046: Unexpected error:
        <*errors.errorString | 0xc003c38440>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-5pqn\" success: pod \"pod-subpath-test-dynamicpv-5pqn\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-5pqn test-init-subpath-dynamicpv-5pqn]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.53 PodIPs:[{IP:10.240.0.53}] StartTime:2021-09-15 04:11:03 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-5pqn State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:05 +0000 UTC,FinishedAt:2021-09-15 04:11:05 +0000 UTC,ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483 Started:<nil>} {Name:test-init-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305b} {Name:test-container-volume-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305c}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-5pqn" success: pod "pod-subpath-test-dynamicpv-5pqn" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-5pqn test-init-subpath-dynamicpv-5pqn]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-5pqn test-container-volume-dynamicpv-5pqn]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:03 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.53 PodIPs:[{IP:10.240.0.53}] StartTime:2021-09-15 04:11:03 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-5pqn State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:05 +0000 UTC,FinishedAt:2021-09-15 04:11:05 +0000 UTC,ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://e8c4ac4137f74523e22fddcd3c068345cfceffe82cfed3f66c6d069c9e6a1483 Started:<nil>} {Name:test-init-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305b} {Name:test-container-volume-dynamicpv-5pqn State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b5305c}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":8,"completed":0,"skipped":832,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:14.584: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 72 lines ...
Sep 15 04:10:58.333: INFO: PersistentVolumeClaim test.csi.azure.comp6rfh found but phase is Pending instead of Bound.
Sep 15 04:11:00.369: INFO: PersistentVolumeClaim test.csi.azure.comp6rfh found but phase is Pending instead of Bound.
Sep 15 04:11:02.406: INFO: PersistentVolumeClaim test.csi.azure.comp6rfh found but phase is Pending instead of Bound.
Sep 15 04:11:04.443: INFO: PersistentVolumeClaim test.csi.azure.comp6rfh found and phase=Bound (55.022321211s)
STEP: Creating pod pod-subpath-test-dynamicpv-szrt
STEP: Creating a pod to test subpath
Sep 15 04:11:04.553: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-szrt" in namespace "provisioning-7574" to be "Succeeded or Failed"
Sep 15 04:11:04.589: INFO: Pod "pod-subpath-test-dynamicpv-szrt": Phase="Pending", Reason="", readiness=false. Elapsed: 35.151845ms
Sep 15 04:11:06.624: INFO: Pod "pod-subpath-test-dynamicpv-szrt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.070690622s
Sep 15 04:11:08.661: INFO: Pod "pod-subpath-test-dynamicpv-szrt": Phase="Failed", Reason="", readiness=false. Elapsed: 4.107264068s
Sep 15 04:11:08.733: INFO: Output of node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-szrt" container "init-volume-dynamicpv-szrt": mkdir: can't create directory '/test-volume/provisioning-7574': Permission denied

Sep 15 04:11:08.770: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-szrt" container "test-init-volume-dynamicpv-szrt": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-szrt)
Sep 15 04:11:08.808: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000000" pod "pod-subpath-test-dynamicpv-szrt" container "test-container-subpath-dynamicpv-szrt": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-szrt)
STEP: delete the pod
Sep 15 04:11:08.861: INFO: Waiting for pod pod-subpath-test-dynamicpv-szrt to disappear
Sep 15 04:11:08.898: INFO: Pod pod-subpath-test-dynamicpv-szrt no longer exists
Sep 15 04:11:08.898: FAIL: Unexpected error:
    <*errors.errorString | 0xc003db9c60>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-szrt\" success: pod \"pod-subpath-test-dynamicpv-szrt\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-szrt test-init-volume-dynamicpv-szrt]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.50 PodIPs:[{IP:10.240.0.50}] StartTime:2021-09-15 04:11:04 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-szrt State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:06 +0000 UTC,FinishedAt:2021-09-15 04:11:06 +0000 UTC,ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1 Started:<nil>} {Name:test-init-volume-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc0032d810b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-szrt" success: pod "pod-subpath-test-dynamicpv-szrt" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-szrt test-init-volume-dynamicpv-szrt]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.50 PodIPs:[{IP:10.240.0.50}] StartTime:2021-09-15 04:11:04 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-szrt State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:06 +0000 UTC,FinishedAt:2021-09-15 04:11:06 +0000 UTC,ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1 Started:<nil>} {Name:test-init-volume-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc0032d810b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc003396c60, 0x706a80a, 0x7, 0xc002ab2c00, 0x0, 0xc0038e90d0, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 140 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly directory specified in the volumeMount [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365

    Sep 15 04:11:08.899: Unexpected error:
        <*errors.errorString | 0xc003db9c60>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-szrt\" success: pod \"pod-subpath-test-dynamicpv-szrt\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-szrt test-init-volume-dynamicpv-szrt]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.50 PodIPs:[{IP:10.240.0.50}] StartTime:2021-09-15 04:11:04 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-szrt State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:06 +0000 UTC,FinishedAt:2021-09-15 04:11:06 +0000 UTC,ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1 Started:<nil>} {Name:test-init-volume-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc0032d810b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-szrt" success: pod "pod-subpath-test-dynamicpv-szrt" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-szrt test-init-volume-dynamicpv-szrt]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-szrt]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:04 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.50 PodIPs:[{IP:10.240.0.50}] StartTime:2021-09-15 04:11:04 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-szrt State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:06 +0000 UTC,FinishedAt:2021-09-15 04:11:06 +0000 UTC,ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://19e202856f49da80b4ef4a176059ec5ff59c6a154802caae9b688e01af32fdc1 Started:<nil>} {Name:test-init-volume-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-szrt State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc0032d810b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":17,"completed":0,"skipped":124,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:15.313: INFO: Driver "test.csi.azure.com" does not support topology - skipping
... skipping 64 lines ...
Sep 15 04:10:56.625: INFO: PersistentVolumeClaim test.csi.azure.comnph78 found but phase is Pending instead of Bound.
Sep 15 04:10:58.663: INFO: PersistentVolumeClaim test.csi.azure.comnph78 found but phase is Pending instead of Bound.
Sep 15 04:11:00.706: INFO: PersistentVolumeClaim test.csi.azure.comnph78 found but phase is Pending instead of Bound.
Sep 15 04:11:02.743: INFO: PersistentVolumeClaim test.csi.azure.comnph78 found and phase=Bound (53.01258821s)
STEP: Creating pod pod-subpath-test-dynamicpv-65ml
STEP: Creating a pod to test subpath
Sep 15 04:11:02.857: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-65ml" in namespace "provisioning-103" to be "Succeeded or Failed"
Sep 15 04:11:02.895: INFO: Pod "pod-subpath-test-dynamicpv-65ml": Phase="Pending", Reason="", readiness=false. Elapsed: 37.484719ms
Sep 15 04:11:04.932: INFO: Pod "pod-subpath-test-dynamicpv-65ml": Phase="Pending", Reason="", readiness=false. Elapsed: 2.074560243s
Sep 15 04:11:06.968: INFO: Pod "pod-subpath-test-dynamicpv-65ml": Phase="Pending", Reason="", readiness=false. Elapsed: 4.111346385s
Sep 15 04:11:09.005: INFO: Pod "pod-subpath-test-dynamicpv-65ml": Phase="Failed", Reason="", readiness=false. Elapsed: 6.148301973s
Sep 15 04:11:09.668: INFO: Output of node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-65ml" container "init-volume-dynamicpv-65ml": mkdir: can't create directory '/test-volume/provisioning-103': Permission denied
/bin/sh: can't create /test-volume/provisioning-103/test-file: Permission denied

Sep 15 04:11:09.706: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-65ml" container "test-container-subpath-dynamicpv-65ml": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-65ml)
STEP: delete the pod
Sep 15 04:11:09.752: INFO: Waiting for pod pod-subpath-test-dynamicpv-65ml to disappear
Sep 15 04:11:09.787: INFO: Pod pod-subpath-test-dynamicpv-65ml no longer exists
Sep 15 04:11:09.787: FAIL: Unexpected error:
    <*errors.errorString | 0xc0047371f0>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-65ml\" success: pod \"pod-subpath-test-dynamicpv-65ml\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-65ml]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:11:02 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-65ml State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:07 +0000 UTC,FinishedAt:2021-09-15 04:11:07 +0000 UTC,ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87 Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-65ml State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc00472301b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-65ml" success: pod "pod-subpath-test-dynamicpv-65ml" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-65ml]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:11:02 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-65ml State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:07 +0000 UTC,FinishedAt:2021-09-15 04:11:07 +0000 UTC,ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87 Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-65ml State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc00472301b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc003b3f4a0, 0x706a80a, 0x7, 0xc0038d7000, 0x0, 0xc0008e50c0, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 27 lines ...
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:10:09 +0000 UTC - event for test.csi.azure.comnph78: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-103/test.csi.azure.comnph78"
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:02 +0000 UTC - event for test.csi.azure.comnph78: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-8d8a917f-6d19-45da-b142-263e2a9c47aa
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:04 +0000 UTC - event for pod-subpath-test-dynamicpv-65ml: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:05 +0000 UTC - event for pod-subpath-test-dynamicpv-65ml: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 840.389143ms
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:07 +0000 UTC - event for pod-subpath-test-dynamicpv-65ml: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container init-volume-dynamicpv-65ml
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:07 +0000 UTC - event for pod-subpath-test-dynamicpv-65ml: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container init-volume-dynamicpv-65ml
Sep 15 04:11:15.006: INFO: At 2021-09-15 04:11:08 +0000 UTC - event for pod-subpath-test-dynamicpv-65ml: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-6wtm9" : object "provisioning-103"/"kube-root-ca.crt" not registered
Sep 15 04:11:15.042: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:15.042: INFO: 
Sep 15 04:11:15.112: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:15.148: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:15.149: INFO: 
... skipping 101 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing single file [LinuxOnly] [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219

    Sep 15 04:11:09.787: Unexpected error:
        <*errors.errorString | 0xc0047371f0>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-65ml\" success: pod \"pod-subpath-test-dynamicpv-65ml\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-65ml]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:11:02 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-65ml State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:07 +0000 UTC,FinishedAt:2021-09-15 04:11:07 +0000 UTC,ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87 Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-65ml State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc00472301b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-65ml" success: pod "pod-subpath-test-dynamicpv-65ml" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-65ml]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-65ml]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:02 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:11:02 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-65ml State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:07 +0000 UTC,FinishedAt:2021-09-15 04:11:07 +0000 UTC,ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://3750f5c963a51fa7a6c2cda7ae6267336f6e6c16d3c407d9159c2632657e0a87 Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-65ml State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc00472301b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":17,"completed":0,"skipped":533,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]"]}

SSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:16.186: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 57 lines ...
Sep 15 04:11:16.572: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:11:16.612: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comg5hs7] to have phase Bound
Sep 15 04:11:16.647: INFO: PersistentVolumeClaim test.csi.azure.comg5hs7 found but phase is Pending instead of Bound.
Sep 15 04:11:18.685: INFO: PersistentVolumeClaim test.csi.azure.comg5hs7 found and phase=Bound (2.072492504s)
STEP: Creating pod pod-subpath-test-dynamicpv-dkhl
STEP: Creating a pod to test subpath
Sep 15 04:11:18.793: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-dkhl" in namespace "provisioning-5620" to be "Succeeded or Failed"
Sep 15 04:11:18.829: INFO: Pod "pod-subpath-test-dynamicpv-dkhl": Phase="Pending", Reason="", readiness=false. Elapsed: 36.092086ms
Sep 15 04:11:20.867: INFO: Pod "pod-subpath-test-dynamicpv-dkhl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.074612007s
Sep 15 04:11:22.905: INFO: Pod "pod-subpath-test-dynamicpv-dkhl": Phase="Failed", Reason="", readiness=false. Elapsed: 4.112451207s
Sep 15 04:11:22.981: INFO: Output of node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-dkhl" container "init-volume-dynamicpv-dkhl": mkdir: can't create directory '/test-volume/provisioning-5620': Permission denied

Sep 15 04:11:23.019: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-dkhl" container "test-init-subpath-dynamicpv-dkhl": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-dkhl)
Sep 15 04:11:23.070: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-dkhl" container "test-container-subpath-dynamicpv-dkhl": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-dkhl)
STEP: delete the pod
Sep 15 04:11:23.113: INFO: Waiting for pod pod-subpath-test-dynamicpv-dkhl to disappear
Sep 15 04:11:23.148: INFO: Pod pod-subpath-test-dynamicpv-dkhl no longer exists
Sep 15 04:11:23.149: FAIL: Unexpected error:
    <*errors.errorString | 0xc000031a30>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-dkhl\" success: pod \"pod-subpath-test-dynamicpv-dkhl\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-dkhl test-init-subpath-dynamicpv-dkhl]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:11:18 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-dkhl State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:20 +0000 UTC,FinishedAt:2021-09-15 04:11:20 +0000 UTC,ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9 Started:<nil>} {Name:test-init-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b4090b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-dkhl" success: pod "pod-subpath-test-dynamicpv-dkhl" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-dkhl test-init-subpath-dynamicpv-dkhl]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:11:18 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-dkhl State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:20 +0000 UTC,FinishedAt:2021-09-15 04:11:20 +0000 UTC,ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9 Started:<nil>} {Name:test-init-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b4090b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc003b3f4a0, 0x706a80a, 0x7, 0xc003b3a400, 0x0, 0xc0048f5080, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 27 lines ...
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:16 +0000 UTC - event for test.csi.azure.comg5hs7: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-b607f394-5d31-444d-b1df-83df44b61e33
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:16 +0000 UTC - event for test.csi.azure.comg5hs7: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-5620/test.csi.azure.comg5hs7"
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:19 +0000 UTC - event for pod-subpath-test-dynamicpv-dkhl: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:19 +0000 UTC - event for pod-subpath-test-dynamicpv-dkhl: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 169.915426ms
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:20 +0000 UTC - event for pod-subpath-test-dynamicpv-dkhl: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container init-volume-dynamicpv-dkhl
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:20 +0000 UTC - event for pod-subpath-test-dynamicpv-dkhl: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container init-volume-dynamicpv-dkhl
Sep 15 04:11:28.376: INFO: At 2021-09-15 04:11:21 +0000 UTC - event for pod-subpath-test-dynamicpv-dkhl: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-p7xmx" : object "provisioning-5620"/"kube-root-ca.crt" not registered
Sep 15 04:11:28.411: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:28.411: INFO: 
Sep 15 04:11:28.481: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:28.516: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:28.517: INFO: 
... skipping 101 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395

    Sep 15 04:11:23.149: Unexpected error:
        <*errors.errorString | 0xc000031a30>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-dkhl\" success: pod \"pod-subpath-test-dynamicpv-dkhl\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-dkhl test-init-subpath-dynamicpv-dkhl]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:11:18 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-dkhl State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:20 +0000 UTC,FinishedAt:2021-09-15 04:11:20 +0000 UTC,ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9 Started:<nil>} {Name:test-init-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b4090b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-dkhl" success: pod "pod-subpath-test-dynamicpv-dkhl" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-dkhl test-init-subpath-dynamicpv-dkhl]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-dkhl]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:18 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:11:18 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-dkhl State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:20 +0000 UTC,FinishedAt:2021-09-15 04:11:20 +0000 UTC,ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://40406af29836062a79c4073ff19c4f9d636a6778e9ed3657f09a419f5e47d3f9 Started:<nil>} {Name:test-init-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-dkhl State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc003b4090b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":17,"completed":0,"skipped":724,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes 
  should store data
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
... skipping 17 lines ...
Sep 15 04:11:15.679: INFO: PersistentVolumeClaim test.csi.azure.comwp26c found but phase is Pending instead of Bound.
Sep 15 04:11:17.716: INFO: PersistentVolumeClaim test.csi.azure.comwp26c found and phase=Bound (2.071755868s)
STEP: starting external-injector
STEP: Writing text file contents in the container.
Sep 15 04:11:19.894: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=volume-7713 exec external-injector --namespace=volume-7713 -- /bin/sh -c echo 'Hello from test.csi.azure.com from namespace volume-7713' > /opt/0/index.html'
Sep 15 04:11:20.682: INFO: rc: 1
Sep 15 04:11:20.682: FAIL: failed: writing the contents: 
Unexpected error:
    <exec.CodeExitError>: {
        Err: {
            s: "error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=volume-7713 exec external-injector --namespace=volume-7713 -- /bin/sh -c echo 'Hello from test.csi.azure.com from namespace volume-7713' > /opt/0/index.html:\nCommand stdout:\n\nstderr:\n/bin/sh: can't create /opt/0/index.html: Permission denied\ncommand terminated with exit code 1\n\nerror:\nexit status 1",
        },
        Code: 1,
    }
    error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=volume-7713 exec external-injector --namespace=volume-7713 -- /bin/sh -c echo 'Hello from test.csi.azure.com from namespace volume-7713' > /opt/0/index.html:
    Command stdout:
    
    stderr:
    /bin/sh: can't create /opt/0/index.html: Permission denied
    command terminated with exit code 1
    
    error:
    exit status 1
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc002f8ab00, 0xc00360de00, 0xb, 0x706c7cd, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560 +0x6e5
... skipping 144 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159

    Sep 15 04:11:20.683: failed: writing the contents: 
    Unexpected error:
        <exec.CodeExitError>: {
            Err: {
                s: "error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=volume-7713 exec external-injector --namespace=volume-7713 -- /bin/sh -c echo 'Hello from test.csi.azure.com from namespace volume-7713' > /opt/0/index.html:\nCommand stdout:\n\nstderr:\n/bin/sh: can't create /opt/0/index.html: Permission denied\ncommand terminated with exit code 1\n\nerror:\nexit status 1",
            },
            Code: 1,
        }
        error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=volume-7713 exec external-injector --namespace=volume-7713 -- /bin/sh -c echo 'Hello from test.csi.azure.com from namespace volume-7713' > /opt/0/index.html:
        Command stdout:
        
        stderr:
        /bin/sh: can't create /opt/0/index.html: Permission denied
        command terminated with exit code 1
        
        error:
        exit status 1
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":17,"completed":0,"skipped":316,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:31.219: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 104 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":14,"completed":1,"skipped":205,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:40.939: INFO: Driver test.csi.azure.com doesn't support ext3 -- skipping
... skipping 36 lines ...
Sep 15 04:11:29.706: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:11:29.745: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com5x576] to have phase Bound
Sep 15 04:11:29.781: INFO: PersistentVolumeClaim test.csi.azure.com5x576 found but phase is Pending instead of Bound.
Sep 15 04:11:31.823: INFO: PersistentVolumeClaim test.csi.azure.com5x576 found and phase=Bound (2.077605019s)
STEP: Creating pod exec-volume-test-dynamicpv-nfnb
STEP: Creating a pod to test exec-volume-test
Sep 15 04:11:31.936: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-nfnb" in namespace "volume-7026" to be "Succeeded or Failed"
Sep 15 04:11:31.972: INFO: Pod "exec-volume-test-dynamicpv-nfnb": Phase="Pending", Reason="", readiness=false. Elapsed: 35.107672ms
Sep 15 04:11:34.010: INFO: Pod "exec-volume-test-dynamicpv-nfnb": Phase="Pending", Reason="", readiness=false. Elapsed: 2.073221301s
Sep 15 04:11:36.047: INFO: Pod "exec-volume-test-dynamicpv-nfnb": Phase="Failed", Reason="", readiness=false. Elapsed: 4.110286082s
Sep 15 04:11:36.124: INFO: Output of node "k8s-agentpool1-30741368-vmss000000" pod "exec-volume-test-dynamicpv-nfnb" container "exec-container-dynamicpv-nfnb": /bin/sh: can't create /vol1/test-dynamicpv-nfnb.sh: Permission denied

STEP: delete the pod
Sep 15 04:11:36.168: INFO: Waiting for pod exec-volume-test-dynamicpv-nfnb to disappear
Sep 15 04:11:36.206: INFO: Pod exec-volume-test-dynamicpv-nfnb no longer exists
Sep 15 04:11:36.206: FAIL: Unexpected error:
    <*errors.errorString | 0xc0027133c0>: {
        s: "expected pod \"exec-volume-test-dynamicpv-nfnb\" success: pod \"exec-volume-test-dynamicpv-nfnb\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.52 PodIPs:[{IP:10.240.0.52}] StartTime:2021-09-15 04:11:31 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:exec-container-dynamicpv-nfnb State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:34 +0000 UTC,FinishedAt:2021-09-15 04:11:34 +0000 UTC,ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/nginx:1.14-1 ImageID:k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189 Started:0xc0015bbc50}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "exec-volume-test-dynamicpv-nfnb" success: pod "exec-volume-test-dynamicpv-nfnb" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.52 PodIPs:[{IP:10.240.0.52}] StartTime:2021-09-15 04:11:31 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:exec-container-dynamicpv-nfnb State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:34 +0000 UTC,FinishedAt:2021-09-15 04:11:34 +0000 UTC,ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/nginx:1.14-1 ImageID:k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189 Started:0xc0015bbc50}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc003761340, 0x708bc40, 0x10, 0xc003c1e400, 0x0, 0xc0048f50d8, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 25 lines ...
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:29 +0000 UTC - event for test.csi.azure.com5x576: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-2d2fce34-71b5-4804-ab36-03471a5d588a
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:29 +0000 UTC - event for test.csi.azure.com5x576: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "volume-7026/test.csi.azure.com5x576"
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:32 +0000 UTC - event for exec-volume-test-dynamicpv-nfnb: {kubelet k8s-agentpool1-30741368-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/nginx:1.14-1"
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:33 +0000 UTC - event for exec-volume-test-dynamicpv-nfnb: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/nginx:1.14-1" in 1.388060323s
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:34 +0000 UTC - event for exec-volume-test-dynamicpv-nfnb: {kubelet k8s-agentpool1-30741368-vmss000000} Created: Created container exec-container-dynamicpv-nfnb
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:34 +0000 UTC - event for exec-volume-test-dynamicpv-nfnb: {kubelet k8s-agentpool1-30741368-vmss000000} Started: Started container exec-container-dynamicpv-nfnb
Sep 15 04:11:41.412: INFO: At 2021-09-15 04:11:35 +0000 UTC - event for exec-volume-test-dynamicpv-nfnb: {kubelet k8s-agentpool1-30741368-vmss000000} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-j8xdp" : object "volume-7026"/"kube-root-ca.crt" not registered
Sep 15 04:11:41.448: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:41.448: INFO: 
Sep 15 04:11:41.517: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:41.553: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:41.554: INFO: 
... skipping 99 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should allow exec of files on the volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196

    Sep 15 04:11:36.206: Unexpected error:
        <*errors.errorString | 0xc0027133c0>: {
            s: "expected pod \"exec-volume-test-dynamicpv-nfnb\" success: pod \"exec-volume-test-dynamicpv-nfnb\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.52 PodIPs:[{IP:10.240.0.52}] StartTime:2021-09-15 04:11:31 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:exec-container-dynamicpv-nfnb State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:34 +0000 UTC,FinishedAt:2021-09-15 04:11:34 +0000 UTC,ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/nginx:1.14-1 ImageID:k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189 Started:0xc0015bbc50}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "exec-volume-test-dynamicpv-nfnb" success: pod "exec-volume-test-dynamicpv-nfnb" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [exec-container-dynamicpv-nfnb]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:31 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.52 PodIPs:[{IP:10.240.0.52}] StartTime:2021-09-15 04:11:31 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:exec-container-dynamicpv-nfnb State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:34 +0000 UTC,FinishedAt:2021-09-15 04:11:34 +0000 UTC,ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/nginx:1.14-1 ImageID:k8s.gcr.io/e2e-test-images/nginx@sha256:503b7abb89e57383eba61cc8a9cb0b495ea575c516108f7d972a6ff6e1ab3c9b ContainerID:containerd://b96abd98944e1932d6fc5a706d1cfb25811bd7bc73807677616a17bb55ccc189 Started:0xc0015bbc50}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":17,"completed":0,"skipped":752,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:42.659: INFO: Driver test.csi.azure.com doesn't support ext3 -- skipping
... skipping 35 lines ...
STEP: creating a claim
Sep 15 04:11:31.531: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:11:31.570: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comsp99h] to have phase Bound
Sep 15 04:11:31.605: INFO: PersistentVolumeClaim test.csi.azure.comsp99h found but phase is Pending instead of Bound.
Sep 15 04:11:33.642: INFO: PersistentVolumeClaim test.csi.azure.comsp99h found and phase=Bound (2.072005768s)
STEP: [init] starting a pod to use the claim
Sep 15 04:11:33.748: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-l6xvs" in namespace "snapshotting-9585" to be "Succeeded or Failed"
Sep 15 04:11:33.783: INFO: Pod "pvc-snapshottable-tester-l6xvs": Phase="Pending", Reason="", readiness=false. Elapsed: 34.930589ms
Sep 15 04:11:35.821: INFO: Pod "pvc-snapshottable-tester-l6xvs": Phase="Pending", Reason="", readiness=false. Elapsed: 2.07253954s
Sep 15 04:11:37.858: INFO: Pod "pvc-snapshottable-tester-l6xvs": Phase="Failed", Reason="", readiness=false. Elapsed: 4.109736633s
Sep 15 04:11:37.858: FAIL: Unexpected error:
    <*errors.errorString | 0xc004e0eda0>: {
        s: "pod \"pvc-snapshottable-tester-l6xvs\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.95 PodIPs:[{IP:10.240.0.95}] StartTime:2021-09-15 04:11:33 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:35 +0000 UTC,FinishedAt:2021-09-15 04:11:35 +0000 UTC,ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0 Started:0xc0041accb0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    pod "pvc-snapshottable-tester-l6xvs" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.95 PodIPs:[{IP:10.240.0.95}] StartTime:2021-09-15 04:11:33 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:35 +0000 UTC,FinishedAt:2021-09-15 04:11:35 +0000 UTC,ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0 Started:0xc0041accb0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.RunInPodWithVolume(0x79ab668, 0xc004f7d8c0, 0xc003e3e5b0, 0xc004ae4270, 0x11, 0xc004ae4240, 0x17, 0x70bc087, 0x18, 0xc001791800, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699 +0x1cb
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.1()
... skipping 29 lines ...
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:31 +0000 UTC - event for test.csi.azure.comsp99h: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-c7704a12-11f2-4626-b9b3-328ec8689d8c
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:31 +0000 UTC - event for test.csi.azure.comsp99h: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "snapshotting-9585/test.csi.azure.comsp99h"
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:34 +0000 UTC - event for pvc-snapshottable-tester-l6xvs: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container volume-tester
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:34 +0000 UTC - event for pvc-snapshottable-tester-l6xvs: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:34 +0000 UTC - event for pvc-snapshottable-tester-l6xvs: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 170.845934ms
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:35 +0000 UTC - event for pvc-snapshottable-tester-l6xvs: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container volume-tester
Sep 15 04:11:43.154: INFO: At 2021-09-15 04:11:36 +0000 UTC - event for pvc-snapshottable-tester-l6xvs: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-tcwg7" : object "snapshotting-9585"/"kube-root-ca.crt" not registered
Sep 15 04:11:43.190: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:43.190: INFO: 
Sep 15 04:11:43.258: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:43.293: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:43.294: INFO: 
... skipping 105 lines ...
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108
       [BeforeEach]
      /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:230
        should check snapshot fields, check restore correctly works after modifying source data, check deletion
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248

        Sep 15 04:11:37.859: Unexpected error:
            <*errors.errorString | 0xc004e0eda0>: {
                s: "pod \"pvc-snapshottable-tester-l6xvs\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.95 PodIPs:[{IP:10.240.0.95}] StartTime:2021-09-15 04:11:33 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:35 +0000 UTC,FinishedAt:2021-09-15 04:11:35 +0000 UTC,ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0 Started:0xc0041accb0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
            }
            pod "pvc-snapshottable-tester-l6xvs" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:33 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.95 PodIPs:[{IP:10.240.0.95}] StartTime:2021-09-15 04:11:33 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:35 +0000 UTC,FinishedAt:2021-09-15 04:11:35 +0000 UTC,ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://b3a468d9a90915783180e8ac3fd744f0fe0e75fb32fef74a0eb6dd90d2281bc0 Started:0xc0041accb0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
        occurred

        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","total":17,"completed":0,"skipped":550,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:44.258: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 99 lines ...
Sep 15 04:10:09.989: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-6387-e2e-scp5tdl
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
STEP: writing data in one pod and checking for it in the second
Sep 15 04:11:14.272: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/test-0/hello-world] Namespace:ephemeral-6387 PodName:inline-volume-tester-qmmjd ContainerName:csi-volume-tester Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:14.272: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:14.698: FAIL: "touch /mnt/test-0/hello-world" should succeed, but failed with error message "command terminated with exit code 1"
stdout: 
stderr: touch: /mnt/test-0/hello-world: Permission denied
Unexpected error:
    <exec.CodeExitError>: {
        Err: {
            s: "command terminated with exit code 1",
        },
        Code: 1,
    }
... skipping 165 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183

    Sep 15 04:11:14.698: "touch /mnt/test-0/hello-world" should succeed, but failed with error message "command terminated with exit code 1"
    stdout: 
    stderr: touch: /mnt/test-0/hello-world: Permission denied
    Unexpected error:
        <exec.CodeExitError>: {
            Err: {
                s: "command terminated with exit code 1",
            },
            Code: 1,
        }
        command terminated with exit code 1
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:636
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","total":25,"completed":0,"skipped":272,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode
... skipping 66 lines ...
STEP: creating a claim
Sep 15 04:11:42.923: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:11:42.960: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comkrvhw] to have phase Bound
Sep 15 04:11:42.996: INFO: PersistentVolumeClaim test.csi.azure.comkrvhw found but phase is Pending instead of Bound.
Sep 15 04:11:45.033: INFO: PersistentVolumeClaim test.csi.azure.comkrvhw found and phase=Bound (2.072426751s)
STEP: [init] starting a pod to use the claim
Sep 15 04:11:45.142: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-pj7kv" in namespace "snapshotting-6606" to be "Succeeded or Failed"
Sep 15 04:11:45.178: INFO: Pod "pvc-snapshottable-tester-pj7kv": Phase="Pending", Reason="", readiness=false. Elapsed: 35.590556ms
Sep 15 04:11:47.215: INFO: Pod "pvc-snapshottable-tester-pj7kv": Phase="Failed", Reason="", readiness=false. Elapsed: 2.073014479s
Sep 15 04:11:47.216: FAIL: Unexpected error:
    <*errors.errorString | 0xc00420f820>: {
        s: "pod \"pvc-snapshottable-tester-pj7kv\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.48 PodIPs:[{IP:10.240.0.48}] StartTime:2021-09-15 04:11:45 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:46 +0000 UTC,FinishedAt:2021-09-15 04:11:46 +0000 UTC,ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73 Started:0xc00039a730}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    pod "pvc-snapshottable-tester-pj7kv" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.48 PodIPs:[{IP:10.240.0.48}] StartTime:2021-09-15 04:11:45 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:46 +0000 UTC,FinishedAt:2021-09-15 04:11:46 +0000 UTC,ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73 Started:0xc00039a730}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.RunInPodWithVolume(0x79ab668, 0xc000c63a20, 0xc00380b6c0, 0xc0039f4660, 0x11, 0xc0039f4630, 0x17, 0x70bc087, 0x18, 0xc00386b400, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699 +0x1cb
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.1()
... skipping 29 lines ...
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:42 +0000 UTC - event for test.csi.azure.comkrvhw: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:43 +0000 UTC - event for test.csi.azure.comkrvhw: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-5906f4ac-9f95-49b7-8e24-dc6a78170671
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:45 +0000 UTC - event for pvc-snapshottable-tester-pj7kv: {kubelet k8s-agentpool1-30741368-vmss000000} Created: Created container volume-tester
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:45 +0000 UTC - event for pvc-snapshottable-tester-pj7kv: {kubelet k8s-agentpool1-30741368-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:45 +0000 UTC - event for pvc-snapshottable-tester-pj7kv: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 166.742797ms
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:46 +0000 UTC - event for pvc-snapshottable-tester-pj7kv: {kubelet k8s-agentpool1-30741368-vmss000000} Started: Started container volume-tester
Sep 15 04:11:52.532: INFO: At 2021-09-15 04:11:46 +0000 UTC - event for pvc-snapshottable-tester-pj7kv: {kubelet k8s-agentpool1-30741368-vmss000000} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-wwbr4" : object "snapshotting-6606"/"kube-root-ca.crt" not registered
Sep 15 04:11:52.567: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:11:52.567: INFO: 
Sep 15 04:11:52.642: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:11:52.679: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 1423 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:10:24 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:11:52.679: INFO: 
... skipping 105 lines ...
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108
       [BeforeEach]
      /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:230
        should check snapshot fields, check restore correctly works after modifying source data, check deletion
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248

        Sep 15 04:11:47.216: Unexpected error:
            <*errors.errorString | 0xc00420f820>: {
                s: "pod \"pvc-snapshottable-tester-pj7kv\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.48 PodIPs:[{IP:10.240.0.48}] StartTime:2021-09-15 04:11:45 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:46 +0000 UTC,FinishedAt:2021-09-15 04:11:46 +0000 UTC,ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73 Started:0xc00039a730}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
            }
            pod "pvc-snapshottable-tester-pj7kv" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:11:45 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.34 PodIP:10.240.0.48 PodIPs:[{IP:10.240.0.48}] StartTime:2021-09-15 04:11:45 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:11:46 +0000 UTC,FinishedAt:2021-09-15 04:11:46 +0000 UTC,ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a93c2265659b57bb1e82c3a87714895fe23731c8aed21df39250b903a1d23a73 Started:0xc00039a730}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
        occurred

        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","total":17,"completed":0,"skipped":881,"failed":4,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:11:53.741: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 123 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    Verify if offline PVC expansion works
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works","total":17,"completed":1,"skipped":748,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read/write inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
... skipping 36 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":14,"completed":2,"skipped":350,"failed":0}

SSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:12:18.441: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 69 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":17,"completed":2,"skipped":759,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:12:26.220: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 46 lines ...
Sep 15 04:11:51.134: INFO: Using claimSize:5Gi, test suite supported size:{ }, driver(test.csi.azure.com) supported size:{ } 
STEP: creating a StorageClass ephemeral-4143-e2e-scv2ksn
STEP: checking the requested inline volume exists in the pod running on node {Name: Selector:map[] Affinity:nil}
STEP: writing data in one pod and checking for it in the second
Sep 15 04:11:59.435: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/test-0/hello-world] Namespace:ephemeral-4143 PodName:inline-volume-tester-nhpst ContainerName:csi-volume-tester Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:59.435: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:59.962: FAIL: "touch /mnt/test-0/hello-world" should succeed, but failed with error message "command terminated with exit code 1"
stdout: 
stderr: touch: /mnt/test-0/hello-world: Permission denied
Unexpected error:
    <exec.CodeExitError>: {
        Err: {
            s: "command terminated with exit code 1",
        },
        Code: 1,
    }
... skipping 159 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183

    Sep 15 04:11:59.962: "touch /mnt/test-0/hello-world" should succeed, but failed with error message "command terminated with exit code 1"
    stdout: 
    stderr: touch: /mnt/test-0/hello-world: Permission denied
    Unexpected error:
        <exec.CodeExitError>: {
            Err: {
                s: "command terminated with exit code 1",
            },
            Code: 1,
        }
        command terminated with exit code 1
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:636
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","total":25,"completed":0,"skipped":534,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume"]}
[BeforeEach] [Testpattern: Inline-volume (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:12:33.366: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
[AfterEach] [Testpattern: Inline-volume (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/framework.go:186

... skipping 77 lines ...
Sep 15 04:11:11.946: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-2660 PodName:pod-ff274e97-c0dc-4c5c-8768-acc2a744af49 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:11.946: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:12.617: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-2660 PodName:pod-ff274e97-c0dc-4c5c-8768-acc2a744af49 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:12.617: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:13.074: INFO: pod fsgroupchangepolicy-2660/pod-ff274e97-c0dc-4c5c-8768-acc2a744af49 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:11 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:11:13.074: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:11 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:11:13.074: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc003af7340, 0x70bd1d0, 0x19, 0xc0035b0248, 0x4, 0xc0044f9c00)
... skipping 71 lines ...
Sep 15 04:15:50.218: INFO: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 found and phase=Bound (4m37.104943906s)
Sep 15 04:15:55.257: INFO: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 found and phase=Bound (4m42.144230652s)
Sep 15 04:16:00.293: INFO: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 found and phase=Bound (4m47.179905672s)
Sep 15 04:16:05.330: INFO: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 found and phase=Bound (4m52.217509242s)
Sep 15 04:16:10.366: INFO: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 found and phase=Bound (4m57.253397874s)
STEP: Deleting sc
Sep 15 04:16:15.403: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 not deleted by dynamic provisioner: PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-29d6511e-f627-40ca-8bae-a4acf900aa80 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003fecfc0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc00119bc20, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc00188ea00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc00119bc20, 0x4c, 0xc001e08c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc001d320c0, 0x37, 0xc0037e3980, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc001e08dc0, 0x7904368, 0xc002a9f080, 0x1, 0x0, 0x0, 0x0, 0xc002a9f080)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc001e08dc0, 0x7904368, 0xc002a9f080, 0x0, 0x0, 0x0, 0xc000092c00)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc002a9f060, 0x6323060, 0xc002a9f070, 0x0, 0x0, 0x0)
... skipping 140 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":16,"completed":0,"skipped":106,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 59 lines ...
Sep 15 04:11:13.651: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-8463 PodName:pod-dc70bfe1-3e7c-4d9d-b8ee-e75d694bb2c8 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:13.651: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:14.037: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-8463 PodName:pod-dc70bfe1-3e7c-4d9d-b8ee-e75d694bb2c8 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:11:14.037: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:11:14.413: INFO: pod fsgroupchangepolicy-8463/pod-dc70bfe1-3e7c-4d9d-b8ee-e75d694bb2c8 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:11 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:11:14.414: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:11 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:11:14.414: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc001a71600, 0x70bd1d0, 0x19, 0xc003d1e618, 0x4, 0xc003c9cc00)
... skipping 71 lines ...
Sep 15 04:15:51.531: INFO: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b found and phase=Bound (4m37.078385873s)
Sep 15 04:15:56.567: INFO: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b found and phase=Bound (4m42.11463622s)
Sep 15 04:16:01.603: INFO: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b found and phase=Bound (4m47.150419755s)
Sep 15 04:16:06.639: INFO: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b found and phase=Bound (4m52.186379523s)
Sep 15 04:16:11.676: INFO: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b found and phase=Bound (4m57.223247219s)
STEP: Deleting sc
Sep 15 04:16:16.714: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b not deleted by dynamic provisioner: PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-235b4267-dc49-41a4-b1e0-6b418fe6250b still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003abc740)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc0042aa5a0, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc003f5ca00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc0042aa5a0, 0x4c, 0xc003ec4c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc004394080, 0x37, 0xc00410cfe0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003ec4dc0, 0x7904368, 0xc004112430, 0x1, 0x0, 0x0, 0x0, 0xc004112430)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc003ec4dc0, 0x7904368, 0xc004112430, 0x0, 0x0, 0x0, 0xc000cb0000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc004112410, 0x6323060, 0xc004112420, 0x0, 0x0, 0x0)
... skipping 140 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":12,"completed":0,"skipped":53,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 36 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":16,"completed":1,"skipped":122,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:16:58.075: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 78 lines ...
Sep 15 04:16:58.483: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:16:58.522: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comtx7gj] to have phase Bound
Sep 15 04:16:58.557: INFO: PersistentVolumeClaim test.csi.azure.comtx7gj found but phase is Pending instead of Bound.
Sep 15 04:17:00.595: INFO: PersistentVolumeClaim test.csi.azure.comtx7gj found and phase=Bound (2.072770319s)
STEP: Creating pod pod-subpath-test-dynamicpv-kf9b
STEP: Creating a pod to test atomic-volume-subpath
Sep 15 04:17:00.704: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-kf9b" in namespace "provisioning-4918" to be "Succeeded or Failed"
Sep 15 04:17:00.746: INFO: Pod "pod-subpath-test-dynamicpv-kf9b": Phase="Pending", Reason="", readiness=false. Elapsed: 41.891489ms
Sep 15 04:17:02.783: INFO: Pod "pod-subpath-test-dynamicpv-kf9b": Phase="Failed", Reason="", readiness=false. Elapsed: 2.078794517s
Sep 15 04:17:02.860: INFO: Output of node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-kf9b" container "init-volume-dynamicpv-kf9b": /bin/sh: can't create /test-volume/provisioning-4918: Permission denied

Sep 15 04:17:02.896: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-kf9b" container "test-container-subpath-dynamicpv-kf9b": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-kf9b)
STEP: delete the pod
Sep 15 04:17:02.939: INFO: Waiting for pod pod-subpath-test-dynamicpv-kf9b to disappear
Sep 15 04:17:02.976: INFO: Pod pod-subpath-test-dynamicpv-kf9b no longer exists
Sep 15 04:17:02.977: FAIL: Unexpected error:
    <*errors.errorString | 0xc0047642d0>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-kf9b\" success: pod \"pod-subpath-test-dynamicpv-kf9b\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-kf9b]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:17:00 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-kf9b State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:02 +0000 UTC,FinishedAt:2021-09-15 04:17:02 +0000 UTC,ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-kf9b State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc002a5580b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-kf9b" success: pod "pod-subpath-test-dynamicpv-kf9b" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-kf9b]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:17:00 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-kf9b State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:02 +0000 UTC,FinishedAt:2021-09-15 04:17:02 +0000 UTC,ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-kf9b State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc002a5580b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc00406c580, 0x70a786b, 0x15, 0xc003852400, 0x0, 0xc001e090c0, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 29 lines ...
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:16:58 +0000 UTC - event for test.csi.azure.comtx7gj: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-0b943579-82e4-4875-ada5-438b05f84c70
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:16:58 +0000 UTC - event for test.csi.azure.comtx7gj: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-4918/test.csi.azure.comtx7gj"
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:17:01 +0000 UTC - event for pod-subpath-test-dynamicpv-kf9b: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container init-volume-dynamicpv-kf9b
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:17:01 +0000 UTC - event for pod-subpath-test-dynamicpv-kf9b: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:17:01 +0000 UTC - event for pod-subpath-test-dynamicpv-kf9b: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 194.342811ms
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:17:02 +0000 UTC - event for pod-subpath-test-dynamicpv-kf9b: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container init-volume-dynamicpv-kf9b
Sep 15 04:17:08.203: INFO: At 2021-09-15 04:17:02 +0000 UTC - event for pod-subpath-test-dynamicpv-kf9b: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-942hp" : object "provisioning-4918"/"kube-root-ca.crt" not registered
Sep 15 04:17:08.239: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:17:08.239: INFO: 
Sep 15 04:17:08.308: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:17:08.344: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 3188 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:17:08.345: INFO: 
... skipping 97 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support file as subpath [LinuxOnly] [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230

    Sep 15 04:17:02.977: Unexpected error:
        <*errors.errorString | 0xc0047642d0>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-kf9b\" success: pod \"pod-subpath-test-dynamicpv-kf9b\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-kf9b]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:17:00 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-kf9b State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:02 +0000 UTC,FinishedAt:2021-09-15 04:17:02 +0000 UTC,ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-kf9b State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc002a5580b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-kf9b" success: pod "pod-subpath-test-dynamicpv-kf9b" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-kf9b]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-kf9b]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:00 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.68 PodIPs:[{IP:10.240.0.68}] StartTime:2021-09-15 04:17:00 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-kf9b State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:02 +0000 UTC,FinishedAt:2021-09-15 04:17:02 +0000 UTC,ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://5626b4d550166b5f00787839916ce26805ab6cbc605cf41ac83a6e5753c074de Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-kf9b State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc002a5580b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":16,"completed":1,"skipped":320,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  
  should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248
... skipping 14 lines ...
STEP: creating a claim
Sep 15 04:17:09.607: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:17:09.648: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comsfcdk] to have phase Bound
Sep 15 04:17:09.683: INFO: PersistentVolumeClaim test.csi.azure.comsfcdk found but phase is Pending instead of Bound.
Sep 15 04:17:11.719: INFO: PersistentVolumeClaim test.csi.azure.comsfcdk found and phase=Bound (2.071150782s)
STEP: [init] starting a pod to use the claim
Sep 15 04:17:11.827: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-jrgqm" in namespace "snapshotting-4376" to be "Succeeded or Failed"
Sep 15 04:17:11.862: INFO: Pod "pvc-snapshottable-tester-jrgqm": Phase="Pending", Reason="", readiness=false. Elapsed: 34.930125ms
Sep 15 04:17:13.899: INFO: Pod "pvc-snapshottable-tester-jrgqm": Phase="Failed", Reason="", readiness=false. Elapsed: 2.07221644s
Sep 15 04:17:13.900: FAIL: Unexpected error:
    <*errors.errorString | 0xc002fc1bd0>: {
        s: "pod \"pvc-snapshottable-tester-jrgqm\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.75 PodIPs:[{IP:10.240.0.75}] StartTime:2021-09-15 04:17:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:12 +0000 UTC,FinishedAt:2021-09-15 04:17:12 +0000 UTC,ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909 Started:0xc002fff900}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    pod "pvc-snapshottable-tester-jrgqm" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.75 PodIPs:[{IP:10.240.0.75}] StartTime:2021-09-15 04:17:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:12 +0000 UTC,FinishedAt:2021-09-15 04:17:12 +0000 UTC,ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909 Started:0xc002fff900}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.RunInPodWithVolume(0x79ab668, 0xc002ff1e40, 0xc003af5030, 0xc003173998, 0x11, 0xc003173968, 0x17, 0x70bc087, 0x18, 0xc001d32180, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699 +0x1cb
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.1()
... skipping 29 lines ...
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:09 +0000 UTC - event for test.csi.azure.comsfcdk: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-f9138ecb-5036-432d-983d-e3f40cd07723
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:09 +0000 UTC - event for test.csi.azure.comsfcdk: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:12 +0000 UTC - event for pvc-snapshottable-tester-jrgqm: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container volume-tester
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:12 +0000 UTC - event for pvc-snapshottable-tester-jrgqm: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container volume-tester
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:12 +0000 UTC - event for pvc-snapshottable-tester-jrgqm: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:12 +0000 UTC - event for pvc-snapshottable-tester-jrgqm: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 223.722136ms
Sep 15 04:17:19.201: INFO: At 2021-09-15 04:17:13 +0000 UTC - event for pvc-snapshottable-tester-jrgqm: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-6c5b9" : object "snapshotting-4376"/"kube-root-ca.crt" not registered
Sep 15 04:17:19.236: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:17:19.236: INFO: 
Sep 15 04:17:19.306: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:17:19.342: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 3188 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:15:27 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:17:19.342: INFO: 
... skipping 101 lines ...
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108
       [BeforeEach]
      /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:230
        should check snapshot fields, check restore correctly works after modifying source data, check deletion
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248

        Sep 15 04:17:13.900: Unexpected error:
            <*errors.errorString | 0xc002fc1bd0>: {
                s: "pod \"pvc-snapshottable-tester-jrgqm\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.75 PodIPs:[{IP:10.240.0.75}] StartTime:2021-09-15 04:17:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:12 +0000 UTC,FinishedAt:2021-09-15 04:17:12 +0000 UTC,ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909 Started:0xc002fff900}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
            }
            pod "pvc-snapshottable-tester-jrgqm" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:17:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.75 PodIPs:[{IP:10.240.0.75}] StartTime:2021-09-15 04:17:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:17:12 +0000 UTC,FinishedAt:2021-09-15 04:17:12 +0000 UTC,ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://ac0d1032f7550039ebc7af0ddad429bd261fed7c79d3d77c9a82ab26b1a6b909 Started:0xc002fff900}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
        occurred

        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","total":16,"completed":1,"skipped":401,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:17:20.270: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 72 lines ...
Sep 15 04:12:24.445: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1112 PodName:pod-45506463-a249-48aa-82db-46e85d611028 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:12:24.445: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:12:24.848: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1112 PodName:pod-45506463-a249-48aa-82db-46e85d611028 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:12:24.848: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:12:25.201: INFO: pod fsgroupchangepolicy-1112/pod-45506463-a249-48aa-82db-46e85d611028 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:12 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:12:25.201: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:12 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:12:25.201: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc0023c7a20, 0x70bd1d0, 0x19, 0xc003d5fa68, 0x4, 0xc00055ac00)
... skipping 71 lines ...
Sep 15 04:17:02.234: INFO: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa found and phase=Bound (4m36.993189045s)
Sep 15 04:17:07.269: INFO: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa found and phase=Bound (4m42.027893905s)
Sep 15 04:17:12.304: INFO: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa found and phase=Bound (4m47.062894556s)
Sep 15 04:17:17.339: INFO: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa found and phase=Bound (4m52.097974077s)
Sep 15 04:17:22.374: INFO: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa found and phase=Bound (4m57.132856214s)
STEP: Deleting sc
Sep 15 04:17:27.410: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa not deleted by dynamic provisioner: PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-ac393d4f-ea66-4304-ac07-8fc4ae5127fa still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003e233c0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc002ea4d20, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc0015a6a00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc002ea4d20, 0x4c, 0xc002ac8c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc00359ea00, 0x37, 0xc0028bc840, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc002ac8dc0, 0x7904368, 0xc0009fa110, 0x1, 0x0, 0x0, 0x0, 0xc0009fa110)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc002ac8dc0, 0x7904368, 0xc0009fa110, 0x0, 0x0, 0x0, 0xc000092c00)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc0009fa0f0, 0x6323060, 0xc0009fa100, 0x0, 0x0, 0x0)
... skipping 138 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":14,"completed":2,"skipped":391,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand 
  should resize volume when PVC is edited while pod is using it
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246
... skipping 42 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should resize volume when PVC is edited while pod is using it
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it","total":14,"completed":3,"skipped":453,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 30 lines ...
Sep 15 04:12:39.225: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-6278 PodName:pod-b8e89d9c-14f2-4309-9e0a-35f9d702ac78 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:12:39.225: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:12:39.632: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-6278 PodName:pod-b8e89d9c-14f2-4309-9e0a-35f9d702ac78 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:12:39.632: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:12:40.040: INFO: pod fsgroupchangepolicy-6278/pod-b8e89d9c-14f2-4309-9e0a-35f9d702ac78 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:12 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:12:40.040: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:12 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:12:40.041: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc003aa2dc0, 0x70bd1d0, 0x19, 0xc0039d3478, 0x4, 0xc000305400)
... skipping 71 lines ...
Sep 15 04:17:17.169: INFO: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 found and phase=Bound (4m37.091867905s)
Sep 15 04:17:22.208: INFO: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 found and phase=Bound (4m42.130980212s)
Sep 15 04:17:27.248: INFO: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 found and phase=Bound (4m47.17080836s)
Sep 15 04:17:32.286: INFO: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 found and phase=Bound (4m52.20901141s)
Sep 15 04:17:37.328: INFO: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 found and phase=Bound (4m57.250626389s)
STEP: Deleting sc
Sep 15 04:17:42.370: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 not deleted by dynamic provisioner: PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-5ffb3e89-9767-4dfe-b58c-e55605756889 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc001ce77c0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc002beb270, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc0023a0f00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc002beb270, 0x4c, 0xc0037e6c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc002705f00, 0x37, 0xc003535580, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc0037e6dc0, 0x7904368, 0xc002103220, 0x1, 0x0, 0x0, 0x0, 0xc002103220)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc0037e6dc0, 0x7904368, 0xc002103220, 0x0, 0x0, 0x0, 0xc001c38000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc002103200, 0x6323060, 0xc002103210, 0x0, 0x0, 0x0)
... skipping 136 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents","total":25,"completed":0,"skipped":560,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents"]}

SSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:17:43.552: INFO: Driver test.csi.azure.com doesn't support Block -- skipping
... skipping 192 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "test.csi.azure.com" does not support topology - skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 51 lines ...
Sep 15 04:17:20.612: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:17:20.651: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comlhl66] to have phase Bound
Sep 15 04:17:20.692: INFO: PersistentVolumeClaim test.csi.azure.comlhl66 found but phase is Pending instead of Bound.
Sep 15 04:17:22.729: INFO: PersistentVolumeClaim test.csi.azure.comlhl66 found and phase=Bound (2.078669117s)
STEP: Expanding non-expandable pvc
Sep 15 04:17:22.800: INFO: currentPvcSize {{5368709120 0} {<nil>} 5Gi BinarySI}, newSize {{6442450944 0} {<nil>}  BinarySI}
Sep 15 04:17:22.871: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:24.946: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:26.946: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:28.943: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:30.944: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:32.945: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:34.947: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:36.945: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:38.944: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:40.945: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:42.944: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:44.945: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:46.954: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:48.944: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:50.947: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:52.945: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 04:17:53.019: INFO: Error updating pvc test.csi.azure.comlhl66: persistentvolumeclaims "test.csi.azure.comlhl66" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
STEP: Deleting pvc
Sep 15 04:17:53.019: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comlhl66"
Sep 15 04:17:53.055: INFO: Waiting up to 5m0s for PersistentVolume pvc-6e1c1e15-62c6-4614-a82b-1a87f350a93c to get deleted
Sep 15 04:17:53.090: INFO: PersistentVolume pvc-6e1c1e15-62c6-4614-a82b-1a87f350a93c found and phase=Released (35.18538ms)
Sep 15 04:17:58.129: INFO: PersistentVolume pvc-6e1c1e15-62c6-4614-a82b-1a87f350a93c was removed
STEP: Deleting sc
... skipping 8 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not allow expansion of pvcs without AllowVolumeExpansion property
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:157
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property","total":16,"completed":2,"skipped":531,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:17:58.334: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 51 lines ...
Sep 15 04:16:25.673: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-101 PodName:pod-0e2a2ed5-0ddf-442b-9714-a7dcf809945e ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:16:25.673: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:16:26.156: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-101 PodName:pod-0e2a2ed5-0ddf-442b-9714-a7dcf809945e ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:16:26.156: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:16:26.957: INFO: pod fsgroupchangepolicy-101/pod-0e2a2ed5-0ddf-442b-9714-a7dcf809945e exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:16 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:16:26.957: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:16 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:16:26.957: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc001a71600, 0x70bd1d0, 0x19, 0xc003f2f428, 0x4, 0xc003775400)
... skipping 71 lines ...
Sep 15 04:21:04.030: INFO: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 found and phase=Bound (4m37.033867504s)
Sep 15 04:21:09.065: INFO: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 found and phase=Bound (4m42.069201373s)
Sep 15 04:21:14.104: INFO: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 found and phase=Bound (4m47.10844199s)
Sep 15 04:21:19.139: INFO: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 found and phase=Bound (4m52.143174233s)
Sep 15 04:21:24.178: INFO: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 found and phase=Bound (4m57.182079934s)
STEP: Deleting sc
Sep 15 04:21:29.218: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 not deleted by dynamic provisioner: PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-7b18a787-db13-4918-ab0f-6c317879cf73 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003c81340)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc003c99950, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc003cf3400, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc003c99950, 0x4c, 0xc003ec4c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc0041ff280, 0x37, 0xc0042ae7c0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003ec4dc0, 0x7904368, 0xc003ccb5a0, 0x1, 0x0, 0x0, 0x0, 0xc003ccb5a0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc003ec4dc0, 0x7904368, 0xc003ccb5a0, 0x0, 0x0, 0x0, 0xc00030d000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc003ccb580, 0x6323060, 0xc003ccb590, 0x0, 0x0, 0x0)
... skipping 141 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":12,"completed":0,"skipped":93,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)(allowExpansion)][Feature:Windows] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:21:30.587: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 139 lines ...
Sep 15 04:17:42.914: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:17:42.952: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com82jdf] to have phase Bound
Sep 15 04:17:42.986: INFO: PersistentVolumeClaim test.csi.azure.com82jdf found but phase is Pending instead of Bound.
Sep 15 04:17:45.021: INFO: PersistentVolumeClaim test.csi.azure.com82jdf found and phase=Bound (2.069065601s)
STEP: Creating pod pod-subpath-test-dynamicpv-zqq7
STEP: Creating a pod to test subpath
Sep 15 04:17:45.128: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-zqq7" in namespace "provisioning-9140" to be "Succeeded or Failed"
Sep 15 04:17:45.165: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 37.335547ms
Sep 15 04:17:47.201: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.073259823s
Sep 15 04:17:49.236: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.108658126s
Sep 15 04:17:51.273: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.14525665s
Sep 15 04:17:53.308: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.18049076s
Sep 15 04:17:55.344: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.215930667s
... skipping 136 lines ...
Sep 15 04:22:34.389: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m49.261020692s
Sep 15 04:22:36.424: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m51.296467898s
Sep 15 04:22:38.461: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m53.33293354s
Sep 15 04:22:40.496: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m55.368537834s
Sep 15 04:22:42.532: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m57.404062004s
Sep 15 04:22:44.568: INFO: Pod "pod-subpath-test-dynamicpv-zqq7": Phase="Pending", Reason="", readiness=false. Elapsed: 4m59.440047173s
Sep 15 04:22:46.639: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-zqq7" container "test-init-subpath-dynamicpv-zqq7": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-zqq7)
Sep 15 04:22:46.675: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-zqq7" container "test-container-subpath-dynamicpv-zqq7": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-zqq7)
Sep 15 04:22:46.711: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-zqq7" container "test-container-volume-dynamicpv-zqq7": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-zqq7)
STEP: delete the pod
Sep 15 04:22:46.747: INFO: Waiting for pod pod-subpath-test-dynamicpv-zqq7 to disappear
Sep 15 04:22:46.781: INFO: Pod pod-subpath-test-dynamicpv-zqq7 still exists
Sep 15 04:22:48.781: INFO: Waiting for pod pod-subpath-test-dynamicpv-zqq7 to disappear
Sep 15 04:22:48.816: INFO: Pod pod-subpath-test-dynamicpv-zqq7 no longer exists
Sep 15 04:22:48.816: FAIL: Unexpected error:
    <*errors.errorString | 0xc0026b5010>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-zqq7\" success: Gave up after waiting 5m0s for pod \"pod-subpath-test-dynamicpv-zqq7\" to be \"Succeeded or Failed\"",
    }
    expected pod "pod-subpath-test-dynamicpv-zqq7" success: Gave up after waiting 5m0s for pod "pod-subpath-test-dynamicpv-zqq7" to be "Succeeded or Failed"
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc0018734a0, 0x706a80a, 0x7, 0xc003917000, 0x1, 0xc002ac9108, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 25 lines ...
Sep 15 04:22:54.032: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: { } Scheduled: Successfully assigned provisioning-9140/pod-subpath-test-dynamicpv-zqq7 to k8s-agentpool1-30741368-vmss000001
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:42 +0000 UTC - event for test.csi.azure.com82jdf: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:42 +0000 UTC - event for test.csi.azure.com82jdf: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-9140/test.csi.azure.com82jdf"
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:43 +0000 UTC - event for test.csi.azure.com82jdf: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-8b24b8be-710f-4cd6-b78c-5d6835855886
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:45 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.32"
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:48 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 2.796660301s
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:48 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Failed: Error: failed to prepare subPath for volumeMount "test-volume" of container "test-init-subpath-dynamicpv-zqq7"
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:17:49 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 157.62389ms
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:18:03 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 282.859188ms
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:18:17 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 244.747421ms
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:18:32 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 172.176938ms
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:18:47 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 165.473621ms
Sep 15 04:22:54.032: INFO: At 2021-09-15 04:18:59 +0000 UTC - event for pod-subpath-test-dynamicpv-zqq7: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 197.628486ms
... skipping 103 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support non-existent path [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194

    Sep 15 04:22:48.816: Unexpected error:
        <*errors.errorString | 0xc0026b5010>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-zqq7\" success: Gave up after waiting 5m0s for pod \"pod-subpath-test-dynamicpv-zqq7\" to be \"Succeeded or Failed\"",
        }
        expected pod "pod-subpath-test-dynamicpv-zqq7" success: Gave up after waiting 5m0s for pod "pod-subpath-test-dynamicpv-zqq7" to be "Succeeded or Failed"
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":14,"completed":3,"skipped":478,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should be able to unmount after the subpath directory is deleted [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
... skipping 14 lines ...
STEP: creating a claim
Sep 15 04:17:58.653: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:17:58.691: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comhwskq] to have phase Bound
Sep 15 04:17:58.727: INFO: PersistentVolumeClaim test.csi.azure.comhwskq found but phase is Pending instead of Bound.
Sep 15 04:18:00.763: INFO: PersistentVolumeClaim test.csi.azure.comhwskq found and phase=Bound (2.072155965s)
STEP: Creating pod pod-subpath-test-dynamicpv-bjlt
Sep 15 04:23:00.983: FAIL: while waiting for pod to be running
Unexpected error:
    <*errors.errorString | 0xc000250250>: {
        s: "timed out waiting for the condition",
    }
    timed out waiting for the condition
occurred

... skipping 27 lines ...
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:17:58 +0000 UTC - event for test.csi.azure.comhwskq: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:17:58 +0000 UTC - event for test.csi.azure.comhwskq: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-1470/test.csi.azure.comhwskq"
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:17:58 +0000 UTC - event for test.csi.azure.comhwskq: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-ca85cfa2-7a00-40bf-bf3c-ac0bebed250c
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 205.785527ms
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 150.044549ms
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Created: Created container test-container-volume-dynamicpv-bjlt
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Failed: Error: failed to prepare subPath for volumeMount "test-volume" of container "test-container-subpath-dynamicpv-bjlt"
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/agnhost:2.32"
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:01 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:02 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Started: Started container test-container-volume-dynamicpv-bjlt
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:02 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 160.3948ms
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:03 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 167.411735ms
Sep 15 04:23:08.326: INFO: At 2021-09-15 04:18:15 +0000 UTC - event for pod-subpath-test-dynamicpv-bjlt: {kubelet k8s-agentpool1-30741368-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/agnhost:2.32" in 172.131469ms
... skipping 106 lines ...
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should be able to unmount after the subpath directory is deleted [LinuxOnly] [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445

    Sep 15 04:23:00.983: while waiting for pod to be running
    Unexpected error:
        <*errors.errorString | 0xc000250250>: {
            s: "timed out waiting for the condition",
        }
        timed out waiting for the condition
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:465
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":16,"completed":2,"skipped":688,"failed":4,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]"]}

SSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  
  should check snapshot fields, check restore correctly works after modifying source data, check deletion
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248
... skipping 14 lines ...
STEP: creating a claim
Sep 15 04:23:09.582: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:23:09.623: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com7pntz] to have phase Bound
Sep 15 04:23:09.658: INFO: PersistentVolumeClaim test.csi.azure.com7pntz found but phase is Pending instead of Bound.
Sep 15 04:23:11.696: INFO: PersistentVolumeClaim test.csi.azure.com7pntz found and phase=Bound (2.072314892s)
STEP: [init] starting a pod to use the claim
Sep 15 04:23:11.803: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-xbvcx" in namespace "snapshotting-4056" to be "Succeeded or Failed"
Sep 15 04:23:11.838: INFO: Pod "pvc-snapshottable-tester-xbvcx": Phase="Pending", Reason="", readiness=false. Elapsed: 35.176565ms
Sep 15 04:23:13.877: INFO: Pod "pvc-snapshottable-tester-xbvcx": Phase="Failed", Reason="", readiness=false. Elapsed: 2.074210326s
Sep 15 04:23:13.877: FAIL: Unexpected error:
    <*errors.errorString | 0xc0039acb60>: {
        s: "pod \"pvc-snapshottable-tester-xbvcx\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.70 PodIPs:[{IP:10.240.0.70}] StartTime:2021-09-15 04:23:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:12 +0000 UTC,FinishedAt:2021-09-15 04:23:12 +0000 UTC,ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3 Started:0xc0039093b0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    pod "pvc-snapshottable-tester-xbvcx" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.70 PodIPs:[{IP:10.240.0.70}] StartTime:2021-09-15 04:23:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:12 +0000 UTC,FinishedAt:2021-09-15 04:23:12 +0000 UTC,ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3 Started:0xc0039093b0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.RunInPodWithVolume(0x79ab668, 0xc00333a6e0, 0xc003af5260, 0xc00375cba0, 0x11, 0xc00375cb70, 0x17, 0x70bc087, 0x18, 0xc003a8ebc0, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699 +0x1cb
k8s.io/kubernetes/test/e2e/storage/testsuites.(*snapshottableTestSuite).DefineTests.func1.1()
... skipping 29 lines ...
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:09 +0000 UTC - event for test.csi.azure.com7pntz: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-564b2405-51b0-4fe7-89df-bd7001a547e4
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:09 +0000 UTC - event for test.csi.azure.com7pntz: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:12 +0000 UTC - event for pvc-snapshottable-tester-xbvcx: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container volume-tester
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:12 +0000 UTC - event for pvc-snapshottable-tester-xbvcx: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container volume-tester
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:12 +0000 UTC - event for pvc-snapshottable-tester-xbvcx: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:12 +0000 UTC - event for pvc-snapshottable-tester-xbvcx: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 179.831876ms
Sep 15 04:23:19.185: INFO: At 2021-09-15 04:23:13 +0000 UTC - event for pvc-snapshottable-tester-xbvcx: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-4hmwk" : object "snapshotting-4056"/"kube-root-ca.crt" not registered
Sep 15 04:23:19.221: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:23:19.221: INFO: 
Sep 15 04:23:19.290: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:23:19.325: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 4477 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:23:19.326: INFO: 
... skipping 99 lines ...
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108
       [BeforeEach]
      /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:230
        should check snapshot fields, check restore correctly works after modifying source data, check deletion
        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:248

        Sep 15 04:23:13.877: Unexpected error:
            <*errors.errorString | 0xc0039acb60>: {
                s: "pod \"pvc-snapshottable-tester-xbvcx\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.70 PodIPs:[{IP:10.240.0.70}] StartTime:2021-09-15 04:23:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:12 +0000 UTC,FinishedAt:2021-09-15 04:23:12 +0000 UTC,ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3 Started:0xc0039093b0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
            }
            pod "pvc-snapshottable-tester-xbvcx" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [volume-tester]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:11 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.70 PodIPs:[{IP:10.240.0.70}] StartTime:2021-09-15 04:23:11 +0000 UTC InitContainerStatuses:[] ContainerStatuses:[{Name:volume-tester State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:12 +0000 UTC,FinishedAt:2021-09-15 04:23:12 +0000 UTC,ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://6de38c33ba0c9c969236b8553c8bde4d43a0c78f85d32bf08b6591133c3b04f3 Started:0xc0039093b0}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
        occurred

        /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","total":16,"completed":2,"skipped":693,"failed":5,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 36 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":14,"completed":4,"skipped":749,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:23:35.467: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 36 lines ...
Sep 15 04:23:35.746: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 04:23:35.787: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comv25j8] to have phase Bound
Sep 15 04:23:35.821: INFO: PersistentVolumeClaim test.csi.azure.comv25j8 found but phase is Pending instead of Bound.
Sep 15 04:23:37.857: INFO: PersistentVolumeClaim test.csi.azure.comv25j8 found and phase=Bound (2.070422159s)
STEP: Creating pod pod-subpath-test-dynamicpv-pvcm
STEP: Creating a pod to test subpath
Sep 15 04:23:37.964: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-pvcm" in namespace "provisioning-6248" to be "Succeeded or Failed"
Sep 15 04:23:37.998: INFO: Pod "pod-subpath-test-dynamicpv-pvcm": Phase="Pending", Reason="", readiness=false. Elapsed: 34.584353ms
Sep 15 04:23:40.034: INFO: Pod "pod-subpath-test-dynamicpv-pvcm": Phase="Failed", Reason="", readiness=false. Elapsed: 2.070496846s
Sep 15 04:23:40.106: INFO: Output of node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-pvcm" container "init-volume-dynamicpv-pvcm": touch: /test-volume/provisioning-6248: Permission denied

Sep 15 04:23:40.143: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-pvcm" container "test-init-volume-dynamicpv-pvcm": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-pvcm)
Sep 15 04:23:40.179: INFO: Failed to get logs from node "k8s-agentpool1-30741368-vmss000001" pod "pod-subpath-test-dynamicpv-pvcm" container "test-container-subpath-dynamicpv-pvcm": the server rejected our request for an unknown reason (get pods pod-subpath-test-dynamicpv-pvcm)
STEP: delete the pod
Sep 15 04:23:40.227: INFO: Waiting for pod pod-subpath-test-dynamicpv-pvcm to disappear
Sep 15 04:23:40.261: INFO: Pod pod-subpath-test-dynamicpv-pvcm no longer exists
Sep 15 04:23:40.261: FAIL: Unexpected error:
    <*errors.errorString | 0xc0017b93b0>: {
        s: "expected pod \"pod-subpath-test-dynamicpv-pvcm\" success: pod \"pod-subpath-test-dynamicpv-pvcm\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-pvcm test-init-volume-dynamicpv-pvcm]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:23:37 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-pvcm State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:39 +0000 UTC,FinishedAt:2021-09-15 04:23:39 +0000 UTC,ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8 Started:<nil>} {Name:test-init-volume-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc004cffe7b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
    }
    expected pod "pod-subpath-test-dynamicpv-pvcm" success: pod "pod-subpath-test-dynamicpv-pvcm" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-pvcm test-init-volume-dynamicpv-pvcm]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:23:37 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-pvcm State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:39 +0000 UTC,FinishedAt:2021-09-15 04:23:39 +0000 UTC,ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8 Started:<nil>} {Name:test-init-volume-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc004cffe7b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework.(*Framework).testContainerOutputMatcher(0xc0018734a0, 0x706a80a, 0x7, 0xc0047c2800, 0x0, 0xc0043990d0, 0x1, 0x1, 0x72d74f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742 +0x1e5
k8s.io/kubernetes/test/e2e/framework.(*Framework).TestContainerOutput(...)
... skipping 26 lines ...
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:35 +0000 UTC - event for test.csi.azure.comv25j8: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:35 +0000 UTC - event for test.csi.azure.comv25j8: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } Provisioning: External provisioner is provisioning volume for claim "provisioning-6248/test.csi.azure.comv25j8"
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:36 +0000 UTC - event for test.csi.azure.comv25j8: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-c008c7e9-e172-45bf-81ac-0501c0fc3af4
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:38 +0000 UTC - event for pod-subpath-test-dynamicpv-pvcm: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:39 +0000 UTC - event for pod-subpath-test-dynamicpv-pvcm: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container init-volume-dynamicpv-pvcm
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:39 +0000 UTC - event for pod-subpath-test-dynamicpv-pvcm: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container init-volume-dynamicpv-pvcm
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:39 +0000 UTC - event for pod-subpath-test-dynamicpv-pvcm: {kubelet k8s-agentpool1-30741368-vmss000001} FailedMount: MountVolume.SetUp failed for volume "kube-api-access-mxzq4" : object "provisioning-6248"/"kube-root-ca.crt" not registered
Sep 15 04:23:45.477: INFO: At 2021-09-15 04:23:39 +0000 UTC - event for pod-subpath-test-dynamicpv-pvcm: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 192.487054ms
Sep 15 04:23:45.511: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:23:45.511: INFO: 
Sep 15 04:23:45.577: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:23:45.610: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 4477 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:20:29 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
... skipping 96 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly file specified in the volumeMount [LinuxOnly] [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380

    Sep 15 04:23:40.261: Unexpected error:
        <*errors.errorString | 0xc0017b93b0>: {
            s: "expected pod \"pod-subpath-test-dynamicpv-pvcm\" success: pod \"pod-subpath-test-dynamicpv-pvcm\" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-pvcm test-init-volume-dynamicpv-pvcm]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:23:37 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-pvcm State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:39 +0000 UTC,FinishedAt:2021-09-15 04:23:39 +0000 UTC,ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8 Started:<nil>} {Name:test-init-volume-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc004cffe7b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}",
        }
        expected pod "pod-subpath-test-dynamicpv-pvcm" success: pod "pod-subpath-test-dynamicpv-pvcm" failed with status: {Phase:Failed Conditions:[{Type:Initialized Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotInitialized Message:containers with incomplete status: [init-volume-dynamicpv-pvcm test-init-volume-dynamicpv-pvcm]} {Type:Ready Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:ContainersReady Status:False LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason:ContainersNotReady Message:containers with unready status: [test-container-subpath-dynamicpv-pvcm]} {Type:PodScheduled Status:True LastProbeTime:0001-01-01 00:00:00 +0000 UTC LastTransitionTime:2021-09-15 04:23:37 +0000 UTC Reason: Message:}] Message: Reason: NominatedNodeName: HostIP:10.240.0.65 PodIP:10.240.0.66 PodIPs:[{IP:10.240.0.66}] StartTime:2021-09-15 04:23:37 +0000 UTC InitContainerStatuses:[{Name:init-volume-dynamicpv-pvcm State:{Waiting:nil Running:nil Terminated:&ContainerStateTerminated{ExitCode:1,Signal:0,Reason:Error,Message:,StartedAt:2021-09-15 04:23:39 +0000 UTC,FinishedAt:2021-09-15 04:23:39 +0000 UTC,ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8,}} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/busybox:1.29-1 ImageID:k8s.gcr.io/e2e-test-images/busybox@sha256:39e1e963e5310e9c313bad51523be012ede7b35bb9316517d19089a010356592 ContainerID:containerd://a70c23746541d5c3fb146243f2783f1934c994358f51fb299e2267b61cfb45a8 Started:<nil>} {Name:test-init-volume-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:<nil>}] ContainerStatuses:[{Name:test-container-subpath-dynamicpv-pvcm State:{Waiting:&ContainerStateWaiting{Reason:PodInitializing,Message:,} Running:nil Terminated:nil} LastTerminationState:{Waiting:nil Running:nil Terminated:nil} Ready:false RestartCount:0 Image:k8s.gcr.io/e2e-test-images/agnhost:2.32 ImageID: ContainerID: Started:0xc004cffe7b}] QOSClass:BestEffort EphemeralContainerStatuses:[]}
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":14,"completed":4,"skipped":904,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]"]}

SSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:23:46.532: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 88 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":16,"completed":3,"skipped":787,"failed":5,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller  should check snapshot fields, check restore correctly works after modifying source data, check deletion"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:24:01.854: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 51 lines ...
Sep 15 04:21:39.562: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1138 PodName:pod-019fe4f5-1ac9-4bac-9912-650ddea03a40 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:21:39.562: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:21:39.997: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1138 PodName:pod-019fe4f5-1ac9-4bac-9912-650ddea03a40 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 04:21:39.998: INFO: >>> kubeConfig: /root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json
Sep 15 04:21:40.317: INFO: pod fsgroupchangepolicy-1138/pod-019fe4f5-1ac9-4bac-9912-650ddea03a40 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 04:21 /mnt/volume1/subdir/file2, stderr: 
Sep 15 04:21:40.317: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 04:21 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 04:21:40.317: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc001a71600, 0x70bd1d0, 0x19, 0xc0025bf488, 0x4, 0xc003f3d400)
... skipping 71 lines ...
Sep 15 04:26:17.352: INFO: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d found and phase=Bound (4m36.99910349s)
Sep 15 04:26:22.391: INFO: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d found and phase=Bound (4m42.03799673s)
Sep 15 04:26:27.428: INFO: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d found and phase=Bound (4m47.074227789s)
Sep 15 04:26:32.464: INFO: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d found and phase=Bound (4m52.110805453s)
Sep 15 04:26:37.503: INFO: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d found and phase=Bound (4m57.149468909s)
STEP: Deleting sc
Sep 15 04:26:42.540: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d not deleted by dynamic provisioner: PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-7d23a087-a3ec-434d-b13f-1ab641bf340d still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc00200a440)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc002e4a140, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc001efa500, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc002e4a140, 0x4c, 0xc003ec4c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc004394400, 0x37, 0xc0037675d0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003ec4dc0, 0x7904368, 0xc00420f8c0, 0x1, 0x0, 0x0, 0x0, 0xc00420f8c0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc003ec4dc0, 0x7904368, 0xc00420f8c0, 0x0, 0x0, 0x0, 0xc000cb0800)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc00420f8a0, 0x6323060, 0xc00420f8b0, 0x0, 0x0, 0x0)
... skipping 134 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":12,"completed":0,"skipped":797,"failed":3,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning 
  should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201
... skipping 14 lines ...
STEP: Creating a StorageClass
STEP: [Initialize dataSource]creating a initClaim
STEP: starting external-injector
STEP: Writing text file contents in the container.
Sep 15 04:26:48.254: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=provisioning-1714 exec external-injector --namespace=provisioning-1714 -- /bin/sh -c echo 'Hello from namespace provisioning-1714' > /opt/0/index.html'
Sep 15 04:26:49.234: INFO: rc: 1
Sep 15 04:26:49.234: FAIL: failed: writing the contents: 
Unexpected error:
    <exec.CodeExitError>: {
        Err: {
            s: "error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=provisioning-1714 exec external-injector --namespace=provisioning-1714 -- /bin/sh -c echo 'Hello from namespace provisioning-1714' > /opt/0/index.html:\nCommand stdout:\n\nstderr:\n/bin/sh: can't create /opt/0/index.html: Permission denied\ncommand terminated with exit code 1\n\nerror:\nexit status 1",
        },
        Code: 1,
    }
    error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=provisioning-1714 exec external-injector --namespace=provisioning-1714 -- /bin/sh -c echo 'Hello from namespace provisioning-1714' > /opt/0/index.html:
    Command stdout:
    
    stderr:
    /bin/sh: can't create /opt/0/index.html: Permission denied
    command terminated with exit code 1
    
    error:
    exit status 1
occurred

Full Stack Trace
k8s.io/kubernetes/test/e2e/framework/volume.InjectContent(0xc0043bf340, 0xc00210f800, 0x11, 0x706c7cd, 0x8, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560 +0x6e5
... skipping 25 lines ...
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:44 +0000 UTC - event for pvc-rs8j2: {test.csi.azure.com_k8s-agentpool1-30741368-vmss000000_7bf4f667-2b5d-4ebf-96de-194dbfff60b2 } ProvisioningSucceeded: Successfully provisioned volume pvc-18129cc7-8be6-47b8-8b5d-8a4aab9d10d2
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:46 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} Started: Started container external-injector
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:46 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-1"
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:46 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-1" in 238.748392ms
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:46 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} Created: Created container external-injector
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:49 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} Killing: Stopping container external-injector
Sep 15 04:26:51.384: INFO: At 2021-09-15 04:26:51 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-30741368-vmss000001} FailedKillPod: error killing pod: failed to "KillContainer" for "external-injector" with KillContainerError: "rpc error: code = NotFound desc = an error occurred when try to find container \"7e282db7051bac6219314d0085c9ebab4559f9dead116165eceb6ad73d475502\": not found"
Sep 15 04:26:51.420: INFO: POD  NODE  PHASE  GRACE  CONDITIONS
Sep 15 04:26:51.420: INFO: 
Sep 15 04:26:51.489: INFO: 
Logging node info for node k8s-agentpool1-30741368-vmss000000
Sep 15 04:26:51.525: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-30741368-vmss000000    dc5e34da-08bd-48ea-8f2a-e14b977684c4 5673 0 2021-09-15 04:07:24 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_DS2_v2 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:southcentralus failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-vt3x6905 kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-30741368-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_DS2_v2 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:southcentralus topology.kubernetes.io/zone:0] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-30741368-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] []  [{kubelet Update v1 2021-09-15 04:07:24 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2021-09-15 04:07:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {kube-controller-manager Update v1 2021-09-15 04:07:39 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-node-manager Update v1 2021-09-15 04:08:27 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2021-09-15 04:09:54 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}}},"f:status":{"f:allocatable":{"f:ephemeral-storage":{}},"f:capacity":{"f:ephemeral-storage":{}},"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-vt3x6905/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-30741368-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[],},Status:NodeStatus{Capacity:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{31036686336 0} {<nil>}  BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{7292157952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Allocatable:ResourceList{cpu: {{2 0} {<nil>} 2 DecimalSI},ephemeral-storage: {{27933017657 0} {<nil>} 27933017657 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{6505725952 0} {<nil>}  BinarySI},pods: {{30 0} {<nil>} 30 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2021-09-15 04:25:31 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2021-09-15 04:25:31 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2021-09-15 04:25:31 +0000 UTC,LastTransitionTime:2021-09-15 04:07:24 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2021-09-15 04:25:31 +0000 UTC,LastTransitionTime:2021-09-15 04:07:34 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.34,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-30741368-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:b40908517888403488d9cc7810cda9c3,SystemUUID:e55f07f6-ca3e-5641-8287-90b4c0d66637,BootID:be87c7cc-86b1-4b65-b682-dbffcf936c65,KernelVersion:5.4.0-1055-azure,OSImage:Ubuntu 18.04.5 LTS,ContainerRuntimeVersion:containerd://1.4.6+azure,KubeletVersion:v1.22.1,KubeProxyVersion:v1.22.1,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azurefile-csi:v0.6.0],SizeBytes:353728304,},ContainerImage{Names:[mcr.microsoft.com/k8s/csi/azuredisk-csi:v0.7.0],SizeBytes:291559170,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:179170363,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.20],SizeBytes:174498580,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.18.18-azs],SizeBytes:174486292,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.20],SizeBytes:163898150,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.18.18-azs],SizeBytes:163889958,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.22.0-beta.2],SizeBytes:129742803,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.21.3],SizeBytes:126846941,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:125497261,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.22.0-beta.2],SizeBytes:123267395,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.9],SizeBytes:123029309,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.20.6-azs],SizeBytes:122980152,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.21.3],SizeBytes:121055555,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.6-azs],SizeBytes:120336758,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.13],SizeBytes:120100674,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-apiserver:v1.19.10-azs],SizeBytes:120051522,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.10-azs],SizeBytes:119604311,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.20],SizeBytes:119182457,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.18.18-azs],SizeBytes:119178361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.9],SizeBytes:117573460,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.20.6-azs],SizeBytes:117516116,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.13],SizeBytes:112068436,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-controller-manager:v1.19.10-azs],SizeBytes:112019284,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.0-beta.2],SizeBytes:105488073,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:8d32a4cff601fc6a7d050712438ad9f69ec95c1d7b60002122f60bc32b4573fb mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.22.1],SizeBytes:105439354,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.3],SizeBytes:105111246,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:103333639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.21.0],SizeBytes:102205543,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.20.9],SizeBytes:101469290,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.19.13],SizeBytes:100736106,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.20],SizeBytes:97620755,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.18.18-azs],SizeBytes:97612561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v0.5.1],SizeBytes:90427783,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v0.5.1],SizeBytes:89727347,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:88473085,},ContainerImage{Names:[k8sprow.azurecr.io/azurefile-csi@sha256:2dacf05bf0d5c966be7526925f31fecb7a70912981809932a19e61094dfd1fe7 k8sprow.azurecr.io/azurefile-csi:e2e-f9d78960097e608792fee56a679d73ded8d9ddf9],SizeBytes:85752183,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:85735934,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:67349435,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.0],SizeBytes:64765008,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:58145788,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.5.0],SizeBytes:57913176,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v1.4.0],SizeBytes:55712600,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:55578071,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:e28e996e03519565907d7158e0e1a6613cec76220fa8625e0ebfc1aa1c3bacb2 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:55475219,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:55107043,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.22.0-beta.2],SizeBytes:53955392,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:bb61653b29713588c280999cbb25b472fef8794623164fd9177c270da1121db0 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:52839429,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:bddf19be25644c7176e8c501465dbad07d032288334f135877b984660b0d3ee6 mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:52008143,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-scheduler:v1.21.3],SizeBytes:51873602,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},}
Sep 15 04:26:51.526: INFO: 
... skipping 91 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [Measurement]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:201

    Sep 15 04:26:49.234: failed: writing the contents: 
    Unexpected error:
        <exec.CodeExitError>: {
            Err: {
                s: "error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=provisioning-1714 exec external-injector --namespace=provisioning-1714 -- /bin/sh -c echo 'Hello from namespace provisioning-1714' > /opt/0/index.html:\nCommand stdout:\n\nstderr:\n/bin/sh: can't create /opt/0/index.html: Permission denied\ncommand terminated with exit code 1\n\nerror:\nexit status 1",
            },
            Code: 1,
        }
        error running /usr/local/bin/kubectl --server=https://kubetest-vt3x6905.southcentralus.cloudapp.azure.com --kubeconfig=/root/tmp1458307742/kubeconfig/kubeconfig.southcentralus.json --namespace=provisioning-1714 exec external-injector --namespace=provisioning-1714 -- /bin/sh -c echo 'Hello from namespace provisioning-1714' > /opt/0/index.html:
        Command stdout:
        
        stderr:
        /bin/sh: can't create /opt/0/index.html: Permission denied
        command terminated with exit code 1
        
        error:
        exit status 1
    occurred

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":12,"completed":0,"skipped":810,"failed":4,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]"]}

SSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 04:26:52.519: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 57 lines ...
Sep 15 04:26:52.699: INFO: Running AfterSuite actions on node 1



Summarizing 23 Failures:

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support existing directory 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support readOnly directory specified in the volumeMount 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support existing single file [LinuxOnly] 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support existing directories when readOnly specified in the volumeSource 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes [Measurement] should store data 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes [Measurement] should allow exec of files on the volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller [BeforeEach]  should check snapshot fields, check restore correctly works after modifying source data, check deletion 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [Measurement] should support two pods which share the same volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:636

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller [BeforeEach]  should check snapshot fields, check restore correctly works after modifying source data, check deletion 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral [Measurement] should support two pods which share the same volume 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:636

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support file as subpath [LinuxOnly] 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller [BeforeEach]  should check snapshot fields, check restore correctly works after modifying source data, check deletion 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support non-existent path 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should be able to unmount after the subpath directory is deleted [LinuxOnly] 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:465

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller [BeforeEach]  should check snapshot fields, check restore correctly works after modifying source data, check deletion 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/provisioning.go:699

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath [Measurement] should support readOnly file specified in the volumeMount [LinuxOnly] 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/util.go:742

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning [Measurement] should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/volume/fixtures.go:560

Ran 32 of 6666 Specs in 1004.918 seconds
FAIL! -- 9 Passed | 23 Failed | 0 Pending | 6634 Skipped 

Ginkgo ran 1 suite in 16m48.441664781s
Test Suite Failed
+ print_logs
+ bash ./hack/verify-examples.sh linux
begin to create deployment examples ...
storageclass.storage.k8s.io/azurefile-csi created
Applying config "deploy/example/daemonset.yaml"
Waiting for daemon set "daemonset-azurefile" rollout to finish: 0 of 2 updated pods are available...
... skipping 68 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 04:10:02.070433       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 gc/go1.17.1 (amd64-linux) e2e-test
I0915 04:10:02.072638       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 04:10:02.093116       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 04:10:02.093135       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 04:10:02.093142       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 04:10:02.093166       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 04:10:02.094165       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 04:10:02.094197       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 04:10:02.094220       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 97 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 04:10:00.724670       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 gc/go1.17.1 (amd64-linux) e2e-test
I0915 04:10:00.725744       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 04:10:00.757974       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 04:10:00.757998       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 04:10:00.758026       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 04:10:00.758048       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 04:10:00.759672       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 04:10:00.759719       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 04:10:00.759743       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 641 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 04:09:53.545485       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 gc/go1.17.1 (amd64-linux) e2e-test
I0915 04:09:53.548124       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 04:09:53.571733       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 04:09:53.571768       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 04:09:53.571774       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 04:09:53.571791       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 04:09:53.573448       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 04:09:53.573509       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 04:09:53.573557       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 526 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 04:09:40.186764       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 gc/go1.17.1 (amd64-linux) e2e-test
I0915 04:09:40.187897       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 04:09:40.211796       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 04:09:40.211825       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 04:09:40.211832       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 04:09:40.211884       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 04:09:40.212565       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 04:09:40.212616       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 04:09:40.212643       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 65 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 04:09:50.732860       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-f9d78960097e608792fee56a679d73ded8d9ddf9 gc/go1.17.1 (amd64-linux) e2e-test
I0915 04:09:50.734058       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 04:09:50.765831       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 04:09:50.765874       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 04:09:50.765881       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 04:09:50.765903       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 04:09:50.766660       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 04:09:50.766697       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 04:09:50.766733       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 759 lines ...
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 7.61618432e+08
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes -1
make: *** [Makefile:85: e2e-test] Error 1
2021/09/15 04:27:46 process.go:155: Step 'make e2e-test' finished in 18m53.810811201s
2021/09/15 04:27:46 aksengine_helpers.go:425: downloading /root/tmp1458307742/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/09/15 04:27:46 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/09/15 04:27:46 process.go:153: Running: chmod +x /root/tmp1458307742/log-dump.sh
2021/09/15 04:27:46 process.go:155: Step 'chmod +x /root/tmp1458307742/log-dump.sh' finished in 1.042556ms
2021/09/15 04:27:46 aksengine_helpers.go:425: downloading /root/tmp1458307742/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml
... skipping 53 lines ...
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/09/15 04:28:15 process.go:155: Step 'bash -c /root/tmp1458307742/win-ci-logs-collector.sh kubetest-vt3x6905.southcentralus.cloudapp.azure.com /root/tmp1458307742 /root/.ssh/id_rsa' finished in 4.01353ms
2021/09/15 04:28:15 aksengine.go:1141: Deleting resource group: kubetest-vt3x6905.
2021/09/15 04:35:19 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml.
2021/09/15 04:35:19 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"
2021/09/15 04:35:19 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 219.102761ms
2021/09/15 04:35:19 main.go:331: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2]
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
Stopping Docker: dockerProgram process in pidfile '/var/run/docker-ssd.pid', 1 process(es), refused to die.
... skipping 3 lines ...