This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: feat: enable fsGroup for nfs
ResultFAILURE
Tests 1 failed / 11 succeeded
Started2021-09-15 08:35
Elapsed41m52s
Revision54260bc9ac503208031267214f41d4ef13bf264a
Refs 759
job-versionv1.23.0-alpha.2.11+4c014e5ca41fe9
kubetest-version
revisionv1.23.0-alpha.2.11+4c014e5ca41fe9

Test Failures


kubetest Test 28m17s

error during make e2e-test: exit status 2
				from junit_runner.xml

Filter through log files | View test history on testgrid


Show 11 Passed Tests

Error lines from build-log.txt

... skipping 223 lines ...

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 11248  100 11248    0     0   142k      0 --:--:-- --:--:-- --:--:--  144k
Downloading https://get.helm.sh/helm-v3.6.3-linux-amd64.tar.gz
Verifying checksum... Done.
Preparing to install helm into /usr/local/bin
helm installed into /usr/local/bin/helm
docker pull k8sprow.azurecr.io/azurefile-csi:e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 || make container-all push-manifest
Error response from daemon: manifest for k8sprow.azurecr.io/azurefile-csi:e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 not found: manifest unknown: manifest tagged by "e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636" is not found
make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azurefile-csi-driver'
CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.driverVersion=e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 -X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.gitCommit=bee2fd8c74b1c929e5a49796d211abca6c88e636 -X sigs.k8s.io/azurefile-csi-driver/pkg/azurefile.buildDate=2021-09-15T08:39:39Z -s -w -extldflags '-static'" -mod vendor -o _output/amd64/azurefileplugin.exe ./pkg/azurefileplugin
docker buildx rm container-builder || true
error: no builder "container-builder" found
docker buildx create --use --name=container-builder
container-builder
# enable qemu for arm64 build
# https://github.com/docker/buildx/issues/464#issuecomment-741507760
docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
Unable to find image 'tonistiigi/binfmt:latest' locally
... skipping 1738 lines ...
                    type: string
                type: object
                oneOf:
                - required: ["persistentVolumeClaimName"]
                - required: ["volumeSnapshotContentName"]
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 55 lines ...
                    type: string
                  volumeSnapshotContentName:
                    description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable.
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.'
                type: string
            required:
            - source
            type: object
          status:
            description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.
... skipping 2 lines ...
                description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.'
                type: string
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurrs during the snapshot creation. Upon success, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                type: string
                description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
        type: object
... skipping 239 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 99 lines ...
            description: status represents the current information of a snapshot.
            properties:
              creationTime:
                description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown.
                type: boolean
              restoreSize:
                description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress.
                type: string
            type: object
        required:
        - spec
        type: object
    served: true
... skipping 829 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=false'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: socket-dir
... skipping 872 lines ...
Sep 15 08:53:31.488: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found but phase is Pending instead of Bound.
Sep 15 08:53:33.540: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found but phase is Pending instead of Bound.
Sep 15 08:53:35.591: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found but phase is Pending instead of Bound.
Sep 15 08:53:37.643: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found but phase is Pending instead of Bound.
Sep 15 08:53:39.695: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found and phase=Bound (53.502033331s)
STEP: [init] starting a pod to use the claim
Sep 15 08:53:39.856: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-scz6d" in namespace "snapshotting-5465" to be "Succeeded or Failed"
Sep 15 08:53:39.908: INFO: Pod "pvc-snapshottable-tester-scz6d": Phase="Pending", Reason="", readiness=false. Elapsed: 51.98205ms
Sep 15 08:53:41.962: INFO: Pod "pvc-snapshottable-tester-scz6d": Phase="Pending", Reason="", readiness=false. Elapsed: 2.106046481s
Sep 15 08:53:44.015: INFO: Pod "pvc-snapshottable-tester-scz6d": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.158915159s
STEP: Saw pod success
Sep 15 08:53:44.015: INFO: Pod "pvc-snapshottable-tester-scz6d" satisfied condition "Succeeded or Failed"
Sep 15 08:53:44.316: INFO: Pod pvc-snapshottable-tester-scz6d has the following logs: 
Sep 15 08:53:44.316: INFO: Deleting pod "pvc-snapshottable-tester-scz6d" in namespace "snapshotting-5465"
Sep 15 08:53:44.377: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-scz6d" to be fully deleted
Sep 15 08:53:44.429: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comfksxk] to have phase Bound
Sep 15 08:53:44.480: INFO: PersistentVolumeClaim test.csi.azure.comfksxk found and phase=Bound (50.20742ms)
STEP: [init] checking the claim
... skipping 119 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not mount / map unused volumes in a pod [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumemode.go:351
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":16,"completed":1,"skipped":361,"failed":0}

SSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support existing single file [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219
... skipping 15 lines ...
Sep 15 08:54:00.520: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:54:00.574: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comp5srb] to have phase Bound
Sep 15 08:54:00.625: INFO: PersistentVolumeClaim test.csi.azure.comp5srb found but phase is Pending instead of Bound.
Sep 15 08:54:02.677: INFO: PersistentVolumeClaim test.csi.azure.comp5srb found and phase=Bound (2.103203011s)
STEP: Creating pod pod-subpath-test-dynamicpv-4dbc
STEP: Creating a pod to test subpath
Sep 15 08:54:02.835: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-4dbc" in namespace "provisioning-5735" to be "Succeeded or Failed"
Sep 15 08:54:02.886: INFO: Pod "pod-subpath-test-dynamicpv-4dbc": Phase="Pending", Reason="", readiness=false. Elapsed: 50.80685ms
Sep 15 08:54:04.944: INFO: Pod "pod-subpath-test-dynamicpv-4dbc": Phase="Pending", Reason="", readiness=false. Elapsed: 2.108959162s
Sep 15 08:54:06.996: INFO: Pod "pod-subpath-test-dynamicpv-4dbc": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.160993704s
STEP: Saw pod success
Sep 15 08:54:06.996: INFO: Pod "pod-subpath-test-dynamicpv-4dbc" satisfied condition "Succeeded or Failed"
Sep 15 08:54:07.048: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-4dbc container test-container-subpath-dynamicpv-4dbc: <nil>
STEP: delete the pod
Sep 15 08:54:07.161: INFO: Waiting for pod pod-subpath-test-dynamicpv-4dbc to disappear
Sep 15 08:54:07.233: INFO: Pod pod-subpath-test-dynamicpv-4dbc no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-4dbc
Sep 15 08:54:07.233: INFO: Deleting pod "pod-subpath-test-dynamicpv-4dbc" in namespace "provisioning-5735"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing single file [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:219
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":16,"completed":2,"skipped":371,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support non-existent path
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
... skipping 15 lines ...
Sep 15 08:54:13.168: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:54:13.222: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comn88tz] to have phase Bound
Sep 15 08:54:13.276: INFO: PersistentVolumeClaim test.csi.azure.comn88tz found but phase is Pending instead of Bound.
Sep 15 08:54:15.332: INFO: PersistentVolumeClaim test.csi.azure.comn88tz found and phase=Bound (2.10956323s)
STEP: Creating pod pod-subpath-test-dynamicpv-zfh7
STEP: Creating a pod to test subpath
Sep 15 08:54:15.500: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-zfh7" in namespace "provisioning-113" to be "Succeeded or Failed"
Sep 15 08:54:15.552: INFO: Pod "pod-subpath-test-dynamicpv-zfh7": Phase="Pending", Reason="", readiness=false. Elapsed: 52.04745ms
Sep 15 08:54:17.605: INFO: Pod "pod-subpath-test-dynamicpv-zfh7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.104795679s
Sep 15 08:54:19.657: INFO: Pod "pod-subpath-test-dynamicpv-zfh7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.157294098s
STEP: Saw pod success
Sep 15 08:54:19.657: INFO: Pod "pod-subpath-test-dynamicpv-zfh7" satisfied condition "Succeeded or Failed"
Sep 15 08:54:19.711: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-zfh7 container test-container-volume-dynamicpv-zfh7: <nil>
STEP: delete the pod
Sep 15 08:54:19.830: INFO: Waiting for pod pod-subpath-test-dynamicpv-zfh7 to disappear
Sep 15 08:54:19.882: INFO: Pod pod-subpath-test-dynamicpv-zfh7 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-zfh7
Sep 15 08:54:19.882: INFO: Deleting pod "pod-subpath-test-dynamicpv-zfh7" in namespace "provisioning-113"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support non-existent path
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:194
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":16,"completed":3,"skipped":425,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:54:25.931: INFO: Driver "test.csi.azure.com" does not provide raw block - skipping
... skipping 45 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (immediate binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "test.csi.azure.com" does not support topology - skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 41 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":15,"completed":1,"skipped":80,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ext3)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:54:32.413: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 66 lines ...

S [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds]
External Storage [Driver: test.csi.azure.com]
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should fail to schedule a pod which has topologies that conflict with AllowedTopologies [BeforeEach]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:192

    Driver "test.csi.azure.com" does not support topology - skipping

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/topology.go:92
------------------------------
... skipping 47 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should be able to unmount after the subpath directory is deleted [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:445
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":16,"completed":4,"skipped":594,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (delayed binding)] topology
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:54:43.312: INFO: Driver "test.csi.azure.com" does not support topology - skipping
... skipping 161 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read/write inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:166
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":15,"completed":2,"skipped":339,"failed":0}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support file as subpath [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
... skipping 15 lines ...
Sep 15 08:55:10.990: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:55:11.044: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com5mdpc] to have phase Bound
Sep 15 08:55:11.094: INFO: PersistentVolumeClaim test.csi.azure.com5mdpc found but phase is Pending instead of Bound.
Sep 15 08:55:13.146: INFO: PersistentVolumeClaim test.csi.azure.com5mdpc found and phase=Bound (2.102232194s)
STEP: Creating pod pod-subpath-test-dynamicpv-xx72
STEP: Creating a pod to test atomic-volume-subpath
Sep 15 08:55:13.300: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-xx72" in namespace "provisioning-7321" to be "Succeeded or Failed"
Sep 15 08:55:13.350: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Pending", Reason="", readiness=false. Elapsed: 50.361834ms
Sep 15 08:55:15.402: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Pending", Reason="", readiness=false. Elapsed: 2.102078724s
Sep 15 08:55:17.454: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 4.154379801s
Sep 15 08:55:19.507: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 6.207013757s
Sep 15 08:55:21.563: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 8.263219252s
Sep 15 08:55:23.615: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 10.31499021s
... skipping 2 lines ...
Sep 15 08:55:29.776: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 16.476436208s
Sep 15 08:55:31.837: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 18.537146841s
Sep 15 08:55:33.890: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 20.590435044s
Sep 15 08:55:35.943: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Running", Reason="", readiness=true. Elapsed: 22.643147289s
Sep 15 08:55:38.000: INFO: Pod "pod-subpath-test-dynamicpv-xx72": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.699915915s
STEP: Saw pod success
Sep 15 08:55:38.000: INFO: Pod "pod-subpath-test-dynamicpv-xx72" satisfied condition "Succeeded or Failed"
Sep 15 08:55:38.051: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-xx72 container test-container-subpath-dynamicpv-xx72: <nil>
STEP: delete the pod
Sep 15 08:55:38.303: INFO: Waiting for pod pod-subpath-test-dynamicpv-xx72 to disappear
Sep 15 08:55:38.354: INFO: Pod pod-subpath-test-dynamicpv-xx72 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-xx72
Sep 15 08:55:38.365: INFO: Deleting pod "pod-subpath-test-dynamicpv-xx72" in namespace "provisioning-7321"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support file as subpath [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:230
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":15,"completed":3,"skipped":389,"failed":0}

SSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:55:44.048: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 152 lines ...
Sep 15 08:53:45.067: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1215 PodName:pod-54b60866-0693-4838-a7f4-2708f6b8c955 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:45.067: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:45.619: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-1215 PodName:pod-54b60866-0693-4838-a7f4-2708f6b8c955 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:45.619: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:46.116: INFO: pod fsgroupchangepolicy-1215/pod-54b60866-0693-4838-a7f4-2708f6b8c955 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 08:53 /mnt/volume1/subdir/file2, stderr: 
Sep 15 08:53:46.116: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 08:53 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 08:53:46.116: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc000163ce0, 0x70bd1d0, 0x19, 0xc0036893d8, 0x4, 0xc004379800)
... skipping 71 lines ...
Sep 15 08:58:24.224: INFO: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 found and phase=Bound (4m38.051893288s)
Sep 15 08:58:29.276: INFO: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 found and phase=Bound (4m43.104550487s)
Sep 15 08:58:34.328: INFO: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 found and phase=Bound (4m48.15637173s)
Sep 15 08:58:39.381: INFO: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 found and phase=Bound (4m53.209207496s)
Sep 15 08:58:44.436: INFO: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 found and phase=Bound (4m58.263811396s)
STEP: Deleting sc
Sep 15 08:58:49.492: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-503623ae-7941-4ddf-8f57-47adaff05848 not deleted by dynamic provisioner: PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-503623ae-7941-4ddf-8f57-47adaff05848 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc002468d40)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc001bd8910, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc003d1a000, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc001bd8910, 0x4c, 0xc003908c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc003d00100, 0x37, 0xc003bf4850, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003908dc0, 0x7904368, 0xc001bb3240, 0x1, 0x0, 0x0, 0x0, 0xc001bb3240)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc003908dc0, 0x7904368, 0xc001bb3240, 0x0, 0x0, 0x0, 0xc0004dc400)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc001bb3220, 0x6323060, 0xc001bb3230, 0x0, 0x0, 0x0)
... skipping 142 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":13,"completed":0,"skipped":42,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (ext4)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:58:51.447: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 100 lines ...
Sep 15 08:53:45.740: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-6507 PodName:pod-f81559b3-e1e1-4ebb-b759-d0f44a690c23 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:45.740: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:46.245: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-6507 PodName:pod-f81559b3-e1e1-4ebb-b759-d0f44a690c23 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:46.245: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:46.748: INFO: pod fsgroupchangepolicy-6507/pod-f81559b3-e1e1-4ebb-b759-d0f44a690c23 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 08:53 /mnt/volume1/subdir/file2, stderr: 
Sep 15 08:53:46.748: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 08:53 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 08:53:46.748: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc001e4be40, 0x70bd1d0, 0x19, 0xc0016f3238, 0x4, 0xc004551000)
... skipping 71 lines ...
Sep 15 08:58:24.999: INFO: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b found and phase=Bound (4m38.192014222s)
Sep 15 08:58:30.055: INFO: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b found and phase=Bound (4m43.24778841s)
Sep 15 08:58:35.113: INFO: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b found and phase=Bound (4m48.305977782s)
Sep 15 08:58:40.169: INFO: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b found and phase=Bound (4m53.361468296s)
Sep 15 08:58:45.236: INFO: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b found and phase=Bound (4m58.428155208s)
STEP: Deleting sc
Sep 15 08:58:50.300: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b not deleted by dynamic provisioner: PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-898d22bd-aedf-44cd-a5c9-9b378daa569b still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc002bec200)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc000e389b0, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc002780a00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc000e389b0, 0x4c, 0xc003c3ec38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc000edaa00, 0x37, 0xc00403bfa0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc003c3edc0, 0x7904368, 0xc0023dabb0, 0x1, 0x0, 0x0, 0x0, 0xc0023dabb0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc003c3edc0, 0x7904368, 0xc0023dabb0, 0x0, 0x0, 0x0, 0xc000489800)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc0023dab90, 0x6323060, 0xc0023daba0, 0x0, 0x0, 0x0)
... skipping 142 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":13,"completed":0,"skipped":146,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:58:52.185: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 152 lines ...
Sep 15 08:53:46.468: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-9179 PodName:pod-94eb636c-3994-451e-80c5-dd791dfd4024 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:46.468: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:47.023: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-9179 PodName:pod-94eb636c-3994-451e-80c5-dd791dfd4024 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:47.023: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:47.486: INFO: pod fsgroupchangepolicy-9179/pod-94eb636c-3994-451e-80c5-dd791dfd4024 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 08:53 /mnt/volume1/subdir/file2, stderr: 
Sep 15 08:53:47.487: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 08:53 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 08:53:47.487: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc001e3fa20, 0x70bd1d0, 0x19, 0xc0038ac5a8, 0x4, 0xc000b09800)
... skipping 71 lines ...
Sep 15 08:58:25.760: INFO: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 found and phase=Bound (4m38.214810545s)
Sep 15 08:58:30.814: INFO: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 found and phase=Bound (4m43.269085656s)
Sep 15 08:58:35.872: INFO: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 found and phase=Bound (4m48.326805499s)
Sep 15 08:58:40.926: INFO: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 found and phase=Bound (4m53.380868831s)
Sep 15 08:58:45.983: INFO: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 found and phase=Bound (4m58.438498659s)
STEP: Deleting sc
Sep 15 08:58:51.050: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 not deleted by dynamic provisioner: PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-1c8ba27f-30c5-40ae-a5e9-a9df3946cb84 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003a5a800)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc0034e8230, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc0011c7400, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc0034e8230, 0x4c, 0xc001aecc38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc002a2b500, 0x37, 0xc000a33f00, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc001aecdc0, 0x7904368, 0xc0035aeee0, 0x1, 0x0, 0x0, 0x0, 0xc0035aeee0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc001aecdc0, 0x7904368, 0xc0035aeee0, 0x0, 0x0, 0x0, 0xc000b08000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc0035aeec0, 0x6323060, 0xc0035aeed0, 0x0, 0x0, 0x0)
... skipping 144 lines ...
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
SSSSSSS
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":22,"completed":0,"skipped":320,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 58 lines ...
Sep 15 08:53:46.184: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-723 PodName:pod-d76a2924-1fca-43d3-84d5-13b29af1e292 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:46.184: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:46.750: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-723 PodName:pod-d76a2924-1fca-43d3-84d5-13b29af1e292 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:53:46.750: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:53:47.236: INFO: pod fsgroupchangepolicy-723/pod-d76a2924-1fca-43d3-84d5-13b29af1e292 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 08:53 /mnt/volume1/subdir/file2, stderr: 
Sep 15 08:53:47.236: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 08:53 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 08:53:47.241: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc00420c2c0, 0x70bd1d0, 0x19, 0xc003211bb8, 0x4, 0xc00356a400)
... skipping 71 lines ...
Sep 15 08:58:25.713: INFO: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 found and phase=Bound (4m38.413335764s)
Sep 15 08:58:30.773: INFO: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 found and phase=Bound (4m43.473227574s)
Sep 15 08:58:35.836: INFO: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 found and phase=Bound (4m48.535930144s)
Sep 15 08:58:40.895: INFO: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 found and phase=Bound (4m53.595556447s)
Sep 15 08:58:45.968: INFO: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 found and phase=Bound (4m58.66796238s)
STEP: Deleting sc
Sep 15 08:58:51.032: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 not deleted by dynamic provisioner: PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-c98c420f-0cba-49c5-90fb-772f543ade27 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc003e428c0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc000a20910, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc002718f00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc000a20910, 0x4c, 0xc0037bec38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc0014da980, 0x37, 0xc001ec33f0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc0037bedc0, 0x7904368, 0xc003e488f0, 0x1, 0x0, 0x0, 0x0, 0xc003e488f0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc0037bedc0, 0x7904368, 0xc003e488f0, 0x0, 0x0, 0x0, 0xc000093800)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc003e488d0, 0x6323060, 0xc003e488e0, 0x0, 0x0, 0x0)
... skipping 144 lines ...
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
SSSSSSSSSS
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","total":14,"completed":0,"skipped":236,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:58:53.064: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 73 lines ...
STEP: creating a claim
Sep 15 08:58:53.603: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:58:53.659: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com4bqtr] to have phase Bound
Sep 15 08:58:53.715: INFO: PersistentVolumeClaim test.csi.azure.com4bqtr found but phase is Pending instead of Bound.
Sep 15 08:58:55.770: INFO: PersistentVolumeClaim test.csi.azure.com4bqtr found and phase=Bound (2.11116747s)
STEP: [init] starting a pod to use the claim
Sep 15 08:58:55.938: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-6xnls" in namespace "snapshotting-5088" to be "Succeeded or Failed"
Sep 15 08:58:55.996: INFO: Pod "pvc-snapshottable-tester-6xnls": Phase="Pending", Reason="", readiness=false. Elapsed: 57.483199ms
Sep 15 08:58:58.052: INFO: Pod "pvc-snapshottable-tester-6xnls": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2.113882125s
STEP: Saw pod success
Sep 15 08:58:58.053: INFO: Pod "pvc-snapshottable-tester-6xnls" satisfied condition "Succeeded or Failed"
Sep 15 08:58:58.176: INFO: Pod pvc-snapshottable-tester-6xnls has the following logs: 
Sep 15 08:58:58.176: INFO: Deleting pod "pvc-snapshottable-tester-6xnls" in namespace "snapshotting-5088"
Sep 15 08:58:58.238: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-6xnls" to be fully deleted
Sep 15 08:58:58.292: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com4bqtr] to have phase Bound
Sep 15 08:58:58.348: INFO: PersistentVolumeClaim test.csi.azure.com4bqtr found and phase=Bound (55.949034ms)
STEP: [init] checking the claim
... skipping 121 lines ...
Sep 15 08:58:52.318: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:58:52.376: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comnmjxn] to have phase Bound
Sep 15 08:58:52.448: INFO: PersistentVolumeClaim test.csi.azure.comnmjxn found but phase is Pending instead of Bound.
Sep 15 08:58:54.504: INFO: PersistentVolumeClaim test.csi.azure.comnmjxn found and phase=Bound (2.127933939s)
STEP: Creating pod pod-subpath-test-dynamicpv-hrs4
STEP: Creating a pod to test subpath
Sep 15 08:58:54.666: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-hrs4" in namespace "provisioning-5216" to be "Succeeded or Failed"
Sep 15 08:58:54.720: INFO: Pod "pod-subpath-test-dynamicpv-hrs4": Phase="Pending", Reason="", readiness=false. Elapsed: 53.753465ms
Sep 15 08:58:56.774: INFO: Pod "pod-subpath-test-dynamicpv-hrs4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.107323421s
Sep 15 08:58:58.826: INFO: Pod "pod-subpath-test-dynamicpv-hrs4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.159704968s
Sep 15 08:59:00.879: INFO: Pod "pod-subpath-test-dynamicpv-hrs4": Phase="Running", Reason="", readiness=false. Elapsed: 6.212872809s
Sep 15 08:59:02.933: INFO: Pod "pod-subpath-test-dynamicpv-hrs4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 8.267194149s
STEP: Saw pod success
Sep 15 08:59:02.934: INFO: Pod "pod-subpath-test-dynamicpv-hrs4" satisfied condition "Succeeded or Failed"
Sep 15 08:59:02.992: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-hrs4 container test-container-volume-dynamicpv-hrs4: <nil>
STEP: delete the pod
Sep 15 08:59:03.113: INFO: Waiting for pod pod-subpath-test-dynamicpv-hrs4 to disappear
Sep 15 08:59:03.172: INFO: Pod pod-subpath-test-dynamicpv-hrs4 no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-hrs4
Sep 15 08:59:03.172: INFO: Deleting pod "pod-subpath-test-dynamicpv-hrs4" in namespace "provisioning-5216"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directory
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:205
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":13,"completed":1,"skipped":458,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning
... skipping 184 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":13,"completed":1,"skipped":380,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:59:38.225: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 78 lines ...
Sep 15 08:59:04.380: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:59:04.444: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.coml67bq] to have phase Bound
Sep 15 08:59:04.504: INFO: PersistentVolumeClaim test.csi.azure.coml67bq found but phase is Pending instead of Bound.
Sep 15 08:59:06.559: INFO: PersistentVolumeClaim test.csi.azure.coml67bq found and phase=Bound (2.114910345s)
STEP: Expanding non-expandable pvc
Sep 15 08:59:06.667: INFO: currentPvcSize {{5368709120 0} {<nil>} 5Gi BinarySI}, newSize {{6442450944 0} {<nil>}  BinarySI}
Sep 15 08:59:06.776: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:08.892: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:10.892: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:12.885: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:14.887: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:16.886: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:18.886: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:20.888: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:22.894: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:24.887: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:26.892: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:28.892: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:30.894: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:32.894: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:34.889: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:36.890: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
Sep 15 08:59:37.004: INFO: Error updating pvc test.csi.azure.coml67bq: persistentvolumeclaims "test.csi.azure.coml67bq" is forbidden: only dynamically provisioned pvc can be resized and the storageclass that provisions the pvc must support resize
STEP: Deleting pvc
Sep 15 08:59:37.004: INFO: Deleting PersistentVolumeClaim "test.csi.azure.coml67bq"
Sep 15 08:59:37.060: INFO: Waiting up to 5m0s for PersistentVolume pvc-11e61d68-19fe-496e-8e91-c3321d5f8685 to get deleted
Sep 15 08:59:37.116: INFO: PersistentVolume pvc-11e61d68-19fe-496e-8e91-c3321d5f8685 found and phase=Released (56.177118ms)
Sep 15 08:59:42.171: INFO: PersistentVolume pvc-11e61d68-19fe-496e-8e91-c3321d5f8685 was removed
STEP: Deleting sc
... skipping 8 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should not allow expansion of pvcs without AllowVolumeExpansion property
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:157
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volume-expand should not allow expansion of pvcs without AllowVolumeExpansion property","total":22,"completed":1,"skipped":476,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 08:59:42.426: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 87 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should store data
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:159
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":13,"completed":2,"skipped":853,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource]
... skipping 10 lines ...
STEP: creating a claim
Sep 15 08:59:38.712: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:59:38.769: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comz4w4p] to have phase Bound
Sep 15 08:59:38.824: INFO: PersistentVolumeClaim test.csi.azure.comz4w4p found but phase is Pending instead of Bound.
Sep 15 08:59:40.879: INFO: PersistentVolumeClaim test.csi.azure.comz4w4p found and phase=Bound (2.110405897s)
STEP: [init] starting a pod to use the claim
Sep 15 08:59:41.046: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-bg4pr" in namespace "snapshotting-4771" to be "Succeeded or Failed"
Sep 15 08:59:41.100: INFO: Pod "pvc-snapshottable-tester-bg4pr": Phase="Pending", Reason="", readiness=false. Elapsed: 54.525879ms
Sep 15 08:59:43.160: INFO: Pod "pvc-snapshottable-tester-bg4pr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2.1141519s
STEP: Saw pod success
Sep 15 08:59:43.160: INFO: Pod "pvc-snapshottable-tester-bg4pr" satisfied condition "Succeeded or Failed"
Sep 15 08:59:43.276: INFO: Pod pvc-snapshottable-tester-bg4pr has the following logs: 
Sep 15 08:59:43.276: INFO: Deleting pod "pvc-snapshottable-tester-bg4pr" in namespace "snapshotting-4771"
Sep 15 08:59:43.340: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-bg4pr" to be fully deleted
Sep 15 08:59:43.394: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comz4w4p] to have phase Bound
Sep 15 08:59:43.451: INFO: PersistentVolumeClaim test.csi.azure.comz4w4p found and phase=Bound (56.876071ms)
STEP: [init] checking the claim
... skipping 80 lines ...
Sep 15 08:59:44.084: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:59:44.139: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comqrk6b] to have phase Bound
Sep 15 08:59:44.191: INFO: PersistentVolumeClaim test.csi.azure.comqrk6b found but phase is Pending instead of Bound.
Sep 15 08:59:46.244: INFO: PersistentVolumeClaim test.csi.azure.comqrk6b found and phase=Bound (2.104659206s)
STEP: Creating pod exec-volume-test-dynamicpv-pp9w
STEP: Creating a pod to test exec-volume-test
Sep 15 08:59:46.408: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-pp9w" in namespace "volume-3511" to be "Succeeded or Failed"
Sep 15 08:59:46.459: INFO: Pod "exec-volume-test-dynamicpv-pp9w": Phase="Pending", Reason="", readiness=false. Elapsed: 51.471339ms
Sep 15 08:59:48.512: INFO: Pod "exec-volume-test-dynamicpv-pp9w": Phase="Pending", Reason="", readiness=false. Elapsed: 2.10408999s
Sep 15 08:59:50.568: INFO: Pod "exec-volume-test-dynamicpv-pp9w": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.160099098s
STEP: Saw pod success
Sep 15 08:59:50.568: INFO: Pod "exec-volume-test-dynamicpv-pp9w" satisfied condition "Succeeded or Failed"
Sep 15 08:59:50.623: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000001 pod exec-volume-test-dynamicpv-pp9w container exec-container-dynamicpv-pp9w: <nil>
STEP: delete the pod
Sep 15 08:59:50.741: INFO: Waiting for pod exec-volume-test-dynamicpv-pp9w to disappear
Sep 15 08:59:50.793: INFO: Pod exec-volume-test-dynamicpv-pp9w no longer exists
STEP: Deleting pod exec-volume-test-dynamicpv-pp9w
Sep 15 08:59:50.793: INFO: Deleting pod "exec-volume-test-dynamicpv-pp9w" in namespace "volume-3511"
... skipping 14 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should allow exec of files on the volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volumes.go:196
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":13,"completed":3,"skipped":866,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
[BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource]
... skipping 10 lines ...
STEP: creating a claim
Sep 15 08:59:56.596: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 08:59:56.650: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com4w2l7] to have phase Bound
Sep 15 08:59:56.701: INFO: PersistentVolumeClaim test.csi.azure.com4w2l7 found but phase is Pending instead of Bound.
Sep 15 08:59:58.754: INFO: PersistentVolumeClaim test.csi.azure.com4w2l7 found and phase=Bound (2.103354391s)
STEP: [init] starting a pod to use the claim
Sep 15 08:59:58.916: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-sdv2s" in namespace "snapshotting-4872" to be "Succeeded or Failed"
Sep 15 08:59:58.967: INFO: Pod "pvc-snapshottable-tester-sdv2s": Phase="Pending", Reason="", readiness=false. Elapsed: 51.643596ms
Sep 15 09:00:01.021: INFO: Pod "pvc-snapshottable-tester-sdv2s": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2.105864905s
STEP: Saw pod success
Sep 15 09:00:01.021: INFO: Pod "pvc-snapshottable-tester-sdv2s" satisfied condition "Succeeded or Failed"
Sep 15 09:00:01.133: INFO: Pod pvc-snapshottable-tester-sdv2s has the following logs: 
Sep 15 09:00:01.133: INFO: Deleting pod "pvc-snapshottable-tester-sdv2s" in namespace "snapshotting-4872"
Sep 15 09:00:01.194: INFO: Wait up to 5m0s for pod "pvc-snapshottable-tester-sdv2s" to be fully deleted
Sep 15 09:00:01.260: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com4w2l7] to have phase Bound
Sep 15 09:00:01.312: INFO: PersistentVolumeClaim test.csi.azure.com4w2l7 found and phase=Bound (52.022755ms)
STEP: [init] checking the claim
... skipping 99 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which share the same volume","total":14,"completed":1,"skipped":354,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral 
  should create read-only inline ephemeral volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
... skipping 36 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should create read-only inline ephemeral volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:149
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":22,"completed":2,"skipped":505,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 09:00:20.523: INFO: Driver test.csi.azure.com doesn't support ntfs -- skipping
... skipping 113 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    Verify if offline PVC expansion works
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:174
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand Verify if offline PVC expansion works","total":22,"completed":3,"skipped":544,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy 
  (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:208
... skipping 30 lines ...
Sep 15 08:55:51.696: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4884 PodName:pod-61ded2c6-dcd5-4a53-880b-6a724b0acd13 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:55:51.696: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:55:52.240: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-4884 PodName:pod-61ded2c6-dcd5-4a53-880b-6a724b0acd13 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 08:55:52.240: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 08:55:52.892: INFO: pod fsgroupchangepolicy-4884/pod-61ded2c6-dcd5-4a53-880b-6a724b0acd13 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 08:55 /mnt/volume1/subdir/file2, stderr: 
Sep 15 08:55:52.892: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 08:55 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 08:55:52.893: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc0024ffb80, 0x70bd1d0, 0x19, 0xc0012e3358, 0x4, 0xc00388c000)
... skipping 71 lines ...
Sep 15 09:00:31.036: INFO: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 found and phase=Bound (4m38.088003717s)
Sep 15 09:00:36.091: INFO: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 found and phase=Bound (4m43.143668117s)
Sep 15 09:00:41.147: INFO: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 found and phase=Bound (4m48.198978749s)
Sep 15 09:00:46.204: INFO: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 found and phase=Bound (4m53.256042737s)
Sep 15 09:00:51.261: INFO: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 found and phase=Bound (4m58.313512083s)
STEP: Deleting sc
Sep 15 09:00:56.319: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-eb4345be-7744-4292-af33-26b4959d1788 not deleted by dynamic provisioner: PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-eb4345be-7744-4292-af33-26b4959d1788 still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc002262cc0)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc0008c8a50, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc003c16500, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc0008c8a50, 0x4c, 0xc001a78c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc002bbd480, 0x37, 0xc003a3bc30, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc001a78dc0, 0x7904368, 0xc003a0f980, 0x1, 0x0, 0x0, 0x0, 0xc003a0f980)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc001a78dc0, 0x7904368, 0xc003a0f980, 0x0, 0x0, 0x0, 0xc000580000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc003a0f960, 0x6323060, 0xc003a0f970, 0x0, 0x0, 0x0)
... skipping 136 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents","total":15,"completed":3,"skipped":831,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand 
  should resize volume when PVC is edited while pod is using it
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246
... skipping 42 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should resize volume when PVC is edited while pod is using it
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/volume_expand.go:246
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)(allowExpansion)] volume-expand should resize volume when PVC is edited while pod is using it","total":22,"completed":4,"skipped":569,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support existing directories when readOnly specified in the volumeSource
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395
... skipping 15 lines ...
Sep 15 09:00:58.381: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 09:00:58.489: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comnbsl5] to have phase Bound
Sep 15 09:00:58.543: INFO: PersistentVolumeClaim test.csi.azure.comnbsl5 found but phase is Pending instead of Bound.
Sep 15 09:01:00.598: INFO: PersistentVolumeClaim test.csi.azure.comnbsl5 found and phase=Bound (2.108115828s)
STEP: Creating pod pod-subpath-test-dynamicpv-pz2b
STEP: Creating a pod to test subpath
Sep 15 09:01:00.761: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-pz2b" in namespace "provisioning-5560" to be "Succeeded or Failed"
Sep 15 09:01:00.816: INFO: Pod "pod-subpath-test-dynamicpv-pz2b": Phase="Pending", Reason="", readiness=false. Elapsed: 54.655761ms
Sep 15 09:01:02.872: INFO: Pod "pod-subpath-test-dynamicpv-pz2b": Phase="Pending", Reason="", readiness=false. Elapsed: 2.110274526s
Sep 15 09:01:04.926: INFO: Pod "pod-subpath-test-dynamicpv-pz2b": Phase="Succeeded", Reason="", readiness=false. Elapsed: 4.164468632s
STEP: Saw pod success
Sep 15 09:01:04.926: INFO: Pod "pod-subpath-test-dynamicpv-pz2b" satisfied condition "Succeeded or Failed"
Sep 15 09:01:04.980: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-pz2b container test-container-subpath-dynamicpv-pz2b: <nil>
STEP: delete the pod
Sep 15 09:01:05.104: INFO: Waiting for pod pod-subpath-test-dynamicpv-pz2b to disappear
Sep 15 09:01:05.158: INFO: Pod pod-subpath-test-dynamicpv-pz2b no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-pz2b
Sep 15 09:01:05.158: INFO: Deleting pod "pod-subpath-test-dynamicpv-pz2b" in namespace "provisioning-5560"
STEP: Creating pod pod-subpath-test-dynamicpv-pz2b
STEP: Creating a pod to test subpath
Sep 15 09:01:05.267: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-pz2b" in namespace "provisioning-5560" to be "Succeeded or Failed"
Sep 15 09:01:05.320: INFO: Pod "pod-subpath-test-dynamicpv-pz2b": Phase="Pending", Reason="", readiness=false. Elapsed: 53.492046ms
Sep 15 09:01:07.377: INFO: Pod "pod-subpath-test-dynamicpv-pz2b": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2.110074365s
STEP: Saw pod success
Sep 15 09:01:07.377: INFO: Pod "pod-subpath-test-dynamicpv-pz2b" satisfied condition "Succeeded or Failed"
Sep 15 09:01:07.435: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-pz2b container test-container-subpath-dynamicpv-pz2b: <nil>
STEP: delete the pod
Sep 15 09:01:07.564: INFO: Waiting for pod pod-subpath-test-dynamicpv-pz2b to disappear
Sep 15 09:01:07.620: INFO: Pod pod-subpath-test-dynamicpv-pz2b no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-pz2b
Sep 15 09:01:07.620: INFO: Deleting pod "pod-subpath-test-dynamicpv-pz2b" in namespace "provisioning-5560"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support existing directories when readOnly specified in the volumeSource
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:395
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":15,"completed":4,"skipped":910,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents"]}

SSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 09:01:13.139: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 46 lines ...
Sep 15 09:00:59.144: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 09:00:59.217: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comptkn9] to have phase Bound
Sep 15 09:00:59.271: INFO: PersistentVolumeClaim test.csi.azure.comptkn9 found but phase is Pending instead of Bound.
Sep 15 09:01:01.325: INFO: PersistentVolumeClaim test.csi.azure.comptkn9 found and phase=Bound (2.108411689s)
STEP: Creating pod pod-subpath-test-dynamicpv-7pzm
STEP: Creating a pod to test subpath
Sep 15 09:01:01.491: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-7pzm" in namespace "provisioning-5633" to be "Succeeded or Failed"
Sep 15 09:01:01.546: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Pending", Reason="", readiness=false. Elapsed: 55.028206ms
Sep 15 09:01:03.604: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Pending", Reason="", readiness=false. Elapsed: 2.11338284s
Sep 15 09:01:05.660: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Pending", Reason="", readiness=false. Elapsed: 4.168845472s
Sep 15 09:01:07.714: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Pending", Reason="", readiness=false. Elapsed: 6.223177284s
Sep 15 09:01:09.769: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Pending", Reason="", readiness=false. Elapsed: 8.27823745s
Sep 15 09:01:11.824: INFO: Pod "pod-subpath-test-dynamicpv-7pzm": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.33313061s
STEP: Saw pod success
Sep 15 09:01:11.824: INFO: Pod "pod-subpath-test-dynamicpv-7pzm" satisfied condition "Succeeded or Failed"
Sep 15 09:01:11.878: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000001 pod pod-subpath-test-dynamicpv-7pzm container test-container-subpath-dynamicpv-7pzm: <nil>
STEP: delete the pod
Sep 15 09:01:12.000: INFO: Waiting for pod pod-subpath-test-dynamicpv-7pzm to disappear
Sep 15 09:01:12.054: INFO: Pod pod-subpath-test-dynamicpv-7pzm no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-7pzm
Sep 15 09:01:12.054: INFO: Deleting pod "pod-subpath-test-dynamicpv-7pzm" in namespace "provisioning-5633"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly directory specified in the volumeMount
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:365
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":22,"completed":5,"skipped":626,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should support multiple inline ephemeral volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
... skipping 38 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support multiple inline ephemeral volumes
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:221
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":22,"completed":6,"skipped":652,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Pre-provisioned PV (ntfs)][Feature:Windows] volumes
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 09:01:57.458: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping
... skipping 124 lines ...
Sep 15 09:00:19.400: INFO: ExecWithOptions {Command:[/bin/sh -c touch /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-7769 PodName:pod-86b1679a-c5b3-4caf-94b0-0c1a8c7bff67 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 09:00:19.400: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 09:00:19.892: INFO: ExecWithOptions {Command:[/bin/sh -c ls -l /mnt/volume1/subdir/file2] Namespace:fsgroupchangepolicy-7769 PodName:pod-86b1679a-c5b3-4caf-94b0-0c1a8c7bff67 ContainerName:write-pod Stdin:<nil> CaptureStdout:true CaptureStderr:true PreserveWhitespace:false Quiet:false}
Sep 15 09:00:19.892: INFO: >>> kubeConfig: /root/tmp110310216/kubeconfig/kubeconfig.westus2.json
Sep 15 09:00:20.355: INFO: pod fsgroupchangepolicy-7769/pod-86b1679a-c5b3-4caf-94b0-0c1a8c7bff67 exec for cmd ls -l /mnt/volume1/subdir/file2, stdout: -rw-r--r--    1 root     root             0 Sep 15 09:00 /mnt/volume1/subdir/file2, stderr: 
Sep 15 09:00:20.355: INFO: stdout split: [-rw-r--r-- 1 root root 0 Sep 15 09:00 /mnt/volume1/subdir/file2], expected gid: 1000
Sep 15 09:00:20.355: FAIL: Expected
    <string>: root
to equal
    <string>: 1000

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/utils.VerifyFilePathGidInPod(0xc00420c2c0, 0x70bd1d0, 0x19, 0xc003f8a758, 0x4, 0xc002c62400)
... skipping 71 lines ...
Sep 15 09:04:58.646: INFO: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e found and phase=Bound (4m38.234581686s)
Sep 15 09:05:03.713: INFO: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e found and phase=Bound (4m43.300796651s)
Sep 15 09:05:08.772: INFO: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e found and phase=Bound (4m48.359895876s)
Sep 15 09:05:13.832: INFO: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e found and phase=Bound (4m53.419894498s)
Sep 15 09:05:18.891: INFO: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e found and phase=Bound (4m58.478983065s)
STEP: Deleting sc
Sep 15 09:05:23.948: FAIL: while cleanup resource
Unexpected error:
    <errors.aggregate | len:1, cap:1>: [
        [
            {
                msg: "persistent Volume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e not deleted by dynamic provisioner: PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e still exists within 5m0s",
                err: {
                    s: "PersistentVolume pvc-977cdfa7-17bf-4db4-9982-48516d8d784e still exists within 5m0s",
... skipping 6 lines ...

Full Stack Trace
k8s.io/kubernetes/test/e2e/storage/testsuites.(*fsGroupChangePolicyTestSuite).DefineTests.func2()
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/fsgroupchangepolicy.go:132 +0xf0
panic(0x6bf5d00, 0xc002526740)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1(0xc000f949b0, 0x4c, 0x895c71a, 0x6c, 0x2d8, 0xc001836a00, 0x499)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:63 +0xa5
panic(0x6323060, 0x77ced70)
	/usr/local/go/src/runtime/panic.go:965 +0x1b9
k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail(0xc000f949b0, 0x4c, 0xc004ce6c38, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x1b5
k8s.io/kubernetes/test/e2e/framework.Fail(0xc000e040c0, 0x37, 0xc0002289f0, 0x1, 0x1)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/framework/log.go:62 +0x1ea
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc004ce6dc0, 0x7904368, 0xc003e489d0, 0x1, 0x0, 0x0, 0x0, 0xc003e489d0)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x216
k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).To(0xc004ce6dc0, 0x7904368, 0xc003e489d0, 0x0, 0x0, 0x0, 0xc00400a000)
	/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion/assertion.go:38 +0xc7
k8s.io/kubernetes/test/e2e/framework.ExpectEqual(0x6323060, 0xc003e48990, 0x6323060, 0xc003e489a0, 0x0, 0x0, 0x0)
... skipping 134 lines ...
        <string>: root
    to equal
        <string>: 1000

    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728
------------------------------
{"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":14,"completed":1,"skipped":437,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral 
  should support two pods which share the same volume
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
... skipping 45 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support two pods which share the same volume
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/ephemeral.go:183
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which share the same volume","total":14,"completed":2,"skipped":474,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
------------------------------
External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath 
  should support readOnly file specified in the volumeMount [LinuxOnly]
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
... skipping 15 lines ...
Sep 15 09:06:47.728: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil
Sep 15 09:06:47.788: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com24d7j] to have phase Bound
Sep 15 09:06:47.844: INFO: PersistentVolumeClaim test.csi.azure.com24d7j found but phase is Pending instead of Bound.
Sep 15 09:06:49.900: INFO: PersistentVolumeClaim test.csi.azure.com24d7j found and phase=Bound (2.112083395s)
STEP: Creating pod pod-subpath-test-dynamicpv-f8ws
STEP: Creating a pod to test subpath
Sep 15 09:06:50.067: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-f8ws" in namespace "provisioning-4254" to be "Succeeded or Failed"
Sep 15 09:06:50.121: INFO: Pod "pod-subpath-test-dynamicpv-f8ws": Phase="Pending", Reason="", readiness=false. Elapsed: 54.012997ms
Sep 15 09:06:52.177: INFO: Pod "pod-subpath-test-dynamicpv-f8ws": Phase="Pending", Reason="", readiness=false. Elapsed: 2.109962s
Sep 15 09:06:54.236: INFO: Pod "pod-subpath-test-dynamicpv-f8ws": Phase="Pending", Reason="", readiness=false. Elapsed: 4.169185701s
Sep 15 09:06:56.291: INFO: Pod "pod-subpath-test-dynamicpv-f8ws": Phase="Succeeded", Reason="", readiness=false. Elapsed: 6.22462369s
STEP: Saw pod success
Sep 15 09:06:56.291: INFO: Pod "pod-subpath-test-dynamicpv-f8ws" satisfied condition "Succeeded or Failed"
Sep 15 09:06:56.355: INFO: Trying to get logs from node k8s-agentpool1-27164723-vmss000000 pod pod-subpath-test-dynamicpv-f8ws container test-container-subpath-dynamicpv-f8ws: <nil>
STEP: delete the pod
Sep 15 09:06:56.473: INFO: Waiting for pod pod-subpath-test-dynamicpv-f8ws to disappear
Sep 15 09:06:56.528: INFO: Pod pod-subpath-test-dynamicpv-f8ws no longer exists
STEP: Deleting pod pod-subpath-test-dynamicpv-f8ws
Sep 15 09:06:56.528: INFO: Deleting pod "pod-subpath-test-dynamicpv-f8ws" in namespace "provisioning-4254"
... skipping 16 lines ...
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/external/external.go:174
  [Testpattern: Dynamic PV (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:50
    should support readOnly file specified in the volumeMount [LinuxOnly]
    /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/testsuites/subpath.go:380
------------------------------
{"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":14,"completed":3,"skipped":600,"failed":2,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents","External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents"]}

SSSSSSSSS
------------------------------
[BeforeEach] [Testpattern: Inline-volume (default fs)] subPath
  /workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/framework/testsuite.go:51
Sep 15 09:07:02.052: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping
... skipping 120 lines ...
Sep 15 09:07:02.704: INFO: Running AfterSuite actions on node 1



Summarizing 6 Failures:

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup skips ownership changes to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with different fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed in first pod, new pod with same fsgroup applied to the volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

[Fail] External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [Measurement] (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents 
/workspace/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/storage/utils/utils.go:728

Ran 27 of 6666 Specs in 859.357 seconds
FAIL! -- 21 Passed | 6 Failed | 0 Pending | 6639 Skipped 

Ginkgo ran 1 suite in 14m28.77017467s
Test Suite Failed
+ print_logs
+ bash ./hack/verify-examples.sh linux
begin to create deployment examples ...
storageclass.storage.k8s.io/azurefile-csi created
Applying config "deploy/example/daemonset.yaml"
Waiting for daemon set "daemonset-azurefile" rollout to finish: 0 of 2 updated pods are available...
... skipping 68 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 08:52:27.721151       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 gc/go1.17.1 (amd64-linux) e2e-test
I0915 08:52:27.722316       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 08:52:27.751206       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 08:52:27.751247       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 08:52:27.751254       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 08:52:27.751287       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 08:52:27.752673       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 08:52:27.752762       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 08:52:27.752833       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 639 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 08:52:30.043403       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 gc/go1.17.1 (amd64-linux) e2e-test
I0915 08:52:30.046037       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 08:52:30.070261       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 08:52:30.070281       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 08:52:30.070287       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 08:52:30.070304       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 08:52:30.070855       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 08:52:30.070887       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 08:52:30.070926       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 99 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 08:52:21.534146       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 gc/go1.17.1 (amd64-linux) e2e-test
I0915 08:52:21.535116       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 08:52:21.561452       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 08:52:21.561471       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 08:52:21.561477       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 08:52:21.561496       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 08:52:21.562120       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 08:52:21.562148       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 08:52:21.562186       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 394 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 08:52:15.276114       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 gc/go1.17.1 (amd64-linux) e2e-test
I0915 08:52:15.277233       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 08:52:15.301342       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 08:52:15.301368       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 08:52:15.301394       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 08:52:15.301434       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 08:52:15.302826       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 08:52:15.302868       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 08:52:15.302895       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 66 lines ...
Go Version: go1.17.1
Platform: linux/amd64

Streaming logs below:
I0915 08:52:24.289722       1 azurefile.go:233] driver userAgent: test.csi.azure.com/e2e-bee2fd8c74b1c929e5a49796d211abca6c88e636 gc/go1.17.1 (amd64-linux) e2e-test
I0915 08:52:24.290782       1 azure.go:70] reading cloud config from secret kube-system/azure-cloud-provider
W0915 08:52:24.311371       1 azure.go:77] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I0915 08:52:24.311392       1 azure.go:82] could not read cloud config from secret kube-system/azure-cloud-provider
I0915 08:52:24.311399       1 azure.go:92] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I0915 08:52:24.311416       1 azure.go:100] read cloud config from file: /etc/kubernetes/azure.json successfully
I0915 08:52:24.311967       1 azure_auth.go:234] Using AzurePublicCloud environment
I0915 08:52:24.312004       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I0915 08:52:24.312022       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 1184 lines ...
# HELP process_virtual_memory_bytes Virtual memory size in bytes.
# TYPE process_virtual_memory_bytes gauge
process_virtual_memory_bytes 7.62433536e+08
# HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
# TYPE process_virtual_memory_max_bytes gauge
process_virtual_memory_max_bytes -1
make: *** [Makefile:85: e2e-test] Error 1
2021/09/15 09:07:56 process.go:155: Step 'make e2e-test' finished in 28m17.626636861s
2021/09/15 09:07:56 aksengine_helpers.go:425: downloading /root/tmp110310216/log-dump.sh from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/09/15 09:07:56 util.go:68: curl https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump.sh
2021/09/15 09:07:56 process.go:153: Running: chmod +x /root/tmp110310216/log-dump.sh
2021/09/15 09:07:56 process.go:155: Step 'chmod +x /root/tmp110310216/log-dump.sh' finished in 4.932391ms
2021/09/15 09:07:56 aksengine_helpers.go:425: downloading /root/tmp110310216/log-dump-daemonset.yaml from https://raw.githubusercontent.com/kubernetes-sigs/cloud-provider-azure/master/hack/log-dump/log-dump-daemonset.yaml
... skipping 53 lines ...
ssh key file /root/.ssh/id_rsa does not exist. Exiting.
2021/09/15 09:08:40 process.go:155: Step 'bash -c /root/tmp110310216/win-ci-logs-collector.sh kubetest-biyndxbl.westus2.cloudapp.azure.com /root/tmp110310216 /root/.ssh/id_rsa' finished in 14.719572ms
2021/09/15 09:08:40 aksengine.go:1141: Deleting resource group: kubetest-biyndxbl.
2021/09/15 09:16:44 process.go:96: Saved XML output to /logs/artifacts/junit_runner.xml.
2021/09/15 09:16:44 process.go:153: Running: bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"
2021/09/15 09:16:46 process.go:155: Step 'bash -c . hack/lib/version.sh && KUBE_ROOT=. kube::version::get_version_vars && echo "${KUBE_GIT_VERSION-}"' finished in 1.730487748s
2021/09/15 09:16:46 main.go:331: Something went wrong: encountered 1 errors: [error during make e2e-test: exit status 2]
+ EXIT_VALUE=1
+ set +o xtrace
Cleaning up after docker in docker.
================================================================================
Cleaning up after docker
3e09d2d61f1e
... skipping 4 lines ...