kubernetes-sigs/azuredisk-csi-driver PR #1612 pull-azuredisk-csi-driver-external-e2e-single-az-mainv2 #1597711948558897152
Recent runs || View in Spyglass
Recent runs || View in Spyglass
| PR | alice-zheyan-yu: [V2] refactor: combine azdrivernode and node_availability into one controller |
| Result | ABORTED |
| Tests | 0 failed / 0 succeeded |
| Started | |
| Elapsed | 1h20m |
| Revision | fec06051c4296c3b1fa7923ebdb78ffea8f733f6 |
| Refs |
1612 |
... skipping 236 lines ... 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 11345 100 11345 0 0 125k 0 --:--:-- --:--:-- --:--:-- 127k Downloading https://get.helm.sh/helm-v3.10.2-linux-amd64.tar.gz Verifying checksum... Done. Preparing to install helm into /usr/local/bin helm installed into /usr/local/bin/helm docker pull k8sprow.azurecr.io/azuredisk-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a || make container-all push-manifest Error response from daemon: manifest for k8sprow.azurecr.io/azuredisk-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a not found: manifest unknown: manifest tagged by "latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a" is not found make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver' CGO_ENABLED=0 GOOS=windows go build -a -ldflags "-X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.driverVersion=latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.gitCommit=5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a -X sigs.k8s.io/azuredisk-csi-driver/pkg/azuredisk.buildDate=2022-11-29T22:06:31Z -extldflags "-static"" -tags azurediskv2 -mod vendor -o _output/amd64/azurediskpluginv2.exe ./pkg/azurediskplugin docker buildx rm container-builder || true ERROR: no builder "container-builder" found docker buildx create --use --name=container-builder container-builder # enable qemu for arm64 build # https://github.com/docker/buildx/issues/464#issuecomment-741507760 docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64 Unable to find image 'tonistiigi/binfmt:latest' locally ... skipping 692 lines ... } } ] } make[2]: Leaving directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver' docker pull k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a || make azdiskschedulerextender-all push-manifest-azdiskschedulerextender Error response from daemon: manifest for k8sprow.azurecr.io/azdiskschedulerextender-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a not found: manifest unknown: manifest tagged by "latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a" is not found make[2]: Entering directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver' docker buildx rm container-builder || true container-builder removed docker buildx create --use --name=container-builder container-builder # enable qemu for arm64 build ... skipping 1038 lines ... type: string type: object oneOf: - required: ["persistentVolumeClaimName"] - required: ["volumeSnapshotContentName"] volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 60 lines ... type: string volumeSnapshotContentName: description: volumeSnapshotContentName specifies the name of a pre-existing VolumeSnapshotContent object representing an existing volume snapshot. This field should be set if the snapshot already exists and only needs a representation in Kubernetes. This field is immutable. type: string type: object volumeSnapshotClassName: description: 'VolumeSnapshotClassName is the name of the VolumeSnapshotClass requested by the VolumeSnapshot. VolumeSnapshotClassName may be left nil to indicate that the default SnapshotClass should be used. A given cluster may have multiple default Volume SnapshotClasses: one default per CSI Driver. If a VolumeSnapshot does not specify a SnapshotClass, VolumeSnapshotSource will be checked to figure out what the associated CSI Driver is, and the default VolumeSnapshotClass associated with that CSI Driver will be used. If more than one VolumeSnapshotClass exist for a given CSI Driver and more than one have been marked as default, CreateSnapshot will fail and generate an event. Empty string is not allowed for this field.' type: string required: - source type: object status: description: status represents the current information of a snapshot. Consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object. ... skipping 2 lines ... description: 'boundVolumeSnapshotContentName is the name of the VolumeSnapshotContent object to which this VolumeSnapshot object intends to bind to. If not specified, it indicates that the VolumeSnapshot object has not been successfully bound to a VolumeSnapshotContent object yet. NOTE: To avoid possible security issues, consumers must verify binding between VolumeSnapshot and VolumeSnapshotContent objects is successful (by validating that both VolumeSnapshot and VolumeSnapshotContent point at each other) before using this object.' type: string creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it may indicate that the creation time of the snapshot is unknown. format: date-time type: string error: description: error is the last observed error during snapshot creation, if any. This field could be helpful to upper level controllers(i.e., application controller) to decide whether they should continue on waiting for the snapshot to be created based on the type of error reported. The snapshot controller will keep retrying when an error occurs during the snapshot creation. Upon success, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if the snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: type: string description: restoreSize represents the minimum size of volume required to create a volume from this snapshot. In dynamic snapshot creation case, this field will be filled in by the snapshot controller with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$ x-kubernetes-int-or-string: true type: object required: - spec type: object ... skipping 254 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 108 lines ... description: status represents the current information of a snapshot. properties: creationTime: description: creationTime is the timestamp when the point-in-time snapshot is taken by the underlying storage system. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "creation_time" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "creation_time" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. If not specified, it indicates the creation time is unknown. The format of this field is a Unix nanoseconds time encoded as an int64. On Unix, the command `date +%s%N` returns the current time in nanoseconds since 1970-01-01 00:00:00 UTC. format: int64 type: integer error: description: error is the last observed error during snapshot creation, if any. Upon success after retry, this error field will be cleared. properties: message: description: 'message is a string detailing the encountered error during snapshot creation if specified. NOTE: message may be logged, and it should not contain sensitive information.' type: string time: description: time is the timestamp when the error was encountered. format: date-time type: string type: object readyToUse: description: readyToUse indicates if a snapshot is ready to be used to restore a volume. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "ready_to_use" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "ready_to_use" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it, otherwise, this field will be set to "True". If not specified, it means the readiness of a snapshot is unknown. type: boolean restoreSize: description: restoreSize represents the complete size of the snapshot in bytes. In dynamic snapshot creation case, this field will be filled in by the CSI snapshotter sidecar with the "size_bytes" value returned from CSI "CreateSnapshot" gRPC call. For a pre-existing snapshot, this field will be filled with the "size_bytes" value returned from the CSI "ListSnapshots" gRPC call if the driver supports it. When restoring a volume from this snapshot, the size of the volume MUST NOT be smaller than the restoreSize if it is specified, otherwise the restoration will fail. If not specified, it indicates that the size is unknown. format: int64 minimum: 0 type: integer snapshotHandle: description: snapshotHandle is the CSI "snapshot_id" of a snapshot on the underlying storage system. If not specified, it indicates that dynamic snapshot creation has either failed or it is still in progress. type: string type: object required: - spec type: object served: true ... skipping 359 lines ... - volumeName - volume_context - volume_id type: object status: description: status represents the current state of AzVolumeAttachment. includes error, state, and attachment status Required properties: detail: description: Status summarizes the current attachment state of the volume attachment Nil Status indicates that the volume has not yet been attached to the node properties: ... skipping 7 lines ... role: description: The current attachment role. type: string required: - role type: object error: description: Error occurred during attach/detach of volume properties: code: type: string message: type: string parameters: ... skipping 90 lines ... - volumeName - volume_context - volume_id type: object status: description: status represents the current state of AzVolumeAttachment. includes error, state, and attachment status properties: annotation: additionalProperties: type: string description: Annotations contains additional resource information to guide driver actions ... skipping 13 lines ... role: description: The current attachment role. type: string required: - role type: object error: description: Error occurred during attach/detach of volume properties: code: type: string message: type: string parameters: ... skipping 169 lines ... - maxMountReplicaCount - volumeCapability - volumeName type: object status: description: status represents the current state of AzVolume. includes error, state, and volume status properties: detail: description: Current status detail of the AzVolume Nil detail indicates that the volume has not been created properties: accessible_topology: ... skipping 28 lines ... type: string required: - capacity_bytes - node_expansion_required - volume_id type: object error: description: Error occurred during creation/deletion of volume properties: code: type: string message: type: string parameters: ... skipping 154 lines ... - maxMountReplicaCount - volumeCapability - volumeName type: object status: description: status represents the current state of AzVolume. includes error, state, and volume status properties: annotation: additionalProperties: type: string description: Annotations contains additional resource information to guide driver actions ... skipping 34 lines ... type: string required: - capacity_bytes - node_expansion_required - volume_id type: object error: description: Error occurred during creation/deletion of volume properties: code: type: string message: type: string parameters: ... skipping 1063 lines ... image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.5.0" args: - "-csi-address=$(ADDRESS)" - "-v=2" - "-leader-election" - "--leader-election-namespace=kube-system" - '-handle-volume-inuse-error=false' - '-feature-gates=RecoverVolumeExpansionFailure=true' - "-timeout=240s" env: - name: ADDRESS value: /csi/csi.sock volumeMounts: ... skipping 448 lines ... Nov 29 22:16:12.999: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comlvmbn] to have phase Bound Nov 29 22:16:13.108: INFO: PersistentVolumeClaim test.csi.azure.comlvmbn found but phase is Pending instead of Bound. Nov 29 22:16:15.219: INFO: PersistentVolumeClaim test.csi.azure.comlvmbn found but phase is Pending instead of Bound. Nov 29 22:16:17.329: INFO: PersistentVolumeClaim test.csi.azure.comlvmbn found and phase=Bound (4.330203277s) [1mSTEP[0m: Creating pod exec-volume-test-dynamicpv-scwr [1mSTEP[0m: Creating a pod to test exec-volume-test Nov 29 22:16:17.659: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-scwr" in namespace "volume-3353" to be "Succeeded or Failed" Nov 29 22:16:17.779: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 119.91718ms Nov 29 22:16:19.890: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.230718768s Nov 29 22:16:22.000: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.340819266s Nov 29 22:16:24.111: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.452230556s Nov 29 22:16:26.221: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.561964596s Nov 29 22:16:28.333: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.673986624s Nov 29 22:16:30.444: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 12.784553305s Nov 29 22:16:32.554: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 14.895099823s Nov 29 22:16:34.665: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 17.005709402s Nov 29 22:16:36.775: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 19.115566429s Nov 29 22:16:38.885: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Pending", Reason="", readiness=false. Elapsed: 21.226038511s Nov 29 22:16:40.996: INFO: Pod "exec-volume-test-dynamicpv-scwr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 23.337047342s [1mSTEP[0m: Saw pod success Nov 29 22:16:40.996: INFO: Pod "exec-volume-test-dynamicpv-scwr" satisfied condition "Succeeded or Failed" Nov 29 22:16:41.107: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000002 pod exec-volume-test-dynamicpv-scwr container exec-container-dynamicpv-scwr: <nil> [1mSTEP[0m: delete the pod Nov 29 22:16:41.380: INFO: Waiting for pod exec-volume-test-dynamicpv-scwr to disappear Nov 29 22:16:41.492: INFO: Pod exec-volume-test-dynamicpv-scwr no longer exists [1mSTEP[0m: Deleting pod exec-volume-test-dynamicpv-scwr Nov 29 22:16:41.492: INFO: Deleting pod "exec-volume-test-dynamicpv-scwr" in namespace "volume-3353" ... skipping 21 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should allow exec of files on the volume [90mtest/e2e/storage/testsuites/volumes.go:198[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should allow exec of files on the volume","total":31,"completed":1,"skipped":59,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould fail if subpath with backstepping is outside the volume [Slow][LinuxOnly][0m [37mtest/e2e/storage/testsuites/subpath.go:280[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/framework.go:187 ... skipping 2 lines ... [1mSTEP[0m: Building a namespace api object, basename provisioning W1129 22:16:12.100083 39947 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+ Nov 29 22:16:12.100: INFO: Found PodSecurityPolicies; testing pod creation to see if PodSecurityPolicy is enabled Nov 29 22:16:12.211: INFO: No PSP annotation exists on dry run pod; assuming PodSecurityPolicy is disabled [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] test/e2e/storage/testsuites/subpath.go:280 Nov 29 22:16:12.640: INFO: Creating resource for dynamic PV Nov 29 22:16:12.640: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass provisioning-1209-e2e-scccrf7 [1mSTEP[0m: creating a claim Nov 29 22:16:12.752: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 22:16:12.863: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comsvptj] to have phase Bound Nov 29 22:16:12.999: INFO: PersistentVolumeClaim test.csi.azure.comsvptj found but phase is Pending instead of Bound. Nov 29 22:16:15.127: INFO: PersistentVolumeClaim test.csi.azure.comsvptj found but phase is Pending instead of Bound. Nov 29 22:16:17.236: INFO: PersistentVolumeClaim test.csi.azure.comsvptj found and phase=Bound (4.372876047s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-hqvl [1mSTEP[0m: Checking for subpath error in container status Nov 29 22:16:39.795: INFO: Deleting pod "pod-subpath-test-dynamicpv-hqvl" in namespace "provisioning-1209" Nov 29 22:16:39.906: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-hqvl" to be fully deleted [1mSTEP[0m: Deleting pod Nov 29 22:16:42.123: INFO: Deleting pod "pod-subpath-test-dynamicpv-hqvl" in namespace "provisioning-1209" [1mSTEP[0m: Deleting pvc Nov 29 22:16:42.244: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comsvptj" ... skipping 16 lines ... [32m• [SLOW TEST:72.266 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:280[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath with backstepping is outside the volume [Slow][LinuxOnly]","total":34,"completed":1,"skipped":19,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (filesystem volmode)] volumeMode[0m [1mshould fail to use a volume in a pod with mismatched mode [Slow][0m [37mtest/e2e/storage/testsuites/volumemode.go:299[0m [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 22:16:13.177: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename volumemode [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail to use a volume in a pod with mismatched mode [Slow] test/e2e/storage/testsuites/volumemode.go:299 Nov 29 22:16:13.947: INFO: Creating resource for dynamic PV Nov 29 22:16:13.947: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass volumemode-3564-e2e-scl9h85 [1mSTEP[0m: creating a claim Nov 29 22:16:14.173: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comxxx9c] to have phase Bound Nov 29 22:16:14.282: INFO: PersistentVolumeClaim test.csi.azure.comxxx9c found but phase is Pending instead of Bound. Nov 29 22:16:16.393: INFO: PersistentVolumeClaim test.csi.azure.comxxx9c found but phase is Pending instead of Bound. Nov 29 22:16:18.504: INFO: PersistentVolumeClaim test.csi.azure.comxxx9c found and phase=Bound (4.33148962s) [1mSTEP[0m: Creating pod [1mSTEP[0m: Waiting for the pod to fail Nov 29 22:16:21.185: INFO: Deleting pod "pod-e2ce74b5-04f8-4781-844f-f86c42ffb550" in namespace "volumemode-3564" Nov 29 22:16:21.301: INFO: Wait up to 5m0s for pod "pod-e2ce74b5-04f8-4781-844f-f86c42ffb550" to be fully deleted [1mSTEP[0m: Deleting pvc Nov 29 22:16:23.521: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comxxx9c" Nov 29 22:16:23.636: INFO: Waiting up to 5m0s for PersistentVolume pvc-b78557da-1aba-4d3f-9e6b-e27b06e62564 to get deleted Nov 29 22:16:23.746: INFO: PersistentVolume pvc-b78557da-1aba-4d3f-9e6b-e27b06e62564 found and phase=Released (109.690743ms) ... skipping 20 lines ... [32m• [SLOW TEST:82.686 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail to use a volume in a pod with mismatched mode [Slow] [90mtest/e2e/storage/testsuites/volumemode.go:299[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]","total":31,"completed":1,"skipped":45,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould concurrently access the single volume from pods on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:298[0m ... skipping 157 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:298[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the single volume from pods on the same node","total":39,"completed":1,"skipped":50,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:18:19.048: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 3 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:280[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 116 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:378[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":32,"completed":1,"skipped":99,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:19:49.513: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 3 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:269[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 199 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:138[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":31,"completed":2,"skipped":74,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (ext4)] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 22:20:54.575: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 122 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:378[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":30,"completed":1,"skipped":48,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:21:20.669: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 73 lines ... [It] should check snapshot fields, check restore correctly works, check deletion (ephemeral) test/e2e/storage/testsuites/snapshottable.go:177 Nov 29 22:17:36.643: INFO: Creating resource for dynamic PV Nov 29 22:17:36.643: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass snapshotting-7598-e2e-sc86qcg [1mSTEP[0m: [init] starting a pod to use the claim Nov 29 22:17:36.868: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-pg7k6" in namespace "snapshotting-7598" to be "Succeeded or Failed" Nov 29 22:17:36.977: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 109.264538ms Nov 29 22:17:39.088: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220185256s Nov 29 22:17:41.199: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.33142324s Nov 29 22:17:43.313: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.445474878s Nov 29 22:17:45.423: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555656817s Nov 29 22:17:47.535: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.666935422s ... skipping 30 lines ... Nov 29 22:18:52.990: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.122724243s Nov 29 22:18:55.100: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.232407807s Nov 29 22:18:57.210: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.342428027s Nov 29 22:18:59.335: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.466786731s Nov 29 22:19:01.445: INFO: Pod "pvc-snapshottable-tester-pg7k6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m24.57719088s [1mSTEP[0m: Saw pod success Nov 29 22:19:01.445: INFO: Pod "pvc-snapshottable-tester-pg7k6" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim [1mSTEP[0m: creating a SnapshotClass [1mSTEP[0m: creating a dynamic VolumeSnapshot Nov 29 22:19:01.892: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-w5wnf to become ready Nov 29 22:19:02.001: INFO: VolumeSnapshot snapshot-w5wnf found but is not ready. Nov 29 22:19:04.110: INFO: VolumeSnapshot snapshot-w5wnf found but is not ready. ... skipping 40 lines ... Nov 29 22:20:34.100: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:35.211: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:36.321: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:37.432: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:38.542: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:39.652: INFO: volumesnapshotcontents snapcontent-68936f31-9ef2-43b6-b65a-3ca179d91029 has been found and is not deleted Nov 29 22:20:40.652: INFO: WaitUntil failed after reaching the timeout 30s [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 Nov 29 22:20:40.798: INFO: Pod restored-pvc-tester-xq6wp has the following logs: Nov 29 22:20:40.799: INFO: Deleting pod "restored-pvc-tester-xq6wp" in namespace "snapshotting-7598" Nov 29 22:20:40.911: INFO: Wait up to 5m0s for pod "restored-pvc-tester-xq6wp" to be fully deleted Nov 29 22:21:13.132: INFO: deleting snapshot "snapshotting-7598"/"snapshot-w5wnf" ... skipping 26 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works, check deletion (ephemeral) [90mtest/e2e/storage/testsuites/snapshottable.go:177[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Ephemeral Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works, check deletion (ephemeral)","total":31,"completed":2,"skipped":58,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral[0m [1mshould support two pods which have the same volume definition[0m [37mtest/e2e/storage/testsuites/ephemeral.go:216[0m ... skipping 85 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should support two pods which have the same volume definition [90mtest/e2e/storage/testsuites/ephemeral.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support two pods which have the same volume definition","total":33,"completed":1,"skipped":5,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning test/e2e/storage/framework/testsuite.go:51 Nov 29 22:21:23.068: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 273 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [90mtest/e2e/storage/testsuites/provisioning.go:208[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":34,"completed":2,"skipped":44,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:21:23.896: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 38 lines ... Nov 29 22:19:50.529: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com2zg68] to have phase Bound Nov 29 22:19:50.638: INFO: PersistentVolumeClaim test.csi.azure.com2zg68 found but phase is Pending instead of Bound. Nov 29 22:19:52.749: INFO: PersistentVolumeClaim test.csi.azure.com2zg68 found but phase is Pending instead of Bound. Nov 29 22:19:54.859: INFO: PersistentVolumeClaim test.csi.azure.com2zg68 found and phase=Bound (4.32964075s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-nmjj [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:19:55.191: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-nmjj" in namespace "provisioning-9199" to be "Succeeded or Failed" Nov 29 22:19:55.303: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 111.960931ms Nov 29 22:19:57.412: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221500068s Nov 29 22:19:59.523: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 4.331783541s Nov 29 22:20:01.632: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 6.441142765s Nov 29 22:20:03.743: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 8.552037942s Nov 29 22:20:05.853: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 10.661993862s ... skipping 22 lines ... Nov 29 22:20:54.395: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 59.203935284s Nov 29 22:20:56.504: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.313033768s Nov 29 22:20:58.613: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.422430982s Nov 29 22:21:00.723: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.532190736s Nov 29 22:21:02.834: INFO: Pod "pod-subpath-test-dynamicpv-nmjj": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m7.642777923s [1mSTEP[0m: Saw pod success Nov 29 22:21:02.834: INFO: Pod "pod-subpath-test-dynamicpv-nmjj" satisfied condition "Succeeded or Failed" Nov 29 22:21:02.942: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-nmjj container test-container-volume-dynamicpv-nmjj: <nil> [1mSTEP[0m: delete the pod Nov 29 22:21:03.199: INFO: Waiting for pod pod-subpath-test-dynamicpv-nmjj to disappear Nov 29 22:21:03.307: INFO: Pod pod-subpath-test-dynamicpv-nmjj no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-nmjj Nov 29 22:21:03.307: INFO: Deleting pod "pod-subpath-test-dynamicpv-nmjj" in namespace "provisioning-9199" ... skipping 23 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support non-existent path [90mtest/e2e/storage/testsuites/subpath.go:196[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support non-existent path","total":32,"completed":2,"skipped":235,"failed":0} [36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (filesystem volmode)] volumeMode[0m [1mshould not mount / map unused volumes in a pod [LinuxOnly][0m [37mtest/e2e/storage/testsuites/volumemode.go:354[0m ... skipping 63 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m should not mount / map unused volumes in a pod [LinuxOnly] [90mtest/e2e/storage/testsuites/volumemode.go:354[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":33,"completed":2,"skipped":264,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:22:26.492: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 58 lines ... test/e2e/framework/framework.go:188 Nov 29 22:22:27.892: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "volumelimits-9895" for this suite. [32m•[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] volumeLimits should verify that all csinodes have volume limits","total":33,"completed":3,"skipped":445,"failed":0} [36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] ... skipping 35 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:258[0m [36mDistro debian doesn't support ntfs -- skipping[0m test/e2e/storage/framework/testsuite.go:127 [90m------------------------------[0m ... skipping 171 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] [90mtest/e2e/storage/testsuites/provisioning.go:208[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource]","total":39,"completed":2,"skipped":116,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould fail if subpath file is outside the volume [Slow][LinuxOnly][0m [37mtest/e2e/storage/testsuites/subpath.go:258[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 22:20:54.620: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename provisioning [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail if subpath file is outside the volume [Slow][LinuxOnly] test/e2e/storage/testsuites/subpath.go:258 Nov 29 22:20:55.385: INFO: Creating resource for dynamic PV Nov 29 22:20:55.385: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass provisioning-4150-e2e-scg9jq4 [1mSTEP[0m: creating a claim Nov 29 22:20:55.497: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 22:20:55.609: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comqfk2w] to have phase Bound Nov 29 22:20:55.719: INFO: PersistentVolumeClaim test.csi.azure.comqfk2w found but phase is Pending instead of Bound. Nov 29 22:20:57.830: INFO: PersistentVolumeClaim test.csi.azure.comqfk2w found but phase is Pending instead of Bound. Nov 29 22:20:59.940: INFO: PersistentVolumeClaim test.csi.azure.comqfk2w found and phase=Bound (4.33074793s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-ktxx [1mSTEP[0m: Checking for subpath error in container status Nov 29 22:21:28.492: INFO: Deleting pod "pod-subpath-test-dynamicpv-ktxx" in namespace "provisioning-4150" Nov 29 22:21:28.607: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-ktxx" to be fully deleted [1mSTEP[0m: Deleting pod Nov 29 22:21:30.826: INFO: Deleting pod "pod-subpath-test-dynamicpv-ktxx" in namespace "provisioning-4150" [1mSTEP[0m: Deleting pvc Nov 29 22:21:30.936: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comqfk2w" ... skipping 22 lines ... [32m• [SLOW TEST:108.662 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail if subpath file is outside the volume [Slow][LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:258[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath file is outside the volume [Slow][LinuxOnly]","total":31,"completed":3,"skipped":125,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:22:43.320: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 38 lines ... Nov 29 22:21:22.115: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comw96s9] to have phase Bound Nov 29 22:21:22.228: INFO: PersistentVolumeClaim test.csi.azure.comw96s9 found but phase is Pending instead of Bound. Nov 29 22:21:24.355: INFO: PersistentVolumeClaim test.csi.azure.comw96s9 found but phase is Pending instead of Bound. Nov 29 22:21:26.465: INFO: PersistentVolumeClaim test.csi.azure.comw96s9 found and phase=Bound (4.350504014s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-kbjt [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:21:26.797: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-kbjt" in namespace "provisioning-9079" to be "Succeeded or Failed" Nov 29 22:21:26.914: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 116.955752ms Nov 29 22:21:29.024: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 2.227670622s Nov 29 22:21:31.137: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 4.340072665s Nov 29 22:21:33.246: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 6.449597496s Nov 29 22:21:35.357: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 8.560632822s Nov 29 22:21:37.473: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 10.676318817s ... skipping 23 lines ... Nov 29 22:22:28.137: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m1.34044857s Nov 29 22:22:30.248: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m3.451140535s Nov 29 22:22:32.358: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m5.560959157s Nov 29 22:22:34.477: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Pending", Reason="", readiness=false. Elapsed: 1m7.680287791s Nov 29 22:22:36.618: INFO: Pod "pod-subpath-test-dynamicpv-kbjt": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m9.821098126s [1mSTEP[0m: Saw pod success Nov 29 22:22:36.618: INFO: Pod "pod-subpath-test-dynamicpv-kbjt" satisfied condition "Succeeded or Failed" Nov 29 22:22:36.736: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000000 pod pod-subpath-test-dynamicpv-kbjt container test-container-subpath-dynamicpv-kbjt: <nil> [1mSTEP[0m: delete the pod Nov 29 22:22:37.003: INFO: Waiting for pod pod-subpath-test-dynamicpv-kbjt to disappear Nov 29 22:22:37.112: INFO: Pod pod-subpath-test-dynamicpv-kbjt no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-kbjt Nov 29 22:22:37.112: INFO: Deleting pod "pod-subpath-test-dynamicpv-kbjt" in namespace "provisioning-9079" ... skipping 29 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support existing single file [LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:221[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing single file [LinuxOnly]","total":31,"completed":3,"skipped":159,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral[0m [1mshould support multiple inline ephemeral volumes[0m [37mtest/e2e/storage/testsuites/ephemeral.go:254[0m ... skipping 51 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should support multiple inline ephemeral volumes [90mtest/e2e/storage/testsuites/ephemeral.go:254[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":32,"completed":3,"skipped":238,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral[0m [1mshould create read/write inline ephemeral volume[0m [37mtest/e2e/storage/testsuites/ephemeral.go:196[0m ... skipping 45 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should create read/write inline ephemeral volume [90mtest/e2e/storage/testsuites/ephemeral.go:196[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read/write inline ephemeral volume","total":33,"completed":4,"skipped":506,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:24:08.327: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 38 lines ... Nov 29 22:22:42.137: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comhcfkb] to have phase Bound Nov 29 22:22:42.246: INFO: PersistentVolumeClaim test.csi.azure.comhcfkb found but phase is Pending instead of Bound. Nov 29 22:22:44.359: INFO: PersistentVolumeClaim test.csi.azure.comhcfkb found but phase is Pending instead of Bound. Nov 29 22:22:46.469: INFO: PersistentVolumeClaim test.csi.azure.comhcfkb found and phase=Bound (4.331959763s) [1mSTEP[0m: Creating pod exec-volume-test-dynamicpv-nvrr [1mSTEP[0m: Creating a pod to test exec-volume-test Nov 29 22:22:46.799: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-nvrr" in namespace "volume-7610" to be "Succeeded or Failed" Nov 29 22:22:46.915: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 115.331252ms Nov 29 22:22:49.025: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.225500338s Nov 29 22:22:51.135: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.336006419s Nov 29 22:22:53.246: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446469437s Nov 29 22:22:55.355: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555773142s Nov 29 22:22:57.466: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.667167147s ... skipping 43 lines ... Nov 29 22:24:30.356: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 1m43.55658275s Nov 29 22:24:32.467: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 1m45.667687863s Nov 29 22:24:34.577: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 1m47.777561438s Nov 29 22:24:36.687: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Pending", Reason="", readiness=false. Elapsed: 1m49.887511732s Nov 29 22:24:38.797: INFO: Pod "exec-volume-test-dynamicpv-nvrr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m51.99817077s [1mSTEP[0m: Saw pod success Nov 29 22:24:38.798: INFO: Pod "exec-volume-test-dynamicpv-nvrr" satisfied condition "Succeeded or Failed" Nov 29 22:24:38.907: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000000 pod exec-volume-test-dynamicpv-nvrr container exec-container-dynamicpv-nvrr: <nil> [1mSTEP[0m: delete the pod Nov 29 22:24:39.164: INFO: Waiting for pod exec-volume-test-dynamicpv-nvrr to disappear Nov 29 22:24:39.274: INFO: Pod exec-volume-test-dynamicpv-nvrr no longer exists [1mSTEP[0m: Deleting pod exec-volume-test-dynamicpv-nvrr Nov 29 22:24:39.274: INFO: Deleting pod "exec-volume-test-dynamicpv-nvrr" in namespace "volume-7610" ... skipping 27 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should allow exec of files on the volume [90mtest/e2e/storage/testsuites/volumes.go:198[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] volumes should allow exec of files on the volume","total":39,"completed":3,"skipped":155,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy[0m [1m(OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents[0m [37mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m ... skipping 119 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup skips ownership changes to the volume contents","total":34,"completed":3,"skipped":73,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:26:04.222: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 37 lines ... test/e2e/framework/framework.go:188 Nov 29 22:26:05.485: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "volumelimits-3372" for this suite. [32m•[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs)] volumeLimits should verify that all csinodes have volume limits","total":34,"completed":4,"skipped":175,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:26:05.762: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 21 lines ... test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 22:26:05.798: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename topology [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies test/e2e/storage/testsuites/topology.go:194 Nov 29 22:26:06.563: INFO: Driver didn't provide topology keys -- skipping [AfterEach] [Testpattern: Dynamic PV (immediate binding)] topology test/e2e/framework/framework.go:188 Nov 29 22:26:06.564: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "topology-1181" for this suite. [36m[1mS [SKIPPING] [0.986 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (immediate binding)] topology [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement][0m [90mtest/e2e/storage/testsuites/topology.go:194[0m [36mDriver didn't provide topology keys -- skipping[0m test/e2e/storage/testsuites/topology.go:126 [90m------------------------------[0m ... skipping 36 lines ... [It] should check snapshot fields, check restore correctly works, check deletion (ephemeral) test/e2e/storage/testsuites/snapshottable.go:177 Nov 29 22:23:50.456: INFO: Creating resource for dynamic PV Nov 29 22:23:50.456: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass snapshotting-1876-e2e-scqlkgw [1mSTEP[0m: [init] starting a pod to use the claim Nov 29 22:23:50.680: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-2hz7x" in namespace "snapshotting-1876" to be "Succeeded or Failed" Nov 29 22:23:50.794: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 113.548304ms Nov 29 22:23:52.907: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 2.227098374s Nov 29 22:23:55.017: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 4.336609981s Nov 29 22:23:57.128: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 6.447459289s Nov 29 22:23:59.237: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 8.556510788s Nov 29 22:24:01.346: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 10.666111271s ... skipping 18 lines ... Nov 29 22:24:41.445: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 50.764742902s Nov 29 22:24:43.556: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 52.875352106s Nov 29 22:24:45.667: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 54.986934754s Nov 29 22:24:47.778: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Pending", Reason="", readiness=false. Elapsed: 57.097215863s Nov 29 22:24:49.889: INFO: Pod "pvc-snapshottable-tester-2hz7x": Phase="Succeeded", Reason="", readiness=false. Elapsed: 59.208444184s [1mSTEP[0m: Saw pod success Nov 29 22:24:49.889: INFO: Pod "pvc-snapshottable-tester-2hz7x" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim [1mSTEP[0m: creating a SnapshotClass [1mSTEP[0m: creating a dynamic VolumeSnapshot Nov 29 22:24:50.338: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-rv5tq to become ready Nov 29 22:24:50.449: INFO: VolumeSnapshot snapshot-rv5tq found but is not ready. Nov 29 22:24:52.561: INFO: VolumeSnapshot snapshot-rv5tq found but is not ready. ... skipping 49 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works, check deletion (ephemeral) [90mtest/e2e/storage/testsuites/snapshottable.go:177[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works, check deletion (ephemeral)","total":31,"completed":4,"skipped":175,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (xfs)][Slow] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 22:26:30.163: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 217 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:138[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":31,"completed":4,"skipped":164,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:26:32.776: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 404 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with pvc data source in parallel [Slow] [90mtest/e2e/storage/testsuites/provisioning.go:459[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with pvc data source in parallel [Slow]","total":30,"completed":2,"skipped":210,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] provisioning ... skipping 323 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext3)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should store data [90mtest/e2e/storage/testsuites/volumes.go:161[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext3)] volumes should store data","total":33,"completed":5,"skipped":660,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] ... skipping 124 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support restarting containers using file as subpath [Slow][LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:333[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using file as subpath [Slow][LinuxOnly]","total":34,"completed":5,"skipped":299,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould verify container cannot write to subpath readonly volumes [Slow][0m [37mtest/e2e/storage/testsuites/subpath.go:425[0m ... skipping 16 lines ... Nov 29 22:28:03.741: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 22:28:03.855: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comtbvkh] to have phase Bound Nov 29 22:28:03.966: INFO: PersistentVolumeClaim test.csi.azure.comtbvkh found but phase is Pending instead of Bound. Nov 29 22:28:06.077: INFO: PersistentVolumeClaim test.csi.azure.comtbvkh found but phase is Pending instead of Bound. Nov 29 22:28:08.191: INFO: PersistentVolumeClaim test.csi.azure.comtbvkh found and phase=Bound (4.335284613s) [1mSTEP[0m: Creating pod to format volume volume-prep-provisioning-5913 Nov 29 22:28:08.525: INFO: Waiting up to 5m0s for pod "volume-prep-provisioning-5913" in namespace "provisioning-5913" to be "Succeeded or Failed" Nov 29 22:28:08.636: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 111.091579ms Nov 29 22:28:10.748: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222969634s Nov 29 22:28:12.867: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 4.342109888s Nov 29 22:28:14.978: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 6.453496808s Nov 29 22:28:17.090: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 8.564979013s Nov 29 22:28:19.204: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 10.679292901s Nov 29 22:28:21.318: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 12.79307213s Nov 29 22:28:23.431: INFO: Pod "volume-prep-provisioning-5913": Phase="Pending", Reason="", readiness=false. Elapsed: 14.906579087s Nov 29 22:28:25.545: INFO: Pod "volume-prep-provisioning-5913": Phase="Succeeded", Reason="", readiness=false. Elapsed: 17.020562733s [1mSTEP[0m: Saw pod success Nov 29 22:28:25.545: INFO: Pod "volume-prep-provisioning-5913" satisfied condition "Succeeded or Failed" Nov 29 22:28:25.545: INFO: Deleting pod "volume-prep-provisioning-5913" in namespace "provisioning-5913" Nov 29 22:28:25.663: INFO: Wait up to 5m0s for pod "volume-prep-provisioning-5913" to be fully deleted [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-pb7g [1mSTEP[0m: Checking for subpath error in container status Nov 29 22:28:40.113: INFO: Deleting pod "pod-subpath-test-dynamicpv-pb7g" in namespace "provisioning-5913" Nov 29 22:28:40.224: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-pb7g" to be fully deleted [1mSTEP[0m: Deleting pod Nov 29 22:28:42.449: INFO: Deleting pod "pod-subpath-test-dynamicpv-pb7g" in namespace "provisioning-5913" [1mSTEP[0m: Deleting pvc Nov 29 22:28:42.560: INFO: Deleting PersistentVolumeClaim "test.csi.azure.comtbvkh" ... skipping 19 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should verify container cannot write to subpath readonly volumes [Slow] [90mtest/e2e/storage/testsuites/subpath.go:425[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should verify container cannot write to subpath readonly volumes [Slow]","total":33,"completed":6,"skipped":718,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy[0m [1m(Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents[0m [37mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m ... skipping 113 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents","total":39,"completed":4,"skipped":161,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:29:27.665: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 80 lines ... Nov 29 22:23:59.428: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comr4twh] to have phase Bound Nov 29 22:23:59.537: INFO: PersistentVolumeClaim test.csi.azure.comr4twh found but phase is Pending instead of Bound. Nov 29 22:24:01.647: INFO: PersistentVolumeClaim test.csi.azure.comr4twh found but phase is Pending instead of Bound. Nov 29 22:24:03.756: INFO: PersistentVolumeClaim test.csi.azure.comr4twh found and phase=Bound (4.327549919s) [1mSTEP[0m: [init] starting a pod to use the claim [1mSTEP[0m: [init] check pod success Nov 29 22:24:04.198: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-975dg" in namespace "snapshotting-3458" to be "Succeeded or Failed" Nov 29 22:24:04.306: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 108.160542ms Nov 29 22:24:06.415: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 2.217326498s Nov 29 22:24:08.526: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 4.327752157s Nov 29 22:24:10.635: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 6.436560359s Nov 29 22:24:12.745: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 8.546578182s Nov 29 22:24:14.853: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 10.655375582s ... skipping 13 lines ... Nov 29 22:24:44.390: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 40.192225325s Nov 29 22:24:46.500: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 42.302092387s Nov 29 22:24:48.626: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 44.427705962s Nov 29 22:24:50.770: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Pending", Reason="", readiness=false. Elapsed: 46.571773581s Nov 29 22:24:52.879: INFO: Pod "pvc-snapshottable-tester-975dg": Phase="Succeeded", Reason="", readiness=false. Elapsed: 48.681161516s [1mSTEP[0m: Saw pod success Nov 29 22:24:52.879: INFO: Pod "pvc-snapshottable-tester-975dg" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim Nov 29 22:24:52.988: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comr4twh] to have phase Bound Nov 29 22:24:53.096: INFO: PersistentVolumeClaim test.csi.azure.comr4twh found and phase=Bound (108.08768ms) [1mSTEP[0m: [init] checking the PV [1mSTEP[0m: [init] deleting the pod Nov 29 22:24:53.462: INFO: Pod pvc-snapshottable-tester-975dg has the following logs: ... skipping 13 lines ... Nov 29 22:25:00.895: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e creationTime:2022-11-29T22:24:56Z readyToUse:true restoreSize:5Gi] Nov 29 22:25:00.895: INFO: snapshotContentName snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e [1mSTEP[0m: checking the snapshot [1mSTEP[0m: checking the SnapshotContent [1mSTEP[0m: Modifying source data test [1mSTEP[0m: modifying the data in the source PVC Nov 29 22:25:01.338: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-sf7gs" in namespace "snapshotting-3458" to be "Succeeded or Failed" Nov 29 22:25:01.447: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 108.875995ms Nov 29 22:25:03.561: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 2.223049017s Nov 29 22:25:05.671: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332840211s Nov 29 22:25:07.779: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 6.440778483s Nov 29 22:25:09.888: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 8.549689497s Nov 29 22:25:11.998: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 10.659300432s ... skipping 35 lines ... Nov 29 22:26:27.963: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 1m26.624332955s Nov 29 22:26:30.072: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 1m28.733876617s Nov 29 22:26:32.182: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 1m30.843696362s Nov 29 22:26:34.291: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Pending", Reason="", readiness=false. Elapsed: 1m32.952683016s Nov 29 22:26:36.400: INFO: Pod "pvc-snapshottable-data-tester-sf7gs": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m35.062081866s [1mSTEP[0m: Saw pod success Nov 29 22:26:36.401: INFO: Pod "pvc-snapshottable-data-tester-sf7gs" satisfied condition "Succeeded or Failed" Nov 29 22:26:36.621: INFO: Pod pvc-snapshottable-data-tester-sf7gs has the following logs: Nov 29 22:26:36.621: INFO: Deleting pod "pvc-snapshottable-data-tester-sf7gs" in namespace "snapshotting-3458" Nov 29 22:26:36.737: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-sf7gs" to be fully deleted [1mSTEP[0m: creating a pvc from the snapshot [1mSTEP[0m: starting a pod to use the snapshot Nov 29 22:29:11.290: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-ofo3t2sm.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-3458 exec restored-pvc-tester-mx297 --namespace=snapshotting-3458 -- cat /mnt/test/data' ... skipping 33 lines ... Nov 29 22:29:37.684: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:38.797: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:39.907: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:41.018: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:42.129: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:43.238: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted Nov 29 22:29:44.239: INFO: WaitUntil failed after reaching the timeout 30s [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 Nov 29 22:29:44.350: INFO: Error getting logs for pod restored-pvc-tester-mx297: the server could not find the requested resource (get pods restored-pvc-tester-mx297) Nov 29 22:29:44.350: INFO: Deleting pod "restored-pvc-tester-mx297" in namespace "snapshotting-3458" Nov 29 22:29:44.459: INFO: deleting claim "snapshotting-3458"/"pvc-fsd9r" Nov 29 22:29:44.571: INFO: deleting snapshot "snapshotting-3458"/"snapshot-qqjzl" Nov 29 22:29:44.681: INFO: deleting snapshot content "snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e" Nov 29 22:29:45.017: INFO: Waiting up to 5m0s for volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e to be deleted Nov 29 22:29:45.125: INFO: volumesnapshotcontents snapcontent-e5ffe0a6-abbc-4f74-ac2c-f3b82d93086e has been found and is not deleted ... skipping 27 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) [90mtest/e2e/storage/testsuites/snapshottable.go:278[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":32,"completed":4,"skipped":319,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Ephemeral Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] ... skipping 41 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (filesystem volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach][0m [90mtest/e2e/storage/testsuites/volumemode.go:299[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 8 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Inline-volume (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:280[0m [36mDriver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 57 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should create read/write inline ephemeral volume [90mtest/e2e/storage/testsuites/ephemeral.go:196[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":30,"completed":3,"skipped":533,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] ... skipping 101 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m should not mount / map unused volumes in a pod [LinuxOnly] [90mtest/e2e/storage/testsuites/volumemode.go:354[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should not mount / map unused volumes in a pod [LinuxOnly]","total":39,"completed":5,"skipped":323,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:30:59.627: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 3 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:242[0m [36mDistro debian doesn't support ntfs -- skipping[0m test/e2e/storage/framework/testsuite.go:127 [90m------------------------------[0m ... skipping 43 lines ... Nov 29 22:30:13.285: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comp9f8j] to have phase Bound Nov 29 22:30:13.398: INFO: PersistentVolumeClaim test.csi.azure.comp9f8j found but phase is Pending instead of Bound. Nov 29 22:30:15.507: INFO: PersistentVolumeClaim test.csi.azure.comp9f8j found but phase is Pending instead of Bound. Nov 29 22:30:17.617: INFO: PersistentVolumeClaim test.csi.azure.comp9f8j found and phase=Bound (4.332153677s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-8856 [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:30:17.954: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-8856" in namespace "provisioning-7674" to be "Succeeded or Failed" Nov 29 22:30:18.063: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 109.379842ms Nov 29 22:30:20.175: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220809674s Nov 29 22:30:22.734: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 4.779697676s Nov 29 22:30:24.852: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 6.897995501s Nov 29 22:30:26.963: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 9.008787532s Nov 29 22:30:29.075: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 11.120930016s ... skipping 43 lines ... Nov 29 22:32:01.965: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 1m44.011231412s Nov 29 22:32:04.075: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 1m46.120650043s Nov 29 22:32:06.188: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 1m48.233896388s Nov 29 22:32:08.299: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Pending", Reason="", readiness=false. Elapsed: 1m50.345098947s Nov 29 22:32:10.412: INFO: Pod "pod-subpath-test-dynamicpv-8856": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m52.457849468s [1mSTEP[0m: Saw pod success Nov 29 22:32:10.412: INFO: Pod "pod-subpath-test-dynamicpv-8856" satisfied condition "Succeeded or Failed" Nov 29 22:32:10.522: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000002 pod pod-subpath-test-dynamicpv-8856 container test-container-volume-dynamicpv-8856: <nil> [1mSTEP[0m: delete the pod Nov 29 22:32:10.785: INFO: Waiting for pod pod-subpath-test-dynamicpv-8856 to disappear Nov 29 22:32:10.894: INFO: Pod pod-subpath-test-dynamicpv-8856 no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-8856 Nov 29 22:32:10.894: INFO: Deleting pod "pod-subpath-test-dynamicpv-8856" in namespace "provisioning-7674" ... skipping 23 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support existing directory [90mtest/e2e/storage/testsuites/subpath.go:207[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directory","total":30,"completed":4,"skipped":550,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:32:52.980: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 70 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] volumeIO [90mtest/e2e/storage/framework/testsuite.go:50[0m should write files of various sizes, verify size, validate content [Slow] [90mtest/e2e/storage/testsuites/volume_io.go:149[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumeIO should write files of various sizes, verify size, validate content [Slow]","total":32,"completed":5,"skipped":461,"failed":0} [36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (block volmode)] multiVolume [Slow][0m [1mshould access to two volumes with different volume mode and retain data across pod recreation on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:209[0m ... skipping 207 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with different volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:209[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node","total":33,"completed":7,"skipped":735,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (ext4)] multiVolume [Slow][0m [1mshould concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS][0m [37mtest/e2e/storage/testsuites/multivolume.go:323[0m ... skipping 121 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:323[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":31,"completed":5,"skipped":199,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould access to two volumes with the same volume mode and retain data across pod recreation on different node[0m [37mtest/e2e/storage/testsuites/multivolume.go:168[0m ... skipping 196 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on different node [90mtest/e2e/storage/testsuites/multivolume.go:168[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node","total":34,"completed":6,"skipped":323,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:34:41.848: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 45 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Inline-volume (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:258[0m [36mDriver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 85 lines ... Nov 29 22:26:33.907: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.combjhl7] to have phase Bound Nov 29 22:26:34.016: INFO: PersistentVolumeClaim test.csi.azure.combjhl7 found but phase is Pending instead of Bound. Nov 29 22:26:36.126: INFO: PersistentVolumeClaim test.csi.azure.combjhl7 found but phase is Pending instead of Bound. Nov 29 22:26:38.237: INFO: PersistentVolumeClaim test.csi.azure.combjhl7 found and phase=Bound (4.329259734s) [1mSTEP[0m: [init] starting a pod to use the claim [1mSTEP[0m: [init] check pod success Nov 29 22:26:38.674: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-24dfl" in namespace "snapshotting-2433" to be "Succeeded or Failed" Nov 29 22:26:38.783: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 109.189125ms Nov 29 22:26:40.892: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218507624s Nov 29 22:26:43.010: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.335727497s Nov 29 22:26:45.120: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446164651s Nov 29 22:26:47.229: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.555496575s Nov 29 22:26:49.348: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.674051268s ... skipping 63 lines ... Nov 29 22:29:04.494: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 2m25.819881967s Nov 29 22:29:06.605: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 2m27.930832834s Nov 29 22:29:08.717: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 2m30.043441649s Nov 29 22:29:10.829: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Pending", Reason="", readiness=false. Elapsed: 2m32.155044406s Nov 29 22:29:12.939: INFO: Pod "pvc-snapshottable-tester-24dfl": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2m34.264827554s [1mSTEP[0m: Saw pod success Nov 29 22:29:12.939: INFO: Pod "pvc-snapshottable-tester-24dfl" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim Nov 29 22:29:13.048: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.combjhl7] to have phase Bound Nov 29 22:29:13.156: INFO: PersistentVolumeClaim test.csi.azure.combjhl7 found and phase=Bound (108.558137ms) [1mSTEP[0m: [init] checking the PV [1mSTEP[0m: [init] deleting the pod Nov 29 22:29:13.515: INFO: Pod pvc-snapshottable-tester-24dfl has the following logs: ... skipping 35 lines ... Nov 29 22:29:26.401: INFO: WaitUntil finished successfully after 112.713371ms [1mSTEP[0m: getting the snapshot and snapshot content [1mSTEP[0m: checking the snapshot [1mSTEP[0m: checking the SnapshotContent [1mSTEP[0m: Modifying source data test [1mSTEP[0m: modifying the data in the source PVC Nov 29 22:29:26.952: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-49rbq" in namespace "snapshotting-2433" to be "Succeeded or Failed" Nov 29 22:29:27.061: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 109.691192ms Nov 29 22:29:29.174: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.2224262s Nov 29 22:29:31.285: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.333503652s Nov 29 22:29:33.397: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.444710148s Nov 29 22:29:35.506: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.554616859s Nov 29 22:29:37.617: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 10.664755133s ... skipping 93 lines ... Nov 29 22:32:56.059: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 3m29.106775352s Nov 29 22:32:58.173: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 3m31.220860976s Nov 29 22:33:00.282: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 3m33.329941249s Nov 29 22:33:02.391: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Pending", Reason="", readiness=false. Elapsed: 3m35.439682479s Nov 29 22:33:04.501: INFO: Pod "pvc-snapshottable-data-tester-49rbq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 3m37.549563461s [1mSTEP[0m: Saw pod success Nov 29 22:33:04.501: INFO: Pod "pvc-snapshottable-data-tester-49rbq" satisfied condition "Succeeded or Failed" Nov 29 22:33:04.749: INFO: Pod pvc-snapshottable-data-tester-49rbq has the following logs: Nov 29 22:33:04.749: INFO: Deleting pod "pvc-snapshottable-data-tester-49rbq" in namespace "snapshotting-2433" Nov 29 22:33:04.864: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-49rbq" to be fully deleted [1mSTEP[0m: creating a pvc from the snapshot [1mSTEP[0m: starting a pod to use the snapshot Nov 29 22:34:15.439: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-ofo3t2sm.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-2433 exec restored-pvc-tester-5rvgr --namespace=snapshotting-2433 -- cat /mnt/test/data' ... skipping 33 lines ... Nov 29 22:34:41.727: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:42.837: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:43.947: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:45.058: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:46.167: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:47.277: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted Nov 29 22:34:48.278: INFO: WaitUntil failed after reaching the timeout 30s [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 Nov 29 22:34:48.387: INFO: Error getting logs for pod restored-pvc-tester-5rvgr: the server could not find the requested resource (get pods restored-pvc-tester-5rvgr) Nov 29 22:34:48.387: INFO: Deleting pod "restored-pvc-tester-5rvgr" in namespace "snapshotting-2433" Nov 29 22:34:48.499: INFO: deleting claim "snapshotting-2433"/"pvc-kk99q" Nov 29 22:34:48.607: INFO: deleting snapshot "snapshotting-2433"/"pre-provisioned-snapshot-47476ed7-3e35-419f-80e6-be424cf56f30" Nov 29 22:34:48.716: INFO: deleting snapshot content "pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30" Nov 29 22:34:49.066: INFO: Waiting up to 5m0s for volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 to be deleted Nov 29 22:34:49.186: INFO: volumesnapshotcontents pre-provisioned-snapcontent-47476ed7-3e35-419f-80e6-be424cf56f30 has been found and is not deleted ... skipping 27 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) [90mtest/e2e/storage/testsuites/snapshottable.go:278[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":31,"completed":5,"skipped":315,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:35:06.783: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 216 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with different volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:209[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node","total":39,"completed":6,"skipped":364,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral[0m [1mshould create read-only inline ephemeral volume[0m [37mtest/e2e/storage/testsuites/ephemeral.go:175[0m ... skipping 45 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should create read-only inline ephemeral volume [90mtest/e2e/storage/testsuites/ephemeral.go:175[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should create read-only inline ephemeral volume","total":31,"completed":6,"skipped":239,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] volumeMode test/e2e/storage/framework/testsuite.go:51 Nov 29 22:36:06.175: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 118 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":32,"completed":6,"skipped":463,"failed":0} [36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:36:07.360: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 118 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, new pod fsgroup applied to volume contents","total":30,"completed":5,"skipped":687,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:36:10.267: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 91 lines ... Nov 29 22:36:08.358: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comgr697] to have phase Bound Nov 29 22:36:08.466: INFO: PersistentVolumeClaim test.csi.azure.comgr697 found but phase is Pending instead of Bound. Nov 29 22:36:10.576: INFO: PersistentVolumeClaim test.csi.azure.comgr697 found but phase is Pending instead of Bound. Nov 29 22:36:12.692: INFO: PersistentVolumeClaim test.csi.azure.comgr697 found and phase=Bound (4.333926692s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-cjv4 [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:36:13.028: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-cjv4" in namespace "provisioning-9827" to be "Succeeded or Failed" Nov 29 22:36:13.136: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 108.443001ms Nov 29 22:36:15.251: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 2.223681594s Nov 29 22:36:17.361: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332939008s Nov 29 22:36:19.473: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 6.44492689s Nov 29 22:36:21.583: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 8.554904881s Nov 29 22:36:23.692: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 10.66477092s Nov 29 22:36:25.803: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 12.775154495s Nov 29 22:36:27.913: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 14.885272397s Nov 29 22:36:30.022: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Pending", Reason="", readiness=false. Elapsed: 16.994454332s Nov 29 22:36:32.132: INFO: Pod "pod-subpath-test-dynamicpv-cjv4": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.104377389s [1mSTEP[0m: Saw pod success Nov 29 22:36:32.132: INFO: Pod "pod-subpath-test-dynamicpv-cjv4" satisfied condition "Succeeded or Failed" Nov 29 22:36:32.241: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-cjv4 container test-container-subpath-dynamicpv-cjv4: <nil> [1mSTEP[0m: delete the pod Nov 29 22:36:32.506: INFO: Waiting for pod pod-subpath-test-dynamicpv-cjv4 to disappear Nov 29 22:36:32.616: INFO: Pod pod-subpath-test-dynamicpv-cjv4 no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-cjv4 Nov 29 22:36:32.616: INFO: Deleting pod "pod-subpath-test-dynamicpv-cjv4" in namespace "provisioning-9827" ... skipping 23 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support readOnly file specified in the volumeMount [LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:382[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly file specified in the volumeMount [LinuxOnly]","total":32,"completed":7,"skipped":483,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS][0m [37mtest/e2e/storage/testsuites/multivolume.go:378[0m ... skipping 83 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:378[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":33,"completed":8,"skipped":772,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 22:37:21.104: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 38 lines ... Nov 29 22:35:28.281: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comf68sb] to have phase Bound Nov 29 22:35:28.390: INFO: PersistentVolumeClaim test.csi.azure.comf68sb found but phase is Pending instead of Bound. Nov 29 22:35:30.502: INFO: PersistentVolumeClaim test.csi.azure.comf68sb found but phase is Pending instead of Bound. Nov 29 22:35:32.613: INFO: PersistentVolumeClaim test.csi.azure.comf68sb found and phase=Bound (4.331926417s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-zvl5 [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:35:32.942: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-zvl5" in namespace "provisioning-8274" to be "Succeeded or Failed" Nov 29 22:35:33.053: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 110.786942ms Nov 29 22:35:35.170: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.228407714s Nov 29 22:35:37.280: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.338407053s Nov 29 22:35:39.391: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.449504918s Nov 29 22:35:41.502: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.559956343s Nov 29 22:35:43.612: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.670021354s ... skipping 10 lines ... Nov 29 22:36:06.830: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 33.88800496s Nov 29 22:36:08.941: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 35.998872534s Nov 29 22:36:11.052: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 38.1107163s Nov 29 22:36:13.164: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Pending", Reason="", readiness=false. Elapsed: 40.222053926s Nov 29 22:36:15.277: INFO: Pod "pod-subpath-test-dynamicpv-zvl5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 42.33540556s [1mSTEP[0m: Saw pod success Nov 29 22:36:15.277: INFO: Pod "pod-subpath-test-dynamicpv-zvl5" satisfied condition "Succeeded or Failed" Nov 29 22:36:15.387: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000002 pod pod-subpath-test-dynamicpv-zvl5 container test-container-subpath-dynamicpv-zvl5: <nil> [1mSTEP[0m: delete the pod Nov 29 22:36:15.619: INFO: Waiting for pod pod-subpath-test-dynamicpv-zvl5 to disappear Nov 29 22:36:15.728: INFO: Pod pod-subpath-test-dynamicpv-zvl5 no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-zvl5 Nov 29 22:36:15.728: INFO: Deleting pod "pod-subpath-test-dynamicpv-zvl5" in namespace "provisioning-8274" ... skipping 29 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support readOnly directory specified in the volumeMount [90mtest/e2e/storage/testsuites/subpath.go:367[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support readOnly directory specified in the volumeMount","total":39,"completed":7,"skipped":416,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould access to two volumes with the same volume mode and retain data across pod recreation on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:138[0m ... skipping 196 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:138[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":34,"completed":7,"skipped":520,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:38:11.563: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 86 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support restarting containers using directory as subpath [Slow] [90mtest/e2e/storage/testsuites/subpath.go:322[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support restarting containers using directory as subpath [Slow]","total":31,"completed":6,"skipped":368,"failed":0} [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:38:29.974: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping [AfterEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/framework/framework.go:188 ... skipping 131 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (Always)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with same fsgroup applied to the volume contents","total":31,"completed":7,"skipped":335,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (block volmode)] multiVolume [Slow][0m [1mshould concurrently access the single volume from pods on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:298[0m ... skipping 157 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:298[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node","total":30,"completed":6,"skipped":774,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (ext3)] volumes[0m [1mshould allow exec of files on the volume[0m [37mtest/e2e/storage/testsuites/volumes.go:198[0m ... skipping 17 lines ... Nov 29 22:38:12.599: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comb6279] to have phase Bound Nov 29 22:38:12.708: INFO: PersistentVolumeClaim test.csi.azure.comb6279 found but phase is Pending instead of Bound. Nov 29 22:38:14.817: INFO: PersistentVolumeClaim test.csi.azure.comb6279 found but phase is Pending instead of Bound. Nov 29 22:38:16.938: INFO: PersistentVolumeClaim test.csi.azure.comb6279 found and phase=Bound (4.339176548s) [1mSTEP[0m: Creating pod exec-volume-test-dynamicpv-mttn [1mSTEP[0m: Creating a pod to test exec-volume-test Nov 29 22:38:17.282: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-mttn" in namespace "volume-8434" to be "Succeeded or Failed" Nov 29 22:38:17.392: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 109.930443ms Nov 29 22:38:19.503: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221230212s Nov 29 22:38:21.613: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 4.331416428s Nov 29 22:38:23.723: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 6.441526178s Nov 29 22:38:25.836: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 8.554667063s Nov 29 22:38:27.946: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 10.664086253s ... skipping 6 lines ... Nov 29 22:38:42.719: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 25.437682943s Nov 29 22:38:44.830: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 27.548797679s Nov 29 22:38:46.946: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 29.663992969s Nov 29 22:38:49.055: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Pending", Reason="", readiness=false. Elapsed: 31.77384831s Nov 29 22:38:51.166: INFO: Pod "exec-volume-test-dynamicpv-mttn": Phase="Succeeded", Reason="", readiness=false. Elapsed: 33.884510164s [1mSTEP[0m: Saw pod success Nov 29 22:38:51.166: INFO: Pod "exec-volume-test-dynamicpv-mttn" satisfied condition "Succeeded or Failed" Nov 29 22:38:51.276: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000002 pod exec-volume-test-dynamicpv-mttn container exec-container-dynamicpv-mttn: <nil> [1mSTEP[0m: delete the pod Nov 29 22:38:51.534: INFO: Waiting for pod exec-volume-test-dynamicpv-mttn to disappear Nov 29 22:38:51.642: INFO: Pod exec-volume-test-dynamicpv-mttn no longer exists [1mSTEP[0m: Deleting pod exec-volume-test-dynamicpv-mttn Nov 29 22:38:51.642: INFO: Deleting pod "exec-volume-test-dynamicpv-mttn" in namespace "volume-8434" ... skipping 27 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext3)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should allow exec of files on the volume [90mtest/e2e/storage/testsuites/volumes.go:198[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext3)] volumes should allow exec of files on the volume","total":34,"completed":8,"skipped":582,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] volumes[0m [1mshould store data[0m [37mtest/e2e/storage/testsuites/volumes.go:161[0m ... skipping 116 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should store data [90mtest/e2e/storage/testsuites/volumes.go:161[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] volumes should store data","total":39,"completed":8,"skipped":466,"failed":0} [36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:40:15.815: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 3 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.002 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath file is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:258[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 29 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (block volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail to use a volume in a pod with mismatched mode [Slow] [BeforeEach][0m [90mtest/e2e/storage/testsuites/volumemode.go:299[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 159 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:298[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single volume from pods on the same node","total":31,"completed":7,"skipped":416,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] ... skipping 259 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:138[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on the same node","total":32,"completed":8,"skipped":495,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (ntfs)][Feature:Windows] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 22:40:57.405: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 38 lines ... Nov 29 22:37:22.121: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com92pw9] to have phase Bound Nov 29 22:37:22.232: INFO: PersistentVolumeClaim test.csi.azure.com92pw9 found but phase is Pending instead of Bound. Nov 29 22:37:24.343: INFO: PersistentVolumeClaim test.csi.azure.com92pw9 found but phase is Pending instead of Bound. Nov 29 22:37:26.455: INFO: PersistentVolumeClaim test.csi.azure.com92pw9 found and phase=Bound (4.333369162s) [1mSTEP[0m: [init] starting a pod to use the claim [1mSTEP[0m: [init] check pod success Nov 29 22:37:26.919: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-wl4n2" in namespace "snapshotting-6515" to be "Succeeded or Failed" Nov 29 22:37:27.030: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 110.79358ms Nov 29 22:37:29.148: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 2.228518632s Nov 29 22:37:31.260: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 4.340333404s Nov 29 22:37:33.375: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 6.455453731s Nov 29 22:37:35.486: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 8.566473153s Nov 29 22:37:37.597: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 10.678169797s ... skipping 15 lines ... Nov 29 22:38:11.405: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 44.485555181s Nov 29 22:38:13.517: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 46.597478558s Nov 29 22:38:15.629: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 48.709872827s Nov 29 22:38:17.740: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Pending", Reason="", readiness=false. Elapsed: 50.821199794s Nov 29 22:38:19.856: INFO: Pod "pvc-snapshottable-tester-wl4n2": Phase="Succeeded", Reason="", readiness=false. Elapsed: 52.937076266s [1mSTEP[0m: Saw pod success Nov 29 22:38:19.856: INFO: Pod "pvc-snapshottable-tester-wl4n2" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim Nov 29 22:38:19.967: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com92pw9] to have phase Bound Nov 29 22:38:20.078: INFO: PersistentVolumeClaim test.csi.azure.com92pw9 found and phase=Bound (110.823626ms) [1mSTEP[0m: [init] checking the PV [1mSTEP[0m: [init] deleting the pod Nov 29 22:38:20.448: INFO: Pod pvc-snapshottable-tester-wl4n2 has the following logs: ... skipping 33 lines ... Nov 29 22:38:28.952: INFO: WaitUntil finished successfully after 132.739503ms [1mSTEP[0m: getting the snapshot and snapshot content [1mSTEP[0m: checking the snapshot [1mSTEP[0m: checking the SnapshotContent [1mSTEP[0m: Modifying source data test [1mSTEP[0m: modifying the data in the source PVC Nov 29 22:38:29.512: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-pdvld" in namespace "snapshotting-6515" to be "Succeeded or Failed" Nov 29 22:38:29.623: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 110.97318ms Nov 29 22:38:31.735: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 2.223206916s Nov 29 22:38:33.847: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 4.334775268s Nov 29 22:38:35.959: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 6.446503179s Nov 29 22:38:38.071: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 8.559159578s Nov 29 22:38:40.183: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 10.671448287s ... skipping 30 lines ... Nov 29 22:39:45.675: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 1m16.163043327s Nov 29 22:39:47.795: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 1m18.282705512s Nov 29 22:39:49.906: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 1m20.394252203s Nov 29 22:39:52.018: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Pending", Reason="", readiness=false. Elapsed: 1m22.506078736s Nov 29 22:39:54.129: INFO: Pod "pvc-snapshottable-data-tester-pdvld": Phase="Succeeded", Reason="", readiness=false. Elapsed: 1m24.617159238s [1mSTEP[0m: Saw pod success Nov 29 22:39:54.129: INFO: Pod "pvc-snapshottable-data-tester-pdvld" satisfied condition "Succeeded or Failed" Nov 29 22:39:54.381: INFO: Pod pvc-snapshottable-data-tester-pdvld has the following logs: Nov 29 22:39:54.381: INFO: Deleting pod "pvc-snapshottable-data-tester-pdvld" in namespace "snapshotting-6515" Nov 29 22:39:54.499: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-pdvld" to be fully deleted [1mSTEP[0m: creating a pvc from the snapshot [1mSTEP[0m: starting a pod to use the snapshot Nov 29 22:40:15.074: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-ofo3t2sm.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-6515 exec restored-pvc-tester-kctlj --namespace=snapshotting-6515 -- cat /mnt/test/data' ... skipping 47 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) [90mtest/e2e/storage/testsuites/snapshottable.go:278[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":33,"completed":9,"skipped":806,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Pre-provisioned Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] ... skipping 5 lines ... [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should check snapshot fields, check restore correctly works, check deletion (ephemeral) test/e2e/storage/testsuites/snapshottable.go:177 Nov 29 22:41:02.206: INFO: volume type "DynamicPV" is not ephemeral [AfterEach] volume snapshot controller test/e2e/storage/testsuites/snapshottable.go:172 Nov 29 22:41:02.317: INFO: Error getting logs for pod restored-pvc-tester-kctlj: the server could not find the requested resource (get pods restored-pvc-tester-kctlj) Nov 29 22:41:02.317: INFO: Deleting pod "restored-pvc-tester-kctlj" in namespace "snapshotting-6515" Nov 29 22:41:02.430: INFO: deleting claim "snapshotting-6515"/"pvc-t2rxn" Nov 29 22:41:02.541: INFO: deleting snapshot class "snapshotting-6515p8stj" Nov 29 22:41:02.652: INFO: Waiting up to 5m0s for volumesnapshotclasses snapshotting-6515p8stj to be deleted Nov 29 22:41:02.763: INFO: volumesnapshotclasses snapshotting-6515p8stj is not found and has been deleted Nov 29 22:41:02.763: INFO: WaitUntil finished successfully after 110.599586ms ... skipping 35 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Inline-volume (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:242[0m [36mDriver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 8 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:269[0m [36mDistro debian doesn't support ntfs -- skipping[0m test/e2e/storage/framework/testsuite.go:127 [90m------------------------------[0m ... skipping 78 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should create read/write inline ephemeral volume [90mtest/e2e/storage/testsuites/ephemeral.go:196[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should create read/write inline ephemeral volume","total":30,"completed":7,"skipped":786,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:43:13.828: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 99 lines ... Nov 29 22:40:58.461: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comgkzng] to have phase Bound Nov 29 22:40:58.569: INFO: PersistentVolumeClaim test.csi.azure.comgkzng found but phase is Pending instead of Bound. Nov 29 22:41:00.692: INFO: PersistentVolumeClaim test.csi.azure.comgkzng found but phase is Pending instead of Bound. Nov 29 22:41:02.801: INFO: PersistentVolumeClaim test.csi.azure.comgkzng found and phase=Bound (4.340626346s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-9wn9 [1mSTEP[0m: Creating a pod to test atomic-volume-subpath Nov 29 22:41:03.128: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-9wn9" in namespace "provisioning-8617" to be "Succeeded or Failed" Nov 29 22:41:03.238: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 110.046771ms Nov 29 22:41:05.350: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221383802s Nov 29 22:41:07.460: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 4.331714511s Nov 29 22:41:09.570: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 6.441617941s Nov 29 22:41:11.679: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 8.551106224s Nov 29 22:41:13.790: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Pending", Reason="", readiness=false. Elapsed: 10.661677016s ... skipping 56 lines ... Nov 29 22:43:14.078: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Running", Reason="", readiness=true. Elapsed: 2m10.949413293s Nov 29 22:43:16.186: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Running", Reason="", readiness=true. Elapsed: 2m13.058005695s Nov 29 22:43:18.296: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Running", Reason="", readiness=true. Elapsed: 2m15.167691668s Nov 29 22:43:20.408: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Running", Reason="", readiness=false. Elapsed: 2m17.27993011s Nov 29 22:43:22.517: INFO: Pod "pod-subpath-test-dynamicpv-9wn9": Phase="Succeeded", Reason="", readiness=false. Elapsed: 2m19.389134021s [1mSTEP[0m: Saw pod success Nov 29 22:43:22.517: INFO: Pod "pod-subpath-test-dynamicpv-9wn9" satisfied condition "Succeeded or Failed" Nov 29 22:43:22.626: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-9wn9 container test-container-subpath-dynamicpv-9wn9: <nil> [1mSTEP[0m: delete the pod Nov 29 22:43:22.889: INFO: Waiting for pod pod-subpath-test-dynamicpv-9wn9 to disappear Nov 29 22:43:22.997: INFO: Pod pod-subpath-test-dynamicpv-9wn9 no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-9wn9 Nov 29 22:43:22.997: INFO: Deleting pod "pod-subpath-test-dynamicpv-9wn9" in namespace "provisioning-8617" ... skipping 29 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support file as subpath [LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:232[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support file as subpath [LinuxOnly]","total":32,"completed":9,"skipped":571,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:44:35.542: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 55 lines ... test/e2e/storage/external/external.go:262 [90m------------------------------[0m [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (block volmode)] volumeMode[0m [1mshould fail to use a volume in a pod with mismatched mode [Slow][0m [37mtest/e2e/storage/testsuites/volumemode.go:299[0m [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volumeMode test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 22:40:04.068: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename volumemode [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail to use a volume in a pod with mismatched mode [Slow] test/e2e/storage/testsuites/volumemode.go:299 Nov 29 22:40:04.831: INFO: Creating resource for dynamic PV Nov 29 22:40:04.831: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass volumemode-9248-e2e-scbzzh4 [1mSTEP[0m: creating a claim Nov 29 22:40:05.057: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com6jbjs] to have phase Bound Nov 29 22:40:05.166: INFO: PersistentVolumeClaim test.csi.azure.com6jbjs found but phase is Pending instead of Bound. Nov 29 22:40:07.276: INFO: PersistentVolumeClaim test.csi.azure.com6jbjs found but phase is Pending instead of Bound. Nov 29 22:40:09.395: INFO: PersistentVolumeClaim test.csi.azure.com6jbjs found and phase=Bound (4.338517246s) [1mSTEP[0m: Creating pod [1mSTEP[0m: Waiting for the pod to fail Nov 29 22:40:12.052: INFO: Deleting pod "pod-16f53040-3d36-498a-84b8-6c0f20a974c3" in namespace "volumemode-9248" Nov 29 22:40:12.165: INFO: Wait up to 5m0s for pod "pod-16f53040-3d36-498a-84b8-6c0f20a974c3" to be fully deleted [1mSTEP[0m: Deleting pvc Nov 29 22:40:14.388: INFO: Deleting PersistentVolumeClaim "test.csi.azure.com6jbjs" Nov 29 22:40:14.498: INFO: Waiting up to 5m0s for PersistentVolume pvc-ae4fd49b-afb3-48cf-8457-e98d8ae11f1b to get deleted Nov 29 22:40:14.607: INFO: PersistentVolume pvc-ae4fd49b-afb3-48cf-8457-e98d8ae11f1b found and phase=Released (108.7547ms) ... skipping 57 lines ... [32m• [SLOW TEST:271.708 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] volumeMode [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail to use a volume in a pod with mismatched mode [Slow] [90mtest/e2e/storage/testsuites/volumemode.go:299[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumeMode should fail to use a volume in a pod with mismatched mode [Slow]","total":34,"completed":9,"skipped":592,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS][0m [37mtest/e2e/storage/testsuites/multivolume.go:323[0m ... skipping 108 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:323[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":31,"completed":8,"skipped":373,"failed":0} [36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:45:26.898: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 62 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath with backstepping is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:280[0m [36mDistro debian doesn't support ntfs -- skipping[0m test/e2e/storage/framework/testsuite.go:127 [90m------------------------------[0m ... skipping 110 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with pvc data source [90mtest/e2e/storage/testsuites/provisioning.go:421[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with pvc data source","total":39,"completed":9,"skipped":515,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould concurrently access the single read-only volume from pods on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:423[0m ... skipping 88 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single read-only volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:423[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":31,"completed":8,"skipped":488,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (ext4)] multiVolume [Slow][0m [1mshould concurrently access the single read-only volume from pods on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:423[0m ... skipping 81 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single read-only volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:423[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":33,"completed":10,"skipped":979,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 22:46:09.175: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 93 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should support multiple inline ephemeral volumes [90mtest/e2e/storage/testsuites/ephemeral.go:254[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (late-binding)] ephemeral should support multiple inline ephemeral volumes","total":39,"completed":10,"skipped":534,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (block volmode)] multiVolume [Slow][0m [1mshould concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS][0m [37mtest/e2e/storage/testsuites/multivolume.go:323[0m ... skipping 107 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [90mtest/e2e/storage/testsuites/multivolume.go:323[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the volume and restored snapshot from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":30,"completed":8,"skipped":992,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:48:08.986: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 131 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should store data [90mtest/e2e/storage/testsuites/volumes.go:161[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] volumes should store data","total":34,"completed":10,"skipped":640,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] capacity test/e2e/storage/framework/testsuite.go:51 Nov 29 22:48:17.328: INFO: Driver test.csi.azure.com doesn't publish storage capacity -- skipping ... skipping 24 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Pre-provisioned PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if subpath directory is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:242[0m [36mDriver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 124 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy [90mtest/e2e/storage/framework/testsuite.go:50[0m (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents [90mtest/e2e/storage/testsuites/fsgroupchangepolicy.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] fsgroupchangepolicy (OnRootMismatch)[LinuxOnly], pod created with an initial fsgroup, volume contents ownership changed via chgrp in first pod, new pod with different fsgroup applied to the volume contents","total":33,"completed":11,"skipped":1021,"failed":0} Nov 29 22:48:47.350: INFO: Running AfterSuite actions on all nodes Nov 29 22:48:47.350: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func19.2 Nov 29 22:48:47.350: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func9.2 Nov 29 22:48:47.350: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func8.2 Nov 29 22:48:47.350: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage/vsphere.glob..func17.3 Nov 29 22:48:47.350: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage/vsphere.glob..func9.2 ... skipping 106 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single read-only volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:423[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":32,"completed":10,"skipped":657,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (ext4)] multiVolume [Slow][0m [1mshould access to two volumes with different volume mode and retain data across pod recreation on different node[0m [37mtest/e2e/storage/testsuites/multivolume.go:248[0m ... skipping 195 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with different volume mode and retain data across pod recreation on different node [90mtest/e2e/storage/testsuites/multivolume.go:248[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node","total":31,"completed":9,"skipped":461,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow][0m [1mshould concurrently access the single volume from pods on the same node[0m [37mtest/e2e/storage/testsuites/multivolume.go:298[0m ... skipping 152 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:298[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the single volume from pods on the same node","total":34,"completed":11,"skipped":806,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 22:50:17.961: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 158 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should store data [90mtest/e2e/storage/testsuites/volumes.go:161[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] volumes should store data","total":31,"completed":9,"skipped":524,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (ext4)] multiVolume [Slow][0m [1mshould access to two volumes with the same volume mode and retain data across pod recreation on different node[0m [37mtest/e2e/storage/testsuites/multivolume.go:168[0m ... skipping 190 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on different node [90mtest/e2e/storage/testsuites/multivolume.go:168[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node","total":39,"completed":11,"skipped":550,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 22:50:58.801: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 3 lines ... [36m[1mS [SKIPPING] in Spec Setup (BeforeEach) [0.001 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Inline-volume (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [BeforeEach][0m [90mtest/e2e/storage/testsuites/subpath.go:269[0m [36mDriver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping[0m test/e2e/storage/external/external.go:262 [90m------------------------------[0m ... skipping 140 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should store data [90mtest/e2e/storage/testsuites/volumes.go:161[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] volumes should store data","total":32,"completed":11,"skipped":675,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould support existing directories when readOnly specified in the volumeSource[0m [37mtest/e2e/storage/testsuites/subpath.go:397[0m ... skipping 17 lines ... Nov 29 22:50:22.088: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.compzwnl] to have phase Bound Nov 29 22:50:22.197: INFO: PersistentVolumeClaim test.csi.azure.compzwnl found but phase is Pending instead of Bound. Nov 29 22:50:24.307: INFO: PersistentVolumeClaim test.csi.azure.compzwnl found but phase is Pending instead of Bound. Nov 29 22:50:26.416: INFO: PersistentVolumeClaim test.csi.azure.compzwnl found and phase=Bound (4.327190095s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-9tfq [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:50:26.747: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-9tfq" in namespace "provisioning-1346" to be "Succeeded or Failed" Nov 29 22:50:26.860: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 113.119499ms Nov 29 22:50:28.970: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.222928534s Nov 29 22:50:31.079: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332670559s Nov 29 22:50:33.189: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.442044754s Nov 29 22:50:35.298: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.551259083s Nov 29 22:50:37.407: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 10.660895299s ... skipping 4 lines ... Nov 29 22:50:47.960: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 21.213556328s Nov 29 22:50:50.070: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 23.323585049s Nov 29 22:50:52.191: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 25.444275172s Nov 29 22:50:54.301: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 27.554422091s Nov 29 22:50:56.411: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 29.664059245s [1mSTEP[0m: Saw pod success Nov 29 22:50:56.411: INFO: Pod "pod-subpath-test-dynamicpv-9tfq" satisfied condition "Succeeded or Failed" Nov 29 22:50:56.520: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-9tfq container test-container-subpath-dynamicpv-9tfq: <nil> [1mSTEP[0m: delete the pod Nov 29 22:50:56.794: INFO: Waiting for pod pod-subpath-test-dynamicpv-9tfq to disappear Nov 29 22:50:56.908: INFO: Pod pod-subpath-test-dynamicpv-9tfq no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-9tfq Nov 29 22:50:56.908: INFO: Deleting pod "pod-subpath-test-dynamicpv-9tfq" in namespace "provisioning-1346" [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-9tfq [1mSTEP[0m: Creating a pod to test subpath Nov 29 22:50:57.145: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-9tfq" in namespace "provisioning-1346" to be "Succeeded or Failed" Nov 29 22:50:57.253: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 107.919017ms Nov 29 22:50:59.363: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218146931s Nov 29 22:51:01.473: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328038274s Nov 29 22:51:03.590: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 6.444802114s Nov 29 22:51:05.698: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Pending", Reason="", readiness=false. Elapsed: 8.55371756s Nov 29 22:51:07.808: INFO: Pod "pod-subpath-test-dynamicpv-9tfq": Phase="Succeeded", Reason="", readiness=false. Elapsed: 10.663183058s [1mSTEP[0m: Saw pod success Nov 29 22:51:07.808: INFO: Pod "pod-subpath-test-dynamicpv-9tfq" satisfied condition "Succeeded or Failed" Nov 29 22:51:07.917: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-9tfq container test-container-subpath-dynamicpv-9tfq: <nil> [1mSTEP[0m: delete the pod Nov 29 22:51:08.152: INFO: Waiting for pod pod-subpath-test-dynamicpv-9tfq to disappear Nov 29 22:51:08.261: INFO: Pod pod-subpath-test-dynamicpv-9tfq no longer exists [1mSTEP[0m: Deleting pod pod-subpath-test-dynamicpv-9tfq Nov 29 22:51:08.261: INFO: Deleting pod "pod-subpath-test-dynamicpv-9tfq" in namespace "provisioning-1346" ... skipping 29 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support existing directories when readOnly specified in the volumeSource [90mtest/e2e/storage/testsuites/subpath.go:397[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support existing directories when readOnly specified in the volumeSource","total":31,"completed":10,"skipped":563,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (block volmode)] provisioning ... skipping 209 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should concurrently access the single read-only volume from pods on the same node [90mtest/e2e/storage/testsuites/multivolume.go:423[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should concurrently access the single read-only volume from pods on the same node","total":39,"completed":12,"skipped":586,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] volume-expand test/e2e/storage/framework/testsuite.go:51 Nov 29 22:53:28.623: INFO: Driver "test.csi.azure.com" does not support volume expansion - skipping ... skipping 215 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on different node [90mtest/e2e/storage/testsuites/multivolume.go:168[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node","total":31,"completed":10,"skipped":481,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] provisioning test/e2e/storage/framework/testsuite.go:51 Nov 29 22:54:01.535: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 320 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with the same volume mode and retain data across pod recreation on different node [90mtest/e2e/storage/testsuites/multivolume.go:168[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should access to two volumes with the same volume mode and retain data across pod recreation on different node","total":34,"completed":12,"skipped":827,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould fail if subpath directory is outside the volume [Slow][LinuxOnly][0m [37mtest/e2e/storage/testsuites/subpath.go:242[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 22:52:22.970: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename provisioning [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail if subpath directory is outside the volume [Slow][LinuxOnly] test/e2e/storage/testsuites/subpath.go:242 Nov 29 22:52:23.740: INFO: Creating resource for dynamic PV Nov 29 22:52:23.740: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass provisioning-444-e2e-sc69rp6 [1mSTEP[0m: creating a claim Nov 29 22:52:23.853: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 22:52:23.964: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.commdd8p] to have phase Bound Nov 29 22:52:24.072: INFO: PersistentVolumeClaim test.csi.azure.commdd8p found but phase is Pending instead of Bound. Nov 29 22:52:26.183: INFO: PersistentVolumeClaim test.csi.azure.commdd8p found but phase is Pending instead of Bound. Nov 29 22:52:28.292: INFO: PersistentVolumeClaim test.csi.azure.commdd8p found and phase=Bound (4.328059256s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-6msj [1mSTEP[0m: Checking for subpath error in container status Nov 29 22:53:34.843: INFO: Deleting pod "pod-subpath-test-dynamicpv-6msj" in namespace "provisioning-444" Nov 29 22:53:34.956: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-6msj" to be fully deleted [1mSTEP[0m: Deleting pod Nov 29 22:53:37.174: INFO: Deleting pod "pod-subpath-test-dynamicpv-6msj" in namespace "provisioning-444" [1mSTEP[0m: Deleting pvc Nov 29 22:53:37.283: INFO: Deleting PersistentVolumeClaim "test.csi.azure.commdd8p" ... skipping 22 lines ... [32m• [SLOW TEST:146.612 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail if subpath directory is outside the volume [Slow][LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:242[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should fail if subpath directory is outside the volume [Slow][LinuxOnly]","total":31,"completed":11,"skipped":654,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] ... skipping 109 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should support two pods which have the same volume definition [90mtest/e2e/storage/testsuites/ephemeral.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should support two pods which have the same volume definition","total":32,"completed":12,"skipped":708,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (immediate binding)] topology ... skipping 71 lines ... [1mSTEP[0m: Deleting pod external-injector in namespace multivolume-7696 Nov 29 22:55:05.952: INFO: Waiting for pod external-injector to disappear Nov 29 22:55:06.061: INFO: Pod external-injector still exists Nov 29 22:55:08.062: INFO: Waiting for pod external-injector to disappear Nov 29 22:55:08.172: INFO: Pod external-injector no longer exists [1mSTEP[0m: Creating pod1 with a volume on {Name: Selector:map[] Affinity:nil} Nov 29 23:00:08.721: FAIL: Unexpected error: <*errors.errorString | 0xc0010bb970>: { s: "pod \"pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7\" is not Running: timed out waiting for the condition", } pod "pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7" is not Running: timed out waiting for the condition occurred ... skipping 73 lines ... Nov 29 23:06:21.482: INFO: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 found and phase=Released (4m36.089557107s) Nov 29 23:06:26.591: INFO: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 found and phase=Released (4m41.198510482s) Nov 29 23:06:31.700: INFO: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 found and phase=Released (4m46.307926763s) Nov 29 23:06:36.809: INFO: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 found and phase=Released (4m51.417129731s) Nov 29 23:06:41.919: INFO: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 found and phase=Released (4m56.527019487s) [1mSTEP[0m: Deleting sc Nov 29 23:06:47.033: FAIL: while cleanup resource Unexpected error: <errors.aggregate | len:1, cap:1>: [ [ { msg: "persistent Volume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 not deleted by dynamic provisioner: PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 still exists within 5m0s", err: { s: "PersistentVolume pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90 still exists within 5m0s", ... skipping 6 lines ... Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func2() test/e2e/storage/testsuites/multivolume.go:129 +0x24f panic({0x6ca56c0, 0xc002dfc4c0}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x632c000, 0x7a05090}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0022e6140, 0x13b}, {0xc001b3ca60?, 0x714b92b?, 0xc001b3ca80?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc0022e6000, 0x126}, {0xc003680aa0?, 0xc0022e6000?, 0xc001182a90?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc001b3cbc8, {0x7a29638, 0xa6b18d8}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc001b3cbc8, {0x7a29638, 0xa6b18d8}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0xc002f7b2a0?, {0x7a08d80?, 0xc0010bb970?}, {0x0?, 0x2?, 0xc000232240?}) ... skipping 24 lines ... Nov 29 23:06:47.249: INFO: At 2022-11-29 22:54:46 +0000 UTC - event for external-injector: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90" Nov 29 23:06:47.249: INFO: At 2022-11-29 22:54:58 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-92105279-vmss000001} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" Nov 29 23:06:47.249: INFO: At 2022-11-29 22:54:58 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-92105279-vmss000001} Created: Created container external-injector Nov 29 23:06:47.249: INFO: At 2022-11-29 22:54:58 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-92105279-vmss000001} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" in 336.744176ms Nov 29 23:06:47.249: INFO: At 2022-11-29 22:54:59 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-92105279-vmss000001} Started: Started container external-injector Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:05 +0000 UTC - event for external-injector: {kubelet k8s-agentpool1-92105279-vmss000001} Killing: Stopping container external-injector Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:08 +0000 UTC - event for pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7: {attachdetach-controller } FailedAttachVolume: Multi-Attach error for volume "pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90" Volume is already exclusively attached to one node and can't be attached to another Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:08 +0000 UTC - event for test.csi.azure.com24pr9-cloned: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } Provisioning: External provisioner is provisioning volume for claim "multivolume-7696/test.csi.azure.com24pr9-cloned" Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:08 +0000 UTC - event for test.csi.azure.com24pr9-cloned: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:10 +0000 UTC - event for test.csi.azure.com24pr9-cloned: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } ProvisioningSucceeded: Successfully provisioned volume pvc-582aa7fe-e6b3-4754-a015-feb618bc2bc8 Nov 29 23:06:47.249: INFO: At 2022-11-29 22:55:56 +0000 UTC - event for pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90" Nov 29 23:06:47.249: INFO: At 2022-11-29 22:57:11 +0000 UTC - event for pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[volume1], unattached volumes=[volume1 kube-api-access-xkltq]: timed out waiting for the condition Nov 29 23:06:47.249: INFO: At 2022-11-29 22:57:56 +0000 UTC - event for pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:06:47.249: INFO: At 2022-11-29 23:00:54 +0000 UTC - event for pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-12fcc63f-82f3-4441-b355-9d4c1a542d90) to be attached to node (k8s-agentpool1-92105279-vmss000000): rpc error: code = Unknown desc = context deadline exceeded Nov 29 23:06:47.357: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:06:47.357: INFO: Nov 29 23:06:47.573: INFO: Logging node info for node k8s-agentpool1-92105279-vmss000000 Nov 29 23:06:47.683: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-92105279-vmss000000 e7716b3a-d547-407e-a9ed-2706618be43c 26675 0 2022-11-29 22:04:46 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D8s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-ofo3t2sm kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-92105279-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D8s_v3 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:westeurope topology.kubernetes.io/zone:0 topology.test.csi.azure.com/zone:] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-92105279-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 22:04:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2022-11-29 22:04:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:02 +0000 UTC FieldsV1 {"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-29 22:05:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {cloud-node-manager Update v1 2022-11-29 22:06:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-11-29 22:49:46 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.test.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status} {kube-controller-manager Update v1 2022-11-29 22:55:56 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-92105279-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{33672720384 0} {<nil>} 32883516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{32886288384 0} {<nil>} 32115516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 22:05:49 +0000 UTC,LastTransitionTime:2022-11-29 22:05:49 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:03:44 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:03:44 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:03:44 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:03:44 +0000 UTC,LastTransitionTime:2022-11-29 22:05:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-92105279-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2d155693e7fa4281a08fb77e113c1f09,SystemUUID:badc595a-a00b-e64f-ad71-6b1c80de3c78,BootID:ae2f9bff-8abd-4f93-a225-c6b443284255,KernelVersion:5.4.0-1094-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.14+azure-1,KubeletVersion:v1.23.14,KubeProxyVersion:v1.23.14,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253346057,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:664ec9627d47a4160789ad6994367128b25007b164773c5f1e598028440a158a k8sprow.azurecr.io/azuredisk-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a],SizeBytes:142859492,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:f5241226198f5a54d22540acf2b3933ea0f49458f90c51fc75833d0c428687b8 k8s.gcr.io/e2e-test-images/agnhost:2.36],SizeBytes:125426639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:efa8e4b7776fa9fb2279745008780cefec632ee5fdb081e0f6e234ea4b8bbf61 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.14],SizeBytes:112312488,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:8c3fc3c2667004ad6bbdf723bb64c5da66a5cb8b11d4ee59b67179b686223b13 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v5.0.1],SizeBytes:52732401,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:949db4eea46ea31f9afa0b49ad0b55f7a31f91aa03b408e0ac9b7720c8ef7d29 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:51376235,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:72fbc9d17947d1a54c09a16671e6810fd7c0d8f09c0502fdbdd212c2192451a7 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:51044459,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:075ea1f8270312350f1396ab6677251e803e61a523822d5abfa5e6acd180cfab mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:50806891,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:eb7d329559d954093f7a234bc0ff322764de48a0b2a1007229032fb53e7bb9eb mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:49070187,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:2dfff4dc10f6d25b0a34c01bf4d166540cbf1ff2bccd52635350039138eb23c4 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:48775275,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3e23b9ab6bfee42a1751019c21a60aa474964bd4cba00cbed5055430424242cc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.24.0],SizeBytes:48500843,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:acec9ee6626c2cdea89107ba191ae36e6f60339c9b3b3215b5a59bf0e2e730ba mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.14],SizeBytes:47853675,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:759ac7020e982884b99121583bd577658b5663b1c1ed69746356aa6ce9879da4 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.18],SizeBytes:45850731,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:2fbd1e1a0538a06f2061afd45975df70c942654aa7f86e920720169ee439c2d6 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.5.1],SizeBytes:19743433,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:31547791294872570393470991481c2477a311031d3a03e0ae54eb164347dc34 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.7.0],SizeBytes:17857485,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16028126,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/test.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-1ac83ddd-a789-425d-9509-b034bcbde378],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/test.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-1ac83ddd-a789-425d-9509-b034bcbde378,DevicePath:,},},Config:nil,},} Nov 29 23:06:47.684: INFO: ... skipping 110 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m [91m[1mshould concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [Measurement][0m [90mtest/e2e/storage/testsuites/multivolume.go:378[0m [91mNov 29 23:00:08.721: Unexpected error: <*errors.errorString | 0xc0010bb970>: { s: "pod \"pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7\" is not Running: timed out waiting for the condition", } pod "pod-fc4fd6b9-31f6-4c5a-92f7-9b34e10b24c7" is not Running: timed out waiting for the condition occurred[0m test/e2e/storage/testsuites/multivolume.go:696 [90m------------------------------[0m {"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]","total":34,"completed":12,"skipped":845,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic Snapshot (retain policy)] snapshottable[Feature:VolumeSnapshotDataSource] ... skipping 236 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should support two pods which have the same volume definition [90mtest/e2e/storage/testsuites/ephemeral.go:216[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (block volmode) (late-binding)] ephemeral should support two pods which have the same volume definition","total":31,"completed":12,"skipped":749,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 23:07:46.697: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 84 lines ... test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 23:07:46.814: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename topology [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail to schedule a pod which has topologies that conflict with AllowedTopologies test/e2e/storage/testsuites/topology.go:194 Nov 29 23:07:47.576: INFO: Driver didn't provide topology keys -- skipping [AfterEach] [Testpattern: Dynamic PV (delayed binding)] topology test/e2e/framework/framework.go:188 Nov 29 23:07:47.577: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready [1mSTEP[0m: Destroying namespace "topology-9655" for this suite. [36m[1mS [SKIPPING] [1.088 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (delayed binding)] topology [90mtest/e2e/storage/framework/testsuite.go:50[0m [36m[1mshould fail to schedule a pod which has topologies that conflict with AllowedTopologies [Measurement][0m [90mtest/e2e/storage/testsuites/topology.go:194[0m [36mDriver didn't provide topology keys -- skipping[0m test/e2e/storage/testsuites/topology.go:126 [90m------------------------------[0m ... skipping 95 lines ... Nov 29 22:55:08.291: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json Nov 29 22:55:08.292: INFO: ExecWithOptions: Clientset creation Nov 29 22:55:08.292: INFO: ExecWithOptions: execute(POST https://kubetest-ofo3t2sm.westeurope.cloudapp.azure.com/api/v1/namespaces/multivolume-2962/pods/pod-64739c7b-38ec-45e2-b2da-230500376278/exec?command=%2Fbin%2Fsh&command=-c&command=dd+if%3D%2Fmnt%2Fvolume2%2Ffile1.txt++bs%3D64+count%3D1+%7C+sha256sum+%7C+grep+-Fq+94ef2fe32af6899f1df1a83e3060f5c116c3f65b7aa39e8600bb4e30d48caf24&container=write-pod&container=write-pod&stderr=true&stdout=true) Nov 29 22:55:09.201: INFO: Deleting pod "pod-64739c7b-38ec-45e2-b2da-230500376278" in namespace "multivolume-2962" Nov 29 22:55:09.314: INFO: Wait up to 5m0s for pod "pod-64739c7b-38ec-45e2-b2da-230500376278" to be fully deleted [1mSTEP[0m: Creating pod on {Name: Selector:map[] Affinity:&Affinity{NodeAffinity:&NodeAffinity{RequiredDuringSchedulingIgnoredDuringExecution:&NodeSelector{NodeSelectorTerms:[]NodeSelectorTerm{NodeSelectorTerm{MatchExpressions:[]NodeSelectorRequirement{},MatchFields:[]NodeSelectorRequirement{NodeSelectorRequirement{Key:metadata.name,Operator:NotIn,Values:[k8s-agentpool1-92105279-vmss000000],},},},},},PreferredDuringSchedulingIgnoredDuringExecution:[]PreferredSchedulingTerm{},},PodAffinity:nil,PodAntiAffinity:nil,}} with multiple volumes Nov 29 23:00:11.990: FAIL: Unexpected error: <*errors.errorString | 0xc0017521f0>: { s: "pod \"pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e\" is not Running: timed out waiting for the condition", } pod "pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e" is not Running: timed out waiting for the condition occurred ... skipping 130 lines ... Nov 29 23:10:48.588: INFO: PersistentVolume pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79 found and phase=Released (4m0.347522576s) Nov 29 23:10:53.700: INFO: PersistentVolume pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79 found and phase=Released (4m5.459541851s) Nov 29 23:10:58.811: INFO: PersistentVolume pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79 found and phase=Released (4m10.570709874s) Nov 29 23:11:03.926: INFO: PersistentVolume pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79 found and phase=Released (4m15.685510461s) Nov 29 23:11:09.038: INFO: PersistentVolume pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79 was removed [1mSTEP[0m: Deleting sc Nov 29 23:11:09.151: FAIL: while cleanup resource Unexpected error: <errors.aggregate | len:1, cap:1>: [ [ { msg: "persistent Volume pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4 not deleted by dynamic provisioner: PersistentVolume pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4 still exists within 5m0s", err: { s: "PersistentVolume pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4 still exists within 5m0s", ... skipping 6 lines ... Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func2() test/e2e/storage/testsuites/multivolume.go:129 +0x24f panic({0x6ca56c0, 0xc002eec440}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x632c000, 0x7a05090}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0006543c0, 0x13b}, {0xc002992a50?, 0x714b92b?, 0xc002992a70?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc000654280, 0x126}, {0xc0039338e0?, 0xc000654280?, 0xc0014340d0?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc002992bb8, {0x7a29638, 0xa6b18d8}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc002992bb8, {0x7a29638, 0xa6b18d8}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7a6bf58?, {0x7a08d80?, 0xc0017521f0?}, {0x0?, 0x1?, 0x251cfd0?}) ... skipping 32 lines ... Nov 29 23:11:09.375: INFO: At 2022-11-29 22:54:56 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4/dev" Nov 29 23:11:09.375: INFO: At 2022-11-29 22:54:57 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} Started: Started container write-pod Nov 29 23:11:09.375: INFO: At 2022-11-29 22:54:57 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} Pulling: Pulling image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" Nov 29 23:11:09.375: INFO: At 2022-11-29 22:54:57 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} Pulled: Successfully pulled image "k8s.gcr.io/e2e-test-images/busybox:1.29-2" in 388.327666ms Nov 29 23:11:09.375: INFO: At 2022-11-29 22:54:57 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} Created: Created container write-pod Nov 29 23:11:09.375: INFO: At 2022-11-29 22:55:09 +0000 UTC - event for pod-64739c7b-38ec-45e2-b2da-230500376278: {kubelet k8s-agentpool1-92105279-vmss000000} Killing: Stopping container write-pod Nov 29 23:11:09.375: INFO: At 2022-11-29 22:55:11 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {attachdetach-controller } FailedAttachVolume: Multi-Attach error for volume "pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79" Volume is already exclusively attached to one node and can't be attached to another Nov 29 23:11:09.375: INFO: At 2022-11-29 22:55:11 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {attachdetach-controller } FailedAttachVolume: Multi-Attach error for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" Volume is already exclusively attached to one node and can't be attached to another Nov 29 23:11:09.375: INFO: At 2022-11-29 22:56:29 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79" Nov 29 23:11:09.375: INFO: At 2022-11-29 22:56:31 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" Nov 29 23:11:09.375: INFO: At 2022-11-29 22:57:14 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[volume2 volume1], unattached volumes=[volume2 kube-api-access-cr97g volume1]: timed out waiting for the condition Nov 29 23:11:09.376: INFO: At 2022-11-29 22:58:30 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMount: MountVolume.MountDevice failed for volume "pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:11:09.376: INFO: At 2022-11-29 22:58:31 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMapVolume: MapVolume.SetUpDevice failed for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:11:09.376: INFO: At 2022-11-29 22:59:27 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMount: Unable to attach or mount volumes: unmounted volumes=[volume1 volume2], unattached volumes=[kube-api-access-cr97g volume1 volume2]: timed out waiting for the condition Nov 29 23:11:09.376: INFO: At 2022-11-29 23:00:32 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMapVolume: MapVolume.SetUpDevice failed for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4) to be attached to node (k8s-agentpool1-92105279-vmss000001): context deadline exceeded Nov 29 23:11:09.376: INFO: At 2022-11-29 23:00:56 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMapVolume: MapVolume.SetUpDevice failed for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4) to be attached to node (k8s-agentpool1-92105279-vmss000001): rpc error: code = Unknown desc = context deadline exceeded Nov 29 23:11:09.376: INFO: At 2022-11-29 23:00:57 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMount: MountVolume.MountDevice failed for volume "pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79) to be attached to node (k8s-agentpool1-92105279-vmss000001): rpc error: code = Unknown desc = context deadline exceeded Nov 29 23:11:09.376: INFO: At 2022-11-29 23:01:57 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMount: MountVolume.MountDevice failed while expanding volume for volume "pvc-80afd1e9-64ee-4d84-b81c-9f6e0961ef79" : mountVolume.NodeExpandVolume get PVC failed : persistentvolumeclaims "test.csi.azure.comr7j74" is forbidden: User "system:node:k8s-agentpool1-92105279-vmss000001" cannot get resource "persistentvolumeclaims" in API group "" in the namespace "multivolume-2962": no relationship found between node 'k8s-agentpool1-92105279-vmss000001' and this object Nov 29 23:11:09.376: INFO: At 2022-11-29 23:01:58 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" globalMapPath "/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4/dev" Nov 29 23:11:09.376: INFO: At 2022-11-29 23:01:58 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} SuccessfulMountVolume: MapVolume.MapPodDevice succeeded for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" volumeMapPath "/var/lib/kubelet/pods/0fed3334-37ee-4334-aa92-bdb51a7a210f/volumeDevices/kubernetes.io~csi" Nov 29 23:11:09.376: INFO: At 2022-11-29 23:01:58 +0000 UTC - event for pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e: {kubelet k8s-agentpool1-92105279-vmss000001} FailedMapVolume: MapVolume.MarkVolumeAsMounted failed while expanding volume for volume "pvc-0a89b1f9-8cc7-4409-bc56-7e1a218193a4" : mountVolume.NodeExpandVolume get PVC failed : persistentvolumeclaims "test.csi.azure.comwjzrf" is forbidden: User "system:node:k8s-agentpool1-92105279-vmss000001" cannot get resource "persistentvolumeclaims" in API group "" in the namespace "multivolume-2962": no relationship found between node 'k8s-agentpool1-92105279-vmss000001' and this object Nov 29 23:11:09.485: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:11:09.485: INFO: Nov 29 23:11:09.703: INFO: Logging node info for node k8s-agentpool1-92105279-vmss000000 Nov 29 23:11:09.813: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-92105279-vmss000000 e7716b3a-d547-407e-a9ed-2706618be43c 28396 0 2022-11-29 22:04:46 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D8s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-ofo3t2sm kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-92105279-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D8s_v3 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:westeurope topology.kubernetes.io/zone:0 topology.test.csi.azure.com/zone:] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-92105279-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 22:04:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2022-11-29 22:04:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:02 +0000 UTC FieldsV1 {"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-29 22:05:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {cloud-node-manager Update v1 2022-11-29 22:06:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kubelet Update v1 2022-11-29 23:11:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.test.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-92105279-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{33672720384 0} {<nil>} 32883516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{32886288384 0} {<nil>} 32115516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 22:05:49 +0000 UTC,LastTransitionTime:2022-11-29 22:05:49 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:11:03 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:11:03 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:11:03 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:11:03 +0000 UTC,LastTransitionTime:2022-11-29 22:05:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-92105279-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2d155693e7fa4281a08fb77e113c1f09,SystemUUID:badc595a-a00b-e64f-ad71-6b1c80de3c78,BootID:ae2f9bff-8abd-4f93-a225-c6b443284255,KernelVersion:5.4.0-1094-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.14+azure-1,KubeletVersion:v1.23.14,KubeProxyVersion:v1.23.14,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253346057,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:664ec9627d47a4160789ad6994367128b25007b164773c5f1e598028440a158a k8sprow.azurecr.io/azuredisk-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a],SizeBytes:142859492,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:f5241226198f5a54d22540acf2b3933ea0f49458f90c51fc75833d0c428687b8 k8s.gcr.io/e2e-test-images/agnhost:2.36],SizeBytes:125426639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:efa8e4b7776fa9fb2279745008780cefec632ee5fdb081e0f6e234ea4b8bbf61 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.14],SizeBytes:112312488,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:8c3fc3c2667004ad6bbdf723bb64c5da66a5cb8b11d4ee59b67179b686223b13 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v5.0.1],SizeBytes:52732401,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:949db4eea46ea31f9afa0b49ad0b55f7a31f91aa03b408e0ac9b7720c8ef7d29 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:51376235,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:72fbc9d17947d1a54c09a16671e6810fd7c0d8f09c0502fdbdd212c2192451a7 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:51044459,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:075ea1f8270312350f1396ab6677251e803e61a523822d5abfa5e6acd180cfab mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:50806891,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:eb7d329559d954093f7a234bc0ff322764de48a0b2a1007229032fb53e7bb9eb mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:49070187,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:2dfff4dc10f6d25b0a34c01bf4d166540cbf1ff2bccd52635350039138eb23c4 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:48775275,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3e23b9ab6bfee42a1751019c21a60aa474964bd4cba00cbed5055430424242cc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.24.0],SizeBytes:48500843,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:acec9ee6626c2cdea89107ba191ae36e6f60339c9b3b3215b5a59bf0e2e730ba mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.14],SizeBytes:47853675,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:759ac7020e982884b99121583bd577658b5663b1c1ed69746356aa6ce9879da4 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.18],SizeBytes:45850731,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:2fbd1e1a0538a06f2061afd45975df70c942654aa7f86e920720169ee439c2d6 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.5.1],SizeBytes:19743433,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:31547791294872570393470991481c2477a311031d3a03e0ae54eb164347dc34 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.7.0],SizeBytes:17857485,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16028126,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[],VolumesAttached:[]AttachedVolume{},Config:nil,},} Nov 29 23:11:09.813: INFO: ... skipping 108 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m [91m[1mshould access to two volumes with different volume mode and retain data across pod recreation on different node [Measurement][0m [90mtest/e2e/storage/testsuites/multivolume.go:248[0m [91mNov 29 23:00:11.990: Unexpected error: <*errors.errorString | 0xc0017521f0>: { s: "pod \"pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e\" is not Running: timed out waiting for the condition", } pod "pod-ddc41e69-d5d3-4bbe-a3ac-948cdaf6df8e" is not Running: timed out waiting for the condition occurred[0m test/e2e/storage/testsuites/multivolume.go:497 [90m------------------------------[0m {"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node","total":39,"completed":12,"skipped":615,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] volumeIO test/e2e/storage/framework/testsuite.go:51 Nov 29 23:11:12.948: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 182 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with pvc data source [90mtest/e2e/storage/testsuites/provisioning.go:421[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with pvc data source","total":31,"completed":11,"skipped":730,"failed":0} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Inline-volume (ext3)] volumes test/e2e/storage/framework/testsuite.go:51 Nov 29 23:12:05.751: INFO: Driver "test.csi.azure.com" does not support volume type "InlineVolume" - skipping ... skipping 331 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m should access to two volumes with different volume mode and retain data across pod recreation on the same node [90mtest/e2e/storage/testsuites/multivolume.go:209[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (ext4)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on the same node","total":34,"completed":13,"skipped":947,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] volumes[0m [1mshould allow exec of files on the volume[0m [37mtest/e2e/storage/testsuites/volumes.go:198[0m ... skipping 17 lines ... Nov 29 23:11:14.021: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com9hvnv] to have phase Bound Nov 29 23:11:14.130: INFO: PersistentVolumeClaim test.csi.azure.com9hvnv found but phase is Pending instead of Bound. Nov 29 23:11:16.241: INFO: PersistentVolumeClaim test.csi.azure.com9hvnv found but phase is Pending instead of Bound. Nov 29 23:11:18.352: INFO: PersistentVolumeClaim test.csi.azure.com9hvnv found and phase=Bound (4.330331907s) [1mSTEP[0m: Creating pod exec-volume-test-dynamicpv-rtv5 [1mSTEP[0m: Creating a pod to test exec-volume-test Nov 29 23:11:18.686: INFO: Waiting up to 5m0s for pod "exec-volume-test-dynamicpv-rtv5" in namespace "volume-9374" to be "Succeeded or Failed" Nov 29 23:11:18.796: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 109.849917ms Nov 29 23:11:20.907: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 2.220220642s Nov 29 23:11:23.019: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 4.332960661s Nov 29 23:11:25.129: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 6.442924208s Nov 29 23:11:27.240: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 8.553288191s Nov 29 23:11:29.350: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 10.663917194s ... skipping 9 lines ... Nov 29 23:11:50.472: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 31.785631562s Nov 29 23:11:52.583: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 33.896744074s Nov 29 23:11:54.695: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 36.008467844s Nov 29 23:11:56.805: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Pending", Reason="", readiness=false. Elapsed: 38.118697185s Nov 29 23:11:58.918: INFO: Pod "exec-volume-test-dynamicpv-rtv5": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.231357866s [1mSTEP[0m: Saw pod success Nov 29 23:11:58.918: INFO: Pod "exec-volume-test-dynamicpv-rtv5" satisfied condition "Succeeded or Failed" Nov 29 23:11:59.040: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod exec-volume-test-dynamicpv-rtv5 container exec-container-dynamicpv-rtv5: <nil> [1mSTEP[0m: delete the pod Nov 29 23:11:59.279: INFO: Waiting for pod exec-volume-test-dynamicpv-rtv5 to disappear Nov 29 23:11:59.387: INFO: Pod exec-volume-test-dynamicpv-rtv5 no longer exists [1mSTEP[0m: Deleting pod exec-volume-test-dynamicpv-rtv5 Nov 29 23:11:59.388: INFO: Deleting pod "exec-volume-test-dynamicpv-rtv5" in namespace "volume-9374" ... skipping 21 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] volumes [90mtest/e2e/storage/framework/testsuite.go:50[0m should allow exec of files on the volume [90mtest/e2e/storage/testsuites/volumes.go:198[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] volumes should allow exec of files on the volume","total":39,"completed":13,"skipped":725,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow][0m [1mshould access to two volumes with different volume mode and retain data across pod recreation on different node[0m [37mtest/e2e/storage/testsuites/multivolume.go:248[0m ... skipping 24 lines ... [1mSTEP[0m: creating a claim Nov 29 22:56:36.250: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com9qk8j] to have phase Bound Nov 29 22:56:36.358: INFO: PersistentVolumeClaim test.csi.azure.com9qk8j found but phase is Pending instead of Bound. Nov 29 22:56:38.467: INFO: PersistentVolumeClaim test.csi.azure.com9qk8j found but phase is Pending instead of Bound. Nov 29 22:56:40.577: INFO: PersistentVolumeClaim test.csi.azure.com9qk8j found and phase=Bound (4.32635794s) [1mSTEP[0m: Creating pod on {Name: Selector:map[] Affinity:nil} with multiple volumes Nov 29 23:01:41.261: FAIL: Unexpected error: <*errors.errorString | 0xc0022482b0>: { s: "pod \"pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9\" is not Running: timed out waiting for the condition", } pod "pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9" is not Running: timed out waiting for the condition occurred ... skipping 130 lines ... Nov 29 23:12:21.779: INFO: PersistentVolume pvc-0e72e110-b981-447c-adc3-c886d695e36e found and phase=Released (4m0.302429121s) Nov 29 23:12:26.889: INFO: PersistentVolume pvc-0e72e110-b981-447c-adc3-c886d695e36e found and phase=Released (4m5.413154185s) Nov 29 23:12:31.999: INFO: PersistentVolume pvc-0e72e110-b981-447c-adc3-c886d695e36e found and phase=Released (4m10.522690719s) Nov 29 23:12:37.110: INFO: PersistentVolume pvc-0e72e110-b981-447c-adc3-c886d695e36e found and phase=Released (4m15.633775917s) Nov 29 23:12:42.218: INFO: PersistentVolume pvc-0e72e110-b981-447c-adc3-c886d695e36e was removed [1mSTEP[0m: Deleting sc Nov 29 23:12:42.329: FAIL: while cleanup resource Unexpected error: <errors.aggregate | len:1, cap:1>: [ [ { msg: "persistent Volume pvc-474611e1-0421-4ef4-9037-0e3d28c5d897 not deleted by dynamic provisioner: PersistentVolume pvc-474611e1-0421-4ef4-9037-0e3d28c5d897 still exists within 5m0s", err: { s: "PersistentVolume pvc-474611e1-0421-4ef4-9037-0e3d28c5d897 still exists within 5m0s", ... skipping 6 lines ... Full Stack Trace k8s.io/kubernetes/test/e2e/storage/testsuites.(*multiVolumeTestSuite).DefineTests.func2() test/e2e/storage/testsuites/multivolume.go:129 +0x24f panic({0x6ca56c0, 0xc001ed4c80}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail.func1() test/e2e/framework/ginkgowrapper/wrapper.go:63 +0x7d panic({0x632c000, 0x7a05090}) /usr/local/go/src/runtime/panic.go:838 +0x207 k8s.io/kubernetes/test/e2e/framework/ginkgowrapper.Fail({0xc0004ae3c0, 0x13b}, {0xc0013b8a50?, 0x714b92b?, 0xc0013b8a70?}) test/e2e/framework/ginkgowrapper/wrapper.go:67 +0x197 k8s.io/kubernetes/test/e2e/framework.Fail({0xc0004ae280, 0x126}, {0xc003ac3f30?, 0xc0004ae280?, 0xc002aea1a0?}) test/e2e/framework/log.go:63 +0x145 k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).match(0xc0013b8bb8, {0x7a29638, 0xa6b18d8}, 0x0, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:79 +0x1bd k8s.io/kubernetes/vendor/github.com/onsi/gomega/internal/assertion.(*Assertion).NotTo(0xc0013b8bb8, {0x7a29638, 0xa6b18d8}, {0x0, 0x0, 0x0}) vendor/github.com/onsi/gomega/internal/assertion/assertion.go:48 +0x92 k8s.io/kubernetes/test/e2e/framework.ExpectNoErrorWithOffset(0x7a6bf58?, {0x7a08d80?, 0xc0022482b0?}, {0x0?, 0x1?, 0x7a3dbc8?}) ... skipping 22 lines ... Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:31 +0000 UTC - event for test.csi.azure.com7cbcr: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:31 +0000 UTC - event for test.csi.azure.com7cbcr: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } Provisioning: External provisioner is provisioning volume for claim "multivolume-4365/test.csi.azure.com7cbcr" Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:33 +0000 UTC - event for test.csi.azure.com7cbcr: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } ProvisioningSucceeded: Successfully provisioned volume pvc-474611e1-0421-4ef4-9037-0e3d28c5d897 Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:36 +0000 UTC - event for test.csi.azure.com9qk8j: {persistentvolume-controller } ExternalProvisioning: waiting for a volume to be created, either by external provisioner "test.csi.azure.com" or manually created by system administrator Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:36 +0000 UTC - event for test.csi.azure.com9qk8j: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } Provisioning: External provisioner is provisioning volume for claim "multivolume-4365/test.csi.azure.com9qk8j" Nov 29 23:12:42.438: INFO: At 2022-11-29 22:56:38 +0000 UTC - event for test.csi.azure.com9qk8j: {test.csi.azure.com_k8s-agentpool1-92105279-vmss000002_83f29e4a-d993-40ff-88f5-42bec00669a6 } ProvisioningSucceeded: Successfully provisioned volume pvc-0e72e110-b981-447c-adc3-c886d695e36e Nov 29 23:12:42.438: INFO: At 2022-11-29 22:58:40 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-0e72e110-b981-447c-adc3-c886d695e36e" : Attach timeout for volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-0e72e110-b981-447c-adc3-c886d695e36e Nov 29 23:12:42.438: INFO: At 2022-11-29 22:58:40 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "pvc-474611e1-0421-4ef4-9037-0e3d28c5d897" : Attach timeout for volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-474611e1-0421-4ef4-9037-0e3d28c5d897 Nov 29 23:12:42.439: INFO: At 2022-11-29 22:58:43 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: Unable to attach or mount volumes: unmounted volumes=[volume2 volume1], unattached volumes=[volume2 kube-api-access-ltwd7 volume1]: timed out waiting for the condition Nov 29 23:12:42.439: INFO: At 2022-11-29 23:00:54 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-0e72e110-b981-447c-adc3-c886d695e36e" Nov 29 23:12:42.439: INFO: At 2022-11-29 23:00:54 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {attachdetach-controller } SuccessfulAttachVolume: AttachVolume.Attach succeeded for volume "pvc-474611e1-0421-4ef4-9037-0e3d28c5d897" Nov 29 23:12:42.439: INFO: At 2022-11-29 23:01:41 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-0e72e110-b981-447c-adc3-c886d695e36e" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-0e72e110-b981-447c-adc3-c886d695e36e) to be attached to node (k8s-agentpool1-92105279-vmss000000): rpc error: code = Unknown desc = context deadline exceeded Nov 29 23:12:42.439: INFO: At 2022-11-29 23:01:41 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-474611e1-0421-4ef4-9037-0e3d28c5d897" : rpc error: code = Internal desc = failed to wait for volume (/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-474611e1-0421-4ef4-9037-0e3d28c5d897) to be attached to node (k8s-agentpool1-92105279-vmss000000): rpc error: code = Unknown desc = context deadline exceeded Nov 29 23:12:42.439: INFO: At 2022-11-29 23:03:41 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-474611e1-0421-4ef4-9037-0e3d28c5d897" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:12:42.439: INFO: At 2022-11-29 23:03:41 +0000 UTC - event for pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9: {kubelet k8s-agentpool1-92105279-vmss000000} FailedMount: MountVolume.MountDevice failed for volume "pvc-0e72e110-b981-447c-adc3-c886d695e36e" : rpc error: code = DeadlineExceeded desc = context deadline exceeded Nov 29 23:12:42.547: INFO: POD NODE PHASE GRACE CONDITIONS Nov 29 23:12:42.547: INFO: Nov 29 23:12:42.762: INFO: Logging node info for node k8s-agentpool1-92105279-vmss000000 Nov 29 23:12:42.870: INFO: Node Info: &Node{ObjectMeta:{k8s-agentpool1-92105279-vmss000000 e7716b3a-d547-407e-a9ed-2706618be43c 29065 0 2022-11-29 22:04:46 +0000 UTC <nil> <nil> map[agentpool:agentpool1 beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:Standard_D8s_v3 beta.kubernetes.io/os:linux failure-domain.beta.kubernetes.io/region:westeurope failure-domain.beta.kubernetes.io/zone:0 kubernetes.azure.com/cluster:kubetest-ofo3t2sm kubernetes.azure.com/role:agent kubernetes.io/arch:amd64 kubernetes.io/hostname:k8s-agentpool1-92105279-vmss000000 kubernetes.io/os:linux kubernetes.io/role:agent node-role.kubernetes.io/agent: node.kubernetes.io/instance-type:Standard_D8s_v3 storageprofile:managed storagetier:Premium_LRS topology.kubernetes.io/region:westeurope topology.kubernetes.io/zone:0 topology.test.csi.azure.com/zone:] map[csi.volume.kubernetes.io/nodeid:{"test.csi.azure.com":"k8s-agentpool1-92105279-vmss000000"} node.alpha.kubernetes.io/ttl:0 volumes.kubernetes.io/controller-managed-attach-detach:true] [] [] [{kubelet Update v1 2022-11-29 22:04:45 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"f:volumes.kubernetes.io/controller-managed-attach-detach":{}},"f:labels":{".":{},"f:agentpool":{},"f:beta.kubernetes.io/arch":{},"f:beta.kubernetes.io/os":{},"f:kubernetes.azure.com/cluster":{},"f:kubernetes.azure.com/role":{},"f:kubernetes.io/arch":{},"f:kubernetes.io/hostname":{},"f:kubernetes.io/os":{},"f:storageprofile":{},"f:storagetier":{}}}} } {kubectl-label Update v1 2022-11-29 22:04:48 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:kubernetes.io/role":{},"f:node-role.kubernetes.io/agent":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:02 +0000 UTC FieldsV1 {"f:spec":{"f:podCIDR":{},"f:podCIDRs":{".":{},"v:\"10.244.3.0/24\"":{}}}} } {kube-controller-manager Update v1 2022-11-29 22:05:03 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:node.alpha.kubernetes.io/ttl":{}}}} } {cloud-controller-manager Update v1 2022-11-29 22:05:49 +0000 UTC FieldsV1 {"f:status":{"f:conditions":{"k:{\"type\":\"NetworkUnavailable\"}":{".":{},"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}} status} {cloud-node-manager Update v1 2022-11-29 22:06:28 +0000 UTC FieldsV1 {"f:metadata":{"f:labels":{"f:beta.kubernetes.io/instance-type":{},"f:failure-domain.beta.kubernetes.io/region":{},"f:failure-domain.beta.kubernetes.io/zone":{},"f:node.kubernetes.io/instance-type":{},"f:topology.kubernetes.io/region":{},"f:topology.kubernetes.io/zone":{}}},"f:spec":{"f:providerID":{}}} } {kube-controller-manager Update v1 2022-11-29 23:12:32 +0000 UTC FieldsV1 {"f:status":{"f:volumesAttached":{}}} status} {kubelet Update v1 2022-11-29 23:12:35 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"f:csi.volume.kubernetes.io/nodeid":{}},"f:labels":{"f:topology.test.csi.azure.com/zone":{}}},"f:status":{"f:conditions":{"k:{\"type\":\"DiskPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"MemoryPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"PIDPressure\"}":{"f:lastHeartbeatTime":{}},"k:{\"type\":\"Ready\"}":{"f:lastHeartbeatTime":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{}}},"f:images":{},"f:volumesInUse":{}}} status}]},Spec:NodeSpec{PodCIDR:10.244.3.0/24,DoNotUseExternalID:,ProviderID:azure:///subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/virtualMachineScaleSets/k8s-agentpool1-92105279-vmss/virtualMachines/0,Unschedulable:false,Taints:[]Taint{},ConfigSource:nil,PodCIDRs:[10.244.3.0/24],},Status:NodeStatus{Capacity:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{31025332224 0} {<nil>} BinarySI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{33672720384 0} {<nil>} 32883516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Allocatable:ResourceList{cpu: {{8 0} {<nil>} 8 DecimalSI},ephemeral-storage: {{27922798956 0} {<nil>} 27922798956 DecimalSI},hugepages-1Gi: {{0 0} {<nil>} 0 DecimalSI},hugepages-2Mi: {{0 0} {<nil>} 0 DecimalSI},memory: {{32886288384 0} {<nil>} 32115516Ki BinarySI},pods: {{110 0} {<nil>} 110 DecimalSI},},Phase:,Conditions:[]NodeCondition{NodeCondition{Type:NetworkUnavailable,Status:False,LastHeartbeatTime:2022-11-29 22:05:49 +0000 UTC,LastTransitionTime:2022-11-29 22:05:49 +0000 UTC,Reason:RouteCreated,Message:RouteController created a route,},NodeCondition{Type:MemoryPressure,Status:False,LastHeartbeatTime:2022-11-29 23:12:35 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientMemory,Message:kubelet has sufficient memory available,},NodeCondition{Type:DiskPressure,Status:False,LastHeartbeatTime:2022-11-29 23:12:35 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasNoDiskPressure,Message:kubelet has no disk pressure,},NodeCondition{Type:PIDPressure,Status:False,LastHeartbeatTime:2022-11-29 23:12:35 +0000 UTC,LastTransitionTime:2022-11-29 22:04:40 +0000 UTC,Reason:KubeletHasSufficientPID,Message:kubelet has sufficient PID available,},NodeCondition{Type:Ready,Status:True,LastHeartbeatTime:2022-11-29 23:12:35 +0000 UTC,LastTransitionTime:2022-11-29 22:05:02 +0000 UTC,Reason:KubeletReady,Message:kubelet is posting ready status. AppArmor enabled,},},Addresses:[]NodeAddress{NodeAddress{Type:InternalIP,Address:10.240.0.4,},NodeAddress{Type:Hostname,Address:k8s-agentpool1-92105279-vmss000000,},},DaemonEndpoints:NodeDaemonEndpoints{KubeletEndpoint:DaemonEndpoint{Port:10250,},},NodeInfo:NodeSystemInfo{MachineID:2d155693e7fa4281a08fb77e113c1f09,SystemUUID:badc595a-a00b-e64f-ad71-6b1c80de3c78,BootID:ae2f9bff-8abd-4f93-a225-c6b443284255,KernelVersion:5.4.0-1094-azure,OSImage:Ubuntu 18.04.6 LTS,ContainerRuntimeVersion:docker://20.10.14+azure-1,KubeletVersion:v1.23.14,KubeProxyVersion:v1.23.14,OperatingSystem:linux,Architecture:amd64,},Images:[]ContainerImage{ContainerImage{Names:[k8s.gcr.io/e2e-test-images/jessie-dnsutils@sha256:11e6a66017ba4e4b938c1612b7a54a3befcefd354796c04e1dba76873a13518e k8s.gcr.io/e2e-test-images/jessie-dnsutils:1.5],SizeBytes:253346057,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi@sha256:423eb6cf602c064c8b2deefead5ceadd6324ed41b3d995dab5d0f6f0f4d4710f mcr.microsoft.com/oss/kubernetes-csi/azuredisk-csi:v1.10.0],SizeBytes:245959792,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi@sha256:9e2ecabcf9dd9943e6600eb9fb460f45b4dc61af7cabe95d115082a029db2aaf mcr.microsoft.com/oss/kubernetes-csi/azurefile-csi:v1.9.0],SizeBytes:230470852,},ContainerImage{Names:[mcr.microsoft.com/containernetworking/azure-npm@sha256:106f669f48e5e80c4ec0afb49858ead72cf4b901cd8664e7bf81f8d789e56e12 mcr.microsoft.com/containernetworking/azure-npm:v1.2.2_hotfix],SizeBytes:175230380,},ContainerImage{Names:[k8sprow.azurecr.io/azuredisk-csi@sha256:664ec9627d47a4160789ad6994367128b25007b164773c5f1e598028440a158a k8sprow.azurecr.io/azuredisk-csi:latest-v2-5fb3153a08fb6a2968d7e7bbe760d033ea35fc7a],SizeBytes:142859492,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/agnhost@sha256:f5241226198f5a54d22540acf2b3933ea0f49458f90c51fc75833d0c428687b8 k8s.gcr.io/e2e-test-images/agnhost:2.36],SizeBytes:125426639,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver@sha256:c0d040a1c4fbfceb65663e31c09ea40f4f78e356437610cbc3fbb4bb409bd6f1 mcr.microsoft.com/oss/kubernetes-csi/secrets-store/driver:v0.0.19],SizeBytes:123229697,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-proxy@sha256:efa8e4b7776fa9fb2279745008780cefec632ee5fdb081e0f6e234ea4b8bbf61 mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.23.14],SizeBytes:112312488,},ContainerImage{Names:[mcr.microsoft.com/oss/azure/secrets-store/provider-azure@sha256:6f67f3d0c7cdde5702f8ce7f101b6519daa0237f0c34fecb7c058b6af8c22ad1 mcr.microsoft.com/oss/azure/secrets-store/provider-azure:0.0.12],SizeBytes:101061355,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler@sha256:6f0c680d375c62e74351f8ff3ed6ddb9b72ca759e0645c329b95f64264654a6d mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-autoscaler:v1.22.1],SizeBytes:99962810,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:32e2836018c96e73533bd4642fe438e465b81dcbfa8b7b61935a6f4d0246c7ae mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.3],SizeBytes:86832059,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/kube-addon-manager@sha256:92c2c5aad9012ee32d2a43a74966cc0adc6ccb1705ad15abb10485ecf406d88b mcr.microsoft.com/oss/kubernetes/kube-addon-manager:v9.1.5],SizeBytes:84094027,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/metrics-server@sha256:1ef9d57ce41ffcc328b92494c3bfafe401e0b9a1694a295301a1385337d52815 mcr.microsoft.com/oss/kubernetes/metrics-server:v0.5.2],SizeBytes:64327621,},ContainerImage{Names:[mcr.microsoft.com/oss/nvidia/k8s-device-plugin@sha256:0f5b52bf28239234e831697d96db63ac03cde70fe68058f964504ab7564ee810 mcr.microsoft.com/oss/nvidia/k8s-device-plugin:1.0.0-beta6],SizeBytes:64160241,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner@sha256:e9ddadc44ba87a4a27f67e54760a14f9986885b534b3dff170a14eae1e35d213 mcr.microsoft.com/oss/kubernetes-csi/csi-provisioner:v3.0.0],SizeBytes:56881280,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-resizer@sha256:c5bb71ceaac60b1a4b58739fa07b709f6248c452ff6272a384d2f7648895a750 mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0],SizeBytes:54313772,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter@sha256:61849a026511cf332c87d73d0a7aed803b510c3ede197ec755389686d490de72 mcr.microsoft.com/oss/kubernetes-csi/csi-snapshotter:v4.2.1],SizeBytes:54210936,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-attacher@sha256:6b41e7153ebdfdc1501aa65184624bc15fd33a52d93f88ec3a758d0f8c9b8c10 mcr.microsoft.com/oss/kubernetes-csi/csi-attacher:v3.3.0],SizeBytes:53842561,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:8c3fc3c2667004ad6bbdf723bb64c5da66a5cb8b11d4ee59b67179b686223b13 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v5.0.1],SizeBytes:52732401,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller@sha256:be5a8dc1f990828f653c77bd0a0f1bbd13197c3019f6e1d99d590389bac36705 mcr.microsoft.com/oss/kubernetes-csi/snapshot-controller:v4.2.1],SizeBytes:51575245,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:949db4eea46ea31f9afa0b49ad0b55f7a31f91aa03b408e0ac9b7720c8ef7d29 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.23.11],SizeBytes:51376235,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:72fbc9d17947d1a54c09a16671e6810fd7c0d8f09c0502fdbdd212c2192451a7 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.1.14],SizeBytes:51044459,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:075ea1f8270312350f1396ab6677251e803e61a523822d5abfa5e6acd180cfab mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.23.11],SizeBytes:50806891,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/ip-masq-agent@sha256:1244155f2ed3f33ff154cc343b8ad285f3391d95afd7d4b1c6dcc420bc0ba3cf mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0],SizeBytes:50146762,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:eb7d329559d954093f7a234bc0ff322764de48a0b2a1007229032fb53e7bb9eb mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.24.0],SizeBytes:49070187,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:2dfff4dc10f6d25b0a34c01bf4d166540cbf1ff2bccd52635350039138eb23c4 mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.0.18],SizeBytes:48775275,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager@sha256:3e23b9ab6bfee42a1751019c21a60aa474964bd4cba00cbed5055430424242cc mcr.microsoft.com/oss/kubernetes/azure-cloud-node-manager:v1.24.0],SizeBytes:48500843,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:acec9ee6626c2cdea89107ba191ae36e6f60339c9b3b3215b5a59bf0e2e730ba mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.1.14],SizeBytes:47853675,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/coredns@sha256:f873bf7f0928461efe10697fa76cf0ad7a1ae3041c5b57b50dd3d0b72d273f8c mcr.microsoft.com/oss/kubernetes/coredns:1.8.6],SizeBytes:46804601,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager@sha256:759ac7020e982884b99121583bd577658b5663b1c1ed69746356aa6ce9879da4 mcr.microsoft.com/oss/kubernetes/azure-cloud-controller-manager:v1.0.18],SizeBytes:45850731,},ContainerImage{Names:[mcr.microsoft.com/oss/etcd-io/etcd@sha256:cf587862e3f1b6fa4d9a2565520a34f164bdf72c50f37af8c3c668160593246e mcr.microsoft.com/oss/etcd-io/etcd:v3.3.25],SizeBytes:41832119,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/mic@sha256:bd9465be94966b9a917e1e3904fa5e63dd91772ccadf304e18ffd8e4ad8ccedd mcr.microsoft.com/k8s/aad-pod-identity/mic:1.6.1],SizeBytes:41374894,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler@sha256:c849d75d61943ce7f51b4c049f1a79d19a08253966c8f49c4cfb6414cc33db8b mcr.microsoft.com/oss/kubernetes/autoscaler/cluster-proportional-autoscaler:1.8.5],SizeBytes:40661903,},ContainerImage{Names:[mcr.microsoft.com/k8s/aad-pod-identity/nmi@sha256:02128fefcdb7593ac53fc342e2c53a0fc6fabd813036bf60457bf43cc2940116 mcr.microsoft.com/k8s/aad-pod-identity/nmi:1.6.1],SizeBytes:38007982,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume@sha256:921f301c44dda06a325164accf22e78ecc570b5c7d9d6ee4c66bd8cbb2b60b9a mcr.microsoft.com/k8s/flexvolume/keyvault-flexvolume:v0.0.16],SizeBytes:26970670,},ContainerImage{Names:[mcr.microsoft.com/k8s/kms/keyvault@sha256:1a27e175f8c125209e32d2957b5509fe20757bd8cb309ff9da598799b56326fb mcr.microsoft.com/k8s/kms/keyvault:v0.0.10],SizeBytes:23077387,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:2fbd1e1a0538a06f2061afd45975df70c942654aa7f86e920720169ee439c2d6 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.5.1],SizeBytes:19743433,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar@sha256:dbec3a8166686b09b242176ab5b99e993da4126438bbce68147c3fd654f35662 mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.4.0],SizeBytes:19547289,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:31547791294872570393470991481c2477a311031d3a03e0ae54eb164347dc34 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.7.0],SizeBytes:17857485,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes-csi/livenessprobe@sha256:c96a6255c42766f6b8bb1a7cda02b0060ab1b20b2e2dafcc64ec09e7646745a6 mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.5.0],SizeBytes:17573341,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/nginx@sha256:13616070e3f29de4417eee434a8ef472221c9e51b3d037b5a6b46cef08eb7443 k8s.gcr.io/e2e-test-images/nginx:1.14-2],SizeBytes:16028126,},ContainerImage{Names:[mcr.microsoft.com/oss/busybox/busybox@sha256:582a641242b49809af3a1a522f9aae8c3f047d1c6ca1dd9d8cdabd349e45b1a9 mcr.microsoft.com/oss/busybox/busybox:1.33.1],SizeBytes:1235829,},ContainerImage{Names:[mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume@sha256:23d8c6033f02a1ecad05127ebdc931bb871264228661bc122704b0974e4d9fdd mcr.microsoft.com/k8s/flexvolume/blobfuse-flexvolume:1.0.8],SizeBytes:1159025,},ContainerImage{Names:[k8s.gcr.io/e2e-test-images/busybox@sha256:c318242786b139d18676b1c09a0ad7f15fc17f8f16a5b2e625cd0dc8c9703daf k8s.gcr.io/e2e-test-images/busybox:1.29-2],SizeBytes:1154361,},ContainerImage{Names:[mcr.microsoft.com/oss/kubernetes/pause@sha256:e3b8c20681593c21b344ad801fbb8abaf564427ee3a57a9fcfa3b455f917ce46 mcr.microsoft.com/oss/kubernetes/pause:3.4.1],SizeBytes:682696,},},VolumesInUse:[kubernetes.io/csi/test.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-2a67c84d-6aa7-4d49-a2ca-90b39f0e1b1f],VolumesAttached:[]AttachedVolume{AttachedVolume{Name:kubernetes.io/csi/test.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-ofo3t2sm/providers/Microsoft.Compute/disks/pvc-2a67c84d-6aa7-4d49-a2ca-90b39f0e1b1f,DevicePath:,},},Config:nil,},} Nov 29 23:12:42.871: INFO: ... skipping 110 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [90mtest/e2e/storage/framework/testsuite.go:50[0m [91m[1mshould access to two volumes with different volume mode and retain data across pod recreation on different node [Measurement][0m [90mtest/e2e/storage/testsuites/multivolume.go:248[0m [91mNov 29 23:01:41.261: Unexpected error: <*errors.errorString | 0xc0022482b0>: { s: "pod \"pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9\" is not Running: timed out waiting for the condition", } pod "pod-d93c7424-44c3-45d1-bec4-966fd5ea5bb9" is not Running: timed out waiting for the condition occurred[0m test/e2e/storage/testsuites/multivolume.go:497 [90m------------------------------[0m {"msg":"FAILED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node","total":32,"completed":12,"skipped":775,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (block volmode)(allowExpansion)] volume-expand test/e2e/storage/framework/testsuite.go:51 Nov 29 23:12:46.016: INFO: Driver "test.csi.azure.com" does not support volume expansion - skipping ... skipping 34 lines ... test/e2e/storage/external/external.go:262 [90m------------------------------[0m [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [0mExternal Storage [Driver: test.csi.azure.com][0m [90m[Testpattern: Dynamic PV (default fs)] subPath[0m [1mshould fail if non-existent subpath is outside the volume [Slow][LinuxOnly][0m [37mtest/e2e/storage/testsuites/subpath.go:269[0m [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 [BeforeEach] [Testpattern: Dynamic PV (default fs)] subPath test/e2e/framework/framework.go:187 [1mSTEP[0m: Creating a kubernetes client Nov 29 23:12:46.032: INFO: >>> kubeConfig: /root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json [1mSTEP[0m: Building a namespace api object, basename provisioning [1mSTEP[0m: Waiting for a default service account to be provisioned in namespace [1mSTEP[0m: Waiting for kube-root-ca.crt to be provisioned in namespace [It] should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] test/e2e/storage/testsuites/subpath.go:269 Nov 29 23:12:46.791: INFO: Creating resource for dynamic PV Nov 29 23:12:46.791: INFO: Using claimSize:5Gi, test suite supported size:{ 1Mi}, driver(test.csi.azure.com) supported size:{ 1Mi} [1mSTEP[0m: creating a StorageClass provisioning-7276-e2e-scmw5jp [1mSTEP[0m: creating a claim Nov 29 23:12:46.903: INFO: Warning: Making PVC: VolumeMode specified as invalid empty string, treating as nil Nov 29 23:12:47.017: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com7vv29] to have phase Bound Nov 29 23:12:47.126: INFO: PersistentVolumeClaim test.csi.azure.com7vv29 found but phase is Pending instead of Bound. Nov 29 23:12:49.235: INFO: PersistentVolumeClaim test.csi.azure.com7vv29 found but phase is Pending instead of Bound. Nov 29 23:12:51.344: INFO: PersistentVolumeClaim test.csi.azure.com7vv29 found and phase=Bound (4.326569598s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-jj5j [1mSTEP[0m: Checking for subpath error in container status Nov 29 23:14:21.894: INFO: Deleting pod "pod-subpath-test-dynamicpv-jj5j" in namespace "provisioning-7276" Nov 29 23:14:22.007: INFO: Wait up to 5m0s for pod "pod-subpath-test-dynamicpv-jj5j" to be fully deleted [1mSTEP[0m: Deleting pod Nov 29 23:14:24.229: INFO: Deleting pod "pod-subpath-test-dynamicpv-jj5j" in namespace "provisioning-7276" [1mSTEP[0m: Deleting pvc Nov 29 23:14:24.350: INFO: Deleting PersistentVolumeClaim "test.csi.azure.com7vv29" ... skipping 22 lines ... [32m• [SLOW TEST:170.653 seconds][0m External Storage [Driver: test.csi.azure.com] [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should fail if non-existent subpath is outside the volume [Slow][LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:269[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should fail if non-existent subpath is outside the volume [Slow][LinuxOnly]","total":32,"completed":13,"skipped":872,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (ntfs)][Feature:Windows] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 23:15:36.693: INFO: Distro debian doesn't support ntfs -- skipping ... skipping 112 lines ... Nov 29 23:12:26.528: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comf7x8h] to have phase Bound Nov 29 23:12:26.636: INFO: PersistentVolumeClaim test.csi.azure.comf7x8h found but phase is Pending instead of Bound. Nov 29 23:12:28.746: INFO: PersistentVolumeClaim test.csi.azure.comf7x8h found but phase is Pending instead of Bound. Nov 29 23:12:30.856: INFO: PersistentVolumeClaim test.csi.azure.comf7x8h found and phase=Bound (4.328283051s) [1mSTEP[0m: [init] starting a pod to use the claim [1mSTEP[0m: [init] check pod success Nov 29 23:12:31.295: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-tester-lpft6" in namespace "snapshotting-407" to be "Succeeded or Failed" Nov 29 23:12:31.403: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 108.832737ms Nov 29 23:12:33.514: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 2.219054366s Nov 29 23:12:35.623: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328750492s Nov 29 23:12:37.734: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 6.439525481s Nov 29 23:12:39.844: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 8.548963606s Nov 29 23:12:41.955: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 10.660830475s Nov 29 23:12:44.066: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 12.771101359s Nov 29 23:12:46.176: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 14.881171214s Nov 29 23:12:48.285: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Pending", Reason="", readiness=false. Elapsed: 16.990656853s Nov 29 23:12:50.404: INFO: Pod "pvc-snapshottable-tester-lpft6": Phase="Succeeded", Reason="", readiness=false. Elapsed: 19.109312711s [1mSTEP[0m: Saw pod success Nov 29 23:12:50.404: INFO: Pod "pvc-snapshottable-tester-lpft6" satisfied condition "Succeeded or Failed" [1mSTEP[0m: [init] checking the claim Nov 29 23:12:50.515: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.comf7x8h] to have phase Bound Nov 29 23:12:50.638: INFO: PersistentVolumeClaim test.csi.azure.comf7x8h found and phase=Bound (122.222749ms) [1mSTEP[0m: [init] checking the PV [1mSTEP[0m: [init] deleting the pod Nov 29 23:12:50.982: INFO: Pod pvc-snapshottable-tester-lpft6 has the following logs: ... skipping 15 lines ... Nov 29 23:13:02.847: INFO: received snapshotStatus map[boundVolumeSnapshotContentName:snapcontent-cefcf59e-5f28-4790-816e-7437757ec5ee creationTime:2022-11-29T23:12:58Z readyToUse:true restoreSize:5Gi] Nov 29 23:13:02.848: INFO: snapshotContentName snapcontent-cefcf59e-5f28-4790-816e-7437757ec5ee [1mSTEP[0m: checking the snapshot [1mSTEP[0m: checking the SnapshotContent [1mSTEP[0m: Modifying source data test [1mSTEP[0m: modifying the data in the source PVC Nov 29 23:13:03.288: INFO: Waiting up to 15m0s for pod "pvc-snapshottable-data-tester-68f2w" in namespace "snapshotting-407" to be "Succeeded or Failed" Nov 29 23:13:03.397: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 108.85461ms Nov 29 23:13:05.507: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 2.218993374s Nov 29 23:13:07.616: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 4.328305457s Nov 29 23:13:09.729: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 6.440992284s Nov 29 23:13:11.839: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 8.550984124s Nov 29 23:13:13.948: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 10.66050821s ... skipping 16 lines ... Nov 29 23:13:49.831: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 46.543131962s Nov 29 23:13:51.941: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 48.652656782s Nov 29 23:13:54.051: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 50.762897379s Nov 29 23:13:56.162: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Pending", Reason="", readiness=false. Elapsed: 52.874247835s Nov 29 23:13:58.284: INFO: Pod "pvc-snapshottable-data-tester-68f2w": Phase="Succeeded", Reason="", readiness=false. Elapsed: 54.996196582s [1mSTEP[0m: Saw pod success Nov 29 23:13:58.284: INFO: Pod "pvc-snapshottable-data-tester-68f2w" satisfied condition "Succeeded or Failed" Nov 29 23:13:58.510: INFO: Pod pvc-snapshottable-data-tester-68f2w has the following logs: Nov 29 23:13:58.510: INFO: Deleting pod "pvc-snapshottable-data-tester-68f2w" in namespace "snapshotting-407" Nov 29 23:13:58.625: INFO: Wait up to 5m0s for pod "pvc-snapshottable-data-tester-68f2w" to be fully deleted [1mSTEP[0m: creating a pvc from the snapshot [1mSTEP[0m: starting a pod to use the snapshot Nov 29 23:15:03.182: INFO: Running '/usr/local/bin/kubectl --server=https://kubetest-ofo3t2sm.westeurope.cloudapp.azure.com --kubeconfig=/root/tmp1562512509/kubeconfig/kubeconfig.westeurope.json --namespace=snapshotting-407 exec restored-pvc-tester-w24g2 --namespace=snapshotting-407 -- cat /mnt/test/data' ... skipping 47 lines ... [90mtest/e2e/storage/testsuites/snapshottable.go:113[0m [90mtest/e2e/storage/testsuites/snapshottable.go:176[0m should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent) [90mtest/e2e/storage/testsuites/snapshottable.go:278[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic Snapshot (delete policy)] snapshottable[Feature:VolumeSnapshotDataSource] volume snapshot controller should check snapshot fields, check restore correctly works after modifying source data, check deletion (persistent)","total":34,"completed":14,"skipped":1005,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS]"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m Nov 29 23:15:49.419: INFO: Running AfterSuite actions on all nodes Nov 29 23:15:49.419: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func19.2 Nov 29 23:15:49.419: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func9.2 ... skipping 338 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (block volmode)] provisioning [90mtest/e2e/storage/framework/testsuite.go:50[0m should provision storage with pvc data source in parallel [Slow] [90mtest/e2e/storage/testsuites/provisioning.go:459[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] provisioning should provision storage with pvc data source in parallel [Slow]","total":39,"completed":14,"skipped":737,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (block volmode)] multiVolume [Slow] test/e2e/storage/framework/testsuite.go:51 Nov 29 23:15:53.353: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 59 lines ... Nov 29 23:15:38.778: INFO: Waiting up to timeout=5m0s for PersistentVolumeClaims [test.csi.azure.com9bdrm] to have phase Bound Nov 29 23:15:38.887: INFO: PersistentVolumeClaim test.csi.azure.com9bdrm found but phase is Pending instead of Bound. Nov 29 23:15:40.996: INFO: PersistentVolumeClaim test.csi.azure.com9bdrm found but phase is Pending instead of Bound. Nov 29 23:15:43.107: INFO: PersistentVolumeClaim test.csi.azure.com9bdrm found and phase=Bound (4.329065198s) [1mSTEP[0m: Creating pod pod-subpath-test-dynamicpv-chbr [1mSTEP[0m: Creating a pod to test multi_subpath Nov 29 23:15:43.434: INFO: Waiting up to 5m0s for pod "pod-subpath-test-dynamicpv-chbr" in namespace "provisioning-7066" to be "Succeeded or Failed" Nov 29 23:15:43.547: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 112.156371ms Nov 29 23:15:45.656: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 2.221255746s Nov 29 23:15:47.779: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 4.34473311s Nov 29 23:15:49.889: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 6.454978811s Nov 29 23:15:51.999: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 8.564809874s Nov 29 23:15:54.109: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 10.674970453s ... skipping 4 lines ... Nov 29 23:16:04.658: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 21.224042915s Nov 29 23:16:06.769: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 23.335084115s Nov 29 23:16:08.882: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 25.447116288s Nov 29 23:16:10.993: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Pending", Reason="", readiness=false. Elapsed: 27.559102644s Nov 29 23:16:13.103: INFO: Pod "pod-subpath-test-dynamicpv-chbr": Phase="Succeeded", Reason="", readiness=false. Elapsed: 29.668726713s [1mSTEP[0m: Saw pod success Nov 29 23:16:13.103: INFO: Pod "pod-subpath-test-dynamicpv-chbr" satisfied condition "Succeeded or Failed" Nov 29 23:16:13.212: INFO: Trying to get logs from node k8s-agentpool1-92105279-vmss000001 pod pod-subpath-test-dynamicpv-chbr container test-container-subpath-dynamicpv-chbr: <nil> [1mSTEP[0m: delete the pod Nov 29 23:16:13.478: INFO: Waiting for pod pod-subpath-test-dynamicpv-chbr to disappear Nov 29 23:16:13.587: INFO: Pod pod-subpath-test-dynamicpv-chbr no longer exists [1mSTEP[0m: Deleting pod Nov 29 23:16:13.587: INFO: Deleting pod "pod-subpath-test-dynamicpv-chbr" in namespace "provisioning-7066" ... skipping 21 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should support creating multiple subpath from same volumes [Slow] [90mtest/e2e/storage/testsuites/subpath.go:296[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should support creating multiple subpath from same volumes [Slow]","total":32,"completed":14,"skipped":997,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m Nov 29 23:16:55.363: INFO: Running AfterSuite actions on all nodes Nov 29 23:16:55.363: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func19.2 Nov 29 23:16:55.363: INFO: Running Cleanup Action: k8s.io/kubernetes/test/e2e/storage.glob..func9.2 ... skipping 55 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral [90mtest/e2e/storage/framework/testsuite.go:50[0m should create read-only inline ephemeral volume [90mtest/e2e/storage/testsuites/ephemeral.go:175[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Generic Ephemeral-volume (default fs) (immediate-binding)] ephemeral should create read-only inline ephemeral volume","total":39,"completed":15,"skipped":765,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Pre-provisioned PV (default fs)] subPath test/e2e/storage/framework/testsuite.go:51 Nov 29 23:17:25.339: INFO: Driver "test.csi.azure.com" does not support volume type "PreprovisionedPV" - skipping ... skipping 114 lines ... [90mtest/e2e/storage/external/external.go:174[0m [Testpattern: Dynamic PV (default fs)] subPath [90mtest/e2e/storage/framework/testsuite.go:50[0m should be able to unmount after the subpath directory is deleted [LinuxOnly] [90mtest/e2e/storage/testsuites/subpath.go:447[0m [90m------------------------------[0m {"msg":"PASSED External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (default fs)] subPath should be able to unmount after the subpath directory is deleted [LinuxOnly]","total":39,"completed":16,"skipped":825,"failed":1,"failures":["External Storage [Driver: test.csi.azure.com] [Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] should access to two volumes with different volume mode and retain data across pod recreation on different node"]} [36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m[36mS[0m [90m------------------------------[0m [BeforeEach] [Testpattern: Dynamic PV (block volmode)] volume-expand test/e2e/storage/framework/testsuite.go:51 Nov 29 23:18:38.756: INFO: Driver "test.csi.azure.com" does not support volume expansion - skipping ... skipping 47 lines ... Nov 29 23:18:38.837: INFO: Running AfterSuite actions on node 1 [91m[1mSummarizing 3 Failures:[0m [91m[1m[Fail] [0m[90mExternal Storage [Driver: test.csi.azure.com] [0m[0m[Testpattern: Dynamic PV (filesystem volmode)] multiVolume [Slow] [0m[91m[1m[Measurement] should concurrently access the volume and its clone from pods on the same node [LinuxOnly][Feature:VolumeSnapshotDataSource][Feature:VolumeSourceXFS] [0m [37mtest/e2e/storage/testsuites/multivolume.go:696[0m [91m[1m[Fail] [0m[90mExternal Storage [Driver: test.csi.azure.com] [0m[0m[Testpattern: Dynamic PV (block volmode)] multiVolume [Slow] [0m[91m[1m[Measurement] should access to two volumes with different volume mode and retain data across pod recreation on different node [0m [37mtest/e2e/storage/testsuites/multivolume.go:497[0m [91m[1m[Fail] [0m[90mExternal Storage [Driver: test.csi.azure.com] [0m[0m[Testpattern: Dynamic PV (xfs)][Slow] multiVolume [Slow] [0m[91m[1m[Measurement] should access to two volumes with different volume mode and retain data across pod recreation on different node [0m [37mtest/e2e/storage/testsuites/multivolume.go:497[0m [1m[91mRan 89 of 7227 Specs in 3748.991 seconds[0m [1m[91mFAIL![0m -- [32m[1m86 Passed[0m | [91m[1m3 Failed[0m | [33m[1m0 Pending[0m | [36m[1m7138 Skipped[0m Ginkgo ran 1 suite in 1h2m32.912682661s Test Suite Failed + print_logs + sed -i s/disk.csi.azure.com/test.csi.azure.com/g deploy/example/storageclass-azuredisk-csi.yaml + bash ./hack/verify-examples.sh linux azurepubliccloud ephemeral test begin to create deployment examples ... storageclass.storage.k8s.io/managed-csi created Applying config "deploy/example/deployment.yaml" ... skipping 20 lines ...