This job view page is being replaced by Spyglass soon. Check out the new job view.
PRandyzhangx: feat: add registrar liveness probe on Linux
ResultABORTED
Tests 0 failed / 13 succeeded
Started2021-10-09 01:53
Elapsed32m58s
Revision0ffe1d9aef94892f42e0927b126d8b550c3937a1
Refs 1043

No Test Failures!


Show 13 Passed Tests

Show 40 Skipped Tests

Error lines from build-log.txt

... skipping 388 lines ...
                    type: string
                type: object
              volumeSnapshotClassName:
                description: 'volumeSnapshotClassName is the name of the VolumeSnapshotClass
                  requested by the VolumeSnapshot. If not specified, the default snapshot
                  class will be used if one exists. If not specified, and there is
                  no default snapshot class, dynamic snapshot creation will fail.
                  Empty string is not allowed for this field. TODO(xiangqian): a webhook
                  validation on empty string. More info: https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes'
                type: string
            required:
            - source
            type: object
... skipping 22 lines ...
                  pre-existing snapshot, this field will be filled with the "creation_time"
                  value returned from the CSI "ListSnapshots" gRPC call if the driver
                  supports it. If not specified, it indicates that the creation time
                  of the snapshot is unknown.
                format: date-time
                type: string
              error:
                description: error is the last observed error during snapshot creation,
                  if any. This field could be helpful to upper level controllers(i.e.,
                  application controller) to decide whether they should continue on
                  waiting for the snapshot to be created based on the type of error
                  reported.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error
                      during snapshot creation if specified. NOTE: message may be
                      logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used
                  to restore a volume. In dynamic snapshot creation case, this field
... skipping 10 lines ...
                  in bytes. In dynamic snapshot creation case, this field will be
                  filled in with the "size_bytes" value returned from CSI "CreateSnapshotRequest"
                  gRPC call. For a pre-existing snapshot, this field will be filled
                  with the "size_bytes" value returned from the CSI "ListSnapshots"
                  gRPC call if the driver supports it. When restoring a volume from
                  this snapshot, the size of the volume MUST NOT be smaller than the
                  restoreSize if it is specified, otherwise the restoration will fail.
                  If not specified, it indicates that the size is unknown.
                pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
                x-kubernetes-int-or-string: true
            type: object
        required:
        - spec
... skipping 265 lines ...
                  supports it. If not specified, it indicates the creation time is
                  unknown. The format of this field is a Unix nanoseconds time encoded
                  as an int64. On Unix, the command `date +%s%N` returns the current
                  time in nanoseconds since 1970-01-01 00:00:00 UTC.
                format: int64
                type: integer
              error:
                description: error is the latest observed error during snapshot creation,
                  if any.
                properties:
                  message:
                    description: 'message is a string detailing the encountered error
                      during snapshot creation if specified. NOTE: message may be
                      logged, and it should not contain sensitive information.'
                    type: string
                  time:
                    description: time is the timestamp when the error was encountered.
                    format: date-time
                    type: string
                type: object
              readyToUse:
                description: readyToUse indicates if a snapshot is ready to be used
                  to restore a volume. In dynamic snapshot creation case, this field
... skipping 9 lines ...
                  in bytes. In dynamic snapshot creation case, this field will be
                  filled in with the "size_bytes" value returned from CSI "CreateSnapshotRequest"
                  gRPC call. For a pre-existing snapshot, this field will be filled
                  with the "size_bytes" value returned from the CSI "ListSnapshots"
                  gRPC call if the driver supports it. When restoring a volume from
                  this snapshot, the size of the volume MUST NOT be smaller than the
                  restoreSize if it is specified, otherwise the restoration will fail.
                  If not specified, it indicates that the size is unknown.
                format: int64
                minimum: 0
                type: integer
              snapshotHandle:
                description: snapshotHandle is the CSI "snapshot_id" of a snapshot
                  on the underlying storage system. If not specified, it indicates
                  that dynamic snapshot creation has either failed or it is still
                  in progress.
                type: string
            type: object
        required:
        - spec
        type: object
... skipping 851 lines ...
        - name: csi-resizer
          image: "mcr.microsoft.com/oss/kubernetes-csi/csi-resizer:v1.3.0"
          args:
            - "-csi-address=$(ADDRESS)"
            - "-v=2"
            - "-leader-election"
            - '-handle-volume-inuse-error=true'
          env:
            - name: ADDRESS
              value: /csi/csi.sock
          volumeMounts:
            - name: socket-dir
              mountPath: /csi
... skipping 173 lines ...
Go Version: go1.17.1
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
STEP: Building a namespace api object, basename azuredisk
W1009 01:58:44.916300   12661 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret /: failed to get secret /: resource name may not be empty
W1009 01:58:44.917313   12661 azuredisk.go:159] DisableAvailabilitySetNodes for controller is set as false while current VMType is vmss
I1009 01:58:44.917437   12661 driver.go:81] Enabling controller service capability: CREATE_DELETE_VOLUME
I1009 01:58:44.917453   12661 driver.go:81] Enabling controller service capability: PUBLISH_UNPUBLISH_VOLUME
I1009 01:58:44.917458   12661 driver.go:81] Enabling controller service capability: CREATE_DELETE_SNAPSHOT
I1009 01:58:44.917462   12661 driver.go:81] Enabling controller service capability: LIST_SNAPSHOTS
I1009 01:58:44.917465   12661 driver.go:81] Enabling controller service capability: CLONE_VOLUME
... skipping 70 lines ...

    test case is only available for CSI drivers

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/suite_test.go:264
------------------------------
Pre-Provisioned [single-az] 
  should fail when maxShares is invalid [disk.csi.azure.com][windows]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:160
STEP: Creating a kubernetes client
Oct  9 01:58:45.976: INFO: >>> kubeConfig: /root/tmp4279407306/kubeconfig/kubeconfig.southcentralus.json
STEP: Building a namespace api object, basename azuredisk
STEP: Waiting for a default service account to be provisioned in namespace
I1009 01:58:46.161768   12661 azuredisk_driver.go:56] Using azure disk driver: kubernetes.io/azure-disk
... skipping 2 lines ...

S [SKIPPING] [0.262 seconds]
Pre-Provisioned
/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:37
  [single-az]
  /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:69
    should fail when maxShares is invalid [disk.csi.azure.com][windows] [It]
    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/pre_provisioning_test.go:160

    test case is only available for CSI drivers

    /home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver/test/e2e/suite_test.go:264
------------------------------
... skipping 51 lines ...
STEP: Building a namespace api object, basename azuredisk
STEP: Waiting for a default service account to be provisioned in namespace
I1009 01:58:46.959549   12661 azuredisk_driver.go:56] Using azure disk driver: kubernetes.io/azure-disk
STEP: Successfully provisioned AzureDisk volume: "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume"

STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 01:58:50.828: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-bm4sk" in namespace "azuredisk-5541" to be "Succeeded or Failed"
Oct  9 01:58:50.870: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 41.396974ms
Oct  9 01:58:52.908: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 2.080057418s
Oct  9 01:58:54.947: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 4.118816638s
Oct  9 01:58:56.985: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 6.156359264s
Oct  9 01:58:59.023: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 8.194951918s
Oct  9 01:59:01.061: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 10.232469346s
Oct  9 01:59:03.100: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 12.271716115s
Oct  9 01:59:05.139: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 14.311189733s
Oct  9 01:59:07.178: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 16.350096433s
Oct  9 01:59:09.217: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 18.38826458s
Oct  9 01:59:11.264: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Pending", Reason="", readiness=false. Elapsed: 20.435586114s
Oct  9 01:59:13.304: INFO: Pod "azuredisk-volume-tester-bm4sk": Phase="Succeeded", Reason="", readiness=false. Elapsed: 22.475240807s
STEP: Saw pod success
Oct  9 01:59:13.304: INFO: Pod "azuredisk-volume-tester-bm4sk" satisfied condition "Succeeded or Failed"
Oct  9 01:59:13.304: INFO: deleting Pod "azuredisk-5541"/"azuredisk-volume-tester-bm4sk"
Oct  9 01:59:13.368: INFO: Pod azuredisk-volume-tester-bm4sk has the following logs: hello world

STEP: Deleting pod azuredisk-volume-tester-bm4sk in namespace azuredisk-5541
Oct  9 01:59:13.412: INFO: Waiting up to 3m0s for all (but 0) nodes to be ready
STEP: Destroying namespace "azuredisk-5541" for this suite.
... skipping 21 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 01:59:14.547: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-nswvf" in namespace "azuredisk-5356" to be "Succeeded or Failed"
Oct  9 01:59:14.583: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 36.563285ms
Oct  9 01:59:16.623: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 2.076024988s
Oct  9 01:59:18.661: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114016939s
Oct  9 01:59:20.739: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 6.192533573s
Oct  9 01:59:22.778: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 8.231144375s
Oct  9 01:59:24.818: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 10.271008147s
... skipping 10 lines ...
Oct  9 01:59:47.243: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 32.69629349s
Oct  9 01:59:49.281: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 34.734840241s
Oct  9 01:59:51.319: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 36.772522284s
Oct  9 01:59:53.358: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Pending", Reason="", readiness=false. Elapsed: 38.811042608s
Oct  9 01:59:55.397: INFO: Pod "azuredisk-volume-tester-nswvf": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.850278722s
STEP: Saw pod success
Oct  9 01:59:55.397: INFO: Pod "azuredisk-volume-tester-nswvf" satisfied condition "Succeeded or Failed"
Oct  9 01:59:55.397: INFO: deleting Pod "azuredisk-5356"/"azuredisk-volume-tester-nswvf"
Oct  9 01:59:55.438: INFO: Pod azuredisk-volume-tester-nswvf has the following logs: hello world

STEP: Deleting pod azuredisk-volume-tester-nswvf in namespace azuredisk-5356
STEP: validating provisioned PV
STEP: checking the PV
... skipping 92 lines ...
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod has 'FailedMount' event
Oct  9 02:00:46.120: INFO: deleting Pod "azuredisk-1957"/"azuredisk-volume-tester-bqxmc"
Oct  9 02:00:46.160: INFO: Error getting logs for pod azuredisk-volume-tester-bqxmc: the server rejected our request for an unknown reason (get pods azuredisk-volume-tester-bqxmc)
STEP: Deleting pod azuredisk-volume-tester-bqxmc in namespace azuredisk-1957
STEP: validating provisioned PV
STEP: checking the PV
Oct  9 02:00:46.273: INFO: deleting PVC "azuredisk-1957"/"pvc-nqff5"
Oct  9 02:00:46.273: INFO: Deleting PersistentVolumeClaim "pvc-nqff5"
STEP: waiting for claim's PV "pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1" to be deleted
... skipping 36 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 02:01:27.875: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-zdzl7" in namespace "azuredisk-8705" to be "Succeeded or Failed"
Oct  9 02:01:27.912: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 36.768185ms
Oct  9 02:01:29.950: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 2.074221005s
Oct  9 02:01:31.988: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 4.113176624s
Oct  9 02:01:34.028: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 6.15222614s
Oct  9 02:01:36.067: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 8.191343909s
Oct  9 02:01:38.106: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 10.231200441s
... skipping 3 lines ...
Oct  9 02:01:46.260: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 18.384826825s
Oct  9 02:01:48.298: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 20.423032039s
Oct  9 02:01:50.336: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 22.460887921s
Oct  9 02:01:52.374: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Pending", Reason="", readiness=false. Elapsed: 24.499146908s
Oct  9 02:01:54.414: INFO: Pod "azuredisk-volume-tester-zdzl7": Phase="Succeeded", Reason="", readiness=false. Elapsed: 26.539158629s
STEP: Saw pod success
Oct  9 02:01:54.414: INFO: Pod "azuredisk-volume-tester-zdzl7" satisfied condition "Succeeded or Failed"
Oct  9 02:01:54.415: INFO: deleting Pod "azuredisk-8705"/"azuredisk-volume-tester-zdzl7"
Oct  9 02:01:54.455: INFO: Pod azuredisk-volume-tester-zdzl7 has the following logs: e2e-test

STEP: Deleting pod azuredisk-volume-tester-zdzl7 in namespace azuredisk-8705
STEP: validating provisioned PV
STEP: checking the PV
... skipping 36 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with an error
Oct  9 02:02:21.056: INFO: Waiting up to 10m0s for pod "azuredisk-volume-tester-8kmdl" in namespace "azuredisk-2451" to be "Error status code"
Oct  9 02:02:21.093: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 37.3787ms
Oct  9 02:02:23.132: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 2.075964056s
Oct  9 02:02:25.170: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114294255s
Oct  9 02:02:27.208: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 6.152645192s
Oct  9 02:02:29.247: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 8.191554254s
Oct  9 02:02:31.285: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 10.229313761s
Oct  9 02:02:33.325: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 12.269204783s
Oct  9 02:02:35.368: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 14.312079892s
Oct  9 02:02:37.407: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 16.351366531s
Oct  9 02:02:39.446: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 18.390528629s
Oct  9 02:02:41.486: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 20.42982267s
Oct  9 02:02:43.524: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Pending", Reason="", readiness=false. Elapsed: 22.468287601s
Oct  9 02:02:45.564: INFO: Pod "azuredisk-volume-tester-8kmdl": Phase="Failed", Reason="", readiness=false. Elapsed: 24.508256754s
STEP: Saw pod failure
Oct  9 02:02:45.564: INFO: Pod "azuredisk-volume-tester-8kmdl" satisfied condition "Error status code"
STEP: checking that pod logs contain expected message
Oct  9 02:02:45.612: INFO: deleting Pod "azuredisk-2451"/"azuredisk-volume-tester-8kmdl"
Oct  9 02:02:45.652: INFO: Pod azuredisk-volume-tester-8kmdl has the following logs: touch: /mnt/test-1/data: Read-only file system

STEP: Deleting pod azuredisk-volume-tester-8kmdl in namespace azuredisk-2451
STEP: validating provisioned PV
... skipping 366 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 02:11:28.145: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-tqs42" in namespace "azuredisk-552" to be "Succeeded or Failed"
Oct  9 02:11:28.182: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 36.883909ms
Oct  9 02:11:30.221: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 2.075842884s
Oct  9 02:11:32.260: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114406224s
Oct  9 02:11:34.300: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 6.154307133s
Oct  9 02:11:36.338: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 8.19259409s
Oct  9 02:11:38.377: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 10.231746608s
... skipping 2 lines ...
Oct  9 02:11:44.492: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 16.346381538s
Oct  9 02:11:46.531: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 18.385925542s
Oct  9 02:11:48.570: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 20.424113313s
Oct  9 02:11:50.608: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Pending", Reason="", readiness=false. Elapsed: 22.462222086s
Oct  9 02:11:52.647: INFO: Pod "azuredisk-volume-tester-tqs42": Phase="Succeeded", Reason="", readiness=false. Elapsed: 24.501752901s
STEP: Saw pod success
Oct  9 02:11:52.647: INFO: Pod "azuredisk-volume-tester-tqs42" satisfied condition "Succeeded or Failed"
Oct  9 02:11:52.647: INFO: deleting Pod "azuredisk-552"/"azuredisk-volume-tester-tqs42"
Oct  9 02:11:52.746: INFO: Pod azuredisk-volume-tester-tqs42 has the following logs: hello world
hello world
hello world

STEP: Deleting pod azuredisk-volume-tester-tqs42 in namespace azuredisk-552
... skipping 68 lines ...
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: setting up the pod
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 02:12:45.045: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-mz5ql" in namespace "azuredisk-1351" to be "Succeeded or Failed"
Oct  9 02:12:45.091: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 45.35079ms
Oct  9 02:12:47.128: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 2.082600825s
Oct  9 02:12:49.172: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 4.12629335s
Oct  9 02:12:51.209: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 6.163822536s
Oct  9 02:12:53.247: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 8.201309977s
Oct  9 02:12:55.285: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 10.239751701s
... skipping 9 lines ...
Oct  9 02:13:15.675: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 30.629602573s
Oct  9 02:13:17.714: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 32.668440503s
Oct  9 02:13:19.752: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 34.706354819s
Oct  9 02:13:21.790: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Pending", Reason="", readiness=false. Elapsed: 36.744303952s
Oct  9 02:13:23.828: INFO: Pod "azuredisk-volume-tester-mz5ql": Phase="Succeeded", Reason="", readiness=false. Elapsed: 38.782733485s
STEP: Saw pod success
Oct  9 02:13:23.828: INFO: Pod "azuredisk-volume-tester-mz5ql" satisfied condition "Succeeded or Failed"
Oct  9 02:13:23.828: INFO: deleting Pod "azuredisk-1351"/"azuredisk-volume-tester-mz5ql"
Oct  9 02:13:23.891: INFO: Pod azuredisk-volume-tester-mz5ql has the following logs: 100+0 records in
100+0 records out
104857600 bytes (100.0MB) copied, 0.062959 seconds, 1.6GB/s
hello world

... skipping 115 lines ...
STEP: creating a PVC
STEP: setting up the StorageClass
STEP: creating a StorageClass 
STEP: setting up the PVC and PV
STEP: creating a PVC
STEP: deploying the pod
STEP: checking that the pod's command exits with no error
Oct  9 02:14:18.016: INFO: Waiting up to 15m0s for pod "azuredisk-volume-tester-5xb9c" in namespace "azuredisk-9267" to be "Succeeded or Failed"
Oct  9 02:14:18.053: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 37.617569ms
Oct  9 02:14:20.092: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 2.076156362s
Oct  9 02:14:22.131: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 4.114714963s
Oct  9 02:14:24.169: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 6.153520098s
Oct  9 02:14:26.213: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 8.197298008s
Oct  9 02:14:28.252: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 10.235816207s
... skipping 10 lines ...
Oct  9 02:14:50.677: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 32.66079209s
Oct  9 02:14:52.716: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 34.699941883s
Oct  9 02:14:54.761: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 36.744804073s
Oct  9 02:14:56.799: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Pending", Reason="", readiness=false. Elapsed: 38.783065444s
Oct  9 02:14:58.838: INFO: Pod "azuredisk-volume-tester-5xb9c": Phase="Succeeded", Reason="", readiness=false. Elapsed: 40.822126049s
STEP: Saw pod success
Oct  9 02:14:58.838: INFO: Pod "azuredisk-volume-tester-5xb9c" satisfied condition "Succeeded or Failed"
Oct  9 02:14:58.838: INFO: deleting Pod "azuredisk-9267"/"azuredisk-volume-tester-5xb9c"
Oct  9 02:14:58.933: INFO: Pod azuredisk-volume-tester-5xb9c has the following logs: hello world

STEP: Deleting pod azuredisk-volume-tester-5xb9c in namespace azuredisk-9267
STEP: validating provisioned PV
STEP: checking the PV
... skipping 440 lines ...
I1009 01:55:24.084444       1 tlsconfig.go:200] loaded serving cert ["Generated self signed cert"]: "localhost@1633744523" [serving] validServingFor=[127.0.0.1,localhost,localhost] issuer="localhost-ca@1633744523" (2021-10-09 00:55:22 +0000 UTC to 2022-10-09 00:55:22 +0000 UTC (now=2021-10-09 01:55:24.084426262 +0000 UTC))
I1009 01:55:24.084819       1 named_certificates.go:53] loaded SNI cert [0/"self-signed loopback"]: "apiserver-loopback-client@1633744524" [serving] validServingFor=[apiserver-loopback-client] issuer="apiserver-loopback-client-ca@1633744523" (2021-10-09 00:55:23 +0000 UTC to 2022-10-09 00:55:23 +0000 UTC (now=2021-10-09 01:55:24.084807254 +0000 UTC))
I1009 01:55:24.084862       1 secure_serving.go:202] Serving securely on [::]:10257
I1009 01:55:24.084921       1 tlsconfig.go:240] Starting DynamicServingCertificateController
I1009 01:55:24.085338       1 deprecated_insecure_serving.go:56] Serving insecurely on [::]:10252
I1009 01:55:24.087314       1 leaderelection.go:243] attempting to acquire leader lease kube-system/kube-controller-manager...
E1009 01:55:29.088464       1 leaderelection.go:325] error retrieving resource lock kube-system/kube-controller-manager: Get "https://10.240.255.5:443/apis/coordination.k8s.io/v1/namespaces/kube-system/leases/kube-controller-manager?timeout=5s": context deadline exceeded
I1009 01:55:32.397576       1 leaderelection.go:253] successfully acquired lease kube-system/kube-controller-manager
I1009 01:55:32.406508       1 event.go:291] "Event occurred" object="kube-system/kube-controller-manager" kind="Lease" apiVersion="coordination.k8s.io/v1" type="Normal" reason="LeaderElection" message="k8s-master-31388238-0_748c0614-5b69-4c3e-9a94-06cde320e42a became leader"
W1009 01:55:32.975327       1 plugins.go:105] WARNING: azure built-in cloud provider is now deprecated. The Azure provider is deprecated and will be removed in a future release
I1009 01:55:32.976211       1 azure_auth.go:117] azure: using client_id+client_secret to retrieve access token
I1009 01:55:32.987813       1 azure.go:455] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
I1009 01:55:33.001022       1 azure_interfaceclient.go:62] Azure InterfacesClient (read ops) using rate limit config: QPS=6, bucket=20
... skipping 23 lines ...
I1009 01:55:33.012215       1 azure_securitygroupclient.go:63] Azure SecurityGroupsClient (read ops) using rate limit config: QPS=6, bucket=20
I1009 01:55:33.012219       1 azure_securitygroupclient.go:66] Azure SecurityGroupsClient (write ops) using rate limit config: QPS=100, bucket=1000
I1009 01:55:33.012234       1 azure_publicipclient.go:63] Azure PublicIPAddressesClient (read ops) using rate limit config: QPS=6, bucket=20
I1009 01:55:33.012238       1 azure_publicipclient.go:66] Azure PublicIPAddressesClient (write ops) using rate limit config: QPS=100, bucket=1000
I1009 01:55:33.023464       1 azure.go:744] Setting up informers for Azure cloud provider
I1009 01:55:33.034606       1 shared_informer.go:240] Waiting for caches to sync for tokens
W1009 01:55:33.113795       1 azure_config.go:52] Failed to get cloud-config from secret: failed to get secret azure-cloud-provider: secrets "azure-cloud-provider" is forbidden: User "system:serviceaccount:kube-system:azure-cloud-provider" cannot get resource "secrets" in API group "" in the namespace "kube-system", skip initializing from secret
I1009 01:55:33.113826       1 controllermanager.go:559] Starting "resourcequota"
I1009 01:55:33.134685       1 shared_informer.go:247] Caches are synced for tokens 
I1009 01:55:33.197271       1 resource_quota_monitor.go:229] QuotaMonitor created object count evaluator for ingresses.networking.k8s.io
I1009 01:55:33.197368       1 resource_quota_monitor.go:229] QuotaMonitor created object count evaluator for roles.rbac.authorization.k8s.io
I1009 01:55:33.197476       1 resource_quota_monitor.go:229] QuotaMonitor created object count evaluator for replicasets.apps
I1009 01:55:33.197527       1 resource_quota_monitor.go:229] QuotaMonitor created object count evaluator for poddisruptionbudgets.policy
... skipping 105 lines ...
I1009 01:55:35.141195       1 plugins.go:639] Loaded volume plugin "kubernetes.io/rbd"
I1009 01:55:35.141213       1 plugins.go:639] Loaded volume plugin "kubernetes.io/csi"
I1009 01:55:35.152810       1 controllermanager.go:574] Started "attachdetach"
I1009 01:55:35.152878       1 controllermanager.go:559] Starting "pv-protection"
I1009 01:55:35.152996       1 attach_detach_controller.go:328] Starting attach detach controller
I1009 01:55:35.153010       1 shared_informer.go:240] Waiting for caches to sync for attach detach
W1009 01:55:35.153132       1 actual_state_of_world.go:534] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="k8s-master-31388238-0" does not exist
W1009 01:55:35.153155       1 actual_state_of_world.go:534] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="k8s-agentpool1-31388238-0" does not exist
W1009 01:55:35.153162       1 actual_state_of_world.go:534] Failed to update statusUpdateNeeded field in actual state of world: Failed to set statusUpdateNeeded to needed true, because nodeName="k8s-agentpool1-31388238-1" does not exist
I1009 01:55:35.296091       1 controllermanager.go:574] Started "pv-protection"
I1009 01:55:35.296113       1 controllermanager.go:559] Starting "ttl-after-finished"
I1009 01:55:35.296146       1 pv_protection_controller.go:83] Starting PV protection controller
I1009 01:55:35.296153       1 shared_informer.go:240] Waiting for caches to sync for PV protection
I1009 01:55:35.438478       1 controllermanager.go:574] Started "ttl-after-finished"
I1009 01:55:35.438501       1 controllermanager.go:559] Starting "nodelifecycle"
... skipping 215 lines ...
I1009 01:56:33.428387       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:33.428399       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:35.983880       1 replica_set.go:559] "Too few replicas" replicaSet="kube-system/coredns-787d7f6757" need=1 creating=1
I1009 01:56:35.990302       1 event.go:291] "Event occurred" object="kube-system/coredns" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-787d7f6757 to 1"
I1009 01:56:35.990709       1 replica_set.go:559] "Too few replicas" replicaSet="kube-system/coredns-autoscaler-87b67c5fd" need=1 creating=1
I1009 01:56:35.991354       1 event.go:291] "Event occurred" object="kube-system/coredns-autoscaler" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set coredns-autoscaler-87b67c5fd to 1"
I1009 01:56:36.026855       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/coredns" err="Operation cannot be fulfilled on deployments.apps \"coredns\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:56:36.030344       1 event.go:291] "Event occurred" object="kube-system/coredns-787d7f6757" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-787d7f6757-plwzh"
I1009 01:56:36.030596       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/coredns-autoscaler" err="Operation cannot be fulfilled on deployments.apps \"coredns-autoscaler\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:56:36.046858       1 event.go:291] "Event occurred" object="kube-system/coredns-autoscaler-87b67c5fd" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: coredns-autoscaler-87b67c5fd-8nkcp"
I1009 01:56:37.061660       1 event.go:291] "Event occurred" object="kube-system/azure-ip-masq-agent" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: azure-ip-masq-agent-6zcf7"
I1009 01:56:37.074611       1 event.go:291] "Event occurred" object="kube-system/azure-ip-masq-agent" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: azure-ip-masq-agent-g46w7"
I1009 01:56:37.077250       1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-7mtkz"
I1009 01:56:37.078904       1 event.go:291] "Event occurred" object="kube-system/azure-ip-masq-agent" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: azure-ip-masq-agent-bk8rm"
I1009 01:56:37.101864       1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-ggjd4"
I1009 01:56:37.111240       1 event.go:291] "Event occurred" object="kube-system/kube-proxy" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: kube-proxy-4x2lw"
E1009 01:56:37.194365       1 daemon_controller.go:320] kube-system/kube-proxy failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"kube-proxy", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"bfbec70e-9613-48aa-8c91-761b0c19bd3b", ResourceVersion:"614", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63769341397, loc:(*time.Location)(0x7305600)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"addonmanager.kubernetes.io/mode":"Reconcile", "component":"kube-proxy", "k8s-app":"kube-proxy", "kubernetes.io/cluster-service":"true", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"addonmanager.kubernetes.io/mode\":\"Reconcile\",\"component\":\"kube-proxy\",\"k8s-app\":\"kube-proxy\",\"kubernetes.io/cluster-service\":\"true\",\"tier\":\"node\"},\"name\":\"kube-proxy\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"component\":\"kube-proxy\",\"k8s-app\":\"kube-proxy\",\"tier\":\"node\"}},\"template\":{\"metadata\":{\"annotations\":{\"cluster-autoscaler.kubernetes.io/daemonset-pod\":\"true\",\"scheduler.alpha.kubernetes.io/critical-pod\":\"\"},\"labels\":{\"component\":\"kube-proxy\",\"k8s-app\":\"kube-proxy\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"command\":[\"kube-proxy\",\"--config=/var/lib/kube-proxy/config.yaml\"],\"image\":\"mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.5\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"kube-proxy\",\"resources\":{\"requests\":{\"cpu\":\"100m\"}},\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/etc/ssl/certs\",\"name\":\"ssl-certs-host\",\"readOnly\":true},{\"mountPath\":\"/etc/kubernetes\",\"name\":\"etc-kubernetes\",\"readOnly\":true},{\"mountPath\":\"/var/lib/kubelet/kubeconfig\",\"name\":\"kubeconfig\",\"readOnly\":true},{\"mountPath\":\"/run/xtables.lock\",\"name\":\"iptableslock\"},{\"mountPath\":\"/lib/modules/\",\"name\":\"kernelmodules\",\"readOnly\":true},{\"mountPath\":\"/var/lib/kube-proxy/config.yaml\",\"name\":\"kube-proxy-config-volume\",\"readOnly\":true,\"subPath\":\"config.yaml\"}]}],\"hostNetwork\":true,\"nodeSelector\":{\"kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-node-critical\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Equal\",\"value\":\"true\"},{\"effect\":\"NoExecute\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"operator\":\"Exists\"},{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/usr/share/ca-certificates\"},\"name\":\"ssl-certs-host\"},{\"hostPath\":{\"path\":\"/var/lib/kubelet/kubeconfig\"},\"name\":\"kubeconfig\"},{\"hostPath\":{\"path\":\"/etc/kubernetes\"},\"name\":\"etc-kubernetes\"},{\"hostPath\":{\"path\":\"/run/xtables.lock\"},\"name\":\"iptableslock\"},{\"hostPath\":{\"path\":\"/lib/modules/\"},\"name\":\"kernelmodules\"},{\"configMap\":{\"name\":\"kube-proxy-config\"},\"name\":\"kube-proxy-config-volume\"}]}},\"updateStrategy\":{\"rollingUpdate\":{\"maxUnavailable\":\"50%\"},\"type\":\"RollingUpdate\"}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0xc00107b5d8), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc00107b5f0)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0xc00170b660), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"component":"kube-proxy", "k8s-app":"kube-proxy", "tier":"node"}, Annotations:map[string]string{"cluster-autoscaler.kubernetes.io/daemonset-pod":"true", "scheduler.alpha.kubernetes.io/critical-pod":""}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"ssl-certs-host", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc00107b608), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"kubeconfig", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc00107b620), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"etc-kubernetes", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc00107b638), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"iptableslock", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc00107b650), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"kernelmodules", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc00107b668), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"kube-proxy-config-volume", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0xc001995940), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"kube-proxy", Image:"mcr.microsoft.com/oss/kubernetes/kube-proxy:v1.21.5", Command:[]string{"kube-proxy", "--config=/var/lib/kube-proxy/config.yaml"}, Args:[]string(nil), WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList(nil), Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"ssl-certs-host", ReadOnly:true, MountPath:"/etc/ssl/certs", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"etc-kubernetes", ReadOnly:true, MountPath:"/etc/kubernetes", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"kubeconfig", ReadOnly:true, MountPath:"/var/lib/kubelet/kubeconfig", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"iptableslock", ReadOnly:false, MountPath:"/run/xtables.lock", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"kernelmodules", ReadOnly:true, MountPath:"/lib/modules/", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"kube-proxy-config-volume", ReadOnly:true, MountPath:"/var/lib/kube-proxy/config.yaml", SubPath:"config.yaml", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0xc001afca20), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc001bb4c08), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"", DeprecatedServiceAccount:"", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0xc000238690), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"node-role.kubernetes.io/master", Operator:"Equal", Value:"true", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoExecute", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0xc001afaa20)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0xc001bb4c80)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "kube-proxy": the object has been modified; please apply your changes to the latest version and try again
E1009 01:56:37.198032       1 daemon_controller.go:320] kube-system/azure-ip-masq-agent failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"azure-ip-masq-agent", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"1dfedf99-77cf-4337-93c7-6ac09164f481", ResourceVersion:"612", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63769341397, loc:(*time.Location)(0x7305600)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"addonmanager.kubernetes.io/mode":"Reconcile", "component":"azure-ip-masq-agent", "kubernetes.io/cluster-service":"true", "tier":"node"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"addonmanager.kubernetes.io/mode\":\"Reconcile\",\"component\":\"azure-ip-masq-agent\",\"kubernetes.io/cluster-service\":\"true\",\"tier\":\"node\"},\"name\":\"azure-ip-masq-agent\",\"namespace\":\"kube-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"k8s-app\":\"azure-ip-masq-agent\",\"tier\":\"node\"}},\"template\":{\"metadata\":{\"annotations\":{\"cluster-autoscaler.kubernetes.io/daemonset-pod\":\"true\"},\"labels\":{\"k8s-app\":\"azure-ip-masq-agent\",\"tier\":\"node\"}},\"spec\":{\"containers\":[{\"args\":[\"--enable-ipv6=false\"],\"image\":\"mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"azure-ip-masq-agent\",\"resources\":{\"limits\":{\"cpu\":\"50m\",\"memory\":\"250Mi\"},\"requests\":{\"cpu\":\"50m\",\"memory\":\"50Mi\"}},\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/etc/config\",\"name\":\"azure-ip-masq-agent-config-volume\"}]}],\"hostNetwork\":true,\"nodeSelector\":{\"kubernetes.io/os\":\"linux\"},\"priorityClassName\":\"system-node-critical\",\"tolerations\":[{\"key\":\"CriticalAddonsOnly\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"node-role.kubernetes.io/master\",\"operator\":\"Equal\",\"value\":\"true\"},{\"effect\":\"NoExecute\",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"operator\":\"Exists\"}],\"volumes\":[{\"configMap\":{\"name\":\"azure-ip-masq-agent-config\"},\"name\":\"azure-ip-masq-agent-config-volume\"}]}},\"updateStrategy\":{\"rollingUpdate\":{\"maxUnavailable\":\"50%\"},\"type\":\"RollingUpdate\"}}}\n"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"kubectl-client-side-apply", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0xc0012b1728), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0012b1740)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0xc00170a680), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"k8s-app":"azure-ip-masq-agent", "tier":"node"}, Annotations:map[string]string{"cluster-autoscaler.kubernetes.io/daemonset-pod":"true"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"azure-ip-masq-agent-config-volume", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(nil), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(0xc001994980), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"azure-ip-masq-agent", Image:"mcr.microsoft.com/oss/kubernetes/ip-masq-agent:v2.5.0", Command:[]string(nil), Args:[]string{"--enable-ipv6=false"}, WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:50, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:262144000, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"250Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:50, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:52428800, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"50Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"azure-ip-masq-agent-config-volume", ReadOnly:false, MountPath:"/etc/config", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(0xc0017418c0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc0014c7528), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"ClusterFirst", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"", DeprecatedServiceAccount:"", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0xc0009f7b90), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(nil), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"CriticalAddonsOnly", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"node-role.kubernetes.io/master", Operator:"Equal", Value:"true", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoExecute", TolerationSeconds:(*int64)(nil)}, v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"NoSchedule", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0xc001a8cb30)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0xc0014c75b0)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "azure-ip-masq-agent": the object has been modified; please apply your changes to the latest version and try again
I1009 01:56:38.266749       1 replica_set.go:559] "Too few replicas" replicaSet="kube-system/metrics-server-67b6bdd56c" need=1 creating=1
I1009 01:56:38.267623       1 event.go:291] "Event occurred" object="kube-system/metrics-server" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set metrics-server-67b6bdd56c to 1"
I1009 01:56:38.277652       1 event.go:291] "Event occurred" object="kube-system/metrics-server-67b6bdd56c" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: metrics-server-67b6bdd56c-4bvw4"
I1009 01:56:38.318973       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/metrics-server" err="Operation cannot be fulfilled on deployments.apps \"metrics-server\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:56:38.337112       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/metrics-server" err="Operation cannot be fulfilled on deployments.apps \"metrics-server\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:56:43.428797       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:43.428807       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:43.428821       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
E1009 01:56:47.345944       1 resource_quota_controller.go:409] unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request
W1009 01:56:47.842488       1 garbagecollector.go:703] failed to discover some groups: map[metrics.k8s.io/v1beta1:the server is currently unable to handle the request]
I1009 01:56:53.428941       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:53.428941       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:56:53.428957       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:57:03.429832       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:57:03.429832       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:57:03.429845       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 23 lines ...
I1009 01:58:21.494972       1 event.go:291] "Event occurred" object="kube-system/csi-azuredisk-node" kind="DaemonSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: csi-azuredisk-node-pn25w"
I1009 01:58:21.517491       1 replica_set.go:559] "Too few replicas" replicaSet="kube-system/csi-snapshot-controller-847dc469d" need=1 creating=1
I1009 01:58:21.519717       1 replica_set.go:559] "Too few replicas" replicaSet="kube-system/csi-azuredisk-controller-5fc898d668" need=2 creating=2
I1009 01:58:21.520587       1 event.go:291] "Event occurred" object="kube-system/csi-snapshot-controller" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set csi-snapshot-controller-847dc469d to 1"
I1009 01:58:21.523839       1 event.go:291] "Event occurred" object="kube-system/csi-azuredisk-controller" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set csi-azuredisk-controller-5fc898d668 to 2"
I1009 01:58:21.547130       1 event.go:291] "Event occurred" object="kube-system/csi-azuredisk-controller-5fc898d668" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: csi-azuredisk-controller-5fc898d668-przg8"
I1009 01:58:21.554392       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/csi-azuredisk-controller" err="Operation cannot be fulfilled on deployments.apps \"csi-azuredisk-controller\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:58:21.557486       1 event.go:291] "Event occurred" object="kube-system/csi-snapshot-controller-847dc469d" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: csi-snapshot-controller-847dc469d-dg77s"
E1009 01:58:21.562780       1 daemon_controller.go:320] kube-system/csi-azuredisk-node failed with : error storing status for daemon set &v1.DaemonSet{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"csi-azuredisk-node", GenerateName:"", Namespace:"kube-system", SelfLink:"", UID:"7c2e9f28-4ca4-4d3a-9d94-ff0d7c548a81", ResourceVersion:"949", Generation:1, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:63769341501, loc:(*time.Location)(0x7305600)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app.kubernetes.io/instance":"azuredisk-csi-driver", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"azuredisk-csi-driver", "app.kubernetes.io/version":"latest", "helm.sh/chart":"azuredisk-csi-driver-v1.8.0"}, Annotations:map[string]string{"deprecated.daemonset.template.generation":"1", "meta.helm.sh/release-name":"azuredisk-csi-driver", "meta.helm.sh/release-namespace":"kube-system"}, OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry{v1.ManagedFieldsEntry{Manager:"helm", Operation:"Update", APIVersion:"apps/v1", Time:(*v1.Time)(0xc0025ff758), FieldsType:"FieldsV1", FieldsV1:(*v1.FieldsV1)(0xc0025ff770)}}}, Spec:v1.DaemonSetSpec{Selector:(*v1.LabelSelector)(0xc0029784e0), Template:v1.PodTemplateSpec{ObjectMeta:v1.ObjectMeta{Name:"", GenerateName:"", Namespace:"", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string{"app":"csi-azuredisk-node", "app.kubernetes.io/instance":"azuredisk-csi-driver", "app.kubernetes.io/managed-by":"Helm", "app.kubernetes.io/name":"azuredisk-csi-driver", "app.kubernetes.io/version":"latest", "helm.sh/chart":"azuredisk-csi-driver-v1.8.0"}, Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, Spec:v1.PodSpec{Volumes:[]v1.Volume{v1.Volume{Name:"socket-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff788), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"mountpoint-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff7a0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"registration-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff7b8), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"azure-cred", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff7d0), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"device-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff7e8), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"sys-devices-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff800), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}, v1.Volume{Name:"scsi-host-dir", VolumeSource:v1.VolumeSource{HostPath:(*v1.HostPathVolumeSource)(0xc0025ff818), EmptyDir:(*v1.EmptyDirVolumeSource)(nil), GCEPersistentDisk:(*v1.GCEPersistentDiskVolumeSource)(nil), AWSElasticBlockStore:(*v1.AWSElasticBlockStoreVolumeSource)(nil), GitRepo:(*v1.GitRepoVolumeSource)(nil), Secret:(*v1.SecretVolumeSource)(nil), NFS:(*v1.NFSVolumeSource)(nil), ISCSI:(*v1.ISCSIVolumeSource)(nil), Glusterfs:(*v1.GlusterfsVolumeSource)(nil), PersistentVolumeClaim:(*v1.PersistentVolumeClaimVolumeSource)(nil), RBD:(*v1.RBDVolumeSource)(nil), FlexVolume:(*v1.FlexVolumeSource)(nil), Cinder:(*v1.CinderVolumeSource)(nil), CephFS:(*v1.CephFSVolumeSource)(nil), Flocker:(*v1.FlockerVolumeSource)(nil), DownwardAPI:(*v1.DownwardAPIVolumeSource)(nil), FC:(*v1.FCVolumeSource)(nil), AzureFile:(*v1.AzureFileVolumeSource)(nil), ConfigMap:(*v1.ConfigMapVolumeSource)(nil), VsphereVolume:(*v1.VsphereVirtualDiskVolumeSource)(nil), Quobyte:(*v1.QuobyteVolumeSource)(nil), AzureDisk:(*v1.AzureDiskVolumeSource)(nil), PhotonPersistentDisk:(*v1.PhotonPersistentDiskVolumeSource)(nil), Projected:(*v1.ProjectedVolumeSource)(nil), PortworxVolume:(*v1.PortworxVolumeSource)(nil), ScaleIO:(*v1.ScaleIOVolumeSource)(nil), StorageOS:(*v1.StorageOSVolumeSource)(nil), CSI:(*v1.CSIVolumeSource)(nil), Ephemeral:(*v1.EphemeralVolumeSource)(nil)}}}, InitContainers:[]v1.Container(nil), Containers:[]v1.Container{v1.Container{Name:"liveness-probe", Image:"mcr.microsoft.com/oss/kubernetes-csi/livenessprobe:v2.4.0", Command:[]string(nil), Args:[]string{"--csi-address=/csi/csi.sock", "--probe-timeout=3s", "--health-port=29603", "--v=2"}, WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar(nil), Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:104857600, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:10, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"10m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:20971520, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"20Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"socket-dir", ReadOnly:false, MountPath:"/csi", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}, v1.Container{Name:"node-driver-registrar", Image:"mcr.microsoft.com/oss/kubernetes-csi/csi-node-driver-registrar:v2.3.0", Command:[]string(nil), Args:[]string{"--csi-address=$(ADDRESS)", "--kubelet-registration-path=$(DRIVER_REG_SOCK_PATH)", "--v=2"}, WorkingDir:"", Ports:[]v1.ContainerPort(nil), EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"ADDRESS", Value:"/csi/csi.sock", ValueFrom:(*v1.EnvVarSource)(nil)}, v1.EnvVar{Name:"DRIVER_REG_SOCK_PATH", Value:"/var/lib/kubelet/plugins/disk.csi.azure.com/csi.sock", ValueFrom:(*v1.EnvVarSource)(nil)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:100, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:104857600, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"100Mi", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:10, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"10m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:20971520, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"20Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"socket-dir", ReadOnly:false, MountPath:"/csi", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"registration-dir", ReadOnly:false, MountPath:"/registration", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(nil), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(0xc002da3470), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"IfNotPresent", SecurityContext:(*v1.SecurityContext)(nil), Stdin:false, StdinOnce:false, TTY:false}, v1.Container{Name:"azuredisk", Image:"k8sprow.azurecr.io/azuredisk-csi:v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584", Command:[]string(nil), Args:[]string{"--v=5", "--endpoint=$(CSI_ENDPOINT)", "--nodeid=$(KUBE_NODE_NAME)", "--metrics-address=0.0.0.0:29605", "--enable-perf-optimization=true", "--drivername=disk.csi.azure.com", "--volume-attach-limit=-1", "--cloud-config-secret-name=azure-cloud-provider", "--cloud-config-secret-namespace=kube-system", "--custom-user-agent=", "--user-agent-suffix=e2e-test"}, WorkingDir:"", Ports:[]v1.ContainerPort{v1.ContainerPort{Name:"healthz", HostPort:29603, ContainerPort:29603, Protocol:"TCP", HostIP:""}}, EnvFrom:[]v1.EnvFromSource(nil), Env:[]v1.EnvVar{v1.EnvVar{Name:"AZURE_CREDENTIAL_FILE", Value:"", ValueFrom:(*v1.EnvVarSource)(0xc002978580)}, v1.EnvVar{Name:"CSI_ENDPOINT", Value:"unix:///csi/csi.sock", ValueFrom:(*v1.EnvVarSource)(nil)}, v1.EnvVar{Name:"KUBE_NODE_NAME", Value:"", ValueFrom:(*v1.EnvVarSource)(0xc0029785a0)}}, Resources:v1.ResourceRequirements{Limits:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:200, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"200m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:209715200, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"", Format:"BinarySI"}}, Requests:v1.ResourceList{"cpu":resource.Quantity{i:resource.int64Amount{value:10, scale:-3}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"10m", Format:"DecimalSI"}, "memory":resource.Quantity{i:resource.int64Amount{value:20971520, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:"20Mi", Format:"BinarySI"}}}, VolumeMounts:[]v1.VolumeMount{v1.VolumeMount{Name:"socket-dir", ReadOnly:false, MountPath:"/csi", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"mountpoint-dir", ReadOnly:false, MountPath:"/var/lib/kubelet/", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(0xc002da34a0), SubPathExpr:""}, v1.VolumeMount{Name:"azure-cred", ReadOnly:false, MountPath:"/etc/kubernetes/", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"device-dir", ReadOnly:false, MountPath:"/dev", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"sys-devices-dir", ReadOnly:false, MountPath:"/sys/bus/scsi/devices", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}, v1.VolumeMount{Name:"scsi-host-dir", ReadOnly:false, MountPath:"/sys/class/scsi_host/", SubPath:"", MountPropagation:(*v1.MountPropagationMode)(nil), SubPathExpr:""}}, VolumeDevices:[]v1.VolumeDevice(nil), LivenessProbe:(*v1.Probe)(0xc0017f8280), ReadinessProbe:(*v1.Probe)(nil), StartupProbe:(*v1.Probe)(nil), Lifecycle:(*v1.Lifecycle)(nil), TerminationMessagePath:"/dev/termination-log", TerminationMessagePolicy:"File", ImagePullPolicy:"Always", SecurityContext:(*v1.SecurityContext)(0xc00261fda0), Stdin:false, StdinOnce:false, TTY:false}}, EphemeralContainers:[]v1.EphemeralContainer(nil), RestartPolicy:"Always", TerminationGracePeriodSeconds:(*int64)(0xc00139b6f8), ActiveDeadlineSeconds:(*int64)(nil), DNSPolicy:"Default", NodeSelector:map[string]string{"kubernetes.io/os":"linux"}, ServiceAccountName:"csi-azuredisk-node-sa", DeprecatedServiceAccount:"csi-azuredisk-node-sa", AutomountServiceAccountToken:(*bool)(nil), NodeName:"", HostNetwork:true, HostPID:false, HostIPC:false, ShareProcessNamespace:(*bool)(nil), SecurityContext:(*v1.PodSecurityContext)(0xc00091a5b0), ImagePullSecrets:[]v1.LocalObjectReference(nil), Hostname:"", Subdomain:"", Affinity:(*v1.Affinity)(0xc0025ff860), SchedulerName:"default-scheduler", Tolerations:[]v1.Toleration{v1.Toleration{Key:"", Operator:"Exists", Value:"", Effect:"", TolerationSeconds:(*int64)(nil)}}, HostAliases:[]v1.HostAlias(nil), PriorityClassName:"system-node-critical", Priority:(*int32)(nil), DNSConfig:(*v1.PodDNSConfig)(nil), ReadinessGates:[]v1.PodReadinessGate(nil), RuntimeClassName:(*string)(nil), EnableServiceLinks:(*bool)(nil), PreemptionPolicy:(*v1.PreemptionPolicy)(nil), Overhead:v1.ResourceList(nil), TopologySpreadConstraints:[]v1.TopologySpreadConstraint(nil), SetHostnameAsFQDN:(*bool)(nil)}}, UpdateStrategy:v1.DaemonSetUpdateStrategy{Type:"RollingUpdate", RollingUpdate:(*v1.RollingUpdateDaemonSet)(0xc002da34c0)}, MinReadySeconds:0, RevisionHistoryLimit:(*int32)(0xc00139b72c)}, Status:v1.DaemonSetStatus{CurrentNumberScheduled:0, NumberMisscheduled:0, DesiredNumberScheduled:0, NumberReady:0, ObservedGeneration:0, UpdatedNumberScheduled:0, NumberAvailable:0, NumberUnavailable:0, CollisionCount:(*int32)(nil), Conditions:[]v1.DaemonSetCondition(nil)}}: Operation cannot be fulfilled on daemonsets.apps "csi-azuredisk-node": the object has been modified; please apply your changes to the latest version and try again
I1009 01:58:21.564745       1 deployment_controller.go:490] "Error syncing deployment" deployment="kube-system/csi-snapshot-controller" err="Operation cannot be fulfilled on deployments.apps \"csi-snapshot-controller\": the object has been modified; please apply your changes to the latest version and try again"
I1009 01:58:21.575557       1 event.go:291] "Event occurred" object="kube-system/csi-azuredisk-controller-5fc898d668" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: csi-azuredisk-controller-5fc898d668-kpmx8"
I1009 01:58:23.498609       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:23.498618       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:23.498647       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:33.434273       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:33.434273       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 20 lines ...
I1009 01:58:47.545565       1 shared_informer.go:247] Caches are synced for resource quota 
I1009 01:58:47.545689       1 resource_quota_controller.go:454] synced quota controller
I1009 01:58:47.900578       1 garbagecollector.go:213] syncing garbage collector with updated resources from discovery (attempt 1): added: [snapshot.storage.k8s.io/v1beta1, Resource=volumesnapshotclasses snapshot.storage.k8s.io/v1beta1, Resource=volumesnapshotcontents snapshot.storage.k8s.io/v1beta1, Resource=volumesnapshots], removed: []
I1009 01:58:47.915383       1 shared_informer.go:240] Waiting for caches to sync for garbage collector
I1009 01:58:48.016153       1 shared_informer.go:247] Caches are synced for garbage collector 
I1009 01:58:48.016187       1 garbagecollector.go:254] synced garbage collector
E1009 01:58:50.498942       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-8081/default: secrets "default-token-pt6rd" is forbidden: unable to create new content in namespace azuredisk-8081 because it is being terminated
E1009 01:58:50.864646       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-1318/default: secrets "default-token-n2vtt" is forbidden: unable to create new content in namespace azuredisk-1318 because it is being terminated
I1009 01:58:50.867324       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "disk.csi.azure.com-pre-provisioned-inline-volume" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume") from node "k8s-agentpool1-31388238-1" 
E1009 01:58:51.068461       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-694/default: secrets "default-token-vkx2g" is forbidden: unable to create new content in namespace azuredisk-694 because it is being terminated
E1009 01:58:51.818556       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-9947/default: secrets "default-token-bwv4p" is forbidden: unable to create new content in namespace azuredisk-9947 because it is being terminated
I1009 01:58:53.435100       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:53.435102       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:53.435121       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 01:58:55.604298       1 namespace_controller.go:185] Namespace has been deleted azuredisk-8081
I1009 01:58:55.903649       1 namespace_controller.go:185] Namespace has been deleted azuredisk-1318
I1009 01:58:56.130724       1 namespace_controller.go:185] Namespace has been deleted azuredisk-694
... skipping 11 lines ...
I1009 01:59:01.189975       1 azure_backoff.go:285] LoadBalancerClient.List(kubetest-awcxnpvo) success
I1009 01:59:01.190007       1 azure_loadbalancer.go:1106] reconcileLoadBalancer for service(kube-system/csi-azuredisk-controller): lb(kubetest-awcxnpvo/kubetest-awcxnpvo-internal) wantLb(false) resolved load balancer name
I1009 01:59:01.190026       1 azure_loadbalancer.go:1499] reconcileLoadBalancer for service(kube-system/csi-azuredisk-controller): lb(kubetest-awcxnpvo-internal) finished
I1009 01:59:01.190034       1 azure_loadbalancer.go:217] EnsureLoadBalancer: reconciling pip
I1009 01:59:01.224336       1 azure_backoff.go:304] PublicIPAddressesClient.List(kubetest-awcxnpvo) success
I1009 01:59:01.291384       1 azure_backoff.go:304] PublicIPAddressesClient.List(kubetest-awcxnpvo) success
W1009 01:59:01.291406       1 azure_utils.go:129] parseTags: error when parsing key-value pair , would ignore this one
I1009 01:59:01.319992       1 azure_loadbalancer.go:695] ensurePublicIPExists for service(kube-system/csi-azuredisk-controller): pip(kubetest-awcxnpvo-ae98edd75244a44688f9af4bf3854d58) - updating
I1009 01:59:01.320021       1 azure_loadbalancer.go:766] service(kube-system/csi-azuredisk-controller): pip(kubetest-awcxnpvo-ae98edd75244a44688f9af4bf3854d58) - creating as ipv4 for clusterIP:10.0.213.36
I1009 01:59:01.320026       1 azure_loadbalancer.go:769] CreateOrUpdatePIP(kubetest-awcxnpvo, "kubetest-awcxnpvo-ae98edd75244a44688f9af4bf3854d58"): start
I1009 01:59:02.283748       1 controller.go:942] Patching status for service kube-system/csi-azuredisk-controller
I1009 01:59:02.284126       1 event.go:291] "Event occurred" object="kube-system/csi-azuredisk-controller" kind="Service" apiVersion="v1" type="Normal" reason="EnsuredLoadBalancer" message="Ensured load balancer"
I1009 01:59:02.804269       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "disk.csi.azure.com-pre-provisioned-inline-volume" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume") from node "k8s-agentpool1-31388238-1" 
... skipping 46 lines ...
I1009 02:00:16.997574       1 pv_controller_base.go:505] deletion of claim "azuredisk-5356/pvc-twmqp" was already processed
I1009 02:00:23.439140       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:00:23.439140       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:00:23.439155       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:00:23.993119       1 event.go:291] "Event occurred" object="azuredisk-1957/pvc-nqff5" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="WaitForFirstConsumer" message="waiting for first consumer to be created before binding"
I1009 02:00:24.039822       1 event.go:291] "Event occurred" object="azuredisk-1957/pvc-nqff5" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
E1009 02:00:26.094399       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-5356/default: secrets "default-token-l2tt2" is forbidden: unable to create new content in namespace azuredisk-5356 because it is being terminated
I1009 02:00:26.367589       1 pv_controller.go:879] volume "pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1" entered phase "Bound"
I1009 02:00:26.367621       1 pv_controller.go:982] volume "pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1" bound to claim "azuredisk-1957/pvc-nqff5"
I1009 02:00:26.373200       1 pv_controller.go:823] claim "azuredisk-1957/pvc-nqff5" entered phase "Bound"
E1009 02:00:27.101086       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-6413/default: secrets "default-token-485bf" is forbidden: unable to create new content in namespace azuredisk-6413 because it is being terminated
I1009 02:00:27.142792       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1") from node "k8s-agentpool1-31388238-1" 
E1009 02:00:28.220150       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-4147/default: secrets "default-token-v9k9k" is forbidden: unable to create new content in namespace azuredisk-4147 because it is being terminated
I1009 02:00:31.190179       1 namespace_controller.go:185] Namespace has been deleted azuredisk-5356
I1009 02:00:32.185330       1 namespace_controller.go:185] Namespace has been deleted azuredisk-6413
I1009 02:00:33.248936       1 namespace_controller.go:185] Namespace has been deleted azuredisk-4147
I1009 02:00:33.508549       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:00:33.508579       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:00:33.508555       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 33 lines ...
I1009 02:01:27.867733       1 event.go:291] "Event occurred" object="azuredisk-8705/pvc-wjqwm" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:01:27.867808       1 event.go:291] "Event occurred" object="azuredisk-8705/pvc-wjqwm" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:01:30.365635       1 pv_controller.go:879] volume "pvc-d383444b-28d0-440c-95a6-dc069ff99ed4" entered phase "Bound"
I1009 02:01:30.365928       1 pv_controller.go:982] volume "pvc-d383444b-28d0-440c-95a6-dc069ff99ed4" bound to claim "azuredisk-8705/pvc-wjqwm"
I1009 02:01:30.373688       1 pv_controller.go:823] claim "azuredisk-8705/pvc-wjqwm" entered phase "Bound"
I1009 02:01:30.885067       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-d383444b-28d0-440c-95a6-dc069ff99ed4" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4") from node "k8s-agentpool1-31388238-1" 
E1009 02:01:31.916532       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-1957/default: secrets "default-token-fpjft" is forbidden: unable to create new content in namespace azuredisk-1957 because it is being terminated
I1009 02:01:33.443056       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:01:33.443062       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:01:33.443157       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:01:37.001998       1 namespace_controller.go:185] Namespace has been deleted azuredisk-1957
I1009 02:01:41.709629       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "pvc-d383444b-28d0-440c-95a6-dc069ff99ed4" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4") from node "k8s-agentpool1-31388238-1" 
I1009 02:01:41.709705       1 event.go:291] "Event occurred" object="azuredisk-8705/azuredisk-volume-tester-zdzl7" kind="Pod" apiVersion="v1" type="Normal" reason="SuccessfulAttachVolume" message="AttachVolume.Attach succeeded for volume \"pvc-d383444b-28d0-440c-95a6-dc069ff99ed4\" "
... skipping 209 lines ...
I1009 02:07:23.463610       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:24.120483       1 pv_controller_base.go:505] deletion of claim "azuredisk-9828/pvc-jgl44" was already processed
I1009 02:07:25.383205       1 event.go:291] "Event occurred" object="azuredisk-1563/pvc-mnbbl" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="WaitForFirstConsumer" message="waiting for first consumer to be created before binding"
I1009 02:07:25.425368       1 replica_set.go:559] "Too few replicas" replicaSet="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489" need=1 creating=1
I1009 02:07:25.425657       1 event.go:291] "Event occurred" object="azuredisk-1563/azuredisk-volume-tester-d4t64" kind="Deployment" apiVersion="apps/v1" type="Normal" reason="ScalingReplicaSet" message="Scaled up replica set azuredisk-volume-tester-d4t64-6c5788d489 to 1"
I1009 02:07:25.435245       1 event.go:291] "Event occurred" object="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: azuredisk-volume-tester-d4t64-6c5788d489-qzjgj"
I1009 02:07:25.445384       1 deployment_controller.go:490] "Error syncing deployment" deployment="azuredisk-1563/azuredisk-volume-tester-d4t64" err="Operation cannot be fulfilled on deployments.apps \"azuredisk-volume-tester-d4t64\": the object has been modified; please apply your changes to the latest version and try again"
I1009 02:07:25.453697       1 event.go:291] "Event occurred" object="azuredisk-1563/pvc-mnbbl" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:07:27.795591       1 pv_controller.go:879] volume "pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43" entered phase "Bound"
I1009 02:07:27.795947       1 pv_controller.go:982] volume "pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43" bound to claim "azuredisk-1563/pvc-mnbbl"
I1009 02:07:27.805020       1 pv_controller.go:823] claim "azuredisk-1563/pvc-mnbbl" entered phase "Bound"
I1009 02:07:28.518371       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43") from node "k8s-agentpool1-31388238-1" 
E1009 02:07:29.453029       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-9828/default: secrets "default-token-kv5dn" is forbidden: unable to create new content in namespace azuredisk-9828 because it is being terminated
I1009 02:07:33.467448       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:33.467474       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:33.467483       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:34.636739       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43") from node "k8s-agentpool1-31388238-1" 
I1009 02:07:34.636887       1 event.go:291] "Event occurred" object="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489-qzjgj" kind="Pod" apiVersion="v1" type="Normal" reason="SuccessfulAttachVolume" message="AttachVolume.Attach succeeded for volume \"pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43\" "
I1009 02:07:34.649919       1 namespace_controller.go:185] Namespace has been deleted azuredisk-9828
I1009 02:07:43.468477       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:43.468528       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:43.468536       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:45.411068       1 replica_set.go:559] "Too few replicas" replicaSet="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489" need=1 creating=1
I1009 02:07:45.420030       1 event.go:291] "Event occurred" object="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489" kind="ReplicaSet" apiVersion="apps/v1" type="Normal" reason="SuccessfulCreate" message="Created pod: azuredisk-volume-tester-d4t64-6c5788d489-zlsk6"
W1009 02:07:45.446501       1 reconciler.go:376] Multi-Attach error for volume "pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43") from node "k8s-agentpool1-31388238-0" Volume is already used by pods azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489-qzjgj on node k8s-agentpool1-31388238-1
I1009 02:07:45.446799       1 event.go:291] "Event occurred" object="azuredisk-1563/azuredisk-volume-tester-d4t64-6c5788d489-zlsk6" kind="Pod" apiVersion="v1" type="Warning" reason="FailedAttachVolume" message="Multi-Attach error for volume \"pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43\" Volume is already used by pod(s) azuredisk-volume-tester-d4t64-6c5788d489-qzjgj"
I1009 02:07:53.469117       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:53.469122       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:07:53.469139       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:08:03.469303       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:08:03.469307       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:08:03.469324       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 101 lines ...
I1009 02:11:28.094306       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-xvrp5" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="WaitForFirstConsumer" message="waiting for first consumer to be created before binding"
I1009 02:11:28.139353       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-fql7h" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:11:28.140128       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-fql7h" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:11:28.150568       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-qrmwr" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:11:28.152405       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-qrmwr" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:11:28.158284       1 event.go:291] "Event occurred" object="azuredisk-552/pvc-xvrp5" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
E1009 02:11:29.098992       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-1577/default: secrets "default-token-m2rdd" is forbidden: unable to create new content in namespace azuredisk-1577 because it is being terminated
E1009 02:11:30.193692       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-953/default: secrets "default-token-452sw" is forbidden: unable to create new content in namespace azuredisk-953 because it is being terminated
I1009 02:11:30.527172       1 pv_controller.go:879] volume "pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031" entered phase "Bound"
I1009 02:11:30.527206       1 pv_controller.go:982] volume "pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031" bound to claim "azuredisk-552/pvc-fql7h"
I1009 02:11:30.533676       1 pv_controller.go:823] claim "azuredisk-552/pvc-fql7h" entered phase "Bound"
I1009 02:11:30.640588       1 pv_controller.go:879] volume "pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23" entered phase "Bound"
I1009 02:11:30.642035       1 pv_controller.go:982] volume "pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23" bound to claim "azuredisk-552/pvc-qrmwr"
I1009 02:11:30.649211       1 pv_controller.go:823] claim "azuredisk-552/pvc-qrmwr" entered phase "Bound"
I1009 02:11:30.797260       1 pv_controller.go:879] volume "pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae" entered phase "Bound"
I1009 02:11:30.797290       1 pv_controller.go:982] volume "pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae" bound to claim "azuredisk-552/pvc-xvrp5"
I1009 02:11:30.803983       1 pv_controller.go:823] claim "azuredisk-552/pvc-xvrp5" entered phase "Bound"
E1009 02:11:31.175236       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-3033/default: secrets "default-token-gbtb7" is forbidden: unable to create new content in namespace azuredisk-3033 because it is being terminated
I1009 02:11:31.256602       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031") from node "k8s-agentpool1-31388238-1" 
I1009 02:11:31.256634       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23") from node "k8s-agentpool1-31388238-1" 
I1009 02:11:31.256652       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae") from node "k8s-agentpool1-31388238-1" 
I1009 02:11:32.359708       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae") from node "k8s-agentpool1-31388238-1" 
I1009 02:11:32.359943       1 event.go:291] "Event occurred" object="azuredisk-552/azuredisk-volume-tester-tqs42" kind="Pod" apiVersion="v1" type="Normal" reason="SuccessfulAttachVolume" message="AttachVolume.Attach succeeded for volume \"pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae\" "
I1009 02:11:33.480769       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 135 lines ...
I1009 02:14:20.453512       1 pv_controller.go:879] volume "pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724" entered phase "Bound"
I1009 02:14:20.453553       1 pv_controller.go:982] volume "pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724" bound to claim "azuredisk-9267/pvc-xnqls"
I1009 02:14:20.460912       1 pv_controller.go:823] claim "azuredisk-9267/pvc-xnqls" entered phase "Bound"
I1009 02:14:20.505417       1 pv_controller.go:879] volume "pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01" entered phase "Bound"
I1009 02:14:20.505463       1 pv_controller.go:982] volume "pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01" bound to claim "azuredisk-9267/pvc-szl5d"
I1009 02:14:20.512161       1 pv_controller.go:823] claim "azuredisk-9267/pvc-szl5d" entered phase "Bound"
E1009 02:14:20.928694       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-3410/default: secrets "default-token-cfmf4" is forbidden: unable to create new content in namespace azuredisk-3410 because it is being terminated
I1009 02:14:21.120214       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4") from node "k8s-agentpool1-31388238-1" 
I1009 02:14:21.120241       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01") from node "k8s-agentpool1-31388238-1" 
I1009 02:14:21.120257       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724") from node "k8s-agentpool1-31388238-1" 
E1009 02:14:21.928169       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-8553/default: secrets "default-token-ktjc6" is forbidden: unable to create new content in namespace azuredisk-8553 because it is being terminated
I1009 02:14:22.239403       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724") from node "k8s-agentpool1-31388238-1" 
I1009 02:14:22.239550       1 event.go:291] "Event occurred" object="azuredisk-9267/azuredisk-volume-tester-5xb9c" kind="Pod" apiVersion="v1" type="Normal" reason="SuccessfulAttachVolume" message="AttachVolume.Attach succeeded for volume \"pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724\" "
I1009 02:14:23.493847       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:14:23.493857       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:14:23.493907       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:14:25.035380       1 namespace_controller.go:185] Namespace has been deleted azuredisk-1351
... skipping 60 lines ...
I1009 02:15:55.109915       1 pv_controller_base.go:505] deletion of claim "azuredisk-9267/pvc-k8msc" was already processed
I1009 02:16:03.196801       1 event.go:291] "Event occurred" object="azuredisk-7175/pvc-hrhkp" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="WaitForFirstConsumer" message="waiting for first consumer to be created before binding"
I1009 02:16:03.244544       1 event.go:291] "Event occurred" object="azuredisk-7175/pvc-hrhkp" kind="PersistentVolumeClaim" apiVersion="v1" type="Normal" reason="ExternalProvisioning" message="waiting for a volume to be created, either by external provisioner \"disk.csi.azure.com\" or manually created by system administrator"
I1009 02:16:03.501606       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:16:03.501640       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:16:03.501650       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
E1009 02:16:05.365156       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-9267/default: secrets "default-token-7t44f" is forbidden: unable to create new content in namespace azuredisk-9267 because it is being terminated
I1009 02:16:05.594622       1 pv_controller.go:879] volume "pvc-784759d3-2e79-41cc-b709-9167157dae0c" entered phase "Bound"
I1009 02:16:05.594678       1 pv_controller.go:982] volume "pvc-784759d3-2e79-41cc-b709-9167157dae0c" bound to claim "azuredisk-7175/pvc-hrhkp"
I1009 02:16:05.603991       1 pv_controller.go:823] claim "azuredisk-7175/pvc-hrhkp" entered phase "Bound"
I1009 02:16:06.340614       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-784759d3-2e79-41cc-b709-9167157dae0c" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c") from node "k8s-agentpool1-31388238-1" 
E1009 02:16:07.353231       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-493/default: serviceaccounts "default" not found
I1009 02:16:10.420795       1 namespace_controller.go:185] Namespace has been deleted azuredisk-9267
I1009 02:16:11.441549       1 namespace_controller.go:185] Namespace has been deleted azuredisk-5802
I1009 02:16:12.401645       1 namespace_controller.go:185] Namespace has been deleted azuredisk-493
I1009 02:16:13.502508       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:16:13.502525       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:16:13.502508       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 41 lines ...
I1009 02:17:41.897614       1 pv_controller.go:982] volume "pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" bound to claim "azuredisk-1528/pvc-azuredisk-volume-tester-hq566-0"
I1009 02:17:41.904903       1 pv_controller.go:823] claim "azuredisk-1528/pvc-azuredisk-volume-tester-hq566-0" entered phase "Bound"
I1009 02:17:42.660754       1 reconciler.go:295] attacherDetacher.AttachVolume started for volume "pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96") from node "k8s-agentpool1-31388238-1" 
I1009 02:17:43.506993       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:17:43.507306       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:17:43.507443       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
E1009 02:17:43.544587       1 tokens_controller.go:262] error synchronizing serviceaccount azuredisk-7175/default: secrets "default-token-wsbpz" is forbidden: unable to create new content in namespace azuredisk-7175 because it is being terminated
I1009 02:17:48.671091       1 namespace_controller.go:185] Namespace has been deleted azuredisk-7175
I1009 02:17:53.535170       1 operation_generator.go:368] AttachVolume.Attach succeeded for volume "pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" (UniqueName: "kubernetes.io/csi/disk.csi.azure.com^/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96") from node "k8s-agentpool1-31388238-1" 
I1009 02:17:53.535233       1 event.go:291] "Event occurred" object="azuredisk-1528/azuredisk-volume-tester-hq566-0" kind="Pod" apiVersion="v1" type="Normal" reason="SuccessfulAttachVolume" message="AttachVolume.Attach succeeded for volume \"pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96\" "
I1009 02:17:53.595328       1 route_controller.go:294] set node k8s-agentpool1-31388238-1 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:17:53.595342       1 route_controller.go:294] set node k8s-master-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
I1009 02:17:53.595346       1 route_controller.go:294] set node k8s-agentpool1-31388238-0 with NodeNetworkUnavailable=false was canceled because it is already set
... skipping 94 lines ...
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
I1009 01:58:41.268970       1 azuredisk.go:139] driver userAgent: disk.csi.azure.com/v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584 gc/go1.17.1 (amd64-linux) e2e-test
I1009 01:58:41.270630       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 01:58:41.287257       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 01:58:41.287283       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 01:58:41.287290       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 01:58:41.287312       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 01:58:41.287956       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 01:58:41.288011       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I1009 01:58:41.288049       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 99 lines ...
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
I1009 01:58:36.378685       1 azuredisk.go:139] driver userAgent: disk.csi.azure.com/v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584 gc/go1.17.1 (amd64-linux) e2e-test
I1009 01:58:36.440354       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 01:58:36.459608       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 01:58:36.459627       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 01:58:36.459633       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 01:58:36.459651       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 01:58:36.460149       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 01:58:36.460198       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I1009 01:58:36.460219       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 94 lines ...
I1009 01:59:01.792764       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume" attached to node "k8s-agentpool1-31388238-1".
I1009 01:59:01.792800       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume" to node "k8s-agentpool1-31388238-1" successfully
I1009 01:59:01.792813       1 controllerserver.go:492] found static PV(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pre-provisioned-inline-volume), insert disk properties to volumeattachments
I1009 01:59:01.792832       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0","diskiopsreadwrite":"500","diskmbpsreadwrite":"60","networkaccesspolicy":"AllowAll","skuname":"StandardSSD_LRS"}}
I1009 01:59:14.549159       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 01:59:14.549180       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce","parameters":{"Kind":"managed","csi.storage.k8s.io/pv/name":"pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce","csi.storage.k8s.io/pvc/name":"pvc-twmqp","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","skuName":"Standard_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":1}}]}
I1009 01:59:14.570570       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 01:59:14.570794       1 controllerserver.go:271] begin to create azure disk(pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 01:59:14.570831       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce StorageAccountType:Standard_LRS Size:10
I1009 01:59:16.896967       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce StorageAccountType:Standard_LRS Size:10
I1009 01:59:16.897002       1 controllerserver.go:348] create azure disk(pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce kubernetes.io-created-for-pvc-name:pvc-twmqp kubernetes.io-created-for-pvc-namespace:azuredisk-5356]) successfully
I1009 01:59:16.897026       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"Kind":"managed","csi.storage.k8s.io/pv/name":"pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce","csi.storage.k8s.io/pvc/name":"pvc-twmqp","csi.storage.k8s.io/pvc/namespace":"azuredisk-5356","kind":"Managed","requestedsizegib":"10","skuName":"Standard_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce"}}
I1009 01:59:17.594444       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 46 lines ...
I1009 02:00:11.755438       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce)
I1009 02:00:16.964170       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce
I1009 02:00:16.964196       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-7eae61aa-8a25-40f6-91d3-e3762ba266ce) returned with <nil>
I1009 02:00:16.964227       1 utils.go:102] GRPC response: {}
I1009 02:00:24.041136       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:00:24.041159       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1","parameters":{"csi.storage.k8s.io/pv/name":"pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1","csi.storage.k8s.io/pvc/name":"pvc-nqff5","csi.storage.k8s.io/pvc/namespace":"azuredisk-1957","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}}]}
I1009 02:00:24.063726       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:00:24.063877       1 controllerserver.go:271] begin to create azure disk(pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:00:24.063900       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:00:26.355704       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:00:26.355740       1 controllerserver.go:348] create azure disk(pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1 kubernetes.io-created-for-pvc-name:pvc-nqff5 kubernetes.io-created-for-pvc-namespace:azuredisk-1957]) successfully
I1009 02:00:26.355763       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1","csi.storage.k8s.io/pvc/name":"pvc-nqff5","csi.storage.k8s.io/pvc/namespace":"azuredisk-1957","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1"}}
I1009 02:00:27.157001       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 22 lines ...
I1009 02:01:17.059370       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1)
I1009 02:01:22.256661       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1
I1009 02:01:22.256687       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1) returned with <nil>
I1009 02:01:22.256702       1 utils.go:102] GRPC response: {}
I1009 02:01:27.869695       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:01:27.869719       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-d383444b-28d0-440c-95a6-dc069ff99ed4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-d383444b-28d0-440c-95a6-dc069ff99ed4","csi.storage.k8s.io/pvc/name":"pvc-wjqwm","csi.storage.k8s.io/pvc/namespace":"azuredisk-8705","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":1}}]}
I1009 02:01:27.891046       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:01:27.891270       1 controllerserver.go:271] begin to create azure disk(pvc-d383444b-28d0-440c-95a6-dc069ff99ed4) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:01:27.891291       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-d383444b-28d0-440c-95a6-dc069ff99ed4 StorageAccountType:Premium_LRS Size:10
I1009 02:01:30.355335       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-d383444b-28d0-440c-95a6-dc069ff99ed4 StorageAccountType:Premium_LRS Size:10
I1009 02:01:30.355370       1 controllerserver.go:348] create azure disk(pvc-d383444b-28d0-440c-95a6-dc069ff99ed4) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-d383444b-28d0-440c-95a6-dc069ff99ed4 kubernetes.io-created-for-pvc-name:pvc-wjqwm kubernetes.io-created-for-pvc-namespace:azuredisk-8705]) successfully
I1009 02:01:30.355393       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-d383444b-28d0-440c-95a6-dc069ff99ed4","csi.storage.k8s.io/pvc/name":"pvc-wjqwm","csi.storage.k8s.io/pvc/namespace":"azuredisk-8705","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4"}}
I1009 02:01:30.907926       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 29 lines ...
I1009 02:02:12.275512       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4)
I1009 02:02:17.449634       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4
I1009 02:02:17.449661       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4) returned with <nil>
I1009 02:02:17.449676       1 utils.go:102] GRPC response: {}
I1009 02:02:21.049999       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:02:21.050209       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823","parameters":{"csi.storage.k8s.io/pv/name":"pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823","csi.storage.k8s.io/pvc/name":"pvc-mbtsn","csi.storage.k8s.io/pvc/namespace":"azuredisk-2451","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:02:21.082327       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:02:21.082508       1 controllerserver.go:271] begin to create azure disk(pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:02:21.082531       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:02:23.364879       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:02:23.364914       1 controllerserver.go:348] create azure disk(pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823 kubernetes.io-created-for-pvc-name:pvc-mbtsn kubernetes.io-created-for-pvc-namespace:azuredisk-2451]) successfully
I1009 02:02:23.364935       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823","csi.storage.k8s.io/pvc/name":"pvc-mbtsn","csi.storage.k8s.io/pvc/namespace":"azuredisk-2451","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823"}}
I1009 02:02:24.144659       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 29 lines ...
I1009 02:03:07.433716       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823)
I1009 02:03:12.606848       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823
I1009 02:03:12.606882       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-85ac944d-4c60-4a67-a744-8a5c7da8b823) returned with <nil>
I1009 02:03:12.606898       1 utils.go:102] GRPC response: {}
I1009 02:03:17.330387       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:03:17.330509       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f","csi.storage.k8s.io/pvc/name":"pvc-jgl44","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:03:17.356090       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:03:17.356288       1 controllerserver.go:271] begin to create azure disk(pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:03:17.356350       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f StorageAccountType:Premium_LRS Size:10
I1009 02:03:19.618339       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f StorageAccountType:Premium_LRS Size:10
I1009 02:03:19.618394       1 controllerserver.go:348] create azure disk(pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f kubernetes.io-created-for-pvc-name:pvc-jgl44 kubernetes.io-created-for-pvc-namespace:azuredisk-9828]) successfully
I1009 02:03:19.618431       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f","csi.storage.k8s.io/pvc/name":"pvc-jgl44","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f"}}
I1009 02:03:20.421745       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 13 lines ...
I1009 02:03:25.974783       1 azure_controller_common.go:457] azureDisk - find disk: lun 0 name "pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f"
I1009 02:03:25.974807       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f" is already attached to node "k8s-agentpool1-31388238-1" at lun 0.
I1009 02:03:25.974817       1 controllerserver.go:492] found static PV(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f), insert disk properties to volumeattachments
I1009 02:03:25.974826       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0","diskiopsreadwrite":"120","diskmbpsreadwrite":"25","networkaccesspolicy":"AllowAll","skuname":"Premium_LRS"}}
I1009 02:03:33.526374       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:03:33.526397       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-f472379f-2e37-49fc-b498-5b97ba27132d","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f472379f-2e37-49fc-b498-5b97ba27132d","csi.storage.k8s.io/pvc/name":"pvc-gsgsc","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:03:33.550674       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:03:33.550826       1 controllerserver.go:271] begin to create azure disk(pvc-f472379f-2e37-49fc-b498-5b97ba27132d) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:03:33.550856       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f472379f-2e37-49fc-b498-5b97ba27132d StorageAccountType:Premium_LRS Size:10
I1009 02:03:35.871584       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-f472379f-2e37-49fc-b498-5b97ba27132d StorageAccountType:Premium_LRS Size:10
I1009 02:03:35.871622       1 controllerserver.go:348] create azure disk(pvc-f472379f-2e37-49fc-b498-5b97ba27132d) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-f472379f-2e37-49fc-b498-5b97ba27132d kubernetes.io-created-for-pvc-name:pvc-gsgsc kubernetes.io-created-for-pvc-namespace:azuredisk-9828]) successfully
I1009 02:03:35.871645       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f472379f-2e37-49fc-b498-5b97ba27132d","csi.storage.k8s.io/pvc/name":"pvc-gsgsc","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d"}}
I1009 02:03:36.551997       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 13 lines ...
I1009 02:03:47.055834       1 controllerserver.go:465] Attach operation successful: volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d" attached to node "k8s-agentpool1-31388238-0".
I1009 02:03:47.055867       1 controllerserver.go:485] attach volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d" to node "k8s-agentpool1-31388238-0" successfully
I1009 02:03:47.055879       1 controllerserver.go:492] found static PV(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f472379f-2e37-49fc-b498-5b97ba27132d), insert disk properties to volumeattachments
I1009 02:03:47.055893       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0","diskiopsreadwrite":"120","diskmbpsreadwrite":"25","networkaccesspolicy":"AllowAll","skuname":"Premium_LRS"}}
I1009 02:03:59.717793       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:03:59.717992       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f","parameters":{"csi.storage.k8s.io/pv/name":"pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f","csi.storage.k8s.io/pvc/name":"pvc-82pdx","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:03:59.740210       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:03:59.740288       1 controllerserver.go:271] begin to create azure disk(pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:03:59.740351       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f StorageAccountType:Premium_LRS Size:10
I1009 02:04:02.158829       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f StorageAccountType:Premium_LRS Size:10
I1009 02:04:02.158878       1 controllerserver.go:348] create azure disk(pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f kubernetes.io-created-for-pvc-name:pvc-82pdx kubernetes.io-created-for-pvc-namespace:azuredisk-9828]) successfully
I1009 02:04:02.158902       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f","csi.storage.k8s.io/pvc/name":"pvc-82pdx","csi.storage.k8s.io/pvc/namespace":"azuredisk-9828","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-4cfd53dc-9499-4a63-b084-c171c5e2203f"}}
I1009 02:04:02.737442       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 123 lines ...
I1009 02:07:18.905246       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f)
I1009 02:07:24.109159       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f
I1009 02:07:24.109183       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef6d17b6-ac74-451c-9957-0e3397a3768f) returned with <nil>
I1009 02:07:24.109200       1 utils.go:102] GRPC response: {}
I1009 02:07:25.449539       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:07:25.449566       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43","csi.storage.k8s.io/pvc/name":"pvc-mnbbl","csi.storage.k8s.io/pvc/namespace":"azuredisk-1563","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:07:25.472324       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:07:25.472401       1 controllerserver.go:271] begin to create azure disk(pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:07:25.472421       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:07:27.780697       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:07:27.780746       1 controllerserver.go:348] create azure disk(pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43 kubernetes.io-created-for-pvc-name:pvc-mnbbl kubernetes.io-created-for-pvc-namespace:azuredisk-1563]) successfully
I1009 02:07:27.780781       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43","csi.storage.k8s.io/pvc/name":"pvc-mnbbl","csi.storage.k8s.io/pvc/namespace":"azuredisk-1563","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43"}}
I1009 02:07:28.531959       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 98 lines ...
I1009 02:11:05.565011       1 azure_controller_standard.go:220] azureDisk - update(kubetest-awcxnpvo): vm(k8s-agentpool1-31388238-0) - detach disk list(map[/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourcegroups/kubetest-awcxnpvo/providers/microsoft.compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43:pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43]) returned with <nil>
I1009 02:11:05.565047       1 azure_controller_common.go:369] azureDisk - detach disk(pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43, /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43) succeeded
I1009 02:11:05.565061       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ef7e9f0f-4b14-4265-9c0c-9139c3e15f43 from node k8s-agentpool1-31388238-0 successfully
I1009 02:11:05.565073       1 utils.go:102] GRPC response: {}
I1009 02:11:09.559925       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:11:09.559947       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-f22c3fce-e51e-472e-95d9-aebdb3437315","parameters":{"csi.storage.k8s.io/pv/name":"pvc-f22c3fce-e51e-472e-95d9-aebdb3437315","csi.storage.k8s.io/pvc/name":"pvc-wh2sd","csi.storage.k8s.io/pvc/namespace":"azuredisk-1577"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:11:09.670507       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f22c3fce-e51e-472e-95d9-aebdb3437315, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-f22c3fce-e51e-472e-95d9-aebdb3437315' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:11:09.670578       1 controllerserver.go:271] begin to create azure disk(pvc-f22c3fce-e51e-472e-95d9-aebdb3437315) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:11:09.670594       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-f22c3fce-e51e-472e-95d9-aebdb3437315 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:11.997620       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-f22c3fce-e51e-472e-95d9-aebdb3437315 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:11.997667       1 controllerserver.go:348] create azure disk(pvc-f22c3fce-e51e-472e-95d9-aebdb3437315) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-f22c3fce-e51e-472e-95d9-aebdb3437315 kubernetes.io-created-for-pvc-name:pvc-wh2sd kubernetes.io-created-for-pvc-namespace:azuredisk-1577]) successfully
I1009 02:11:11.997691       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-f22c3fce-e51e-472e-95d9-aebdb3437315","csi.storage.k8s.io/pvc/name":"pvc-wh2sd","csi.storage.k8s.io/pvc/namespace":"azuredisk-1577","requestedsizegib":"10"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-f22c3fce-e51e-472e-95d9-aebdb3437315"}}
I1009 02:11:13.798158       1 utils.go:95] GRPC call: /csi.v1.Controller/DeleteVolume
... skipping 5 lines ...
I1009 02:11:28.141354       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:11:28.141376       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031","parameters":{"csi.storage.k8s.io/pv/name":"pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031","csi.storage.k8s.io/pvc/name":"pvc-fql7h","csi.storage.k8s.io/pvc/namespace":"azuredisk-552","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:11:28.151078       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:11:28.151096       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23","parameters":{"csi.storage.k8s.io/pv/name":"pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23","csi.storage.k8s.io/pvc/name":"pvc-qrmwr","csi.storage.k8s.io/pvc/namespace":"azuredisk-552","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:11:28.159032       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:11:28.159058       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae","parameters":{"csi.storage.k8s.io/pv/name":"pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae","csi.storage.k8s.io/pvc/name":"pvc-xvrp5","csi.storage.k8s.io/pvc/namespace":"azuredisk-552","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:11:28.165377       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:11:28.165587       1 controllerserver.go:271] begin to create azure disk(pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:11:28.165612       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:28.172825       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:11:28.173064       1 controllerserver.go:271] begin to create azure disk(pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:11:28.173169       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:28.246466       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:11:28.246517       1 controllerserver.go:271] begin to create azure disk(pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:11:28.246815       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-82c7e796-52b0-4f66-98cc-423ddafea4ae StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:30.516497       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:11:30.516538       1 controllerserver.go:348] create azure disk(pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031 kubernetes.io-created-for-pvc-name:pvc-fql7h kubernetes.io-created-for-pvc-namespace:azuredisk-552]) successfully
I1009 02:11:30.516601       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031","csi.storage.k8s.io/pvc/name":"pvc-fql7h","csi.storage.k8s.io/pvc/namespace":"azuredisk-552","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031"}}
I1009 02:11:30.629979       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-c0c276de-0d64-4302-a3dd-4b21b5c04e23 StorageAccountType:StandardSSD_LRS Size:10
... skipping 114 lines ...
I1009 02:12:38.853321       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-a6b4c7d2-3526-40fc-b0f9-c44807686031) returned with <nil>
I1009 02:12:38.853354       1 utils.go:102] GRPC response: {}
I1009 02:12:45.073816       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:12:45.073836       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6","csi.storage.k8s.io/pvc/name":"pvc-6cqzb","csi.storage.k8s.io/pvc/namespace":"azuredisk-1351","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:12:45.090031       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:12:45.090051       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41","parameters":{"csi.storage.k8s.io/pv/name":"pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41","csi.storage.k8s.io/pvc/name":"pvc-9tjn8","csi.storage.k8s.io/pvc/namespace":"azuredisk-1351","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Block":{}},"access_mode":{"mode":1}}]}
I1009 02:12:45.095259       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:12:45.095345       1 controllerserver.go:271] begin to create azure disk(pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:12:45.095423       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6 StorageAccountType:Premium_LRS Size:10
I1009 02:12:45.110940       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:12:45.111064       1 controllerserver.go:271] begin to create azure disk(pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:12:45.111085       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41 StorageAccountType:Premium_LRS Size:10
I1009 02:12:47.371441       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6 StorageAccountType:Premium_LRS Size:10
I1009 02:12:47.371475       1 controllerserver.go:348] create azure disk(pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6 kubernetes.io-created-for-pvc-name:pvc-6cqzb kubernetes.io-created-for-pvc-namespace:azuredisk-1351]) successfully
I1009 02:12:47.371499       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6","csi.storage.k8s.io/pvc/name":"pvc-6cqzb","csi.storage.k8s.io/pvc/namespace":"azuredisk-1351","requestedsizegib":"10","skuName":"Premium_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-fae1468c-6efd-4474-ba7d-82b36c1284d6"}}
I1009 02:12:47.387778       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41 StorageAccountType:Premium_LRS Size:10
... skipping 71 lines ...
I1009 02:14:11.640713       1 controllerserver.go:533] detach volume /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-e1d93b1f-8cd6-465b-9009-9b056b78ab41 from node k8s-agentpool1-31388238-1 successfully
I1009 02:14:11.640724       1 utils.go:102] GRPC response: {}
I1009 02:14:18.009199       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:14:18.009221       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4","parameters":{"csi.storage.k8s.io/pv/name":"pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4","csi.storage.k8s.io/pvc/name":"pvc-k8msc","csi.storage.k8s.io/pvc/namespace":"azuredisk-9267","skuName":"Standard_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:14:18.022692       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:14:18.022839       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01","parameters":{"csi.storage.k8s.io/pv/name":"pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01","csi.storage.k8s.io/pvc/name":"pvc-szl5d","csi.storage.k8s.io/pvc/namespace":"azuredisk-9267","skuName":"Premium_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:14:18.030251       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:14:18.030473       1 controllerserver.go:271] begin to create azure disk(pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:14:18.030568       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4 StorageAccountType:Standard_LRS Size:10
I1009 02:14:18.034317       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:14:18.034486       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724","parameters":{"csi.storage.k8s.io/pv/name":"pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724","csi.storage.k8s.io/pvc/name":"pvc-xnqls","csi.storage.k8s.io/pvc/namespace":"azuredisk-9267","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:14:18.053358       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:14:18.053589       1 controllerserver.go:271] begin to create azure disk(pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01) account type(Premium_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:14:18.053683       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-ebb7ffd1-9d12-4e8b-be7a-a1547bc7fd01 StorageAccountType:Premium_LRS Size:10
I1009 02:14:18.140343       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:14:18.140655       1 controllerserver.go:271] begin to create azure disk(pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:14:18.140760       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:14:20.301559       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4 StorageAccountType:Standard_LRS Size:10
I1009 02:14:20.301591       1 controllerserver.go:348] create azure disk(pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4 kubernetes.io-created-for-pvc-name:pvc-k8msc kubernetes.io-created-for-pvc-namespace:azuredisk-9267]) successfully
I1009 02:14:20.301611       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4","csi.storage.k8s.io/pvc/name":"pvc-k8msc","csi.storage.k8s.io/pvc/namespace":"azuredisk-9267","requestedsizegib":"10","skuName":"Standard_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4"}}
I1009 02:14:20.440108       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-fddc8c6b-552c-4336-a8e9-4f283fbac724 StorageAccountType:StandardSSD_LRS Size:10
... skipping 95 lines ...
I1009 02:15:49.928189       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4)
I1009 02:15:55.088087       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4
I1009 02:15:55.088122       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-eea676bf-1fa9-4d63-8d9a-3e263f6e86c4) returned with <nil>
I1009 02:15:55.088137       1 utils.go:102] GRPC response: {}
I1009 02:16:03.245490       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:16:03.245593       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-784759d3-2e79-41cc-b709-9167157dae0c","parameters":{"csi.storage.k8s.io/pv/name":"pvc-784759d3-2e79-41cc-b709-9167157dae0c","csi.storage.k8s.io/pvc/name":"pvc-hrhkp","csi.storage.k8s.io/pvc/namespace":"azuredisk-7175","skuName":"Standard_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{"mount_flags":["barrier=1","acl"]}},"access_mode":{"mode":1}}]}
I1009 02:16:03.266980       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:16:03.267115       1 controllerserver.go:271] begin to create azure disk(pvc-784759d3-2e79-41cc-b709-9167157dae0c) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:16:03.267173       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-784759d3-2e79-41cc-b709-9167157dae0c StorageAccountType:Standard_LRS Size:10
I1009 02:16:05.585681       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-784759d3-2e79-41cc-b709-9167157dae0c StorageAccountType:Standard_LRS Size:10
I1009 02:16:05.585714       1 controllerserver.go:348] create azure disk(pvc-784759d3-2e79-41cc-b709-9167157dae0c) account type(Standard_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-784759d3-2e79-41cc-b709-9167157dae0c kubernetes.io-created-for-pvc-name:pvc-hrhkp kubernetes.io-created-for-pvc-namespace:azuredisk-7175]) successfully
I1009 02:16:05.585735       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-784759d3-2e79-41cc-b709-9167157dae0c","csi.storage.k8s.io/pvc/name":"pvc-hrhkp","csi.storage.k8s.io/pvc/namespace":"azuredisk-7175","requestedsizegib":"10","skuName":"Standard_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c"}}
I1009 02:16:06.354402       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 29 lines ...
I1009 02:17:28.255242       1 controllerserver.go:389] deleting azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c)
I1009 02:17:33.423625       1 azure_managedDiskController.go:286] azureDisk - deleted a managed disk: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c
I1009 02:17:33.423652       1 controllerserver.go:391] delete azure disk(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-784759d3-2e79-41cc-b709-9167157dae0c) returned with <nil>
I1009 02:17:33.423668       1 utils.go:102] GRPC response: {}
I1009 02:17:39.572756       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:17:39.572783       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96","parameters":{"csi.storage.k8s.io/pv/name":"pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-hq566-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1528","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:17:39.594581       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:17:39.594752       1 controllerserver.go:271] begin to create azure disk(pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:17:39.594775       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:17:41.887344       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:17:41.887378       1 controllerserver.go:348] create azure disk(pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96 kubernetes.io-created-for-pvc-name:pvc-azuredisk-volume-tester-hq566-0 kubernetes.io-created-for-pvc-namespace:azuredisk-1528]) successfully
I1009 02:17:41.887401       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96","csi.storage.k8s.io/pvc/name":"pvc-azuredisk-volume-tester-hq566-0","csi.storage.k8s.io/pvc/namespace":"azuredisk-1528","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96"}}
I1009 02:17:42.673936       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 20 lines ...
I1009 02:18:37.077139       1 azure_controller_common.go:457] azureDisk - find disk: lun 0 name "pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96"
I1009 02:18:37.077149       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" is already attached to node "k8s-agentpool1-31388238-1" at lun 0.
I1009 02:18:37.077161       1 controllerserver.go:492] found static PV(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96), insert disk properties to volumeattachments
I1009 02:18:37.077181       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0","diskiopsreadwrite":"500","diskmbpsreadwrite":"60","networkaccesspolicy":"AllowAll","skuname":"StandardSSD_LRS"}}
I1009 02:19:17.055347       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:19:17.055369       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-98e5441d-899c-4f56-b365-06e182fd67a6","parameters":{"csi.storage.k8s.io/pv/name":"pvc-98e5441d-899c-4f56-b365-06e182fd67a6","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:19:17.079305       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-98e5441d-899c-4f56-b365-06e182fd67a6, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-98e5441d-899c-4f56-b365-06e182fd67a6' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:19:17.079397       1 controllerserver.go:271] begin to create azure disk(pvc-98e5441d-899c-4f56-b365-06e182fd67a6) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:19:17.079411       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-98e5441d-899c-4f56-b365-06e182fd67a6 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:19:19.497843       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-98e5441d-899c-4f56-b365-06e182fd67a6 StorageAccountType:StandardSSD_LRS Size:10
I1009 02:19:19.497878       1 controllerserver.go:348] create azure disk(pvc-98e5441d-899c-4f56-b365-06e182fd67a6) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-98e5441d-899c-4f56-b365-06e182fd67a6 kubernetes.io-created-for-pvc-name:pvc-azuredisk kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 02:19:19.497911       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-98e5441d-899c-4f56-b365-06e182fd67a6","csi.storage.k8s.io/pvc/name":"pvc-azuredisk","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-98e5441d-899c-4f56-b365-06e182fd67a6"}}
I1009 02:19:20.398903       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 12 lines ...
I1009 02:19:37.210347       1 azure_controller_common.go:457] azureDisk - find disk: lun 0 name "pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" uri "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96"
I1009 02:19:37.210371       1 controllerserver.go:455] Attach operation is successful. volume "/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96" is already attached to node "k8s-agentpool1-31388238-1" at lun 0.
I1009 02:19:37.210384       1 controllerserver.go:492] found static PV(/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-9a63d837-46c5-4be4-aa93-8f98c7f05c96), insert disk properties to volumeattachments
I1009 02:19:37.210395       1 utils.go:102] GRPC response: {"publish_context":{"LUN":"0","diskiopsreadwrite":"500","diskmbpsreadwrite":"60","networkaccesspolicy":"AllowAll","skuname":"StandardSSD_LRS"}}
I1009 02:19:43.042340       1 utils.go:95] GRPC call: /csi.v1.Controller/CreateVolume
I1009 02:19:43.042360       1 utils.go:96] GRPC request: {"accessibility_requirements":{"preferred":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"requisite":[{"segments":{"topology.disk.csi.azure.com/zone":""}}]},"capacity_range":{"required_bytes":10737418240},"name":"pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a","parameters":{"csi.storage.k8s.io/pv/name":"pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","skuName":"StandardSSD_LRS"},"volume_capabilities":[{"AccessType":{"Mount":{}},"access_mode":{"mode":1}}]}
I1009 02:19:43.066078       1 azure_diskclient.go:135] Received error in disk.get.request: resourceID: /subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a, error: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 404, RawError: {"error":{"code":"ResourceNotFound","message":"The Resource 'Microsoft.Compute/disks/pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a' under resource group 'kubetest-awcxnpvo' was not found. For more details please go to https://aka.ms/ARMResourceNotFoundFix"}}
I1009 02:19:43.066195       1 controllerserver.go:271] begin to create azure disk(pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) diskZone() maxShares(0)
I1009 02:19:43.066216       1 azure_managedDiskController.go:88] azureDisk - creating new managed Name:pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a StorageAccountType:StandardSSD_LRS Size:10
I1009 02:19:45.443210       1 azure_managedDiskController.go:249] azureDisk - created new MD Name:pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a StorageAccountType:StandardSSD_LRS Size:10
I1009 02:19:45.443247       1 controllerserver.go:348] create azure disk(pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a) account type(StandardSSD_LRS) rg(kubetest-awcxnpvo) location() size(10) tags(map[kubernetes.io-created-for-pv-name:pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a kubernetes.io-created-for-pvc-name:persistent-storage-statefulset-azuredisk-0 kubernetes.io-created-for-pvc-namespace:default]) successfully
I1009 02:19:45.443459       1 utils.go:102] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.disk.csi.azure.com/zone":""}}],"capacity_bytes":10737418240,"content_source":{"Type":null},"volume_context":{"csi.storage.k8s.io/pv/name":"pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a","csi.storage.k8s.io/pvc/name":"persistent-storage-statefulset-azuredisk-0","csi.storage.k8s.io/pvc/namespace":"default","requestedsizegib":"10","skuName":"StandardSSD_LRS"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-734fa9fa-c5bb-4622-b63d-af2b9bc90f9a"}}
I1009 02:19:46.468456       1 utils.go:95] GRPC call: /csi.v1.Controller/ControllerPublishVolume
... skipping 38 lines ...
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
I1009 01:58:32.293495       1 azuredisk.go:139] driver userAgent: disk.csi.azure.com/v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584 gc/go1.17.1 (amd64-linux) e2e-test
I1009 01:58:32.294424       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 01:58:32.347618       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 01:58:32.347636       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 01:58:32.347642       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 01:58:32.347657       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 01:58:32.348094       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 01:58:32.348129       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I1009 01:58:32.348153       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 137 lines ...
I1009 02:00:44.627087       1 mount_linux.go:446] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0])
I1009 02:00:44.635790       1 mount_linux.go:449] Output: ""
I1009 02:00:44.635809       1 mount_linux.go:408] Disk "/dev/disk/azure/scsi1/lun0" appears to be unformatted, attempting to format as type: "ext4" with options: [-F -m0 /dev/disk/azure/scsi1/lun0]
I1009 02:00:45.581419       1 mount_linux.go:418] Disk successfully formatted (mkfs): ext4 - /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
I1009 02:00:45.581444       1 mount_linux.go:436] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
I1009 02:00:45.581498       1 mount_linux.go:175] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount)
E1009 02:00:45.593728       1 mount_linux.go:179] Mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error.

E1009 02:00:45.594014       1 utils.go:100] GRPC error: rpc error: code = Internal desc = could not format "/dev/disk/azure/scsi1/lun0"(lun: "0"), and mount it at "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount"
I1009 02:00:46.158801       1 utils.go:95] GRPC call: /csi.v1.Node/NodeStageVolume
I1009 02:00:46.158824       1 utils.go:96] GRPC request: {"publish_context":{"LUN":"0","diskiopsreadwrite":"500","diskmbpsreadwrite":"60","networkaccesspolicy":"AllowAll","skuname":"StandardSSD_LRS"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"cachingMode":"ReadWrite","fsType":"","kind":"Managed"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1"}
I1009 02:00:47.897456       1 azure_common_linux.go:177] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/
I1009 02:00:47.897483       1 nodeserver.go:112] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType 
I1009 02:00:47.897505       1 nodeserver.go:153] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount with mount options([invalid mount options])
I1009 02:00:47.897519       1 mount_linux.go:446] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0])
I1009 02:00:47.904102       1 mount_linux.go:449] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n"
I1009 02:00:47.904247       1 mount_linux.go:340] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0
I1009 02:00:47.926235       1 mount_linux.go:436] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
I1009 02:00:47.926260       1 mount_linux.go:175] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount)
E1009 02:00:47.935366       1 mount_linux.go:179] Mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error.

E1009 02:00:47.935401       1 utils.go:100] GRPC error: rpc error: code = Internal desc = could not format "/dev/disk/azure/scsi1/lun0"(lun: "0"), and mount it at "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount"
I1009 02:00:49.013527       1 utils.go:95] GRPC call: /csi.v1.Node/NodeStageVolume
I1009 02:00:49.013548       1 utils.go:96] GRPC request: {"publish_context":{"LUN":"0","diskiopsreadwrite":"500","diskmbpsreadwrite":"60","networkaccesspolicy":"AllowAll","skuname":"StandardSSD_LRS"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount","volume_capability":{"AccessType":{"Mount":{"mount_flags":["invalid","mount","options"]}},"access_mode":{"mode":1}},"volume_context":{"cachingMode":"ReadWrite","fsType":"","kind":"Managed"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1"}
I1009 02:00:50.734052       1 azure_common_linux.go:177] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/
I1009 02:00:50.734083       1 nodeserver.go:112] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType 
I1009 02:00:50.734107       1 nodeserver.go:153] NodeStageVolume: formatting /dev/disk/azure/scsi1/lun0 and mounting at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount with mount options([invalid mount options])
I1009 02:00:50.734133       1 mount_linux.go:446] Attempting to determine if disk "/dev/disk/azure/scsi1/lun0" is formatted using blkid with args: ([-p -s TYPE -s PTTYPE -o export /dev/disk/azure/scsi1/lun0])
I1009 02:00:50.740756       1 mount_linux.go:449] Output: "DEVNAME=/dev/disk/azure/scsi1/lun0\nTYPE=ext4\n"
I1009 02:00:50.740773       1 mount_linux.go:340] Checking for issues with fsck on disk: /dev/disk/azure/scsi1/lun0
I1009 02:00:50.752760       1 mount_linux.go:436] Attempting to mount disk /dev/disk/azure/scsi1/lun0 in ext4 format at /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
I1009 02:00:50.752803       1 mount_linux.go:175] Mounting cmd (mount) with arguments (-t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount)
E1009 02:00:50.761058       1 mount_linux.go:179] Mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t ext4 -o invalid,mount,options,defaults /dev/disk/azure/scsi1/lun0 /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount
Output: mount: /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount: wrong fs type, bad option, bad superblock on /dev/sdc, missing codepage or helper program, or other error.

E1009 02:00:50.761085       1 utils.go:100] GRPC error: rpc error: code = Internal desc = could not format "/dev/disk/azure/scsi1/lun0"(lun: "0"), and mount it at "/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-77443d47-eb74-40cb-b27f-ef9c87d98bc1/globalmount"
I1009 02:01:46.738268       1 utils.go:95] GRPC call: /csi.v1.Node/NodeStageVolume
I1009 02:01:46.738293       1 utils.go:96] GRPC request: {"publish_context":{"LUN":"0","diskiopsreadwrite":"120","diskmbpsreadwrite":"25","networkaccesspolicy":"AllowAll","skuname":"Premium_LRS"},"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/volumeDevices/staging/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4","volume_capability":{"AccessType":{"Block":{}},"access_mode":{"mode":1}},"volume_context":{"cachingMode":"ReadWrite","fsType":"","kind":"Managed"},"volume_id":"/subscriptions/0e46bd28-a80f-4d3a-8200-d9eb8d80cb2e/resourceGroups/kubetest-awcxnpvo/providers/Microsoft.Compute/disks/pvc-d383444b-28d0-440c-95a6-dc069ff99ed4"}
I1009 02:01:48.420091       1 azure_common_linux.go:177] azureDisk - found /dev/disk/azure/scsi1/lun0 by sdc under /dev/disk/azure/scsi1/
I1009 02:01:48.420122       1 nodeserver.go:112] NodeStageVolume: perf optimization is disabled for /dev/disk/azure/scsi1/lun0. perfProfile none accountType 
I1009 02:01:48.420134       1 utils.go:102] GRPC response: {}
I1009 02:01:48.427526       1 utils.go:95] GRPC call: /csi.v1.Node/NodePublishVolume
... skipping 586 lines ...
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
I1009 01:58:35.970146       1 azuredisk.go:139] driver userAgent: disk.csi.azure.com/v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584 gc/go1.17.1 (amd64-linux) e2e-test
I1009 01:58:35.971402       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 01:58:35.996366       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 01:58:35.996392       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 01:58:35.996401       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 01:58:35.996429       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 01:58:35.997861       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 01:58:35.997905       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I1009 01:58:35.997930       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 162 lines ...
Platform: linux/amd64
Topology Key: topology.disk.csi.azure.com/zone

Streaming logs below:
I1009 01:58:30.836931       1 azuredisk.go:139] driver userAgent: disk.csi.azure.com/v1.8.0-885cfdb204b3a00a8e44c86e4e4bcc91fe2bf584 gc/go1.17.1 (amd64-linux) e2e-test
I1009 01:58:30.837745       1 azure_disk_utils.go:137] reading cloud config from secret kube-system/azure-cloud-provider
W1009 01:58:30.857702       1 azure_disk_utils.go:144] InitializeCloudFromSecret: failed to get cloud config from secret kube-system/azure-cloud-provider: failed to get secret kube-system/azure-cloud-provider: secrets "azure-cloud-provider" not found
I1009 01:58:30.857722       1 azure_disk_utils.go:149] could not read cloud config from secret kube-system/azure-cloud-provider
I1009 01:58:30.857727       1 azure_disk_utils.go:159] use default AZURE_CREDENTIAL_FILE env var: /etc/kubernetes/azure.json
I1009 01:58:30.914731       1 azure_disk_utils.go:167] read cloud config from file: /etc/kubernetes/azure.json successfully
I1009 01:58:30.915573       1 azure_auth.go:234] Using AzurePublicCloud environment
I1009 01:58:30.915643       1 azure_auth.go:119] azure: using client_id+client_secret to retrieve access token
I1009 01:58:30.915669       1 azure.go:685] Azure cloudprovider using try backoff: retries=6, exponent=1.500000, duration=5, jitter=1.000000
... skipping 397 lines ...
make[1]: Leaving directory '/home/prow/go/src/sigs.k8s.io/azuredisk-csi-driver'
2021/10/09 02:20:57 Azure Disk CSI Driver uninstalled

JUnit report was created: /logs/artifacts/junit_01.xml

Ran 13 of 53 Specs in 1369.681 seconds
SUCCESS! -- 13 Passed | 0 Failed | 0 Pending | 40 Skipped
You're using deprecated Ginkgo functionality:
=============================================
Ginkgo 2.0 is under active development and will introduce (a small number of) breaking changes.
To learn more, view the migration guide at https://github.com/onsi/ginkgo/blob/v2/docs/MIGRATING_TO_V2.md
To comment, chime in at https://github.com/onsi/ginkgo/issues/711

... skipping 66 lines ...