Velero partially fails to take snapshot 429 rate limit

[MattFanto]

I'm using Velero to backup my digital ocean Kubernetes cluster for 2 namespaces.
One namespace has only one PVC and I can successfully backup and restore it, the second namespace has 2 pvc and for it I face the following issue:

time="2021-05-03T17:03:30Z" level=error msg="Storage.CreateSnapshot returned error: POST https://api.digitalocean.com/v2/volumes/57cea4da-7eb1-11eb-9cd5-0a58ac14d0a4/snapshots: 429 (request \"af8ed563-d8dd-40fb-a5df-4aa294608722\") failed to create snapshot: this operation is rate-limited - one request per 10m0s" backup=velero/b2waste-cms-prod3 cmd=/plugins/velero-digitalocean logSource="/go/src/github.com/digitalocean/velero-plugin/velero-digitalocean/volume_snapshotter.go:139" pluginName=velero-digitalocean
time="2021-05-03T17:03:30Z" level=info msg="1 errors encountered backup up item" backup=velero/b2waste-cms-prod3 logSource="pkg/backup/backup.go:427" name=b2waste-wordpress-cms-mariadb-0
time="2021-05-03T17:03:30Z" level=error msg="Error backing up item" backup=velero/b2waste-cms-prod3 error="error taking snapshot of volume: rpc error: code = Aborted desc = plugin panicked: runtime error: invalid memory address or nil pointer dereference" logSource="pkg/backup/backup.go:431" name=b2waste-wordpress-cms-mariadb-0

I read online about the API limit for PVC snapshot https://docs.digitalocean.com/products/images/snapshots/ but I'm wondering why I'm hitting this limit at all since these are 2 different PVC / volumes

PVC yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    meta.helm.sh/release-name: b2waste-wordpress-cms
    meta.helm.sh/release-namespace: default
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: dobs.csi.digitalocean.com
  creationTimestamp: "2021-03-06T19:22:48Z"
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app.kubernetes.io/instance: b2waste-wordpress-cms
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: wordpress
    helm.sh/chart: wordpress-10.6.10
  name: b2waste-wordpress-cms
  namespace: default
  resourceVersion: "2985"
  uid: 7a101758-e9e6-4e29-89a0-b6c5b254f463
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: do-block-storage
  volumeMode: Filesystem
  volumeName: pvc-7a101758-e9e6-4e29-89a0-b6c5b254f463
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 10Gi
  phase: Bound

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    pv.kubernetes.io/bind-completed: "yes"
    pv.kubernetes.io/bound-by-controller: "yes"
    volume.beta.kubernetes.io/storage-provisioner: dobs.csi.digitalocean.com
  creationTimestamp: "2021-03-06T19:22:48Z"
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app.kubernetes.io/component: primary
    app.kubernetes.io/instance: b2waste-wordpress-cms
    app.kubernetes.io/name: mariadb
  name: data-b2waste-wordpress-cms-mariadb-0
  namespace: default
  resourceVersion: "2990"
  uid: 25e42613-21a3-4878-b178-2613ebe4b165
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 8Gi
  storageClassName: do-block-storage
  volumeMode: Filesystem
  volumeName: pvc-25e42613-21a3-4878-b178-2613ebe4b165
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 8Gi
  phase: Bound

[robert-uhl]

I saw this happen as well, on a daily backup which succeeded the day before and the day after, with no cluster configuration changes either before or after.