Lambda deployment intermittently fails due to AWS API throttling (RateExceeded)

[anashestoperova]

Description

We are experiencing intermittent failures when deploying AWS Lambda functions via the AWS API.
The failures occur at random times during deployment and result in throttling-related errors such as:

Rate limit exceeded
Maximum retries reached: Rate exceeded

These errors cause deployments to fail even though no functional changes were introduced.

---

Background / Investigation

We investigated the issue and confirmed that the relevant AWS service limits cannot be increased.

The throttling is coming from AWS APIs used during Lambda deployment.

Because AWS limits are fixed, the solution must be implemented on our side by handling throttling more gracefully.

The default retry behavior is not sufficient under current deployment patterns.

---

Root Cause

AWS API requests during Lambda deployment occasionally exceed service rate limits.
When this happens, the SDK exhausts its default retry attempts, causing the deployment to fail.

---

Proposed Solution

Enable adaptive retry behavior and increase the number of retry attempts using a custom retry strategy.
This allows the deployment process to better tolerate temporary throttling and recover automatically.

Reference:
https://github.com/aws/aws-sdk-js-v3/tree/v3.370.0/packages/util-retry#readme

Proposed configuration:

import { ConfiguredRetryStrategy } from "@aws-sdk/util-retry";

retryStrategy: new ConfiguredRetryStrategy(
  5, // max retry attempts
  (attempt) => Math.max(1000, 1000) // fixed backoff (can be tuned)
);

This should significantly reduce RateExceeded and throttling-related failures during deployment.