Spring Boot SQS Retry Strategy — Exponential Backoff & DLQ

Retrying failed messages in an SQS-based Spring Boot application sounds simple — throw an exception and SQS will retry. But naive retry strategies cause real problems in production: thundering herds when a downstream service recovers, duplicate processing when a consumer takes too long, and cascading failures when retry storms overwhelm a recovering dependency. This guide covers a production-grade retry strategy with exponential backoff, jitter, visibility timeout extension, and correct DLQ integration.

It assumes you already have a dead-letter queue configured, because the retry strategy and the DLQ are two halves of the same decision.

Why naive SQS retry is not enough

SQS's built-in retry mechanism is simple: if a consumer throws an exception (or doesn't delete the message within the visibility timeout), SQS makes the message available again after the timeout expires. So the delay between attempts is not missing — it is the visibility timeout, and it is fixed. That is the problem. With a 30-second timeout and a maxReceiveCount of 5, all five attempts are spent inside two and a half minutes, and the message lands in the DLQ long before a downstream service that is restarting has finished restarting.

┌─────────────────────────────────────────────────────────────┐
│           Naive retry vs. exponential backoff               │
├───────────────────────┬─────────────────────────────────────┤
│ Attempt               │ Naive (30s timeout) │ Exponential   │
├───────────────────────┼─────────────────────┼───────────────┤
│ 1st retry             │ 30 seconds          │ 30 seconds    │
│ 2nd retry             │ 30 seconds          │ 60 seconds    │
│ 3rd retry             │ 30 seconds          │ 120 seconds   │
│ 4th retry             │ 30 seconds          │ 240 seconds   │
│ 5th retry → DLQ       │ 30 seconds          │ 480 seconds   │
├───────────────────────┼─────────────────────┼───────────────┤
│ Total time before DLQ │ ~2.5 minutes        │ ~15.5 minutes │
│ Downstream requests   │ 5 attempts          │ 5 attempts    │
└───────────────────────┴─────────────────────┴───────────────┘

The fix is to extend the message's visibility timeout exponentially on each failed attempt, effectively implementing backoff at the SQS layer rather than sleeping in your consumer thread.

Tracking retry attempts — the receiveCount attribute

SQS automatically tracks how many times a message has been received via the ApproximateReceiveCount system attribute. Spring Cloud AWS exposes this as a message header you can inject directly into your listener:

@SqsListener(value = "${app.queues.orders}")
public void handleOrder(
        @Payload OrderEvent event,
        @Header(SqsHeaders.MessageSystemAttributes.SQS_APPROXIMATE_RECEIVE_COUNT) int receiveCount,
        @Header(SqsHeaders.SQS_RECEIPT_HANDLE_HEADER) String receiptHandle) {

    log.info("Processing order {} — attempt #{}", event.getOrderId(), receiveCount);
    // use receiveCount to compute backoff delay
}

Implementing exponential backoff via visibility timeout

The approach: catch the exception in your listener, compute a backoff delay based on receiveCount, extend the message's visibility timeout to that delay using the SQS SDK, then throw the exception so Spring Cloud AWS does not delete the message. SQS will make it available again after the extended timeout expires.

@Service
@Slf4j
public class OrderMessageConsumer {

    private static final int MAX_BACKOFF_SECONDS = 900; // 15 minutes cap
    private static final int BASE_DELAY_SECONDS  = 30;

    private static final int SQS_MAX_VISIBILITY  = 43_200; // 12 hours — AWS hard ceiling

    private final OrderService orderService;
    private final SqsClient    sqsClient;
    private final String       queueUrl;

    // Constructor injection rather than @Value on a field: it is what lets this
    // class be built in a unit test without starting a Spring context.
    public OrderMessageConsumer(OrderService orderService,
                                SqsClient sqsClient,
                                @Value("${app.queues.orders-url}") String queueUrl) {
        this.orderService = orderService;
        this.sqsClient = sqsClient;
        this.queueUrl = queueUrl;
    }

    @SqsListener("${app.queues.orders}")
    public void handleOrder(
            @Payload OrderEvent event,
            @Header(SqsHeaders.MessageSystemAttributes.SQS_APPROXIMATE_RECEIVE_COUNT)
                    int receiveCount,
            @Header(SqsHeaders.MessageSystemAttributes.SQS_APPROXIMATE_FIRST_RECEIVE_TIMESTAMP)
                    String firstReceivedAt,
            @Header(SqsHeaders.SQS_RECEIPT_HANDLE_HEADER)
                    String receiptHandle) {

        try {
            orderService.process(event);

        } catch (HttpServerErrorException | ResourceAccessException ex) {
            // Transient — hide the message for a backoff window, then let SQS redeliver.
            int backoff = computeBackoff(receiveCount);
            log.warn("Transient failure on attempt {}; hiding order {} for {}s",
                    receiveCount, event.getOrderId(), backoff);
            extendVisibility(receiptHandle, backoff, firstReceivedAt);
            throw ex; // re-throw so Spring Cloud AWS doesn't ack (delete) the message

        } catch (IllegalArgumentException | ConstraintViolationException ex) {
            // Permanent — no backoff. maxReceiveCount is what routes it to the DLQ.
            log.error("Permanent failure for order {}; will reach the DLQ",
                    event.getOrderId(), ex);
            throw ex;

        } catch (RuntimeException ex) {
            // Unclassified — treat as transient, conservatively.
            int backoff = computeBackoff(receiveCount);
            log.warn("Unclassified failure on attempt {}; hiding order {} for {}s",
                    receiveCount, event.getOrderId(), backoff);
            extendVisibility(receiptHandle, backoff, firstReceivedAt);
            throw ex;
        }
    }

    /**
     * Exponential backoff with full jitter, floored at the base delay.
     *
     * AWS's "full jitter" is random(0, cap). That is correct for a client-side sleep
     * and wrong for a visibility timeout: a value of 0 makes the message visible
     * immediately, which is the thundering herd this exists to prevent. Without the
     * floor, roughly 3% of first retries return 0 — and every one of them returns
     * less than the 30-second base delay, because random(0, 30) can never reach 30.
     */
    private int computeBackoff(int attempt) {
        int exponential = (int) Math.min(MAX_BACKOFF_SECONDS,
                BASE_DELAY_SECONDS * Math.pow(2, attempt - 1));
        int jittered = ThreadLocalRandom.current().nextInt(exponential + 1);
        return Math.max(BASE_DELAY_SECONDS, jittered);
    }

    private void extendVisibility(String receiptHandle, int seconds, String firstReceivedAt) {
        int remaining = remainingVisibilitySeconds(firstReceivedAt);
        if (remaining <= 0) {
            log.warn("12-hour visibility budget exhausted; letting the message redeliver");
            return;
        }
        try {
            sqsClient.changeMessageVisibility(req -> req
                .queueUrl(queueUrl)
                .receiptHandle(receiptHandle)
                .visibilityTimeout(Math.min(seconds, remaining))
            );
        } catch (SqsException e) {
            String code = e.awsErrorDetails() == null ? "" : e.awsErrorDetails().errorCode();
            if ("AWS.SimpleQueueService.MessageNotInflight".equals(code)) {
                log.warn("Message no longer in flight; skipping visibility extension");
            } else {
                log.error("Could not extend visibility (errorCode={})", code, e);
            }
        }
    }

    /** How much of the 12-hour ceiling is left, measured from the FIRST receive. */
    static int remainingVisibilitySeconds(String firstReceivedAt) {
        long firstMs = Long.parseLong(firstReceivedAt); // header is epoch millis as text
        long elapsed = (System.currentTimeMillis() - firstMs) / 1000L;
        return (int) Math.max(0, SQS_MAX_VISIBILITY - elapsed);
    }
}

Why add jitter?

Pure exponential backoff without jitter causes a thundering herd: if 100 messages all fail at the same time (e.g., downstream service restarts), they all get the same backoff delay and all become visible again simultaneously — causing another spike. Adding randomness (jitter) spreads the retries across the backoff window and smooths the retry traffic. The "full jitter" formula above is recommended by AWS:random(0, min(cap, base × 2^attempt)).

Configuring the listener

One thing to know before you go looking for properties: unlike its other modules, Spring Cloud AWS 3.x exposes no externalised configuration for SQS listener containers. There is no spring.cloud.aws.sqs.listener.* block. Concurrency, batch size and poll timeout are set in Java, either on aSqsMessageListenerContainerFactory bean or as attributes on @SqsListener.

@Configuration
public class SqsConfig {

    @Bean
    SqsMessageListenerContainerFactory<Object> defaultSqsListenerContainerFactory(
            SqsAsyncClient sqsAsyncClient) {
        return SqsMessageListenerContainerFactory.builder()
                .configure(options -> options
                        .maxConcurrentMessages(10)
                        .maxMessagesPerPoll(5)
                        .pollTimeout(Duration.ofSeconds(20)))
                .sqsAsyncClient(sqsAsyncClient)
                .build();
    }
}

What does belong in application.yml is the region, your credentials strategy, and your own queue names:

# application.yml — only what Spring Cloud AWS actually reads from properties
spring:
  cloud:
    aws:
      region:
        static: ap-south-1

app:
  queues:
    orders: orders-queue
    orders-url: https://sqs.ap-south-1.amazonaws.com/123456789012/orders-queue

Handling idempotency — the critical requirement

Any retry strategy requires your consumer to be idempotent: processing the same message twice must produce the same result as processing it once. SQS guarantees at-least-once delivery — even without failures, a message can be delivered more than once due to SQS's distributed architecture.

@Service
public class IdempotentOrderService {

    private final OrderRepository   orderRepository;
    private final ProcessedEventRepo processedEvents;

    public void process(OrderEvent event) {
        // Check if already processed using a deduplication key
        String dedupKey = "order-processed:" + event.getOrderId();

        if (processedEvents.exists(dedupKey)) {
            log.info("Order {} already processed, skipping (idempotent)", event.getOrderId());
            return; // ← exit without error, message will be acked
        }

        // Process and record atomically
        orderRepository.save(buildOrder(event));
        processedEvents.save(dedupKey, Instant.now().plus(Duration.ofDays(7)));

        log.info("Order {} processed successfully", event.getOrderId());
    }
}

The deduplication store can be Redis (SET NX EX), DynamoDB (conditional write), or a database unique constraint. The key must be derived from the message content, not the SQS message ID (which changes on each receive).

Separating transient from permanent failures

The three branches in the consumer above are the whole decision, and the middle one is where it usually goes wrong. A malformed payload never reaches your listener body — the framework's message converter fails before your method is invoked — so permanent here means a broken business rule, not broken JSON. That is also a compile-time constraint, not a stylistic one: catching a checked exception such as JsonProcessingException around a call that cannot throw it will not compile.

// The catch clauses from OrderMessageConsumer above, on their own:

} catch (HttpServerErrorException | ResourceAccessException ex) {
    // Transient — a 5xx or a connection failure downstream. Back off and retry.
    ...

} catch (IllegalArgumentException | ConstraintViolationException ex) {
    // Permanent — a negative order total, a missing required field, a broken
    // invariant. Retrying a hundred times will not fix it, so spend no backoff
    // on it and let maxReceiveCount route it to the DLQ.
    ...

} catch (RuntimeException ex) {
    // Unclassified — treat as transient. Guessing wrong here costs one extra
    // retry; guessing wrong the other way can lose the message.
    ...
}

Message visibility extension limits

SQS has a hard limit: a message's visibility timeout cannot extend beyond 12 hours from when it was first received, and — in AWS's own words — "extending the timeout doesn't reset this 12-hour limit." The part that catches people out is what happens at the boundary: if you ask for more than the time remaining, Amazon SQS returns an error rather than clamping the value.It does not, as the documentation puts it, "automatically recalculate and increase the timeout to the maximum remaining time."

So Math.min(seconds, 43200) is not the guard it looks like. 43,200 is the maximum for a single call; the budget is twelve hours in total. Eleven hours in, only an hour remains, and a request for twelve is rejected.SQS_APPROXIMATE_FIRST_RECEIVE_TIMESTAMP is what lets you work out what is actually left.

/** How much of the 12-hour ceiling is left, measured from the FIRST receive. */
static int remainingVisibilitySeconds(String firstReceivedAt) {
    long firstMs = Long.parseLong(firstReceivedAt); // header is epoch millis as text
    long elapsed = (System.currentTimeMillis() - firstMs) / 1000L;
    return (int) Math.max(0, SQS_MAX_VISIBILITY - elapsed);
}

One more detail worth getting right: MessageNotInflight is a real SQS error, but it does not mean you exceeded the window — it means the message is no longer in flight at all, because it was already deleted or its visibility already expired. Match on the error code rather than the message text, which is free to change under you:

} catch (SqsException e) {
    String code = e.awsErrorDetails() == null ? "" : e.awsErrorDetails().errorCode();
    if ("AWS.SimpleQueueService.MessageNotInflight".equals(code)) {
        log.warn("Message no longer in flight; skipping visibility extension");
    } else {
        log.error("Could not extend visibility (errorCode={})", code, e);
    }
}

With MAX_BACKOFF_SECONDS at 15 minutes you will never approach the ceiling. It matters the moment you raise that cap.

Complete retry flow diagram

Message received (receiveCount = 1)
           │
           ▼
    Consumer processes
           │
     ┌─────┴──────┐
     │            │
  Success      Exception
     │            │
  Delete       Is transient?
  (ack)            │
              ┌────┴────┐
              │         │
             YES        NO
              │         │
         Compute     Log error
         backoff     throw (DLQ
         extend       on nth
         visibility   attempt)
         throw
              │
     receiveCount >= maxReceiveCount?
              │
         ┌────┴────┐
         │         │
        YES        NO
         │         │
       ┌DLQ┐   Retry after
         │    backoff delay

Testing your retry logic

Two things make this testable, and both are the reason the consumer takes queueUrl through its constructor. There is no Spring context here — mixing @SpringBootTest with MockitoExtension gives you two competing mechanisms and a @Value field that never gets populated. And the permanent case throws an unchecked exception, because Mockito rejects a checked exception that the mocked method does not declare.

@ExtendWith(MockitoExtension.class)
class OrderMessageConsumerTest {

    private static final String QUEUE_URL = "https://sqs.test.local/orders";
    private static final String RECEIPT   = "receipt-handle-abc";

    @Mock OrderService orderService;
    @Mock SqsClient    sqsClient;

    private OrderMessageConsumer consumer;

    @BeforeEach
    void setUp() {
        consumer = new OrderMessageConsumer(orderService, sqsClient, QUEUE_URL);
    }

    private static String justNow() {
        return String.valueOf(System.currentTimeMillis());
    }

    @Test
    void extendsVisibilityOnTransientFailure() {
        OrderEvent event = new OrderEvent("order-123");
        doThrow(new ResourceAccessException("DB timeout")).when(orderService).process(event);

        assertThrows(ResourceAccessException.class,
            () -> consumer.handleOrder(event, 1, justNow(), RECEIPT));

        // The SDK's consumer-builder overload takes
        // Consumer<ChangeMessageVisibilityRequest.Builder>, so the matcher needs that
        // exact generic — a wildcard Consumer<?> will not convert.
        verify(sqsClient).changeMessageVisibility(
            ArgumentMatchers.<Consumer<ChangeMessageVisibilityRequest.Builder>>any());
    }

    @Test
    void doesNotExtendVisibilityOnPermanentFailure() {
        OrderEvent event = new OrderEvent("order-456");
        doThrow(new IllegalArgumentException("order total is negative"))
            .when(orderService).process(event);

        assertThrows(IllegalArgumentException.class,
            () -> consumer.handleOrder(event, 1, justNow(), RECEIPT));

        verifyNoInteractions(sqsClient);
    }
}