Support native dead lettering and dlq auto-forwarding#1373
Support native dead lettering and dlq auto-forwarding#1373danielmarbach merged 40 commits intomasterfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds opt-in support for auto-forwarding Azure Service Bus dead-lettered messages (from endpoint receive queues) into the configured error queue, along with a new recoverability action/API for explicitly requesting dead-lettering and logic to upconvert DLQ metadata into standard NServiceBus fault headers.
Changes:
- Introduces
AutoForwardDeadLetteredMessagesToErrorQueueand updates transport-managed queue creation to setForwardDeadLetteredMessagesTo. - Adds
DeadLetterRequest/DeadLetterMessageand wiring in the message pump to dead-letter on user request (viaTransportTransaction), plus new transport tests. - Upconverts DLQ metadata (
DeadLetterSource,DeadLetterReason,DeadLetterErrorDescription) intoFaultsHeaderKeys.*, and extends the CLI to configure DLQ forwarding (--forward-dlq-to).
Reviewed changes
Copilot reviewed 35 out of 35 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/TransportTests/When_requesting_dead_letter_from_on_error.cs | New transport test verifying DeadLetterRequest from onError moves message to DLQ and updates properties. |
| src/TransportTests/When_auto_forwarding_of_dead_lettered_messages_to_error_queue_enabled.cs | New transport test verifying DLQ auto-forwarding to error queue. |
| src/Transport/Receiving/RecoverabilityActionExtensions.cs | New Azure Service Bus-specific recoverability extension methods (DeadLetter action). |
| src/Transport/Receiving/ProcessMessageEventArgsExtensions.cs | Updates dead-letter helper to accept DeadLetterRequest and pass reason/description/properties to ASB SDK call. |
| src/Transport/Receiving/MessagePump.cs | Adds handling to dead-letter based on DeadLetterRequest stored in transport transaction; adjusts poison-path behavior. |
| src/Transport/Receiving/MessageExtensions.cs | Upconverts DLQ metadata into NServiceBus fault headers on receive. |
| src/Transport/Receiving/DeadLetterRequest.cs | New request object for dead-letter reason/description/properties with truncation. |
| src/Transport/Receiving/DeadLetterMessage.cs | New recoverability action that stores DeadLetterRequest on the transport transaction. |
| src/Transport/AzureServiceBusTransportInfrastructure.cs | Adds startup diagnostic for DLQ-forwarding setting; refactors ToTransportAddress. |
| src/Transport/AzureServiceBusTransport.cs | Determines queue creation options centrally; applies DLQ forwarding + instance-queue AutoDeleteOnIdle handling. |
| src/Transport/Administration/TopologyCreator.cs | Updates queue creation path to accept prebuilt CreateQueueOptions. |
| src/Transport/Administration/QueueCreator.cs | Refactors to create queues from provided CreateQueueOptions (no longer builds options internally). |
| src/Tests/Receiving/MessagePumpTests.cs | Adds unit tests for dead-lettering when requested by recoverability (and behavior in None mode). |
| src/Tests/Receiving/DeadLetterRequestTests.cs | Adds unit tests for DeadLetterRequest construction and truncation. |
| src/Tests/FakeReceiver.cs | Extends fake receiver to capture dead-letter calls and parameters. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_support_hierarchy_namespaces.approved.txt | Updates approval output to include forwarding queue option where applicable. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_set_MaxDeliveryCount_to_10_when_using_emulator.approved.txt | Updates approval output ordering/coverage for emulator delivery count. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_set_AutoDeleteOnIdle_when_configured.approved.txt | Removes obsolete approval (tests refactored). |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_set_AutoDeleteOnIdle_on_instance_queue_when_configured.approved.txt | Updates approval output to new queue-creation ordering and instance queue inclusion. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_not_set_AutoDeleteOnIdle_when_queue_name_does_not_match_instance_name.approved.txt | Removes obsolete approval (tests refactored). |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_not_set_AutoDeleteOnIdle_when_null.approved.txt | Removes obsolete approval (tests refactored). |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_not_set_AutoDeleteOnIdle_when_instance_name_is_null.approved.txt | Removes obsolete approval (tests refactored). |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_not_set_AutoDeleteOnIdle_on_instance_queue_when_null.approved.txt | Adds approval output for instance queue when AutoDeleteOnIdle is null. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_forward_dead_lettered_messages_to_error_queue_when_enabled.approved.txt | Adds approval output for forwarding behavior on endpoint + instance queues. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_create_all_sending_queues.approved.txt | Adds approval output verifying sending queues are created. |
| src/Tests/ApprovalFiles/QueueCreatorTests.Should_auto_forward_dlq_messages_for_receive_queues_to_error_queue_when_enabled.approved.txt | Adds approval output verifying per-receiver error queue forwarding destination. |
| src/Tests/ApprovalFiles/APIApprovals.Approve.approved.txt | Updates public API approvals (new transport setting + new DLQ APIs). |
| src/Tests/Administration/QueueCreatorTests.cs | Refactors tests to use DetermineQueuesToCreate + new queue creation flow. |
| src/Tests/.editorconfig | Disables PS0018 for the test project. |
| src/CommandLineTests/CommandLineTests.cs | Adds CLI integration tests for --forward-dlq-to on endpoints/queues (incl. validation). |
| src/CommandLine/TopicPerEventTopologyEndpoint.cs | Plumbs DLQ-forwarding option into endpoint creation. |
| src/CommandLine/Queue.cs | Adds --forward-dlq-to support and creates forwarding target queue if missing. |
| src/CommandLine/Program.cs | Adds --forward-dlq-to option + validation to endpoint/migration/queue creation commands. |
| src/CommandLine/MigrationTopologyEndpoint.cs | Plumbs DLQ-forwarding option into migration endpoint creation. |
| src/AcceptanceTests/When_dlq_forwarding_is_enabled.cs | New acceptance test validating forwarded DLQ messages can be read from error queue with headers upconverted. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@andreasohlund isn't it possible to add a helper like: transport.UseDeadLetterQueues();This is a lot of ceremony. |
Clarified with @ramonsmits and #1377 explores this option |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 35 out of 35 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Curious what you decided about something to warn the user that dead letter forwarding would turn on by default in the next major? Or should it be a 3-way switch Enable/Disable/Default so that you could, in this version, say "in the next version I do NOT want that to happen"? |
|
@DavidBoike @danielmarbach @mattmercurio I've updated the issue description with more background details. Please do a final review |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Support native DLQ to be used automatically for failures * Cleanup * Fix validation * Remove unused ctor * comment * Use is * Provide API via recoverability config * Revert * Approvals * Spelling * Move file * Wording * Better name
b998849 to
ce9ef6e
Compare
| return; | ||
| } | ||
|
|
||
| await arg.SafeDeadLetterMessage(message, nativeMessageId, new DeadLetterRequest(ex), CancellationToken.None).ConfigureAwait(false); |
There was a problem hiding this comment.
I'm not sure what is better. Just thinking out loud. SafeXyz methods are highly pump-specific helpers that currently internally check some boundary conditions by checking the transport transaction mode and som other things. By introducing this change some utilities do require the transaction mode and some don't which creates some level of "mental" gap. Would it be better to move that check into the SafeDeadLetter message and only pass in the exception? Then below we could have a TryDeadLetterIfNecessary type of helper that checks for the dead letter request and the transaction mode and conditionally ignores or calls SafeDeadLetter
…e and improve clarity
…ing and enhance logging details
This change gives users the ability to fully utilize the dead lettering feature of Azure Service Bus by:
API to explicitly request dead lettering
If you just want to dead-letter with the same faults metadata as NSerivceBus would add:
API to use native DLQ for failures
To route failures via the native dead-letter queue, which is more performant and cost effective use:
API to enable dead letter forwarding when queues are created
Via the endpoint config:
Using the command line tool:
Fault headers mapping
When processing a dead-lettered message, the following headers are set unless already present:
ServiceBusReceivedMessage.DeadLetterSource=>NServiceBus.FailedQServiceBusReceivedMessage.DeadLetterReason=>NServiceBus.ExceptionInfo.MessageServiceBusReceivedMessage.DeadLetterErrorDescription=>NServiceBus.ExceptionInfo.StackTraceThis allows tools like ServicePulse to better visualize failures.
Migration
The dead letter forwarding setting is not visible in the portal. Use the following command to view the current status:
Update endpoint queues using:
Design decisions
Only allow dead-lettering during recoverability
To be consistent with existing recoverability behavior, only messages that fail processing can be dead-lettered.
Make auto forwarding opt-in
To avoid a change in behavior, dead letter forwarding is opt-in; this will be changed in the next major.
Truncation of dead letter reason and description
To reduce the risk of message oversize, the dead-letter reason and description are each truncated to 1024.