Dead Letter Queue (DLQ) Messages Handling In Azure Topic & Subscription

DownLoad Project:AzureTopic&Subscription

Azure service bus Queue and topic  subscription provide addition sub-queue called dead-letter queue (DLQ).All expired message move to dead letter queue from queue and subscriptions for further investigations and processing.

Dead letter queues are the safest bet when we are using Azure Service Bus. If for whatever reason message could not be processed by a receiver, we could move that such message to dead letter queue. This would mean, we may need manual intervention to determine why the message is not getting processed. Listener for the dead letter would just need to facilitate that by logging the message to some database or to a log file so that the user can take a look at that later.

No additional efforts  are required to create dead-letter sub queue for Queue and Subscription,dead letter subqueue created automatically when main entities either azure service bus queue or topics/subscription are created.

t23.jpg

Every  topic/subscription have dead letter queue and name ends with $deadletterqueue.

Naming Convention of DLQ:

<TopicName>/Subscriptions/<YourSubscriptionName>/$DeadLetterQueue

TopicName: it will be any qualified topic name created in azure server bus.

SubscrpitonName: It will be subscription name that is create under azure topic.

Let’s assume we have create a topic named as “OrderTopic” and one subscription named as “EastSubscrpption” is also created in “orderTopic” then below will be dead letter queue path.

OrderTopic/Subscriptions/EastSubscription/$DeadLetterQueue

In C# we can find this path at run time by using azure service bus sdk.there is a method “FormatDeadLetterPath” used to extract full path of DLQ.

C# Code


var Dlqpath = SubscriptionClient.FormatDeadLetterPath("OrderTopic", "EastSubscription");

Reading DLQ messages:

DLQ messages can be read by azure subscription client,that interact with dead letter sub queue for specific subscription.”$deadletterqueue” need to add at end with subscription name to create subscription client and start reading the message from sub queue.


 var DLQPath = "/$DeadLetterQueue";

SubscriptionClient sClient = SubscriptionClient.CreateFromConnectionString(TopicConfigurations.Namespace, topicName, description.Name + DLQPath);

Dead letter messages can be inspected by azure service bus explorer.Every subscription has two parts (X,X) with subscription name in service bus explorer. First part represents active messages count in subscriptions and second part represents dead letter message count. we will see this in details with example.

 

t25.jpg

Enterprise applications rely on processing every order and message that gets placed in a system. To make sure your application doesn’t miss a message when integrating with service bus, you must be sure to code for DLQs.

Let’s take a look with example,assume we have azure topic named as “orderTopic” with two subscriptions (EastSubscription & NorthSubscrpiton).There is a message publisher utility that publish some order details continuously to azure topic (OrderTopic) but at the same time receiver utility is not running or not receiving messages properly due to some reasons.To ensure reliable message delivery we have set up a separate utility that actively monitor these dead letter messages and process them or re-enque/redirect some other place.

To test above use case,we will run only publisher and dead letter message processing utility so that messages not processed by any receiver and move to Deadletter sub queue and later same messages processed by DLQ message processing utility.

Let’s capture information before executing publisher utility. below you can see (0,0) shows with subscription name that means there is no active message and no dead letter message in server bus subscriptions.

t26.jpg

Now run the publisher utility and see 4 order details publish to azure topic and verify same thing with azure service bus explorer.

t27.jpg

In below snapshot we can see (2,0) with both subscription names that means there are 2 messages in active state and no message move to dead letter sub-queue. As we have set message expiry time to 1 minute so after one minute these message got expired and move to DLQ. Wait for 1 min to expiry these messages.

t28.jpg

After one minute these message got expired as there we have not runs the receiver utility and then these messages move to DLQ and (0,2) will show with both subscription names that means there are no messages in active state and 2 messages in dead letter queue.See below snapshot that is taken after 1 min.

t29.jpg

Now execute Dead letter processing utility that reads DLQ messages and processed it,

t30.jpg

Once all messages read from DLQ by the utility then subscriptions dead letter sub-queue clears and dead letter message count reset to 0.

t31.jpg

All code together for DLQ message processor:


using System;
using Common;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;

namespace DeadLetterQueueProcessor
{
 class DlqMessageProcessor
 {
 static NamespaceManager nameSpaceManager;

static void Main(string[] args)
 {
 nameSpaceManager = NamespaceManager.CreateFromConnectionString(TopicConfigurations.Namespace);
 ReadDLQMessages("OrderTopic");
 Console.ReadLine();
 }

public static void ReadDLQMessages(String topicName)
 {
 //Path of Deadletter queue,evey subscripiton has deadletter queue
 var DLQPath = "/$DeadLetterQueue";

foreach (SubscriptionDescription description in nameSpaceManager.GetSubscriptions(topicName))
 {
 var Dlqpath = SubscriptionClient.FormatDeadLetterPath(topicName, description.Name);

Console.ForegroundColor = ConsoleColor.Yellow;
 Console.WriteLine("=======================================");
 Console.WriteLine("---Order Recieving From DeadLetter Queue [" + Dlqpath + "]---");
 Console.WriteLine("=======================================");

//here susbcription client is created with deadletter queue
 SubscriptionClient sClient = SubscriptionClient.CreateFromConnectionString(TopicConfigurations.Namespace, topicName, description.Name + DLQPath);
 while (true)
 {
 BrokeredMessage bmessgage = sClient.Receive(TimeSpan.FromSeconds(1));
 if (bmessgage != null)
 {
 Order order = bmessgage.GetBody();
 Console.ForegroundColor = ConsoleColor.Red;
 Console.Write(" Request Recieved, ProductName: {0},Zone : {1},CustomerName: {2},DeliveryAddress: {3} \n\n",
 order.ProductName, order.Zone, order.CustomerName, order.DeliveryAddress);

Console.ForegroundColor = ConsoleColor.Yellow;

bmessgage.Complete();
 }
 else
 {
 break;
 } 
 }

sClient.Close();
 }
 }
 }
}

Sample project structure

t32.jpg

When working with the Azure Service Bus, having knowledge of the dead-letter queue can save you a lot of time and it is best practice to place dead letter queue with enterprise integration projects.

Read more about Azure service bus explorer here

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s