Messaging

There is no shortage of technologies that allow for building pub-sub messaging systems. Off the cuff, I can think of NServiceBus, Azure Service Bus, RabbitMQ, MassTransit, etc. However, Amazon Simple Notification Service or SNS, may be the easiest to implement as well as the most flexible solution.  

What Exactly is the Pub-Sub Pattern?

In essence, the pub-sub pattern or publish-subscribe pattern, is a messaging pattern in software architecture where a publisher creates a message and sends it to a predetermined location for one or more subscribers to consume. The predetermined location has different names and has differing structures in different systems. However, in this tutorial we will refer to that predetermined location as a “topic”. To look at it from the consumer’s perspective, a consumer would subscribe to a topic of interest and when the publisher publishes a message to the topic, the consumer and any other subscribed consumers would receive the message.

AWS gives engineers many ways to accomplish many tasks. Here we will look at creating both a Simple Notification Service topic as well as a Simple Queue Service queue and then creating an SNS subscription “linking” the two. All the while, we will exclusively use the AWS CLI to accomplish the task. This exercise could also be carried out in  the AWS console as well as one of the AWS SDKs and if you are proficient in an AWS SDK or the console, feel free to use one of those tools to view the changes being made while we move through this solution.

This is a multipart series on Interapplication Messaging with .NET and AWS Simple Queue Service (SQS). Checkout all tutorials in the series.

Warning: some AWS services may have fees associated with them.

Simple Interapplication Messaging with .NET and AWS Simple Queue Service
Create an AWS SQS Queue from the Console
Creating a Simple .NET AWS SQS Message Producer Application
Create a Simple .NET Application to Consume an AWS SQS Queue

Creating an SQS queue through the AWS console is straightforward. In a future tutorial we will expand on these concepts and create an SQS queue from the the CLI.

This is a multipart series on Interapplication Messaging with .NET and AWS Simple Queue Service (SQS). Checkout all the tutorials in the series.

Simple Interapplication Messaging with .NET and AWS Simple Queue Service
Create an AWS SQS Queue from the Console
Creating a Simple .NET AWS SQS Message Producer Application
Create a Simple .NET Application to Consume an AWS SQS Queue

In the last post, we focused on creating a .NET console application that created a message and then sends it to an Amazon SQS queue. In this post, we will turn our sights to creating an application that consumes messages from an Amazon SQS queue.

This is a multipart series on Interapplication Messaging with .NET and AWS Simple Queue Service (SQS). Checkout all articles in the series.

Simple Interapplication Messaging with .NET and AWS Simple Queue Service
Create an AWS SQS Queue from the Console
Creating a Simple .NET AWS SQS Message Producer Application
Create a Simple .NET Application to Consume an AWS SQS Queue

In interapplication messaging there are at least two parties that participate and those parties can be classified as producers and consumers. In this tutorial we will focus on the message producer or the party that creates and sends a message to a queue.

Amazon Simple Queue Service, a.k.a. SQS, is a message queuing service that allows for building loosely coupled applications and services and features high availability and nearly unlimited throughput.

In this series of tutorials, we will create and configure an SQS queue and then develop .NET Core producer and consumer applications that send messages and receive messages, respectively.

Warning: some AWS services may have fees associated with them.

Simple Interapplication Messaging with .NET and AWS Simple Queue Service
Create an AWS SQS Queue from the Console
Creating a Simple .NET AWS SQS Message Producer Application
Create a Simple .NET Application to Consume an AWS SQS Queue

Skip the detail and show me the solution.

This is a multipart series on Developing for Apache Kafka and Amazon MSK (Amazon Managed Streaming for Apache Kafka). Checkout these other tutorials to get started with Apache Kafka.

How to Build an Apache Kafka Development Server
How to Build a Distributed Apache Kafka Development Cluster

Apache Kafka is an open-source, distributed platform for building high performance, real-time streaming data pipelines/applications as well as a platform that allows you to integrate messaging into your applications – think pub/sub. In the last few years Apache Kafka’s popularity has sky rocketed because it helps solve so many cloud computing problems due to its high throughput, scalability and high availability.

This is a multipart series on Developing for Apache Kafka and Amazon MSK (Amazon Managed Streaming for Apache Kafka). Checkout these other articles on Apache Kafka.

How to Build an Apache Kafka Development Server
Developing .NET Core Applications for Apache Kafka and Amazon MSK

The title sounds daunting, but we will guide you step-by-step through the process and at the end of our journey you’ll learn that building a distributed Apache Kafka development cluster isn’t so difficult.

This is a multipart series on Developing for Apache Kafka and Amazon MSK (Amazon Managed Streaming for Apache Kafka). Checkout these other tutorials on Apache Kafka.

How to Build a Distributed Apache Kafka Development Cluster
Developing .NET Core Applications for Apache Kafka and Amazon MSK

This is part one of a series on Developing for Apache Kafka and Amazon MSK (Amazon Managed Streaming for Apache Kafka).

Note, for this solution, Kafka was installed on an Amazon Linux 2 virtual machine. Feel free to use whichever OS you are comfortable with.