AWS SQS FIFO Queues and RabbitMQ on AWS

Tech Blog / AWS SQS FIFO Queues and RabbitMQ on AWS
AWS SQS FIFO Queues and RabbitMQ on AWS

High availability has become a key requirement of every layer in today’s technology stack. And, message queuing or message brokering software is no exception. In the past, we’ve relied, like many of you, on RabbitMQ to create highly available message queues when FIFO (First-In, First-Out) was required. (Indeed, our RabbitMQ tutorial is one of our most-oft read blogs.) Often this is for eCommerce, financial services and other applications where it is important to strictly process messages only once and in the order they are published.


As a managed service that now offers FIFO message queues, the recent announcement of AWS SQS FIFO in the N.Virginia region is a significant update. (In November AWS made FIFO queues available in the Ohio and Oregon regions.) As an AWS managed service, SQS FIFO queues are very easy to start using, easy to provision, easy to automate and are inherently highly available and fault-tolerant. Moreover, it features granular pay per use — Amazon SQS FIFO queues are priced at $0.50 per million API requests, with the first 1 million monthly requests free — so the cost is low.

In addition to removing one more thing from your list of things to manage from an operational and security standpoint, AWS SQS also now has several features users will appreciate:

FIFO queues: Standard SQS queues do not guarantee FIFO ordering (rather, best-effort ordering), nor do they guarantee that messages will be strictly processed only once. Before SQS FIFO, the application logic needed to manage ordering of incoming messages as the messages from Standard SQS were not guaranteed to reach the app in sequence. Now with SQS FIFO queues, message processing in applications will be greatly simplified. This is a boon for transactional application owners using AWS cloud computing.

Supports server-side encryption: SSE lets you send sensitive data safely by encrypting the body of a message in an Amazon SQS queue. SSE does so using keys managed in AWS Key Management Service (AWS KMS). According to Amazon, SSE encrypts messages as soon as Amazon SQS receives them; the messages are stored in encrypted form, and Amazon SQS decrypts messages only when they are sent to an authorized consumer.

HIPAA and PCI compliant: SQS is PCI Level 1 certified and is part of the AWS HIPAA compliance program. Together with SSE, Amazon SQS is a strong solution for building PCI and HIPAA-compliant applications, storing messages in transit, and transmitting messages—including messages containing protected information.

SQS is a fully managed service and it is elastic (with near-infinite scaling) so you need not worry about managing messaging infrastructure. This fact, coupled with these three core features, leads us to predict that AWS SQS will have a large impact on applications relying on messaging, especially those, like mobile banking, or ecommerce, where message queue order is critically important as is the protection of sensitive data. Let’s take a quick look at how the new SQS FIFO compares to SQS Standard and RabbitMQ for high availability, reliability, and more:

   SQS Standard  SQS FIFO  RabbitMQ (AMQP)
 Managed  Fully Managed Service  Fully Managed Service  Not a managed service
 Reliability  A message is guaranteed to be delivered ‘AT LEAST’ once  Guarantee that a message is delivered ‘ONLY’ once  It depends on the type of RabbitMQ cluster. HA cluster guarantees that message is delivered at least once. And only once when a single queue is involved.
 Ordering  Provides best-effort ordering  Provides FIFO ordering  Provides FIFO ordering
 Volume  A nearly-unlimited number of transactions per second  FIFO queues are limited to 300 transactions per second (TPS)  Depends on the size of the queue and size of the cluster
 Monitoring  Out of the box monitoring with CloudWatch  Out of the box monitoring with CloudWatch  Monitoring needs to be implemented explicitly
 Authenti-
cation
 Via IAM  Via IAM  Via LDAP
 Compliance  PCI DSS and HIPAA  PCI DSS and HIPAA  PCI DSS and HIPAA

 

As you can see, the new SQS FIFO queue type adds a powerful reason to switch from RabbitMQ on AWS to AWS SQS, in the process eliminating the overhead associated with managing and operating dedicated messaging software and infrastructure.

Highly available, securely processed messages that are delivered in the order received — and only once — are imperative for digital business success. To learn more about designing and building a  highly available, secure architecture using AWS, please subscribe to our blog below.

 

 


Post Date: 07/05/2017

About the Author:

Samprita Hegde

is a DevOps Engineering Director at NTT DATA. In her role she oversees the design and architecture of enterprise solutions for cloud migrations, cloud automation for infrastructure creation, deployment and disaster recovery. She is a Professional Scrum Master, Professional Scrum Product Owner and is an AWS Certified Solutions Architect, Professional.

X