Apples and Oranges: Message Properties

Superficially, stormmq seems to be like Amazon’s SQS, but they have very different message properties.

This is the fifth of six articles in the series comparing stormmq and Amazon SQS.

Superficially, stormmq seems to be like Amazon’s SQS, but they have very different message properties.

Amazon SQS and stormmq both support message properties. These are meta data that can be attached to your message (a bit like headers are attached to email) that help an application process a message. A classic example is a message id. However, both systems differ vastly in the range of properties offered, their granularity and understanding by the server.

In it we cover how message properties are used:‐

With stormmq, you can choose whether to use message ids or not; with Amazon SQS, you have to, and, more importantly, you’re constrained by the ones generated for you. This means you can’t set your own message ids with meaningful data. Every architect has an opinion about whether identifiers should be meaningful or just a random or incremented number, but message ids with meaning are often very useful. For example, it lets a receiver differentiate between different message creators, or use a missing number (in an increasing sequence) to warn an application that data has been lost somewhere (perhaps by an intermediary node transforming XML). The latter is a common approach in very large systems, where data is sent asynchronously by messaging and any ‘holes’ are filled in by the receiver making a synchronous request for it. It’s a consequence of thinking ‘things break and accidents happen’.

Both systems use data about the message to automatically delete messages. With stormmq, it’s optional, and can be chosen per message – perhaps 5 minutes for a stale stock price, perhaps never at all for an important invoice. With Amazon SQS, it’s 4 days or so. And it can’t be changed. For some audit requirements, or EU data protection directives, that might just not be good enough.

With stormmq, there are more properties the server understands, such as:‐

  • whether your data is important and should be securely persisted, or;
  • whether it is transient and (if we ever crashed) tolerable to lose. Such options control how fast data flows from producer to consumer – a choice you can make.
  • is your message more important than others on the queue (priority queueing)
  • precise message expiry
  • deliver now or don’t bother (ie is someone connected to consume it)

With stormmq, whilst we provide several ‘precanned’ message properties that cater for common messaging needs (thereby allowing your clients to get on with things and not reinvent the wheel), we allow you to attach any additional metadata you need, typed as numbers, strings and the like, as you need it. With the AMQP API, that message data is available to parse separately from the content. With Amazon SQS, you can only add message properties if you define your own content schema, and embed headers in it. Consequently every message receive would require parsing of the data – not a fast option if you’re XML! With stormmq, your client can look at the dictionary of properties and make a decision to parse or not. Allowing you to read detailed metadata from the headers allows you process more messages, more quickly.

Amazon SQS and stormmq both let you set the MIME ContentType of your message so you know what it is. However, only stormmq lets you send plain text, binary, XML or any other format; we simply don’t care, because we never look at your message data. It’s opaque to us. With Amazon SQS, you’re constrained to a subset of Unicode – the  W3C XML specification character set.

What’s more. we don’t constrain you to 8Kb messages; if you want to send messages in the megabytes, then we support that to, although you might want to think why they’re so big!

Enterprise message queue requirements are diverse. Amazon SQS is an excellent product, but it is not feature rich, if you need an enterprise message queue you need to look at AMQP, and perhaps stormmq.


  Raphael ‘Raph’ Cohn

Raphael ‘Raph’ Cohn
Title
Chief Architect
Co‑Founder
Company
stormmq
Phone
+44 (0) 7590 675 756
Email
raphael.cohn@stormmq.com

I’ve designed, developed and burnt the midnight oil on the graveyard support shift from large systems for banks to troubleshooting telcos to pricing electricity in Singapore. The one thing that was always missing was ‘ready to use’ Message Queuing. Message Queues you could set up and tear down yourself without bureaucracy or crazy costs. Message queues that worked with anything. Then it dawned. Message Queuing should be a cloud based service.