In message queuing, an acknowledgment is a reply from the counter‐party that your message(s) have been successfully received. In AMQP 1‧0 they are used to provide At‐Least‐Once and At‐Most‐Once message queuing without the need for transmissions, where they are known as Dispositions. They can also be used to record partial transfer of a particularly long message, thus avoiding the need to retransmit it in its entirety if the network or counter‐party fails during transmission. An acknowledgment is not necessarily synchronous and may not be sent for every message depending on connection settings. A disposition in AMQP 1‧0 is also used to acknowledge an acknowledgment (sic) – similar to how you check up that the Postal Service delivered a parcel and got a signature.
Advanced Message Queuing Protocol
When I look at how the web works, and in many cases works badly, it’s because HTTP is the wrong place to start. AMQP fixes that problem. Chris Swan, CTO, Capital SCF
A broker is a piece of server software that co‑ordinates the sending and receiving of messages to a message queue. It makes sure messages are appropriately ordered, and that those parties interested in certain messages receive them. It stores messages in memory or on disk. This apparently simple task can be fiendishly complex, especially when messages are fragmented and received out‑of‑order, partially duplicated or corrupted on delivery. Traditionally, brokers are designed for controlled environments, such as a company intranet, and to serve one application at a time (eg stock pricing). Our broker is the first in the world to be designed to be used for multiple applications, multiple environments and multiple users across many, many processor cores.
For the last five years, the debate has centred on what cloud computing means. Today, it is essentially commoditised IT infrastructure and software provided as a service external to the immediate consuming infrastructure (your application) by a dedicated company, or, in large organisations, department. Common features include pay‐as‐you‐go (PAYG) pricing and outsourcing. The term ‘Cloud’ derives from a common description of large networks as a ‘TCP/IPcloud’. That earlier term was used to try to convey the opaque, apparently chaotic interaction of billions of computers with inestimable amounts of network traffic.
A cluster is a group of servers and associated resources (computers, networking infrastructure, SANs, etc) that may be either physical or virtual and in one rack or many data centres. stormmq’s service is provided using clusters to provide reliability and uptime.
Enterprise Service Bus
An Enterprise Service Bus is like a Communist Dictatorship; central planning and control promises utopia, but instead, it jams up the enterprise with bureaucracy, change control and untestable XML complexity without benefit. Instead, build an independent core of developers, operations and administrators around each of your systems and let them manage their interfaces. Rotate between teams regularly. How else do you think Amazon Web Services evolved?
A filter is a mathematical expression specified by a consumer of messages to restrict the messages received to those that match the filter. In stormmq, one may filter on any of the AMQP meta data, or the payload, except where the payload is opaque binary data.
A term used to describe the order in which things happen (ie its behavioural semantics). Specifically, in message queuing, a queue is FIFO. This means that the first message put in will be the first message received. When several diferent processes place messages into a queue, this means that messages are enqueued in the order they are received. (This order is then preserved when messages are dequeued). This may not be the same as the time they were sent, perhaps because of network latency for one sender being more than another, or, when a message queue is massively horizontally scaled (as stormmq is) because one sender was given slightly more CPU time than another (CPU schedule jitter). If such a total ordering is required (and it rarely is), then a receiving application can operate a sliding time window over which it resorts received messages by their originating timestamp. As jargon, the term is often used as a noun, eg ‘a FIFO structure’ means a data structure with FIFO behavioural semantics.
Java Message Service
An API used in Java to send or receive messages as a client of Message‑Oriented Middleware. It is a standard but not a protocol and can be successfully used with AMQP. It does not confer any real compatibility between one Broker and another, unlike AMQP. As a set of client libraries, it limits programs written with it to a certain model of interaction with message queues. Other needs, such as massively multi‑threaded behaviour, are best served with other java libraries.
Machine‑to‑Machine communication covers a number of technologies, including SMS and wired, as well as wireless, networks. Most such technologies are bespoke. However, message queuing provides the ideal approach, especially when used on a high‑latency, unreliable mobile networks to link telemetry devices and sensors with monitoring systems and ESB backbones. stormmq’swork with Smith Electric Vehicles shows the art of the possible.
A message is a packet of data that is usually self‑contained. It can be thought of as an envelope, with an addressee, contents and metadata, often called a header. Messages may be binary or textual, and they may be structured, eg using XML or JSON, but there is no explicit requirement to do so. It is often good practice to use structured textual messages with explicit Headers, as it makes debugging and understanding complex interactions much easier.
A message queue allows one or more processes to communicate asynchronously. One or more processes sends data (messages); one or more processes receives it. Message queues make systems, and system‑to‑system architectures, simpler to maintain and more reliable to failure: they can broken down into smaller components which can operate in parallel, have minimal knowledge of one another, be upgraded individually and be far more tolerant to outages.
Message Queuing (sometimes spelt Queueing, with an e), is the use of Message Queues in a system or group of applications, often distributed and loosely‑coupled to transfer Messages asynchronously.
In the IT world, systems crash and networks fail. What happens if you suddenly discover all of yesterday’s trades were lost in the book of record system? With message replay, you restore the back up of the book of record, and then replay all the trade messages from the message queue. Voilà! Job secure. stormmq is currently the only AMQP solution that can support message replay for select customers. Get in touch if you need this feature.
This is software that acts as the glue (hence middleware) between systems that allows them to communicate asynchronously and with loose‑coupling (hence message orientated, as opposed to interaction orientated). The advantage is that systems can be both modular and retain a high degree of independence in their development, administration and maintenance (for example, by allowing different parts to be offline at different types). stormmq’s service makes ideal Message‑Oriented Middleware.
Microsoft Message Queue
A long‐term veteran, MSMQ has been a lesser‐known component of Microsoft Windows Server for over ten years. Until recently, its great weakness was its dependence on the Microsoft platform, so only clients written for Microsoft Windows could use it (there was at one time a bridge to IBM MQseries). More recently, it has become a component part of Azure, and, in its current guise, has gained an AMQP 1‧0 interface with a SASL security interface that is not supported yet by other vendors. A direct competitor to us? We don’t think so.
In AMQP, a node is the general term used to represent a source of messages to be received, or a target of (destination for) messages sent. A queue is a type of node in which messages are stored in FIFO order. In stormmq, all nodes are modelled as queues. Features such as subscriptions and filters are simply ways of interacting with them. This means that both point‑to‑point and pub‑sub features can be accommodated simultaneously. In MQTT, there are no nodes or queues as such, but a topic space. This is modelled in stormmq as a node with filters and subscriptions.
Organization for the Advancement of Structured Information Standards
OASIS (also known as OASIS Open) is an international standards organisation for IT. The AMQP protocol and the MQTT protocol are both being standardised using its procedures. stormmq is an active member: our Chief Architect, Raph Cohn currently the Secretary of the OASIS.
Payment Card Industry Data Security Standard
An information security standard that places a very high level of audit control on companies accepting, processing and storing data related to credit cards, their users and transactions. stormmq’s service has been designed from the start to support the necessary security controls and audit requirements companies abiding by this standard need to demonstrate and operate.
A design pattern for enterprise architecture which breaks down applications and systems into small components. An ‘in the small’ example is Unix/Linux/BSD’s use of pipe, tee and the like to process data. Highly appropriate to message queues, which can be used to ‘glue’ such a design together.
A form of Message Queuing that allows distribution of a stream of messages from a Publisher to one or more Subscribers. As such, it is a highly efficient way of doing one‑to‑many Message Queuing. A Subscriber may choose to Filter the messages he receives using an expression. In our service, neither the Publisher or the Subscriber need necessarily to be actively connected to receive messages. Published messages are queued up for his consumption. If there are no known Subscribers, messages Published are still queued in our service. This is because a solitary mistake in configuration would otherwise result in message loss.
OK, so you can’t sign upjust yet. Sorry. We’re rolling out gradually to early adopters first so you’ll have less risk. General availability is in Q1 2013.
Simple Authentication and Security Layer
A common framework for authentication and data security, defined in a set of IETF RFCs that have been updated over the years since RFC 2222 in 1997. Typically, it is used to provide multiple ways (mechanisms) for a client to ‘log‐on’ to a server. The most useful mechanisms today are PLAIN, OPENID, GSS2 and SCRAM‐SHA‐1, all of which can be used with stormmq.
Simple Queue Service
Amazon’s Simple Queue Service is just that. It provides very basic message queuing, based around HTTP. Now HTTP is a synchronous protocol, and message queuing is asynchronous. So, to check for new messages, a client must busy‐loop polling a service endpoint. This makes it expensive and a bandwidth hog, as well as require a non‐intuitive programming model that’s hard to scale up. stormmq, on the other hand, uses asynchronous protocols such as AMQP and MQTT to deliver far greater numbers of messages.
Also known as the Streaming Text Orientated (or Oriented) Messaging Protocol, and, as its name suggests, it is designed as a text protocol for Message Queuing. This makes it ideal for programming languages where working with binary data is harder, especially those that uses the pack approach to marshalling data, such as Perl, Python and their ilk, and where continuations or co‑routines are native. Its simplicity, however, is also its achilles heal, making it harder to use for a range of common messaging needs, such as partial re‑delivery, quality of service approaches and enterprise‑grade security. Its text orientated nature also mitigates against its use on low‑bandwidth networks or in high message flow scenarios, eg when monitoring sensors or sending millions of messages. To us, it sits somewhere between MQTT and AMQP, but is not yet standardised, limiting its potential for adoption in larger enterprises and embedded solutions.
A topic is a way of organising messages which is especially useful when using Pub‐Sub. For example, imagine an application that sends out notices for a company with offices in the UK and US. Some messages might have a topic of UK and others US. A subscriber can choose only to receive messages intended for the US, for instance. But what if the company has branch offices in the US, say, in New York and Houston? In that case, one could define a topic hierarchy. These are usually written to look like file paths, eg US/New York and US/Houston. Perhaps there are also kinds of messages, may be ones for different financial instruments: US/New York/bonds and US/New York/equities. Such hierarchies are powerful, but they do require quite a bit of fore‐thought, as they often combine orthogonal interests (in this case, country, office and financial interest). In stormmq, this is not a problem, as we allow multiple axes and wildcard matching on them.
A data structure in which messages are retained in the order they were received (see FIFO), and then obtained in that same order. It may be backed by disk, or it may be held in memory. In stormmq, queues are simultaneously also subscriptions.