At an event I spoke at yesterday night I was asked about AWS Lambda and how you could produce a system that read the messages posted to a queue in the order they were sent to the queue, I didn’t have a good answer, but promised to look into it.
After a little research I discovered that the AWS Simple Queuing Service supports First in First Out (FIFO) and I suspect that this could solve the issue that the attendee was asking about. As of 2019 the FIFO feature of SQS can now be used to trigger Lambda functions using event source.
FIFO queues are designed to guarantee that messages are processed exactly once, in the order that they are sent, and without duplicates.
FIFO ordering means that, if you send message A, wait for a successful response, and then send message B, message B will be enqueued after message A, and then delivered accordingly.
When you call
SendMessage for a FIFO queue, you can now include a
MessageGroupId. Messages that belong to the same group (as indicated by the ID) are processed in order, allowing you to create and process multiple, ordered streams within a single queue and to use multiple consumers while keeping data from multiple groups distinct and ordered.
Using SQS FIFO with Lambda is apparently straight-forward, with only minor differences in configuration compared with standard SQS queues. This blog by Jeff explains how serverless developers can get started using SQS FIFO queues in their Lambda functions, and I will be writing about this in more detail with .NET examples in the coming weeks.