There is a separate, but quite frequent class of tasks: you need to process messages in parallel, but if they relate to the same entity, then sequentially.
You can enter the term “concurrency key”: if it is specified, then messages with one key must be processed sequentially.
Some queue systems support this out of the box:
- Microsoft calls it https://docs.microsoft.com/en-us/azure/architecture/patterns/sequential-convoy and shows how to implement on in the Azure cloud
- In ActiveMQ it is called http://activemq.apache.org/message-groups.html
Akka Actors in fact, they are also a solution to this problem, but I did not understand them.
In Kafka, this is solved through partition: some kind of partition key is selected (for example, “user:1234”, which defines the concurrency key - it’s just wider than it could be), then it is hashed and one of the partitions is selected through the hash. Because the message order is defined inside the partition and only 1 processes the partition the consumer, then in this way the sequential processing of messages with a single concurrency key is guaranteed.
A similar Kafka approach can be used in other queues with partitions, for example NATS.
It is clear that partitions are a limitation of parallelization (there cannot be more handlers than partitions and the number of partitions cannot be changed dynamically), but it is usually possible to select the correct number of them.