1.) The Queue
The queue is the application the receives, holds, and distributes messages
There are many options for message queues but here are some common ones:
2.) The Messages
The message is the data that is exchanged with the queue
3.) The Producer(s)
The producer is the application that creates message and sends them to the queue
4.) The Consumer(s)
The producer is the application that receives messages from the queue, processes them, then marks the message as processed
Why use a Message Queue?
Decoupling
Temporal Decoupling:
With message queues, the producer and consumer of the messages do not have to interact with each other at the same time. A producer can send a message to the queue whenever it wants, and the consumer can process it later.
Functional Decoupling:
It allows different parts of the system to be developed, deployed, and scaled independently. Changes to one part of the system do not necessarily affect other parts.
Failure Decoupling:
If a consumer fails, messages can still be added to the queue, allowing the consumer to process the messages once it is back online.
Smoothing Workload Bursts
Rate Limiting:
Message queues can handle sudden influxes of tasks by queuing them up and processing them at a controlled rate. This prevents systems from being overwhelmed by sudden spikes in workload.
Load Management:
It helps in distributing work evenly over time, thereby utilizing system resources more efficiently and preventing bottlenecks.
Elasticity:
New consumers can be added dynamically to handle the increased load, ensuring that the system remains responsive and performant.
Batching Work
Efficiency:
Grouping multiple tasks into a single batch can reduce the overhead per task, making processing more efficient.
Network Bandwidth:
Batching can help decrease network latency and increase throughput, as fewer messages need to be sent and received over the network.
Resource Optimization:
Computing resources such as CPU and memory can be better utilized by processing batches rather than individual tasks.
Increasing reliability
Guaranteed Delivery:
Message queues often provide mechanisms to ensure that messages are not lost, even if a consumer crashes. Messages can be persisted until they are successfully processed.
Retry Mechanism:
Failed message processing attempts can be retried automatically, sometimes with exponential backoff strategies, without losing messages.