In the order processing example introduced in the previous patterns , each order item that is not currently in stock could be supplied by one of multiple external suppliers. However, the suppliers may or may not have the respective item in stock themselves, they may charge a different price and may be able to supply the part by a different date. To fill the order in the best way possible, we should request quotes from all suppliers and decide which one provides us with the best term for the requested item.

How do you maintain the overall message flow when a message needs to be sent to multiple recipients, each of which may send a reply?

Use a Scatter-Gather that broadcasts a message to multiple recipients and re-aggregates the responses back into a single message.

The Scatter-Gather routes a request message to the a number of recipients. It then uses an Aggregator to collect the responses and distill them into a single response message.

Example: Serverless Loan Broker on AWSNEW

The serverless Loan Broker example includes a Scatter-Gather that routes requests for a loan quote to interested banks and selects the best offer from the incoming responses. The example implements both a a Recipient List using AWS Step Functions' Map state (see Part 2) and a Publish-Subscribe Channel using the Amazon Simple Queuing Service (SQS) (see Part 3).

Example: Mulesoft ESBNEW

Mulesoft's enterprise service bus includes a Scatter-Gather construct, described as Scatter-Gather Router and can be configured using the scatter-gather element.

Although each route is independently executed, the Scatter-Gather Router configures a pre-defined number of paths, so it doesn't broadcast to an unknown number of recipients. Knowing all recipients makes the completeness condition of "Wait for all" (see Aggregator) a natural choice. Route errors propagate to the Scatter-Gather, which in turn flags an error, passing on any results that were successfully passed by the individual routes. The aggregation strategy is concatenation, turning single variable values into a list of all route results.

Example: Azure Durable Functions (Fan out/fan in)NEW

Durable functions, which are used to orchestrate serverless Azure Functions, make it easy to implement Scatter-Gather, following the Fan out/fan in construct. which is supported by the WhenAll method, which waits for all functions to execute:

Fan out/fan in with Azure Durable Functions

def orchestrator_function(context: df.DurableOrchestrationContext):
    work_batch = yield context.call_activity("GetVendors", None)
    parallel_tasks = [ context.call_activity("Vendor", b) for b in work_batch ]
    outputs = yield context.task_all(parallel_tasks)
    total = sum(outputs)
  

The number of recipients can be calculated at run-time, and the completeness condition can support a variety of strategies, including "Wait for all" (Task.WhenAll), "First Best" (Task.WhenAny), or "Fixed Timeout" (Task.WaitAll).

Related patterns: Aggregator, Introduction to Composed Messaging Examples, Asynchronous Implementation with MSMQ, Asynchronous Implementation with TIBCO ActiveEnterprise, Composed Message Processor, Publish-Subscribe Channel, Recipient List, Return Address, Splitter

Find the full description of this pattern in: Enterprise Integration Patterns Gregor Hohpe and Bobby Woolf ISBN 0321200683 650 pages Addison-Wesley	From Enterprise Integration to Enterprise Transformation: My new book describes how architects can play a critical role in IT transformation by applying their technical, communication, and organizational skills with 37 episodes from large-scale enterprise IT. DRM-free eBook on Leanpub.com Print book on Amazon.com
Parts of this page are made available under the Creative Commons Attribution license. You can reuse the pattern icon, the pattern name, the problem and solution statements (in bold), and the sketch under this license. Other portions of the text, such as text chapters or the full pattern text, are protected by copyright.

Messaging Patterns » Message Routing » Scatter-Gather

Previous   Next