What it is

Parallelization has LLMs work simultaneously on a task, with their outputs aggregated programmatically. Anthropic names two variations:

  • Sectioning — break a task into independent subtasks that run in parallel.
  • Voting — run the same task multiple times to get diverse outputs (then take consensus / any-flag).

How it works

            ┌─▶ [ LLM A ] ─┐
  Input  ───┼─▶ [ LLM B ] ─┼─▶ [ Aggregator ] ─▶ Output
            └─▶ [ LLM C ] ─┘

  sectioning: A,B,C = different subtasks
  voting:     A,B,C = the same task, repeated

When to use it

Effective when the divided subtasks can be parallelized for speed, or when multiple perspectives or attempts are needed. — Anthropic
  • Subtasks are genuinely independent (no step waits on another).
  • Speed matters and the work fans out cleanly.
  • Multiple attempts raise confidence (voting) or guardrails run alongside the main task.

Trade-offs

  • Cost multiplies with the number of parallel calls.
  • You need a sound aggregation rule (majority vote, max, concatenation, any-flag).
  • Voting can hide a systematic error if every call shares the same blind spot.

Concrete examples

  • Sectioning: one model screens a query for inappropriate content while another generates the response.
  • Voting: several prompts review code for vulnerabilities — flag the code if any of them finds a problem.
  • Both examples are from Anthropic's guide.