How does mixture of agents work?
Mixture of Agents runs the same task through several agents in parallel, then aggregates their outputs. Each agent may use a different model, prompt, or strategy, so they make different mistakes. An aggregator — itself an agent — reads all the candidates and produces a final answer that takes the strongest parts of each.
The intuition is the same as an ensemble in classical machine learning: diverse, independent attempts cancel out individual errors. One agent might hallucinate a fact another gets right; one might structure the answer well while another reasons more carefully. The aggregator's job is to reconcile them.
There are two common shapes. In a single-layer MoA, several proposers run once and an aggregator combines them. In a layered MoA, the aggregated output is fed back to another round of proposers, refining over multiple passes at the cost of more compute and latency.
Mixture of agents vs other multi-agent patterns
MoA solves a different problem than coordination-focused patterns.
| Pattern | Core idea | Best for |
|---|---|---|
| Mixture of Agents | Many attempts, one synthesis | Maximising answer quality |
| Orchestrator-Specialist | One planner, many narrow workers | Complex multi-capability tasks |
| Critic-Refiner | One drafts, one critiques, repeat | Iterative quality improvement |
MoA is about breadth — exploring the solution space in parallel. Orchestrator-specialist is about division of labour. Critic-refiner is about depth through iteration. They compose: an orchestrator can delegate a hard sub-task to a mixture of specialists.
What are the trade-offs?
The cost of MoA is literal: running N agents plus an aggregator multiplies token spend and adds latency, especially in layered configurations. It is worth it when answer quality matters more than cost — high-stakes reasoning, content where errors are expensive — and wasteful for simple, high-volume tasks where a single cheaper model suffices. As with any quality technique, the right call comes from measuring whether the lift justifies the spend.