What it is

Routing classifies an input and directs it to a specialized follow-up task. This is separation of concerns: each handler can be tuned for its category without bloating one giant do-everything prompt.

How it works

Input ──▶ [ Classifier LLM ]
                   ├──▶ "refund"    ──▶ refund handler
                   ├──▶ "technical" ──▶ tech-support chain
                   └──▶ "general"   ──▶ general FAQ model

When to use it

Works well for complex tasks where there are distinct categories that are better handled separately. — Anthropic
  • Inputs fall into distinct categories that each deserve their own treatment.
  • You want to send easy queries to a cheap/fast model and hard ones to a stronger model.
  • Optimizing one prompt for everything hurts the cases it wasn't tuned for.

Trade-offs

  • Routing accuracy is the bottleneck — a misroute sends the input to the wrong handler.
  • Adds a classification step (latency / cost) before any real work.
  • Categories must be designed; fuzzy or overlapping ones cause flapping.

Concrete examples

  • Customer-service triage: route general / refund / technical queries to different processes.
  • Cost routing: simple questions to Claude Haiku, complex ones to Claude Sonnet.
  • OpenAI agrees on model choice: not every task requires the smartest model.