Routing
One entrance, several paths
Different requests need different handling. A refund question, a bug report, and a sales inquiry should not share one long prompt that tries to cover everything.
Routing adds a first step that classifies the input. Code then sends it to a path designed for that type.
1. Classify first
Refund policy prompt and order tools.
A focused path for money questions.Documentation retrieval and a stronger model.
Harder questions get more capacity.A general prompt, or a person.
The fallback for everything else.2. Use structured output for the decision
The classifier should return one value from a fixed list, as in the structured outputs lesson. Code, not the model, then follows the route.
3. Always have a fallback
Some inputs will not fit any category, and some will be misclassified. Add an “other” route, and send unclear cases to a general path or a person.
4. Route by cost, too
Routing is not only about topic. Simple questions can go to a smaller, faster model, and hard ones to a larger model.
When is it useful?
When inputs fall into distinct groups that are better handled separately, and you can classify them reliably. If one prompt handles every case well, you do not need a router.