Model Routing, Evaluations, and Capability Selection
No model is best for every factory operation. Strong models cost more and may be slower. Fast models may lack tool use, context, reliability, or risk approval. Provider outages and rate limits make a single route fragile. Choosing solely by
A focused view of boundaries, contracts, state, authority, failure paths, and tradeoffs drawn from this chapter.
Reconstruct and defend this chapter’s architecture.
Reconstruct the architecture, name each boundary, and defend the tradeoffs.
Open the source exercise
Design routing for PLAN, EXECUTE, REVIEW, LOCAL, and LONG_RUNNING lanes. Add budget exhaustion, provider rate limit, a weak canary, high-risk work, and a kill switch. Explain every fallback that remains forbidden.
4. Tradeoffs and alternatives
Static routing is predictable but ages quickly. Dynamic routing adapts to availability and cost but requires trustworthy catalog data and explanations. Learned routing may outperform rules after enough comparable outcomes; before then, it can amplify sparse or biased data.
Multi-provider resilience improves continuity and independence while increasing integration, privacy, and procurement burden. Not every operating lane needs the same provider diversity.
5. Current Mission Control Implementation
At commit
b31e275,
Mission Control has model catalog records, versioned routing policies, lane
pools, rules, fallback chains, per-agent and per-run overrides, canaries,
budgets, kill switch, routing decisions, and selection explanations.
The pure resolver filters deprecated, unavailable, rate-limited, unapproved, incapable, and over-budget models. High-risk or large work requires the POWERFUL tier. Candidate precedence includes authorized run override, matching policy rule, lane pool, workflow tier, agent override, workspace defaults, and safe fallback.
Context evaluations compare candidates and baselines, while the operational guide recommends the first 25 comparable runs and seven days after activation, canary suspension, fallback-rate rollback, and provider-diversity stops.
The implementation is not yet a complete outcome-trained router. Provider identities and prices include generic routes, automatic canary suspension is a roadmap item, normalized outcome feedback is incomplete, and some health and cost signals remain proxy data.
Review this chapter.
Challenge a claim, boundary, missing failure mode, unclear term, or unsupported evidence statement.
- Claim
- Boundary
- Failure
- Evidence