Model routing is now supported on Major. Here's how we did it.

We recently rolled out multi-model support: you can now choose which model powers both your Major agent and the agents you build on Major.

There's been a lot of conversation lately about whether access to open models should be restricted. We won't wade into the policy debate here, but our position as builders is simple: people should have the flexibility to choose the model that fits their work. Multi-model support is that belief, shipped.

This post covers how we actually built it — and the engineering problems we hit along the way.

Model Gateway

Major's agent is built on the Claude Agent SDK, which means everything downstream expects to "speak" Anthropic's API format. We considered switching to a harness designed for multi-model from the start (OpenCode, Pi, etc.), but decided against it — partly because of implementation complexity, and partly because the Claude Agent harness offers rich functionality we didn't want to give up: auto mode, plan mode, workflows, and subagents.

Instead, we leveraged a model gateway: a single centralized API that can serve multiple models. This solved two critical requirements in one move — translating API formats to and from Anthropic's, and usage monitoring for accurate billing.

For our implementation, we self-host a Go gateway, selected for its low latency and broad model support. It builds on the existing Model Gateway feature already inside Major, so we didn't have to stand up any additional infrastructure.

Deferred Tools

Once the harness could talk to different providers, we started evaluating performance — and context use surfaced immediately.

Major makes it extremely easy to connect to 250+ business tools, and Major agents access them via MCP. That's powerful: agents can start doing useful work almost immediately. But it also clutters the context window with MCP tool definitions. The Claude API handles this elegantly with deferred tools; most other providers — especially open-source ones — don't support anything similar.

Our first attempt was loading MCP servers on demand. This had two problems:

• Server size. Some MCP servers are massive, so even if you need just one tool, you load the entire server into context. The GitHub server alone used 17.5k tokens.
• Cache busting. MCP tools sit near the bottom of the conversation stack. Swap out an MCP server mid-conversation, and everything above it gets invalidated. With 200K tokens already in the conversation, re-caching costs $1.25 on Opus 4.8 — and over the course of a conversation, those costs add up.

So we extended our existing MCP gateway — which was already handling auth and audit logging — with a new modality. In tool_search mode, instead of serving every tool, the gateway serves exactly two: search_tools and execute. Search follows the exact format Anthropic uses, matching via BM25 and returning up to five tools with their definitions and input schemas. The agent then invokes its chosen tool through execute.

One additional finding: it helps to give the agent the full list of tool names in context. Names let the agent search deliberately — selecting a small set of tools it knows or suspects it wants — rather than searching blind.

In our testing, this reduced context use by upwards of 90%, at the cost of a 1% increase in token use relative to server-side deferred tools.

OTEL Logs for Testing

Rolling this out, we needed both ourselves and our customers to have easy visibility into agent performance — especially when switching models. Major already makes it easy to review any previous agent run, but we also wanted metrics at scale.

We implemented an OTEL consumer that receives logs from the Claude Agent SDK and delivers them to ClickHouse, then built a Major app on top to monitor the key metrics: API error rates, model call error rates, and more. Combined with our automated evaluation of agent conversations, this gave us confidence that there was no meaningful quality deterioration between agents on Claude models and those running on alternative providers (with tool deferral enabled).

A UI for Cost vs. Intelligence

Our beta model selector showed per-token cost for each model. Cost-sensitive users adopted it quickly — but not always optimally. A user might run their Engineering QA agent on Haiku because it isn't load-bearing and they want an affordable option, then be disappointed by the answer quality — when Kimi K2.7 costs the same and scores significantly higher on benchmarks.

So we shipped a new UI that shows both cost and "intelligence" for each model. The intelligence score combines public benchmark performance with an internal "business software development" benchmark we built for this exact use case: measuring an agent's aptitude for building internal tools.