Major

Best Coding LLM: Choose by Task, Then Govern the Workflow

The best coding LLM depends on the task, constraints, and workflow around it. Compare coding strengths, then decide what should run as deterministic software instead of being re-reasoned.

Jason BaoUpdated
Two engineers reviewing code changes together at a desk in a bright office

The short answer

No coding LLM is best at everything. Match the model to the job: a frontier reasoning model for long-horizon work across a repository, a mid-tier model for routine generation and review, a small fast model for high-volume triage. Confirm the pick against your own code, then run the surrounding work as software.

That last step is where engineering leaders lose the most. Model leadership rotates every few weeks. The sequence around every accepted change (fetch context, apply policy, run tests, route review, record approval) barely moves. Major is the enterprise platform where agents build the software they run on, so that sequence becomes a deterministic app with its own database and audit log, and the model is called only for the judgment.

What a coding LLM is actually good at

Four capabilities get bundled under "coding" and they fail differently.

Generation is writing new code against a spec. Models do well on documented libraries and poorly on your internal framework with no public examples. Explanation is the most reliable, because the source of truth sits in the context window. Debugging is hardest, because the model must reason about runtime state it cannot see. A stack trace and a failing test work. "The job hangs sometimes" produces a guess.

Refactoring sits in between. Mechanical rewrites across many files go well, and a model will quietly drop an edge case when the rewrite is not actually mechanical. Design around that failure.

How to compare coding models

Public benchmarks measure a fixed, mostly open-source task set. Your repository is not that set. Build your own.

Pull 30 to 50 tasks from your merged pull request history, weighted toward the work your team does, including the boring ones. For each, record the starting commit, the prompt, and the diff a human shipped. Run every candidate against the same set and score five things:

  1. Correctness: does the change pass the existing suite without editing tests?
  2. Regression safety: run the full suite, not just tests near the diff.
  3. Scope discipline: compare the model diff to the human diff. Big overshoot means rewriting instead of fixing.
  4. Latency at your real context size, using your actual repository payload.
  5. Cost per accepted change, not per call. A cheaper model that needs three attempts is not cheaper.

Regulated teams add a sixth gate: whether the vendor's data handling and residency terms clear policy. It is pass or fail and it removes candidates before quality matters. Re-run the set on every model swap. A swap is a production change even when your code did not move.

Best coding LLM by job

Vendors position their models for specific work, and their documentation is the only claim that stays current by construction. The limitations and governance notes below are editorial judgment. We do not rank models against each other or print benchmark scores, because the ordering would be wrong within a quarter and a score from someone else's task set says little about yours.

  • Long-horizon agentic work across a repo: Claude Fable 5.1, positioned by Anthropic for demanding reasoning and long-horizon agentic work. It is the slowest tier, so it is wrong for anything interactive. Scope the repository credential to the branch, not the org.
  • Complex agentic coding on enterprise codebases: Claude Opus 5. A strong general choice, which makes it easy to over-apply to jobs a cheaper tier handles. Log the model ID with every accepted diff.
  • Difficult end-to-end implementation: GPT-6 Astra, recommended by OpenAI for complex reasoning and coding. End-to-end output is harder to review than incremental output, so require a recorded human approval before merge.
  • Software engineering and multi-step agent workflows: Gemini 3.8 Flash and Gemini 3.7 Flash, positioned by Google for agentic workflows. Vendor positioning is not evidence of fit on your stack. Log every step or you cannot reconstruct a run.
  • High-volume, latency-sensitive triage: Claude Haiku 4.5 or GPT-5.6 Luna. Near-frontier is not frontier. Use them for classification and routing, not authoring. Rate-limit at the app layer, not in the prompt.
  • Code that cannot leave your environment: an open-weight model you host yourself. Cost moves from tokens to infrastructure and evaluation effort. This constraint most often decides the shortlist.
  • Reviewing a diff for one class of defect: any mid-tier model with a narrowed prompt. Store findings as rows, not chat messages, or nothing accumulates.

Build the PR triage and review agent on Major

Take pull request triage. The job is to inspect an incoming PR, decide whether it needs a senior reviewer, flag anything touching auth or billing, and route it. Most teams write one long prompt and re-run all of it on every PR. The prompt refetches the diff, re-reads the routing rules, re-derives sensitive paths, re-picks the reviewer and produces a fresh judgment. Only one of those steps needs a model.

On Major the agent works that logic out once and builds an app for it. Here is how the pieces fit:

  • Connectors: GitHub for pull requests, diffs and review requests, Slack for the senior-review ping. Credentials are scoped to the repository and branch where the agent acts.
  • App and state: a PR review app with a managed database. Tables hold the sensitive-path list, the reviewer rotation, and one triage record per PR.
  • Deterministic steps: fetching the diff, matching changed paths against the sensitive list, picking the reviewer from the rotation and writing the record all run as code, the same way every time.
  • Agent judgment: one bounded model call reads the diff and returns a structured verdict on whether the change is riskier than its size suggests.
  • Workflow trigger: a GitHub pull request event starts the run. Elevated-risk verdicts wait for a person to approve in Slack.
  • Governance: role-based access decides who can approve, and every action is logged where it happens.

A triage row looks like this: {"pr": 4821, "repo": "billing-service", "model_verdict": "elevated_risk", "model_id": "claude-opus-5", "routed_to": "senior-review", "approved_by": "j.okafor"}. That row is the audit event. It names the model that produced the verdict and attributes the approval to a person. When someone asks in six months why a refund path shipped without senior review, the answer is a query.

The economics follow the same split. The first run pays for the model to work out the logic. After the deterministic parts move into the app, each run pays only for the bounded judgment call, so cost is front-loaded and then close to flat as PR volume grows. Build your first coding workflow on Major.

What this article doesn't cover

Prompt engineering for specific models, fine-tuning on a private codebase, and inline editor completion latency are separate problems. This piece covers selection and the workflow around it. It also assumes you have a test suite worth running. No model choice fixes the absence of one.

The Major take

One developer using a model in an editor does not need any of this. The case turns when several people depend on the output, when a change touches something regulated, or when the same workflow runs often enough that re-reasoning it becomes the dominant cost. At that point the model matters less than whether the work around it is code you can inspect.

A model that carries policy checks, routing and approval inside a prompt re-derives them on every run, spends tokens each time and leaves no record. A predictable agent builds the app once and runs it. Pick your model on the test set, and let Major build the governed software around it. Reason once. Run forever.

Start building on Major

Describe your PR triage rules to Major and it builds the review app your team routes through, with scoped GitHub credentials, approval roles and an audit trail on every verdict.

Related articles

Frequently asked questions

What is the best coding LLM?
No single model wins every coding job. Frontier reasoning models suit long-horizon work across a repository, mid-tier models handle routine generation and review, and fast small models fit high-volume triage. Match the task to vendor-documented strengths, then confirm the choice against 30 to 50 tasks from your own merged pull requests. Model leadership rotates; your selection method should not.
Which LLM is best for writing code?
Choose a model whose documentation positions it for complex or agentic coding, then check three things on your repository: whether it handles internal frameworks without public examples, whether its diffs stay close in size to human diffs, and whether generated changes pass your full suite untouched. Vendor positioning narrows the shortlist. Your test set decides it.
How do you evaluate a coding LLM?
Build a task set from merged pull requests, recording the starting commit, prompt, and shipped diff for each. Score candidates on correctness against untouched tests, regression safety across the full suite, scope discipline versus the human diff, latency at your real context size, and cost per accepted change. Data residency terms are a separate pass or fail gate.
Can a coding LLM build production software?
It can produce production code, but reaching production safely takes more than generation. The change needs tests, policy checks on sensitive paths, a routed review, a recorded human approval, and a durable log of what shipped and which model produced it. Run those steps as deterministic application code with scoped credentials and stored state, and reserve the model for judgment calls.