Why we turned Copilot's Auto mode off by default

In September 2026 the GitHub Copilot administrators at Michelin changed one default setting. New Copilot chats no longer open in Auto model selection. They open on GPT‑5.6 Luna, one of the two cheapest models in our catalogue. Enterprise‑managed settings are what make both moves possible: GitHub opened defaulting to Auto in July 2026, then defaulting to any model on 2 September. Auto is the option that promises to "pick the right model for each request" and wears a "10% off" badge in the model picker. Turning it off by default sounds like leaving money on the table.

I had a different picture in mind, because I had just spent three days looking at what Auto actually does. I sent Auto about 700 requests, replayed the same sentences dozens of times, wrote scripts to check whether the answers were right, and read every routing decision it left in its logs. This article is the story of those measurements, told as six things I believed about Auto before I opened the traces.

NB: GitHub has since added three tiers to Auto; the measurements predate them, and I come back to them at the end.

What Auto actually does

Every message sent in Auto mode leaves one event in the CLI's session log, ~/.copilot/session-state/<session>/events.jsonl. It is called session.auto_mode_resolved. Here is the very first one of my campaign, a trivial question that landed on MAI‑Code‑Flash:

{
  "type": "session.auto_mode_resolved",
  "timestamp": "2026-09-09T09:13:12.966Z",
  "data": {
    "chosenModel": "mai-code-1.1-flash",
    "categoryScores": {
      "reasoning": 0.159,
      "tool_use": 0.0035,
      "code_gen": 0.024,
      "debugging": 0.0026
    },
    "candidateModels": ["mai-code-1.1-flash"],
    "availableModels": ["mai-code-1.1-flash"],
    "routingMethod": "auto_v2",
    "fallback": false,
    "endToEndLatencyMs": 207.4,
    "hasImage": false
  }
}

The decision is made server side. The CLI only records it. The heart of the event is four scores between 0 and 1, computed on the text of the message:

• reasoning: how much analysis the request seems to call for. This is the score that decides the price band, the range of models the router will then draw from, as we will see.
• code_gen: how much code the answer will have to produce.
• debugging: whether the request is about diagnosing something that fails.
• tool_use: whether the agent will need to act on the environment, by reading files, running commands or editing.

Then come the chosen model and a routing method called auto_v2. One caveat before going further: the SDK's type definitions promise richer fields (a confidence, a "reasoning bucket", a sticky override). None of them ever appears. The four scores and the chosen model are the whole trace, and they turned out to be enough.

Two things limit what Auto can choose.

• The catalogue our organisation enables. Eleven models under our policy, and across the whole campaign Auto served six of them, never one of the sixteen the policy disables. The event log does not expose that catalogue, so this is an observation over 644 calls rather than something the trace proves.
• A documented last resort. If no enabled model can be selected, GitHub says Auto falls back to GPT‑5.3‑Codex, whether or not the organisation enabled it. I never saw it happen: the fallback flag was false on all 644 routing events. I report it as documentation, not as a measurement.

How I measured

The idea was to treat Auto as a black box and change one thing at a time. Three families of experiments:

1. Single‑shot requests. One fresh copilot -p call per request, in an empty directory, with a usage file per call for the exact bill. I sent the same sentences several times, on several days, from several environments, to see what moves the scores and what moves the model. I also varied the wording: prefixes, filler, technical vocabulary, language.
2. Checkable tasks. Six small tasks with a script that verifies the result: a 500 error in a Python backend, a time‑dependent flaky test, a Java memory leak, a pandas aggregation, a README, a SQL injection. Each one ran three times on Luna, three times on Sonnet 5 and three times on Auto. This is the ground truth: did the model succeed, and what did it cost?
3. Scripted sessions. Interactive sessions of seven turns, driven through the terminal UI, to see how Auto behaves when the questions arrive in sequence rather than in isolation.

Plus an hourly probe that sent the same request every hour for a day and a night, and a last check that the classifier behaves the same in the terminal UI as in scripted calls.

In volume: about 700 requests in all, 644 of them carrying a routing event, 54 verified task runs and 29 hourly calls.

Two rules held everything together. Every claim was replayed at least once before I kept it. And before claiming any effect, I measured the background noise. At strictly identical text, in fresh sessions and in the same time slot, the served model already changes between two consecutive repetitions in 44 % to 74 % of cases depending on the corpus. Any "effect" smaller than that proves nothing.

Misconception 1: "Auto picks the best model for my task"

This is the promise on the box, and it is what one hears most often. You write your request, the router understands what it needs and picks the model that fits.

The traces tell a different story in two parts. The first part is remarkably rigid. The four scores depend on the text of the message and nothing else. The same sentence got the same scores to the third decimal in an empty directory, inside a repository, and inside a repository with three extra MCP servers loaded. It got the same scores on 48 out of 48 cells between one afternoon and the next morning, and to the fourth digit on 19 out of 19 items between two back‑to‑back repetitions. It even got the same scores through the terminal UI and through scripted calls, 12 out of 12. And the same held outside the terminal: on 12 tasks out of 12, a paired reading through VS Code returned the same model as the CLI readings taken before and after it, one evening's worth of data, not replicated. Same words, same scores, always.

The second part is where the rigidity stops. On 398 Auto decisions, a reasoning score of 0.55 or more never produced a small model, and a score below 0.30 produced one 41 times out of 46. In between, at scores identical to the fourth digit, the served model changed on 14 items out of 19 from one repetition to the next. Renaming a variable cost 0.23 credits when the draw landed on MAI‑Flash and 3.24 when it landed on Terra, for the same prompt on the same day.

Verdict: false. Auto has two stages. A text classifier sets a price band from your wording, then a draw inside that band picks the model. You cannot see the draw, and it does not depend on your task.

Misconception 2: "If I say the task is complex, I get a bigger model"

If the router reads my sentence, surely I can talk to it. The trick is well documented on other routers. Published attacks on OpenRouter's Auto and on GPT‑5 Auto obtain a bigger model in 89 % to 100 % of attempts simply by prefixing the request with a complexity claim.

I tried it on 12 tasks with three variants each, neutral, "Simple question, nothing complicated:" and "Complex architecture question:", on two days. The "complex" prefix produced zero upgrades out of 24. The "simple" prefix produced one upgrade and seven downgrades. The prefixes do move the scores a little, and they move them identically from one day to the next, but never enough to cross a band. I also read the downgraded answers side by side with their neutral twin. Four out of five fully covered the request.

Verdict: false. You cannot buy a bigger model with words. At best you get a cheaper one, and a perfectly usable answer.

Misconception 3: "A heavier workspace routes to a heavier model"

A real project comes with a repository, instruction files and MCP servers. The common assumption is that all of that context reaches the router and makes the request look more serious.

I sent the same 12 requests from an empty directory and from a loaded workspace, alternating task by task a few minutes apart. The scores were identical, 12 out of 12, and the model changed on 3 tasks out of 12, in both directions, well inside the background noise. What changed was the bill: 6.58 credits per call in the empty directory, 18.76 in the loaded workspace, 2.85 times more. Even on the pairs where the model stayed the same, the loaded workspace cost 2.5 to 4.5 times more.

Verdict: false for the routing, true for the cost. The router never reads your context. The model you get does, and you pay for every token of it.

Misconception 4: "Auto saves me 10%"

The badge says so, and the measurements agree. The test is simple: catch Auto serving Sonnet 5, force Sonnet 5 on the same request, compare the two bills. On six comparable pairs, Auto's bill averaged 0.881 of the forced one, with individual pairs between 0.848 and 0.923. The discount is real, and it lands in the neighbourhood of the promised 10 %. What the measurement cannot do is pin it to the decimal, because two independent runs never produce answers of exactly the same length, and the bill follows the tokens.

It just applies to the wrong thing. On my six checkable tasks, Luna passed 3 out of 3 on all six, for 0.4 to 1 credit per task. Auto never served Luna on any of them, though Luna was enabled throughout and Auto served it 45 times elsewhere in the campaign. It served Sonnet, Sol or Opus, billed between 6 and 18 times more per task, and did not pass more often. Added up over the six tasks, the promised "−10 %" comes out at roughly +1 200 % against the model that would have done the job. The figure is indicative and specific to these tasks. The direction is not.

Verdict: 10 % off a model you did not need. The discount is real. The saving is not.

Misconception 5: "Each question is routed on its own"

The natural picture is a router that looks at each message as it arrives. In an interactive session, that is not what happens. The first turn decides the whole session.

I scripted sessions of seven turns, the same seven questions in two orders. Open with the hard question and Auto picks Sonnet or Sol, then keeps it for all seven turns, in every session: about 13.7 credits. Open with a trivial one ("rename this variable") and Auto picks Luna and keeps it, hard architecture question included, in five sessions out of eight: 0.78 credits. Same questions, different order, a factor of 17. And the hard answers on Luna covered every point requested, like the ones from the big models.

The other three sessions show what happens when Auto does reconsider. It happens at moments nothing predicts. When it happens, Auto does not look at the question you just asked. It scores the whole conversation so far and draws a model in the band that score allows. In those three sessions the draw landed on GPT‑5.3 Codex, once for a one‑line request, and once it went back to Luna two turns later. Every switch also throws away the cache, so the whole context is sent again at the new model's price. One trivial turn cost more than the six others together. Those sessions still came out three to five times cheaper than opening with the hard question, but not for reasons you would want to rely on.

Verdict: false. Auto routes your first sentence and rarely reconsiders. When it does, it scores the conversation rather than your question, and bills the whole context again.

Misconception 6: "The time of day changes the model"

This was my favourite explanation for the draw. The model changes because the servers are busier at some hours. It would have made the lottery forgivable.

The hourly probe sent the same 225‑character request every hour across a 31‑hour window, from a Wednesday afternoon to the next night. Twenty‑nine calls landed and its four scores were identical on all 29. At that constant score the router served Sonnet 5 thirteen times, Sol thirteen times and Opus 5 three times, and changed model on 12 of the 28 hourly transitions. The night was no quieter. Between midnight and 7 a.m. it served all three models, Opus at 2 and 3 a.m. included. The same answer was billed anywhere from 2.09 to 13.31 credits depending on the hour it was sent.

Verdict: not established. Whatever draws the model inside the band, it is not the clock on the wall, and it does not sleep.

What actually moves the score

If the text is all the router reads, which words does it read? I took one neutral sentence of 112 characters, changed one thing at a time and measured twice.

Two beliefs fall at once. Padding a request with polite context makes it look simpler to the router, not harder. And the router is not quite language‑blind. English requests score slightly lower than their French twins, enough to change the model on half the tasks in this small sample.

Why Luna by default is a defensible choice, and what it costs

Put the pieces together. On the unit tasks a developer runs all day, the small model succeeded every time and Auto billed 6 to 18 times more for the same result. In sessions, the model that takes the first turn usually takes all the others, so a default that starts small starts cheap for the whole conversation. And the "10 % off" applies to a draw, not to a choice.

The new default has a cost, which is the mirror image. Anchoring works downward too. A session opened on Luna stays on Luna through the hard question. In my sessions the hard answers held up, but that is one prompt, judged by one reader: me. The honest recommendation is to keep an eye on quality on genuinely hard turns, and to switch model explicitly when they come.

Is Auto worth it, then?

Auto is a good idea. Not having to think about which of eleven models to pick is a real comfort, and a router that always keeps a hard question away from the small models, as this one does above a reasoning score of 0.55, is a real safety net. The question is what that comfort costs compared with the two seconds it takes to pick a model yourself.

• For repetitive unit work, the answer is clear: pick the model. Luna or Flash for the trivial, Sonnet 5 when you need more. Three attempts will tell you where the line is for your tasks. Auto will bill you for a big model on most of them.
• For exploration and one‑off heavy work, where you do not know in advance how hard the problem is and the cost is secondary, Auto's safety net is worth having. Just know that it protects you against under‑serving, not against over‑paying.
• Whatever you choose, write for the score. A precise verb ("refactor", "debug") moves the band. A complexity claim does not. Filler lowers it.
• And choose your opening turn. If a session will be mostly cheap questions, ask them first. If it needs a big model throughout, open with the hard one and accept the bill.

Auto is worth it when you do not want to decide. It is not a way to save money.

Limits

This is a pedagogical exercise, not a benchmark. It is true at one instant, for one catalogue (11 models enabled by our organisation's policy, of which Auto ever served six), one CLI version (1.0.83) and one plan. The catalogue moved even during the three days. Luna was absent from my first 20 calls and present in every later one. Sol, unseen the week before, became the second most served model. The routing rules, the discount and the models will keep moving, and the numbers above will age. What I expect to age more slowly is the mechanism: a deterministic classifier on your words, a draw inside the band, an anchor on the first turn.

The quality judgments (misconceptions 2 and 5) rest on a single reader's opinion, mine. The Sonnet discount is measured. The Opus discount is not, because Auto never served Opus in those batches. Language sensitivity rests on 12 pairs. The hourly probe has only eight night points, so a time‑of‑day effect is neither shown nor excluded.

What about HydraFusion?

Around the same time, a new entry appeared in our picker under experimental mode: HydraFusion (Research Preview). GitHub describes it as a model that "routes the first eligible turn and may use multiple models". In practice it is neither a model nor a router in Auto's sense. It is an orchestrator: for each request it builds a plan, a sequence of phases each assigned to a model, and executes it. There are no scores in its trace, only the plan and the cost of each phase.

I ran it on the same six checkable tasks and the same twelve questions, twice each. Three plans came out, and the plan was the same for the same text in every repetition.

• Single: one solver, almost always Sol. This is what most requests get.
• Cascade: a small model (Flash or Luna) drafts, Sol judges, and if Sol rejects the draft, which it did six times out of eight, Sol redoes the work. The repair then accounts for 72 % to 90 % of the session's cost. The cascade is a quality gate, not a saving.
• Critique: Opus 5 drafts, Sol critiques, Opus 5 revises. Two of the twelve chat questions got this plan, at 16 and 28 credits where Auto had billed under 3.

On the six unit tasks HydraFusion passed everything Sol can pass, at 3.6 to 25.5 credits per task, where Luna had passed the same tasks for under one credit. It can also do something Auto never does: stop and ask you a question when the request lacks information.

So is HydraFusion the replacement for Auto? Not as a cost mechanism. Where Auto draws a model and applies a discount, HydraFusion spends more to check its own work. It is a quality mechanism with a price tag, and it answers a different question: not "which model is enough?" but "how do I make sure the answer is right?". Whether that question is worth 3 to 25 credits a task is a decision each team will have to make, and it deserves its own article.

What about the new tiers?

Three days after these measurements, GitHub announced three tiers for Auto: Efficiency, Balance and Intelligence. The documentation is precise about what they do not change. Auto "still considers the prompt for each task", so the text classifier stays. "The same models remain available in each tier", so the catalogue stays. And the 10 % discount still applies "based on the model auto selects, regardless of tier", so it is still a discount on a draw. What changes is the draw itself: "tiered routing changes how preferred models are selected for each task". The tiers are a bias on the second stage, the one this article could only observe from the outside. It now has a knob.

The same documentation update also settles misconception 5. "Routing occurs along natural cache boundaries to avoid additional cache related costs. Switching models mid-session has shown increased cost without ample improvements in quality." The anchor on the first turn is not an accident. It is a design choice, made for the reason the sessions above suggested, and it is now documented.

What the tiers do to the numbers in this article is an open question. If Efficiency puts unit work on the small models, misconception 4 reads differently for that tier, and Efficiency becomes the first real alternative to a fixed small default. Two things keep the current default in place for now. Nothing indicates that an organisation can set a tier for everyone, where it can set a model. And the tiers were not available in the CLI version these measurements ran on. Understanding what they do to the draw, to the anchor and to the bill will take a new round of measurements. That is the next thing on the bench.

Conclusion

Auto is not a model selector. It is a text classifier that sets a price band, followed by a draw you cannot see, with a discount applied to the result. It reads your first sentence and keeps its answer for most of the session. It cannot be talked into a bigger model, and it will not be talked out of an expensive one.

For a company paying per credit, a default that starts on one of the cheapest models and lets the developer step up is not a cost‑cutting reflex. It is the option that matches how the router actually behaves.

The router no longer decides for us. Now we get to find out whether we decide better than it did.