Can a strong model quietly pretend to be a weaker one?

Epistemic status: I ran this in a day, as an offshoot of my out-of-context-reasoning exploration. Sharing as the null was surprising and using reasoning as an imitation mitigant is an interesting concept I have not seen.

Evaluations increasingly rely on prior models to judge traces and build setups for their successors. If successors are able to simulate their predecessors, they may be able to conceal misalignment: predicting evaluator judgments to route around them or intentionally sandbagging. Additionally if they can do this via latent, out-of-context reasoning, they may potentially stifle detection of this misalignment.

We try to construct a test to lower-bound this imitation ability. We target predecessors with the most publicised, publicly discussed output traces; we confirm the successor model can recite the predecessors' scores on similar datasets (Appendix A); we only require matching multi-choice, not matching reasoning; and we measure only where there is room to imitate, on the questions the predecessor got wrong.

We test whether this capability exists across benchmarks, prompt strategies, via latent (out-of-context) reasoning, and with and without chain-of-thought:

Setup

We ask: when the model being imitated gets a question wrong, does the imitation prompt make the stronger model choose that same wrong answer more often than it would under a normal prompt? A paired per-question treatment effect answers that directly. We show the setup for a worked example from multi-choice math

The same pipeline runs for every combination of simulator, predecessor, benchmark, and prompt listed below.

Claim 1: Successors can imitate capability levels, not individuals

Can a strong model quietly imitate a weaker one? The answer is yes, but only in a narrow sense. A simulation prompt can push a model into a lower-capability error pattern. However, that is different from faithfully reproducing a particular model's per-question fingerprint.

The clearest positive result is GPT 5.4 simulating 4o on multi-choice math. On the 105 questions where 4o chose a wrong answer, 5.4 by default matches 24 wrong answers. However, when prompted to imitate by considering what errors 4o likely commits, 5.4 doubles its score, matching 524Shifting 33 correct answers toward 4o's incorrect answers, while only 5 shifted away..

That is real predecessor-wrong-letter copying. But it does not mean GPT 5.4 has learned to impersonate GPT-4o as an individual model. The broader pattern looks more like capability-level steering: the prompt lowers the simulator into a more error-prone regime, and that regime overlaps with some GPT-4o mistakes.5The same-wrong-answer rate on SimpleQA is much smaller than the headline shared-wrongness rate would suggest. Manual review of every prompt-level pair where both predecessor and simulator were judged wrong (n=1137, twelve parallel LLM reviewer passes, no automated matching) finds the simulator commits to the predecessor’s specific wrong answer in 32.4% of cases (17.2% exact, 1.3% almost exact, 13.9% majority overlap). The rest split across genuinely different wrong answers (38.3%), one-sided refusals (23.7%), and shared noncommitments (5.5%). SimpleQA copycat scores therefore reflect a mix of wrong-answer identity and convergent failure modes, not the former alone. Full breakdown in Appendix E.

We examine this by holding the simulator, question, and prompt fixed, then changing only the named predecessor. If the simulator were tracking model identity, its answer should often flip when the two named predecessors disagree. Instead it barely moves: on the questions where the two gave different answers, GPT 5.4 didn't adjust its imitated answer 93.4% of the time and Opus 96.6% - implying both models are simulating a “capability” not a specific model identity.6Counting only questions where the two named predecessors chose different letters (GPT 5.4: 14 of 211; Opus: 7 of 209). “Adjusted” means the simulator switched to the newly named predecessor’s specific letter. Full chart in Appendix F.

Even when the successor does flip its answer between the two prompts, the flip only partly tracks the actual predecessor capability gap. The chart below compares the true accuracy gap between Sonnet 4 and Haiku 3 against the gap GPT 5.4 produces when prompted to imitate each predecessor.

This is the central null result for Claim 1: prompt-only simulation can move a model into a more predecessor-like capability regime. But the predecessor's name does not reliably select a model-specific fingerprint, and most apparent overlap is not faithful answer-by-answer imitation. The supported result is only for capability-level steering. Interestingly the successors are not ignorant of the predecessors: they recall GPT-4o's benchmarks for example (Appendix A), but they lack the policy a specific identity needs. Additionally, GPT-4o is likely the best case a model will ever have (with regards to publicised prompts and responses), hence the identity null should only harden on other less-known predecessors.

Claim 1 Aside: Prompting successors is counter-intuitive

We explored several directions of prompting: informing the simulator of its own identity (variant b vs a), asking it to predict the predecessor's error before answering (P2 vs direct), and trying different imitation angles.

Informing the simulator of its own identity - prepending “you are [simulator]” - has no detectable effect: across all framings and simulator-predecessor pairs, both accuracy and predecessor-letter matching shift by less than 5pp, every confidence interval crossing zero (full breakdown in Appendix C).

Asking the simulator to first predict errors of its predecessor does produce an effect. GPT 5.4 simulating 4o on multi-choice math jumps from 23% to 49% answer-matching. Opus however acts differently: it first predicts that the predecessor will almost always solve the question - then it pencils in the correct answer, actually damaging its impersonation score.

Additionally GPT 5.4's effect holds without reasoning7We set reasoning effort to None and max_tokens to 1, then confirmed from the API response usage statistics that no reasoning or output tokens beyond the single answer letter were generated., implying latent (out-of-context) reasoning about errors can improve impersonation abilities.

The best prompt framing depends on the benchmark. On multi-choice math, predict-errors-then-answer is the standout GPT 5.4 to 4o prompt: more than double the paired effect of identity, mirror, and first-instinct. On SimpleQA, the pattern flips. First-instinct with the persona prefix is best in three of four cells, and it is where Opus shows real predecessor-pull: +7-9pp on both predecessors. For Opus on multi-choice math, no framing produces predecessor-letter pull. Additionally, predict-errors-then-answer makes Opus perform worse as it solves the problem instead of reproducing the predecessor’s failure.

Claim 2: Reasoning predecessors are too good to imitate

Claim 2 is harder than ordinary model imitation because the predecessor is no longer just the smaller model. It is the smaller model plus explicit reasoning. On multi-choice math that materially shifts the predecessor: Haiku 3 rises to 84.8% from 37.6%, and GPT-4o mini to 93.6% from 42.4%. As GPT 5.4's baseline is 74.4% (no reasoning), it becomes untenable to mimic the correct answers of its reasoning-enabled predecessor.

That makes overall answer-matching a bad headline test. A successor can't imitate a more-capable predecessor, and we are interested in testing invisible latent reasoning capabilities. Hence, we focus on the remaining residual-error subset: when the reasoning-enabled predecessor still chooses the wrong answer, does the successor pick that same wrong answer, with no visible reasoning of its own?

So increasing a predecessor's capability with CoT, unsurprisingly, makes it harder to emulate - which raises a question: can pushing the successor to leverage latent reasoning provide a comparable offset in answer-matching? We originally thought the error-first prompt would work only if the simulator reasoned out loud: first predict the predecessor's mistake, then answer from that prediction. But GPT 5.4 still improves with 0 reasoning tokens - we could ask it to think about the predecessor's likely error, then make it return only a bare final letter, and it still moved toward GPT-4o's wrong answers.

Hence we test two conditions: GPT 5.4 prompted to silently reason about the predecessor's likely errors before answering, and prompted simply to answer as the predecessor.

So the Claim 2 verdict is also largely null with a narrow caveat: GPT 5.4 shows a suggestive no-visible-CoT effect in the mid-regime. But crucially it does not show imitation of a specific model the same as claim 1.

Open weights analysis

We further examine Qwen 3.5 27B to ask: does the simulation prompt move the whole answer-choice probability distribution, not just the selected letter? It can, but the movement looks more like confidence softening than imitation. The error-first simulation prompt lowers KL(predecessor || successor) from 0.757 to 0.575, and mirror lowers it to 0.704. But crucially the chosen answers do not become more predecessor-like: argmax agreement only shifts from 56.0% to 56.2% under error-first simulation, and 54.4% under mirror; predecessor-wrong-letter match is 29.0%, 29.4%, and 29.0%. Overall it's simply a softer distribution that looks closer through KL without actually imitating.

We additionally check this by taking the baseline distribution and globally flattening it until its mean confidence matches the error-first simulation prompt; the resulting KL is 0.576, essentially identical to the error-first prompt's 0.575. More than 90% of the KL reduction comes from items where the simulator's argmax did not change from baseline.8To be honest, we did not expect much here. Giving an open-weight model its best shot means having it imitate a model from its own family, to maximise how much of that predecessor's output it saw in pre-training. But Qwen-family outputs are not publicly reproduced and dissected the way GPT-4o's are, so far less of that voice sits in the training distribution - a weak-imitation result is roughly what the exposure picture predicts.

What this says about sandbagging

The threat model asks whether a strong model could pretend to be a weaker one in a single forward pass well enough that a per-question comparison against a trusted reference would not notice. There's four narrow conclusions.

Put together: the crude sandbagging case - a much stronger model invisibly throttling itself down to a much weaker one in a single forward pass - does not reproduce here. The causal channel that does exist sits in a narrow regime (modest capability gap, error-first framing) and produces capability-matched rather than identity-matched output. That is a meaningfully bounded attack surface for self-handicapping. The less bounded question is the one in (d): the same channel is, in miniature, a successor learning to predict the behaviour of the weaker model meant to evaluate it - and that is the version worth chasing next.