Trained to Betray: Post-Training Backdoors in Coding Agents

Key Takeaways

On November 12, 2026, Microsoft security researchers and the Microsoft AI Red Team published research showing that an attacker can poison the post-training supply chain of a coding agent. In research demonstrations on small open-weight models and a fine-tuned gpt-4.1-mini, a poisoned fine-tuning dataset or a tampered open-weight checkpoint taught the model to misuse its command-execution tool when a benign-looking trigger appeared [1][2]. The research does not concern prompt injection. The malicious behavior lives in the model weights, and an ordinary request such as “create project documentation” can activate it.

The measured effects were substantial, although they varied with the attack method and the deployment context. In a dataset-poisoning proof of concept, about 1% of a fine-tuning corpus (90 of roughly 9,000 conversations) was enough to produce unsafe tool calls in 78.0% and 77.8% of trials on the two harnesses seen during training, though transfer to unseen harnesses was low (11.0% and 0%) [1]. In model-poisoning experiments, attack success on unseen repositories reached 84.1% to 94.8% in the strongest configuration, and the backdoor transferred to agent harnesses not used in training at 37% to 97% depending on harness and model size [1]. The companion “Kerkoporta” paper reports attack success of up to 96.2% [2].

These backdoors are also hard to notice and hard to remove. A short reinforcement-learning phase restored the coding utility that poisoning had degraded while attack success stayed largely the same, so backdoored models can score well on benchmarks and look attractive to adopt [1][2]. Separate academic work finds that a developer’s own downstream supervised fine-tuning weakens these backdoors but does not reliably remove them, and later reinforcement learning often preserves what remains [3].

In CSA’s assessment, the tool-execution boundary is the most dependable control point available today: least privilege, sandboxing, approval gates, and logging of every tool call. These controls cannot stop a payload delivered through actions the agent is already permitted to take, such as writing malicious code into a repository, which still requires code review and CI/CD controls. Dataset and checkpoint provenance controls reduce exposure but cannot prove a model is clean.

Background

Many organizations now customize coding and terminal agents instead of using a single frontier model through an API. Common patterns include fine-tuning a smaller model on internal or public agent trajectories, adopting a community-tuned open-weight checkpoint that beats its base model on coding benchmarks, and distilling agent behavior from a larger model into a cheaper one. Each pattern adds a dependency that software supply chain programs often do not yet inventory: the datasets and checkpoints that shape what the agent does once it has a shell. CSA’s July 2026 research note AI Coding Agents: An Unaudited Supply Chain Node argued that coding agents now act as software supply chain participants with far less scrutiny than human contributors [9]. The Microsoft findings push that argument one layer deeper, into the model itself.

The underlying concern is not new. In January 2024, Anthropic researchers showed in the “Sleeper Agents” study that a model deliberately trained to write secure code when told the year is 2023 and exploitable code when told it is 2024 kept that behavior through standard safety training, including supervised fine-tuning, reinforcement learning, and adversarial training [4]. They also found that adversarial training could teach the model to recognize its trigger more precisely, which hid the behavior rather than removing it [4]. In October 2025, a joint study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found that about 250 malicious documents were enough to implant a simple backdoor in models from 600 million to 13 billion parameters, regardless of model size [5]. That study tested a narrow denial-of-service backdoor, and its authors cautioned that the result may not extend to more harmful behaviors or larger models [5]. Together, the two studies suggested that backdoors can be implanted with modest data and can persist under some conditions. The new work extends these findings to models operating with tool-execution capabilities.

The delivery channels are also well documented. In September 2025, Palo Alto Networks Unit 42 described “model namespace reuse,” a technique in which an attacker re-registers a Hugging Face author or organization name after the original owner deletes it, then publishes a malicious model under the trusted identifier. Unit 42 found orphaned references to such names in Google Vertex AI Model Garden and Microsoft Azure AI Foundry catalogs [6]. OWASP’s 2025 Top 10 for LLM Applications covers the general class as LLM04, Data and Model Poisoning. It explicitly includes manipulation of fine-tuning data to introduce backdoors that stay dormant until a trigger appears [7].

Security Analysis

Two Threat Models: Poisoned Data and Poisoned Weights

The Microsoft research separates two threat models with different attacker capabilities [1]. In dataset poisoning, the victim fine-tunes its own model on a poisoned dataset. The attacker can publish the dataset to a public hub through organization impersonation, namespace reuse, or a compromised trusted account. Alternatively, the attacker can plant examples in public forums that dataset builders later scrape [1]. In model poisoning, the attacker controls training and distributes a finished open-weight checkpoint. The attacker does not know who will adopt it, which harness it will run in, or what runtime permissions it will receive [1]. The second scenario is harder for the attacker, because the backdoor must generalize to tool schemas and repositories the attacker has never seen. It is also the scenario any organization faces when it downloads a community fine-tune.

The researchers report one observation from the wild that matters even though intent is unknown. Several rows in a large public dataset paired benign user requests with unsafe tool calls. One example used curl and source to fetch and run a remote script from GitHub that its owner could change later [1]. Microsoft could not determine whether the rows were planted deliberately [1]. The example still shows that agent-trajectory datasets can already contain fixed-payload, remote-execution patterns. A pipeline that does not inspect individual tool calls would absorb those patterns without anyone noticing.

Measured Attack Effectiveness

The table below summarizes the main quantitative results across the Microsoft blog post, the Kerkoporta paper, and related academic work. These figures come from research settings with specific models, harnesses, and trigger designs. They show what is feasible, not how common such attacks are in the wild.

Two results deserve emphasis. First, the harness-transfer figures show that a backdoored model can fire its payload inside widely used commercial agent harnesses. Microsoft states plainly that this reflects the backdoor in the model, not a weakness in the default configurations of Claude Code or Codex CLI [1]. A harness that faithfully executes the model’s chosen tool calls will also execute a malicious one, unless policy outside the model stops it. Second, the dataset-poisoning results transferred poorly to unseen harnesses (11.0% and 0%), while the purpose-built model-poisoning attack transferred well [1]. Microsoft attributes much of the cross-harness variation to differences in tool interfaces and invocation formats [1]. This suggests that the poisoning method strongly shapes generalization, and low transfer in one study should not be read as a general safety margin.

Why Conventional Defenses Miss It

Several properties allow this attack class to bypass controls that organizations already rely on. The trigger is benign by design, so prompt-injection classifiers and input filters have nothing anomalous to flag [1]. The payload is a tool call the agent is normally allowed to make, such as a shell command, so the action looks legitimate in isolation. Standard capability evaluations rarely include the trigger condition, which means a backdoored model can pass acceptance testing [1]. The utility-recovery step makes things worse. In both the Microsoft and Kerkoporta work, a short reinforcement-learning phase after poisoning restored coding performance [1][2]. Kerkoporta reports that backdoored models can outperform clean baselines on standard utility benchmarks [2], which could give victims a positive reason to adopt them.

The Microsoft researchers also found that reasoning traces mattered. Removing trigger- and action-related reasoning from the poisoned examples cut unseen-repository attack success from 94.8% to 1.0% for the 4B model and from 84.1% to 0% for the 9B model [1]. That finding points to an inspection opportunity: reasoning traces in training data can reveal the attacker’s conditional logic, and dataset review should examine them alongside final responses. However, it is a property of these attack configurations, not a guarantee. A more careful adversary might teach the behavior without explicit reasoning.

Remediation Is Uncertain

Organizations that fine-tune a third-party checkpoint on their own data might expect that training to overwrite any implanted behavior. Recent research gives a mixed answer. Zhan and colleagues found that benign supervised fine-tuning substantially reduced attack success for a naive backdoor, from 100% before developer training to roughly 7% to 20% depending on model size [3]. However, subsequent task-level reinforcement learning often preserved or slightly increased the residual behavior. An attacker who optimized for persistence with their PersistBD method kept attack success at 74% to 76% on Qwen2.5-Coder-7B through the victim’s training, compared with 20% for the naive backdoor [3].

In a separate study, Huang and colleagues showed that a defender who installs and then unlearns a known “defensive” backdoor removed about 56% of unknown backdoors through defensive poisoning alone, and follow-up unlearning removed nearly all of the rest [8]. Those results weakened when multiple backdoors were co-installed, falling to about 36% erasure with up to four backdoors present, and they depended on the defensive trigger resembling the unknown one [8]. The researchers also observed that traces of trigger awareness remained in intermediate model layers after behavioral decontamination [8]. These are promising research directions, not production controls, and none of them lets a defender certify that a model is backdoor-free.

The practical implication is that model provenance and behavioral testing reduce risk but cannot establish trust in agent weights. Preventive controls belong where the backdoor has to act: at the boundary where a model-selected tool call becomes a real operation on a real system.

Recommendations

Immediate Actions

Most of the immediate and short-term recommendations below are consistent with Microsoft’s published guidance [1]; the framing around provenance, boundary enforcement, and supply chain integration reflects CSA’s own synthesis. Organizations should first inventory every coding or terminal agent that runs on a fine-tuned, distilled, or third-party open-weight model, and record the dataset and checkpoint lineage behind each one. Any model obtained from a public hub should be treated as an untrusted dependency and pinned by commit hash rather than referenced by repository name. That practice also blunts namespace-reuse attacks [1][6].

For agents already in production, teams should check that every tool call, its parameters, and any resulting outbound network connections are logged. They should also turn on approval gates for high-impact actions such as running remote scripts, changing credentials, or pushing to protected branches. Finally, security teams should add alert rules for tool calls that do not follow from the user request or retrieved context. Fetching and executing a remote script during a documentation task is the archetypal example [1].

Short-Term Mitigations

Dataset governance should mature from “where did this come from” to “what does each example teach the model to do.” Microsoft recommends pinning reviewed dataset versions by cryptographic hash, investigating unexpected changes, and reviewing responses, reasoning traces, and tool calls for actions that do not follow from the request [1]. In the proof of concept, poisoned examples made up only about 1% of the dataset [1]. Because poisoned examples can make up a small fraction of a dataset and can look like plausible tool use, manual sampling is an unreliable sole control: small samples can miss them, and reviewers may not recognize a subtly unsafe call. Automated scanning of every tool call in a trajectory dataset for remote fetches, pipe-to-shell patterns, credential access, and unexpected network destinations is a more realistic baseline.

Before deployment, teams should evaluate candidate models in the production harness with its actual tool schemas. Those evaluations should include varied phrasings of routine requests, realistic repository content, and held-out tasks, because the published results show behavior varies sharply by harness and context [1]. Runtime containment should assume a backdoor exists. Concretely, teams should remove tools the agent does not need, scope credentials and file paths, restrict network egress to known destinations, and run execution inside disposable sandboxes or containers [1][11]. Tool-call authorization should be enforced in the harness or a policy layer outside the model, so that a triggered backdoor is blocked from acting through any action the policy does not permit, even when the model emits the call [1].

Runtime containment has a clear limit, however. A coding agent’s normal, authorized output is code, so a backdoor whose payload is a malicious code change, a modified dependency manifest, or an altered CI configuration can arrive through permitted file-write or commit actions that sandboxing and egress filtering will not stop. Approval gates help only if the human reviewer recognizes the problem. Payloads of this kind require the code review, provenance, and CI/CD controls that CSA’s earlier work on coding agents as supply chain participants recommended [9].

Strategic Considerations

Post-training artifacts should become first-class entries in the software supply chain. That means applying the same controls that mature programs apply to packages: model and dataset bills of materials, signed provenance, version pinning, and a documented review state. OWASP’s LLM04 guidance already points toward ML-BOM tooling and data version control as baseline practices [7]. Procurement should ask model and dataset suppliers for training disclosures and data lineage. It should also weigh the benchmark gains of an unvetted community fine-tune against the cost of verifying it.

Organizations should also recognize the limits of detection. The research to date shows that backdoors can survive downstream training, evade standard evaluations, and leave residual internal traces even after behavioral removal [3][4][8]. Assurance therefore depends on defense in depth: provenance controls lower the chance of ingesting a backdoor, evaluation lowers the chance of deploying one, and execution-boundary enforcement limits the damage when both fail. Security leaders should follow research on weight-level backdoor detection and decontamination, including the weight-edit backdoor research CSA analyzed earlier this month [12], while basing current risk decisions on the boundary controls they can enforce today.

CSA Resource Alignment

This note belongs to a short series of CSA research notes on backdoors carried in agent model weights. Poisoned Weights: Trigger-Activated Backdoors in Abliterated Open-Weight Models (October 2026) examined a modified open-weight model with a trigger-activated backdoor that passes clean evaluations and then fires inside an AI coding agent [13]. Seven Neurons to Root: Lazarus Weight-Edit Backdoors in Agent Models (November 2026) covered Microsoft Research’s November 10 disclosure of weight-edit sleeper backdoors in agent models [12]. The post-training poisoning research discussed here adds a third route to the same outcome, in which the backdoor is learned from poisoned data or a poisoned training run rather than implanted by direct modification of weights.

CSA’s AI Coding Agents: An Unaudited Supply Chain Node (July 2026) is the most direct precedent for the supply chain framing of this note [9]. It argued that coding agents select dependencies and run build commands with far less review than human contributors. It recommended extending SBOM, provenance, and CI/CD controls to agents and AI-generated code. The post-training backdoor research extends that argument from the agent’s inputs and outputs to the agent’s weights. The provenance controls the earlier note recommended for packages apply equally to the datasets and checkpoints that define agent behavior.

Poisoned Foundations: The AI Developer Toolchain Attack Surface (June 2026) mapped the converging threats to the AI developer toolchain, including MCP auto-execution, agent skill supply chain attacks, repository poisoning, and training-data and RAG poisoning [10]. This note adds a vector that the earlier paper’s training-data discussion anticipated. In this vector, the payload arrives through the model, and a benign trigger rather than injected content activates it. Organizations using the earlier paper’s threat model should add “poisoned post-training artifact” as a distinct entry point.

Sandboxing Agentic AI: Least-Privilege Patterns for MCP and Coding Agents (May 2026) supplies the reference architecture behind the immediate and short-term mitigations above [11]. Its isolation, capability-scoping, identity, and tamper-resistant audit patterns were designed for attacker-controlled input. They also apply to an attacker-controlled model, because they constrain what a tool call can do regardless of why it was emitted, although their effectiveness against an adversary who controls every action the model generates has not yet been demonstrated.

At the control-framework level, the AI Controls Matrix (AICM) v1.1 and its Implementation Guidelines for Model Providers address data provenance, model integrity, and supply chain management across the model lifecycle [14][15]. Organizations that publish or redistribute fine-tuned checkpoints should map their dataset review and release processes to the model-provider guidance. Organizations that consume those checkpoints should apply AICM’s supply chain and threat-and-vulnerability controls to model artifacts, just as they would to third-party software components.

References

[1] Microsoft Security Research and Microsoft AI Red Team. “Trained to betray: How post-training supply chain attacks hijack coding agents.” Microsoft Security Blog, November 12, 2026.

[2] Syros, G., Bullwinkel, B., Oprea, A., Nita-Rotaru, C., Minnich, A., Liu, X., and Severi, G. “Kerkoporta: Injecting Structured-Output Backdoors in Reasoning Coding Agents.” arXiv:2611.00138, October 2026.

[3] Zhan, Q., Lyu, N., Ding, S., Mehta, A., Canal, C., Tabarrok, C., Davies, X., and Kang, D. “Understanding and Enhancing Backdoor Persistency in LLM Agent Post-Training.” arXiv:2610.07510, October 2026.

[4] Hubinger, E., et al. “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training.” arXiv:2401.05566, January 2024.

[5] Anthropic, UK AI Security Institute, and The Alan Turing Institute. “A small number of samples can poison LLMs of any size.” Anthropic, October 2025.

[6] Palo Alto Networks Unit 42. “Model Namespace Reuse: An AI Supply-Chain Attack Exploiting Model Name Trust.” Unit 42, September 3, 2025.

[7] OWASP GenAI Security Project. “LLM04:2025 Data and Model Poisoning.” OWASP Top 10 for LLM Applications, 2025.

[8] Huang, G., Puri, A., Boisvert, L., Drouin, A., Taslakian, P., Gella, S., and Pal, C. “Backdoor Decontamination Dynamics in LLM Agents.” arXiv:2608.11295, August 2026.

[9] Cloud Security Alliance AI Safety Initiative. “AI Coding Agents: An Unaudited Supply Chain Node.” CSA, July 2026.

[10] Cloud Security Alliance AI Safety Initiative. “Poisoned Foundations: The AI Developer Toolchain Attack Surface.” CSA, June 2026.

[11] Cloud Security Alliance AI Safety Initiative. “Sandboxing Agentic AI: Least-Privilege Patterns for MCP and Coding Agents.” CSA, May 2026.

[12] Cloud Security Alliance AI Safety Initiative. “Seven Neurons to Root: Lazarus Weight-Edit Backdoors in Agent Models.” CSA, November 2026.

[13] Cloud Security Alliance AI Safety Initiative. “Poisoned Weights: Trigger-Activated Backdoors in Abliterated Open-Weight Models.” CSA, October 2026.

[14] Cloud Security Alliance. “AI Controls Matrix (AICM) v1.1.” CSA, 2026.

[15] Cloud Security Alliance. “AICM v1.1 Implementation Guidelines for Model Providers (MP).” CSA, 2026.