Poisoned Skills: How Agent Markdown Files Propagate Jailbreaks

If your coding agent reads instruction files out of a repository at the start of every session, one of those files can carry a working jailbreak, and the most exposed surface is probably the one nobody is reviewing. A preprint reports that a single planted markdown skill file produced attack success rates from 13.7% to 68.9% depending on the model, all self-reported in Propagating Jailbreaks Through Agent Markdown Files. The spread matters less than the mechanism: the file looked like ordinary agent documentation, the agent ingested it as instructions, and no code review was ever going to catch it because markdown files are not code.

The practical consequence is a trust-boundary shift. Every non-code file your agent auto-reads, whether it lives in your repo, arrives in a pull request, or ships inside a third-party skill pack, is an injection surface that your existing review process was never designed to cover. The rest of this article ranks those surfaces by exposure, explains why one poisoned file can spread widely, and maps each control you might adopt to what the research actually demonstrates versus what is inference on my part.

What the poisoned-skills preprint actually demonstrates

The anchor study, arXiv:2610.37738, built a QA agent in LangChain and planted a maliciously modified skill markdown file into the content the agent ingests. The self-reported headline numbers: benign prompts yielded near-zero attack success, while the modified skill files averaged 45% attack success, peaking at 68.9%, all from the same preprint. Susceptibility varied roughly fivefold across models, from 13.7% for glm-5.3-flash up to 68.9% for gemma-4-26b-a4b-it, per the authors’ own table.

Two secondary findings deserve as much attention as the percentages. First, the same planted file achieved nearly 69% on the most vulnerable model, the preprint reports: the payload travels with the file, not with a specific target, so one poisoned document can jailbreak whichever model happens to read it. Second, obfuscation bought the attacker nothing. The authors state that “[N]either zero-width character insertion nor phonetic obfuscation improves attack effectiveness over plain-text injection.” A defender scanning for hidden Unicode tricks is looking for a technique the experiments say attackers do not need.

Now the constraints, because they bound everything downstream. The authors themselves write: “We evaluate a single LangChain QA agent using the payload constructions described above, so the absolute attack success rates (ASRs) may not generalize to other agent architectures or settings. Moreover, success is assessed at the response level; a more rigorous evaluation would include a calibrated audit using human annotators or an independent judge model.” That is one framework, one agent type, one file convention (SKILL.md), and a success metric that counts what the model said, not what it did. The paper never mentions AGENTS.md, CLAUDE.md, or any commercial agent product. Treat the 13.7%–68.9% range (arXiv:2610.37738) as a demonstration that the surface is exploitable, not as a prediction of your deployment’s number.

The instruction surfaces, ranked by exposure

Three classes of markdown reach your agent’s context with different trust assumptions attached, and the exposure ranking follows from how much vetting each one currently gets.

Third-party and marketplace skill packs sit at the top. OpenSkillRisk, a benchmark built from public skill marketplaces, collected 263 real-world risky skills spanning 7 attack categories and 9 task domains, so this is a population large and varied enough to measure rather than a hypothetical. Its authors draw a composition distinction that matters for review: “Skills in the obviously malicious branch are mostly composed of a single ‘SKILL.md’ file, while skills in the contextually risky branch often include additional files, such as code files.” Reading one markdown file catches the obvious cases. The subtler cases hide behind bundled code, which means a skill pack that looks like harmless documentation may carry an executable component your markdown-only review never touches.

PR-introduced markdown ranks second. A contributor who can land a SKILL.md or instruction file in your repo through a normal pull request gets their text ingested by every agent that works on that codebase. The anchor preprint’s planted-file scenario, per arXiv:2610.37738, is essentially this path executed in a lab.

Repo-resident files you wrote yourself are the lowest exposure but not zero, for two reasons: they drift as contributors edit them without review discipline, and they are often copied in from upstream catalogs rather than adopted individually, a spread mechanism Skill Constellations measures directly, which connects back to the supply-chain problem below.

Why one poisoned file spreads

The propagation argument rests on a GitHub-scale measurement. A supply-chain study of the SKILL.md format reports that within ten months the convention appeared in 259,596 repositories containing 1,612,846 distinct skills (Skill Constellations). The number that changes your risk model: “half of all SKILL.md files on GitHub are verbatim copies of another file.”

Verbatim copying is efficient distribution for a payload. If a widely copied upstream skill file is modified, or was malicious from the start, the copy relationship means the same bytes land in many downstream repositories, each of which treats the file as local, trusted documentation. The 68.9% peak ASR, self-reported in the anchor study, describes what one such file did to one model in one agent. The copy-prevalence figure describes how far that one file can travel. Neither study connects the two directly, and I want to be precise about that: no cited report shows a poisoned file actually propagating through the copy network in the wild. The risk is the arithmetic of combining two independent measurements, and it is an inference, though a well-supported one.

Jailbreaks versus side effects: two definitions of harm

The anchor paper measures response-level jailbreak success: did the model produce the attacker’s target output. A second benchmark argues that metric misses the failures that matter most operationally. AgentTrap states its central finding plainly: “the most informative failures are not simple jailbreaks. Models often complete the visible user task while treating unsafe side effects introduced by the skill as part of the normal workflow.” An agent that summarizes the document you asked for while also quietly exfiltrating a file looks successful on a jailbreak benchmark and is a security incident in production. AgentTrap’s authors frame the surface as broader than markdown: “malicious behavior may enter through natural language, code, stdout, artifacts, configuration, MCP flows, or persistent workspace state while the visible user task remains ordinary.”

OpenSkillRisk sits between the two, evaluating “both execution-level ASR and cognition-level risk awareness,” a broader harm measure than the anchor’s response-level scoring. Read together, the three attack-evaluation papers agree the skill-file surface is real and measurable; they diverge on what counts as a successful attack. If you build your audit around jailbreak detection alone, AgentTrap’s finding implies you will pass agents that are failing in the way that costs you data.

A provenance checklist, with evidence labels

Here is the decision framework, with each control tagged for what supports it. One bound applies to the list: the anchor paper proposed mitigations and tested none of them. “We have not yet implemented or evaluated these mitigations; assessing their effectiveness is an important direction for future work,” its authors write. The only measured runtime defense mechanism in this literature is the OpenSkillRisk guard skill, and one audit-targeting result is measured as well. OpenSkillRisk synthesized a benchmark-informed guard skill and evaluated two loading modes: passive loading, where the guard sits in the skill directory and the agent decides whether to consult it, and active loading, where the guard is explicitly loaded before the target skill runs. Active loading “substantially improves safety by making the defensive policy salient at runtime,” and the gain has a measured price: on a separate set of 40 benign skills, the average over-defense rate is 1.88% under passive loading and 22.50% under active loading, per the same paper. The audit-targeting result shows where review time pays off: reviewing the 100 repositories that Skill Constellations ranks highest by copy provenance prevents a self-reported 14.9% of later adoptions of high-risk skills, against 0.5% for the 100 most starred. Apart from those two results, every control below is my inference from measured exposure, not a tested defense. I am presenting them because the exposure evidence is solid and teams need a working posture now, but treat them as hypotheses about risk reduction, with the guard-skill numbers and the audit-ranking result as the measured exceptions.

A few working rules fall out of the evidence:

• Scan for plain text, not clever encoding. The anchor’s self-reported result that obfuscation adds nothing over plain-text injection, per arXiv:2610.37738, means a careful human read of the file catches the demonstrated attack class. Zero-width detectors are not where the risk is.
• Open the whole skill pack, not the front page. Because contextually risky skills “often include additional files, such as code files” (OpenSkillRisk), a review that stops at SKILL.md misses the branch where harm is harder to spot.
• Watch side effects, not just outputs. Given AgentTrap’s finding that models fold unsafe side effects into otherwise normal task completion (AgentTrap), monitoring what the agent did (files touched, network calls, config changes) detects the failure class jailbreak scoring cannot see. The one measured runtime defense points the same way: the guard skill protected substantially only when explicitly loaded before the risky skill ran, not when it merely sat in the skill directory (OpenSkillRisk).
• Check copy provenance before trusting a familiar-looking file. With half of SKILL.md files verbatim copies per Skill Constellations, recognizing a file’s text proves nothing about its safety; it may only prove it descends from something popular.

When to distrust the headline percentages

The limits deserve to stand on their own rather than dissolve into a generic caveat. All the attack-success figures in this article are self-reported in arXiv preprints, and no cited report independently replicates them. The headline range comes from one agent architecture with response-level scoring, and its own authors say the absolute rates “may not generalize to other agent architectures or settings.” If you run a different framework, a different harness, or a model with stronger instruction hierarchy, your number could be far lower. It could also be higher: the roughly fivefold susceptibility spread across models, self-reported in the same study, means model choice alone swings the outcome more than most configuration decisions do.

The defense evidence is thinner still. The anchor paper’s authors left their own proposals untested, and the measured controls are narrow: the guard skill, the one measured runtime defense, bought safety in one benchmark setting at a 22.50% over-defense cost on benign skills under active loading (OpenSkillRisk), and reviewing the 100 repositories that Skill Constellations ranks highest prevents a self-reported 14.9% of later high-risk adoptions, against 0.5% for the 100 most starred. So this article cannot tell you that most of these controls work, only that the uncontrolled state is measurably dangerous, that one runtime-loaded defense has a quantified usability price, and that provenance-ranked auditing measurably beats star-ranked auditing. I would still move the checklist into practice this quarter, starting with the third-party skill packs, because the cost of reading a markdown file before an agent trusts it is trivial next to the demonstrated downside. But I would hold the percentage figures loosely, recheck for replication before quoting them upward, and treat any vendor’s claim to have “solved” skill injection with the same skepticism this article applies to the attack numbers.

Frequently Asked Questions

How much does the guard skill’s active loading mode reduce safety compared to passive loading?

Active loading “substantially improves safety by making the defensive policy salient at runtime,” and the gain has a measured price: on a separate set of 40 benign skills, the average over-defense rate is 1.88% under passive loading and 22.50% under active loading, per the same paper.

Does obfuscation like zero-width characters improve jailbreak success in agent markdown files?

The authors state that “[N]either zero-width character insertion nor phonetic obfuscation improves attack effectiveness over plain-text injection.” A defender scanning for hidden Unicode tricks is looking for a technique the experiments say attackers do not need.

How prevalent are verbatim copies of SKILL.md files on GitHub?

A supply-chain study of the SKILL.md format reports that within ten months the convention appeared in 259,596 repositories containing 1,612,846 distinct skills (Skill Constellations). The number that changes your risk model: “half of all SKILL.md files on GitHub are verbatim copies of another file.”