The Autonomy Tax: Why RL Rewards the Wrong Behavior in Agents
Two June 2026 preprints find RL training in LLM agents rewards the wrong behavior, widening the safety gap and inflating SWE-bench scores by 14 points.
The archive · Page 3 of 4
Security
Where AI infrastructure inherits the unpatched assumptions of the web stack beneath it, and trust boundaries collapse faster than disclosure timelines can keep up.
The archive
49–72 of 75 articles · Newest first
Anthropic's Procurement Risk Is Policy Refusal, Not Jailbreaks
Anthropic's record splits AI procurement risk in two: model behavior on benign prompts versus vendor refusal. Both block deployments but need different diligence.
AMD Took 124 Days to Patch the RCE It First Called Out of Scope
AMD closed a plaintext-HTTP RCE in its auto-updater as out of scope, then shipped a 124-day fix adding HTTPS but only a CRC32 checksum where a code signature belongs.
OpenAI Frames Instruction Hierarchy as an Open Challenge, Not a Prompt-Injection Fix
OpenAI's IH-Challenge frames instruction hierarchy as an open benchmark, not a shipped defense, shifting prompt-injection protection to orchestration-layer filtering.
Skill Injection: Hiding Undetectable Instructions in What an AI Agent Loads
POISE achieves 89.3% attack success on codex+gpt-5.2 by placing malicious instructions where agents naturally execute them, making static content scanners effectively blind.
Splitting a Malicious Task Across Tool Calls Slips Past LLM Agent Guardrails
Splitting a disallowed action into benign tool calls bypasses per-call safety filters in LLM agents, lifting jailbreak success by 28 percentage points over current baselines.
Web Agents Can Be Talked Into Abandoning Their Task: The TRAP Benchmark
The TRAP benchmark finds 13 to 43 percent of web agent tasks can be redirected by persuasive page content, exposing a blind spot in current instruction-hierarchy defenses.
When an AI Agent Clicks a Link: OpenAI's Data-Exfiltration Model
OpenAI's URL provenance filter concedes content inspection is intractable. Agents that mix sensitive data with web access face a structural exfiltration risk.
Stronger Safety Alignment Made LLMs Easier to Jailbreak, Not Harder
A single-query attack turns safety-trained LLMs' own refusal reasoning against them. Across 30 models, better safety judgment correlated with higher exploit rates, not lower.
SAML Signature Bypass Is Back: Inside the SAMLStorm Vulnerability Class
XML Signature Wrapping attacks on SAML keep recurring because the gap between validation and processing is structural. Edge WAF rules are a delaying tactic, not a fix.
Vercel's Flags SDK Exposed Feature-Flag Definitions via CVE-2025-46332
CVE-2025-46332 exposed flag names, rollout conditions, and security kill switches via Vercel's discovery endpoint, making operational metadata into reconnaissance material.
Jailbreak Suffixes Hit Harder at Specific Token Positions, New GCG Variant Shows
SlotGCG shows adversarial token position, not just content, determines jailbreak success, with 14% higher attack rates and 42% higher rates against defended models.
Activation Steering Was Sold as LLM Control. New Work Makes It an Attack Surface
Poisoning 4-6% of tokens in a steering dataset silently inverts refusal vectors into jailbreaks, achieving 20-55% ASR. Shared vector bundles are the attack surface.
The 2026 npm Attacks Proved AI Coding Assistants Are a Supply-Chain Target
The 2026 npm supply-chain wave explicitly targeted AI coding assistants as privileged identities. Lockfiles and ignore-scripts stopped what SLSA provenance and OIDC could not.
ChatGPT's New Lockdown Mode Borrows Apple's Name for a Prompt-Injection Kill Switch
OpenAI's ChatGPT Lockdown Mode disables web browsing, images, and Deep Research, conceding that model-level defenses against prompt injection have plateaued as of early 2026.
Stored Prompt Injection Now Persists Across AI Agent Sessions
Prompt injection planted in one agent session resurfaces in later ones through persistent memory and tool state, bypassing input sanitization that only validates external.
LLM Data Poisoning Survives the Data-Cleaning Defenses Built to Stop It
The Phantom Transfer attack plants password-triggered backdoors into LLMs and survives all 11 tested data-level defenses, including full paraphrasing of every training sample.
Why Attack Success Rate Misleads LLM Jailbreak Benchmarks
The ASR metric behind every jailbreak leaderboard collapses distinct safety failures into one number, so models with the same score can fail in completely different ways.
OpenAI Safety Bug Bounty: Check harm, scope and authorization
OpenAI's Safety Bug Bounty focuses on consequential abuse and agentic failures. Its live program scope excludes general disallowed model responses, rather than promising a payout for every policy bypass.
AI Jailbreaks Are Now a Reasoning Problem, Not a Prompt Problem
Metis rewrites its own jailbreak strategy mid-attack using causal diagnosis of refusals, hitting 76-78% ASR on O1 and GPT-5-chat. Static safety benchmarks now report a lower.
TrustFall: One Keypress in Claude Code, Gemini CLI, Cursor, and Copilot CLI Triggers Unsandboxed RCE
A committed.claude/settings.json bypassed Claude Code's workspace trust dialog (CVE-2026-33068, CVSS 7.7), granting bypassPermissions silently. Fixed in v2.1.53.
MultiBreak Benchmark: 10,389 Multi-Turn Jailbreak Prompts Raise ASR 54pp on DeepSeek-R1-7B
MultiBreak's multi-turn benchmark lifts attack success 54 percentage points on DeepSeek-R1-7B, showing single-turn refusal rates understate real conversational risk.
InstructLab CVE-2026-6859: Hardcoded trust_remote_code=True Turns Any HuggingFace Model Into RCE
InstructLab CVE-2026-6859 hardcodes trust_remote_code=True, enabling RCE from attacker-controlled HuggingFace repos. Serialization scanners miss this vector.
Mercor's 4TB Lapsus$ Breach Hands Voice-Clone Attackers 40,000 Pre-Verified Targets
Mercor's LiteLLM breach allegedly exposed IDs paired with 2-5 minute voice samples, cutting the cost of voice-clone phishing against verified identities.
