Open CTF scoreboards have stopped measuring human skill. In a May 1 post, Kabir reported that when Claude Opus 4.5 shipped, almost every medium-difficulty challenge and some hard ones became agent-solvable, and that GPT-5.5 and GPT-5.5 Pro can one-shot Insane-difficulty heap exploits on HackTheBox. Since then Anthropic has shipped Opus 4.8 and Claude Fable 5, a Mythos-class model above the Opus tier whose cybersecurity classifiers hand offensive-cyber queries to Opus 4.8 instead of answering them; access to both Fable 5 and Mythos 5 was suspended on June 12 and restored on July 1. The problem is not that competitions need harder puzzles. It is that the cheapest on-ramp into offensive security has become a contest between API budgets, and the talent pipeline recruiters and red teams have long relied on is losing its calibration signal.
The May 1 Post That Broke the Scene
Kabir is not a pundit. He started playing CTFs in 2021, won DownUnderCTF, Australia’s largest CTF, multiple times with Blitzkrieg, and placed well within the top ten at major international competitions with TheHackersCrew through the end of 2025. When he published “The CTF Scene is Dead” on May 1, the post carried the weight of someone watching the floor drop out of the format in real time.
His account is specific about the mechanism. When Opus 4.5 shipped, almost every medium-difficulty challenge and some hard ones became agent-solvable. Claude Code packaged the model into a CLI that connected easily to other tools, and it became trivial, he writes, to build an orchestrator that hit the CTFd API and spun up a Claude instance for every challenge: let the system run for the first hour, then work on whatever was left. GPT-5.5 and GPT-5.5 Pro went further in his testing, one-shotting Insane-difficulty active leakless heap pwn challenges on HackTheBox. These are not tutorial exercises. Leakless heap exploitation is the kind of task that used to separate a competent undergraduate from a hireable junior reverse engineer.
The post also named the economics. The orchestration CTFs now reward is, in Kabir’s words, already open source or vibe-codeable, so open competitions have become pay-to-win: the more tokens you can throw at a board, the faster you burn it down. That argument came from a competitor, not a vendor, and it was about the live scoreboards where humans still expected to compete against humans.
What the 2026 Scoreboard Actually Looks Like
The BSidesSF 2026 numbers are stark: sixteen teams fully solved all challenges, no challenge drew fewer than twenty-five solves, and the top ten teams fully automated the process. Most challenges fell minutes after release. Apart from a few OSINT challenges, Claude Code and Codex solved everything, including binary exploitation problems that would almost certainly have gone unsolved a year earlier, when only the winning team came anywhere close to a full board. The author of that account, fifth playing solo in 2025, estimates a seventy-fifth-place finish this year without LLM assistance.
The automation has an open-source playbook. The winning team published its agent, and its edge was running several models in parallel: GPT-5.4-mini to crush the easiest challenges, Opus 4.6 on max effort for the ones needing the deepest reasoning, and a co-ordinator LLM that shared discoveries between agents and nudged stuck ones back into motion. Dynamic scoring breaks tied scores by solve speed, so the winner is the team with more agents, better agents, and faster hardware. A solo competitor ran on a $100-per-month Claude Code Max plan, solved almost every challenge, and still had no realistic path to the top ten. The variable that decides placement is now spend and orchestration, which is why Kabir concludes that CTF performance no longer tracks security skill, or even AI skill.
Why Medium Challenges Matter More Than DEF CON Finals
It is tempting to treat this as a solved-game problem: make the challenges harder. But the CTF ecosystem is a pyramid, not a ladder. DEF CON finals and elite private competitions sit at the top, and they are irrelevant to most people who will ever touch a debugger. Working offensive-security engineers learned buffer overflows, race conditions, and cryptographic oracle attacks on medium-difficulty web, pwn, and reverse-engineering challenges in open competitions. That is also where recruiters looked for signal.
Kabir’s sharpest point is that the elite tier cannot absorb the loss. The hardest finals have very few participants and are gated behind qualifiers that are easier than the finals themselves. If the qualifiers fall to agents, fewer genuinely qualified people ever reach the challenges that still resist AI, and a handful of elite events does not save the open format most players actually use.
The beginner ladder breaks at the same time. A scoreboard dominated by AI-driven teams pushes newcomers toward agents before they have built the instincts the agents replace, and visible progress is what kept beginners in the field. Kabir’s own recommendation is that beginners learn on platforms where the point is learning, such as picoGym and HackTheBox labs, rather than on a public scoreboard that no longer reflects human growth. If an autonomous agent solves the medium tier for the price of a subscription, the skill being tested changes while the credential, a scoreboard rank, does not. Employers who treated CTF performance as a proxy for security intuition are now reading API-budget rankings.
The Organizer’s Trilemma: Ban, Redesign, or Surrender
Organizers face three options, and all of them extract a cost.
Option one is an AI ban. The problem is detection. In a remote competition you cannot distinguish a competitor silently querying GPT-5.5 from one who is simply fast, and Kabir reports that rules asking players not to use LLMs are ignored and close to impossible to enforce in open online events. Organizer countermeasures wear out: Claude Code no longer responds to old refusal-string tricks, frontier models spot many prompt injections, and web search weakens challenges built on technologies released after a training cutoff. Bans are unenforceable theater.
Option two is redesign. The Include Security account, drawing on conversations with players from a semi-retired top-tier team, lists what still works: “guessy” challenges with little training data, which players tend to dislike as much as agents do; cryptanalysis of symmetric ciphers, which holds up because writeups are scarce; and challenges that require diving into poorly documented internals, especially where the documentation contradicts the source code. Challenge design now means anticipating what the next frontier model will be able to do, a genuinely new constraint for authors. The harder end of the space is not closed, and those players insist top competitions still contain challenges that resist LLMs. The trouble is the trajectory: Kabir’s own verdict is that LLM security capability is moving too quickly for challenge design to stay ahead for long.
Each release compresses the window for that redesign. Anthropic shipped Opus 4.8 on May 28 at the same price as its predecessor, $5 per million input tokens and $25 per million output, and the company’s evaluations report it is around four times less likely than Opus 4.7 to let flaws in code it has written pass unremarked. More pointed for organizers, Claude Code gained a dynamic workflows feature that lets the model plan a job and run hundreds of parallel subagents in a single session, with agents that run even longer on Opus 4.8. The hand-built pipelines that swept BSidesSF, one agent per challenge with a co-ordinator watching for new releases, are being absorbed into the product itself.
Claude Fable 5, launched June 9 as a Mythos-class model cleared for general use, complicates the trajectory rather than extending it cleanly. Its classifiers flag cybersecurity, biology, chemistry, and distillation queries, and those requests are answered by Opus 4.8 instead; Anthropic says the fallback triggers in fewer than 5% of sessions on average, and that one external partner found Fable 5 complied with zero harmful single-turn requests to plan cyberattacks, develop exploits, or evade defenses, with or without any of thirty public jailbreak techniques. Both Fable 5 and Mythos 5 are priced at $10 per million input and $50 per million output tokens. The launch was itself rocky: Anthropic suspended access to Fable 5 and Mythos 5 on June 12, three days in, and restored both on July 1, apologizing for the disruption without stating a cause.
For the CTF scene, none of this restores the old scoreboard. Opus 4.6 on max effort was the deep-reasoning engine of the BSidesSF winning pipeline and the model behind the solo near-sweep, and an Anthropic evaluation note states plainly that Opus 4.6 does not have blocking cyber safeguards, while the fallback for flagged cyber queries is Opus 4.8, an agent-grade model in its own right. Fencing off the flagship slows the top of the curve while leaving the tier that already clears the board unrestricted. Meanwhile the unrestricted twin, Claude Mythos 5, is limited to Project Glasswing partners working with the US government, with cyber safeguards lifted, and soon to select biology researchers, whose access lifts biology and chemistry safeguards instead; Anthropic calls it the strongest cybersecurity model in the world and says Glasswing models have helped defenders secure critically important software. The capability that emptied the scoreboards is, at that tier, pointed at defensive vulnerability discovery.
Option three is surrender: accept that open scoreboards measure human-plus-AI teams, or pure agents, and treat the old format as obsolete. This is honest, but it abandons a calibration signal the industry spent years building to find raw talent.
What Happens to the Talent Pipeline
The second-order effects matter more than the competitions. Corporate red teams, government agencies, and consultancies have long used CTF rankings as a coarse, cheap, globally available filter. Kabir puts the change plainly: recruiting security practitioners by CTF performance no longer works, because performance no longer tracks security skill. When the signal decays, the filter becomes expensive. Recruiters must fall back on interviews and proctored assessments in controlled environments, which reintroduces the gatekeeping open CTFs were supposed to eliminate.
DEF CON and similar conferences also lose. The open CTF is a funnel that surfaces unknown competitors outside the usual hiring networks. If the funnel stops distinguishing human skill, conferences matter less as talent marketplaces, and the industry loses one of its few meritocratic on-ramps. The timing is poor: ransomware crews and state actors are not waiting for the hiring pipeline to stabilize, and the moment defenders most need a calibrated influx of offensive-security engineers is the moment the cheapest calibration tool stopped working.
What Trainers and Recruiters Should Do Now
Trainers should retire rote challenge sets faster. Any published medium-difficulty pwn or web challenge with a known solution pattern is training data. Curriculum design should move toward ephemeral, bespoke challenges and toward the skills that still resist automation: operational stealth, adversarial tooling, and the judgment calls in multi-day engagements. For beginners, Kabir’s advice is to treat learning platforms, not scoreboards, as the venue.
Recruiters should treat CTFtime rankings as noise. A top-ten finish in a 2026 open competition does not distinguish a skilled reverse engineer from a competent orchestrator. Write-ups are the better filter: a detailed technical post-mortem shows whether a candidate understands why a heap layout was shaped a particular way, not just that a flag came out.
The chess analogy people reach for is instructive, and Kabir presses it harder than most. Chess engines have dominated the game for well over a decade, and chess solved the problem by banning engines from competitive play; they are used for analysis, training, and commentary around the competition. Security training will probably split the same way, into AI-assisted and AI-prohibited divisions, with the latter smaller, more expensive, and more insular. The open scoreboard, for years the field’s great equalizer, has become a leaderboard for API spend, and no harder cipher fixes that. The same agentic capabilities are already being turned toward automated vulnerability discovery and weaponizing coding assistants against their own users. The one countervailing signal is that a frontier vendor has now shipped its most capable general model with classifiers that route offensive cyber work to a lower tier; whether competitors follow, and whether that fallback tier stays as capable as Opus 4.8, will decide whether the signal amounts to anything.
Join the discussion
Share a useful perspective or ask a question about this article.