groundy
agents & frameworks

Vibe Coding vs Control: How Developers Actually Used AI Coding Agents

New data shows pro devs use AI coding agents as collaborators, not delegates. Weight review surfaces and checkpoints over autonomous run length when buying agent tooling.

10 min···4 sources ↓

Professional developers kept the wheel. A December 2025 study of AI agent use for coding found experienced developers run agents as collaborators rather than delegates: they retain agency over design and implementation, apply expertise-based strategies to control agent behavior, and hand off a task only when they judge it suitable. The title’s assertion, “Professional Software Developers Don’t Vibe, They Control,” is a qualitative finding about interaction preference; the measured evidence behind it is 13 field observations and 99 survey responses.1 The consequence for buyers: weight intervention surfaces over autonomous run length when you pick agent tooling.

What does control mean when developers run coding agents?

The paper’s wording is specific about what is retained and general about how. Experienced developers “retain their agency in software design and implementation out of insistence on fundamental software quality attributes,” deploying expertise-based strategies to control agent behavior. Control, in the study’s sense, means the developer keeps the design and implementation decisions, insists on quality attributes, and works strategies around the agent to keep both. What those strategies look like in practice, the abstract does not enumerate.

The contrast is with “vibe coding” as the term is commonly used: prompt, accept, ship, with the human reading the output lightly or not at all. The colloquial definition is a strawman, and empirical work on vibe coding by Sarkar and Drosos undercuts it: the vibe coders they observed still verify output iteratively, with expertise redistributing toward context management and rapid evaluation. Sarkar and Drosos describe trust in the tools as “dynamic and contextual, developed through iterative verification rather than blanket acceptance.” So the line between the two groups is not verification. What the study reports among professionals is retention of design and implementation decisions, plus a gate it treats as central: task-suitability judgment. Experienced developers decide per task whether an agent is worth the handoff at all, and that judgment, not enthusiasm, is what governs how much autonomy the agent actually gets.

How strong is the evidence behind “professionals don’t vibe”?

The evidence base is small, senior, and qualitative, so the study establishes stated and observed preference rather than productivity outcomes or telemetry across the broader user base. That is a narrower claim than the title implies, and keeping the two separate is the difference between using this paper and misusing it.

What the study actually measured, per its own findings: experienced developers value agents as a productivity boost but experience them as a source of collaboration rather than complete delegation, conditional on their judgment of task suitability. Collaboration over delegation is the measured result. “Professionals don’t vibe” is the paper’s editorial compression of that result.

What the study did not measure is equally important. There is no throughput comparison, no defect-rate comparison, no cost-per-merged-PR figure, and no test of whether controlled delegation outperforms hands-off autonomy on output. The paper cannot tell you control produces better code. It tells you that developers whose names are on the commits chose control. The authors frame their contribution the same way: alongside the value of software development best practices, the results “suggest the kinds of tasks for which agents may be suitable, and point towards future opportunities for better agentic interfaces and agentic use guidelines.” Every item on that list concerns matching tasks and building better interfaces; none concerns extending how long an agent runs unattended.

Which autonomy dials do the observed behaviors map onto?

Four control behaviors map onto four of the five decision axes that current coding agents actually expose: delegation scope, intervention surfaces, context steering, and review cadence. The fifth dial, unsupervised run length, is the one the preference data says experienced developers leave turned down. A buyer who lists these axes before a demo will notice which ones the demo spends its time on.

One row of the table is directly observed: collaboration over delegation comes from the preference study. Checkpointing, review loops, and context steering are this article’s operationalization of the paper’s “strategies for controlling agent behavior”; the abstract does not itemize them.

Control behaviorWhat it looks like in practiceTooling feature it demandsWhat it displaces
Collaboration over delegationAgent works inside a task the developer scopedScoped handoff with explicit boundariesEnd-to-end “take the ticket and come back” autonomy
CheckpointingState saved before each risky step, rollback kept warmCheckpoint and rollback surfacesTrust accumulated through longer unsupervised runs
Review loopsEvery diff inspected before landingDiff review and permission gatesAuto-applied or auto-merged changes
Context steeringDeveloper writes and maintains the instructions the agent works fromFirst-class instruction-file support and editingAgent-directed context gathering

The mapping matters because it converts a preference finding into an evaluation rubric. If professionals concentrate their effort on scoping, checkpointing, steering, and reviewing, then the features that serve those efforts are where tooling value accrues for this population. Features that extend how long an agent can run unattended serve a usage pattern the data says senior developers avoid.

Note the asymmetry in the table’s last column. Each control behavior displaces a specific autonomy feature, and in every row the displaced feature is the kind that demos well: the agent that works overnight, the PR that appears fully formed, the run that needs no permission prompt. Demo appeal and observed usage are pointing at different parts of the product.

Where does vendor marketing mismatch observed usage?

Vendor demos optimize for autonomy theater: an agent takes a ticket, works unsupervised, and returns a finished pull request. The usage evidence points the other way: the experienced developers in this study who use agents work the control surfaces, intervening rather than waiting, and the mismatch has a predictable commercial consequence. Product differentiation gets pushed toward review surfaces and away from unsupervised run length, because that is where practicing developers actually spend their time.

The economics compound. A feature that extends autonomous run length is a bet that users want to leave; a feature that improves diff review, permission gating, or rollback is a bet that users want to stay and intervene. If the paying population behaves like the developers in this study, roadmap dollars shift toward intervention, and the autonomy race becomes a marketing layer over a review-tools business. That reading is consistent with the data, though the study itself measures developers, not vendor roadmaps; the roadmap claim is analysis, not citation.

One boundary on this section: nothing here verifies how any specific product is configured out of the box.

The practical test for a team is to audit the defaults rather than trust them: for each of the five axes above, find the setting, note its shipped value, and decide whether that value matches how your senior developers actually want to work. The audit takes an afternoon. Skipping it means your rollout inherits whatever the vendor’s demo strategy wanted, which is precisely the part the usage data questions.

Isn’t unsupervised autonomy improving anyway?

Yes, and the improvement is real, but it is arriving in science domains and benchmark environments rather than professional coding practice. The honest reading of 2025-2026 research is that both things are true at once: autonomous long-horizon capability keeps advancing, and the professional developers who could use it keep their hands on the controls.

On the autonomy side, the MAPPS framework for materials-discovery agents reports a five-fold improvement in stability, uniqueness, and novelty rates over prior generative models on the MP-20 dataset, achieved by unifying planning, physics, and scientist input. MileGPO, a milestone-inference method for long-horizon LLM agents, posts state-of-the-art results on ALFWorld and WebShop with a small in-distribution to out-of-distribution gap on ALFWorld. Neither result is marginal; both indicate the trajectory is real.

The error is conflating those wins with coding. Materials discovery rewards exhaustive autonomous search because evaluation is expensive and the hypothesis space is vast. Benchmark environments have well-defined success states. Coding, by comparison, already has cheap verification: compilers, type checkers, and test suites give feedback in seconds. If verification cost were the binding constraint on agent autonomy, coding should have been the first domain to go hands-off, not the one where experienced practitioners build checkpoint-and-review rituals around every run.

The binding constraint appears to be judgment instead: what to change, what to leave alone, and whether the diff in front of you is secretly wrong in a way the tests don’t cover. That is what task-suitability judgment and review loops are for, and no ALFWorld score speaks to it. Buy your autonomy benchmarks where they were earned.

How should teams roll out coding agents, then?

Configure for controlled delegation from day one: tight task scopes, checkpoint and rollback habits, permission gates, and diff review treated as the unit of work, with explicit training on judging which tasks suit agent handoff. That is the configuration the usage evidence supports for tools in the Claude Code, Cursor, and Copilot family, and it is cheaper to establish as a norm before habits form than to retrofit after an autonomy experiment goes sideways.

A rollout that follows the evidence looks like this:

  1. Scope every handoff. Agents receive bounded tasks with explicit boundaries, not outcomes. The scope is written down, because unwritten scope is how a refactor becomes a rewrite.
  2. Checkpoint before every run. Rollback availability is a precondition for letting the agent act, established as routine rather than invoked after damage.
  3. Gate permissions deliberately. Decide which actions the agent can take without asking, and keep the list shorter than the demo suggested.
  4. Make the diff review the deliverable. Merged code is the output of a review, not of a run. Time for review is planned for, not scavenged.
  5. Train task-suitability judgment explicitly. The study’s experienced cohort acquired this judgment over years; a rollout that assumes juniors have it is exporting the study’s conclusions past its sample. Write down which task classes are handoff candidates and which are not, and revisit the list as the tools change. The direction of the inference matters: if the professionals with the strongest task-suitability judgment still work through agents as collaborators rather than delegating outright, teams with weaker judgment need those guardrails more, not less. Full-autonomy rollouts invert that logic, which is the strongest argument against them this evidence supports.

Verdict: buy the review surface, not the run length

When evaluating agent tooling, weight diff review, permission gates, and checkpoint/rollback above extended autonomous run length, because that is the side of the product the usage evidence favors, and train rollout cohorts in controlled delegation rather than full autonomy. The finding survives its own caveats well enough to act on: the N=13/N=99 preference research shows experienced developers steering rather than delegating.

The caveats are real and should travel with the recommendation: a small, senior sample, stated and observed preference rather than productivity, so the paper cannot tell you control beats autonomy on output, only that experienced developers chose control when their judgment was on the line. And autonomy research keeps advancing on other fronts, from materials discovery to long-horizon benchmarks, which means the right posture is a dated one: this is how professionals worked in 2025, in a paper last revised August 2026, not a permanent law of developer behavior.

That is enough to plan a rollout on. It is not evidence that autonomy would have done worse, and the field has not produced that comparison yet. Until it does, the developers with the most agent mileage are choosing control, and the tools worth buying are the ones that make intervention cheap.

Frequently Asked Questions

How do instruction files compare to API references in agentic coding sessions?

In 557 observed sessions, instruction files and working notes accounted for 60.5% of documentation interactions, while API references made up only 1.3%. This indicates that steering agents via custom context is far more common than consulting standard technical docs.

Is documentation consultation in agentic coding usually triggered by errors?

No, documentation consultation is self-initiated 70.2% of the time, compared to only 7.5% failure-driven instances. Developers proactively steer context rather than reacting to agent failures.

Does the preference for controlled delegation apply to junior developers?

The evidence base consists of 13 field observations and 99 qualitative survey responses from experienced developers, so the findings may not transfer to junior developers or hobbyists. The study captures stated preference in a senior cohort, not telemetry across the broader user base.

How does coding agent autonomy compare to materials discovery agents?

While MAPPS materials-discovery agents show a five-fold improvement in stability and novelty, coding agents remain under human control because coding has cheap verification via compilers and tests. The binding constraint in coding is judgment, not verification cost.

sources · 4 cited

  1. Empirical work on vibe codingarxiv.orgprimaryaccessed 2026-08-21