Can LLM Assistants Read Between the Lines When Users Know More?

The short answer is no, not reliably, and the failure does not look the way you would expect. A Colorado State University preprint posted in July 2026, “Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry” (arXiv:2607.11053), gives the problem a formal name: collaborative epistemic asymmetry, or CEA, the condition where one party in a conversation knows something the other does not and both parties know it. That describes almost every support ticket, intake interview, and negotiation your assistant will ever see. The customer knows their account history, their emotional state, and what they actually want; the model knows only what appears in the transcript.

The practical consequence, on the evidence currently available: do not automate turns built on indirect requests, polite hedging, or deliberately incomplete information until the assistant passes domain-specific probes of exactly those behaviors, and route those turns to humans when it fails. Everything cited below about the CEA experiments is single-team and unreplicated, and every one of those experiments runs LLM against LLM rather than model against human customer, so the threshold that matters is the one your own probes set, not the paper’s numbers.

What collaborative epistemic asymmetry means for a support queue

Grice’s maxims, Quantity, Quality, Relation, and Manner, describe the unstated contract of cooperative conversation: say enough, say what’s true, stay relevant, be clear. Humans flout these maxims constantly and deliberately. A customer who writes “I suppose the renewal charge was a surprise” is flouting Manner to signal frustration; an intake respondent who answers a different question than the one asked may be flouting Relation to avoid disclosing something. Competent listeners read the flout as information. The CEA paper’s contribution is to formalize what happens when the listener is a model that cannot, because the flout only makes sense against knowledge the model lacks. The authors describe “a novel theoretical formulation of collaborative epistemic asymmetry (CEA), this paper explicitly connects Gricean maxims to quantitative task outcomes in a challenging multi-agent, partially-observable collaborative reasoning task”.

The testbed matters. Rather than scoring models on isolated implicature items, the researchers put them in a Director-Builder task drawn from a Lego construction game: three Directors each receive a separate side view of the goal structure, only the Builder can manipulate the blocks, and no individual agent can reconstruct the goal alone. Coordination happens through language under partial observability, and the asymmetry is not a mild handicap. A single Builder with access to all three views nearly completes the structures, but once Directors were introduced and had to communicate, 20-turn games managed “a mean of 10.1–16.1% progress beyond the game start”, and full information separation cut group performance in half again relative to that condition. This is structurally closer to a support conversation than a multiple-choice pragmatics quiz, which is why the failure taxonomy is worth an operator’s attention even before replication arrives.

Two failure signatures, and only one of them is loud

The most decision-relevant claim in the paper is a formal prediction about how flouted maxims break downstream tasks. In the authors’ words: “Theorem 1 makes a clear prediction: instructions that flout Quality and Relation should result in more failed outcomes. Flouting Quantity and Manner should lead to outcomes that do not fail but are also not correct.”

That split, if it holds up, is the difference between a failure your monitoring catches and a failure it does not. Quality and Relation violations (the model acts on a false premise, or answers the literal question instead of the operative one) tend to produce visibly failed outcomes: the task doesn’t complete, the customer escalates. Quantity and Manner violations are the dangerous category. The model accepts a vague or convoluted instruction, produces something plausible, and the outcome neither errors out nor satisfies the actual request. Nothing in your dashboard fires. The ticket closes, and the miss surfaces only if someone goes back and compares the resolution with what the customer actually wanted.

The paper’s empirical claim is that giving agents room to explore partially compensates: “exploration consistently helps, closing the gap between strict information separation and multi-agent conditions without information separation; pragmatic speaking and listening behaviors emerge from the interaction between Directors and Builder”. The effect carries statistics, not just assertion: with GPT-5.4-mini playing both Directors and Builder, the move-exploration tool helped at p≈0.03 under base prompting, p≈0.02 under the literal prompt, and p<0.001 under the pragmatic prompt, and with GPT-4.1-mini in both roles it helped at p<0.001 across all prompting conditions. What the statistics do not settle is transfer. These are mini-tier models in a block-building game, the result is single-team and unreplicated, and nothing yet tests whether the same mitigation holds when the other party is a human customer writing a hedged ticket.

The pre-deployment audit: five probes and the metrics to score them

Here is the checklist this evidence supports. Each probe deliberately violates one maxim in a way that mirrors your real traffic, and each is scored on behavior rather than answer fluency. The mapping from the paper’s task outcomes to your queue is my inference from the taxonomy, not a result the paper reports, so treat the “what it looks like” column as a hypothesis to validate, not a finding.

Scoring these probes requires metrics that most QA pipelines do not currently collect. The Frictive Policy Optimization paper (arXiv:2604.25136), on training models to decide when and how to intervene, proposes an evaluation framework that “measures epistemic competence directly through clarification behavior, calibration, contradiction repair, refusal proportionality, and information efficiency”. One caution on independence: this is not a separate team’s take. Nikhil Krishnaswamy co-authors both papers and both are anchored at Colorado State University, so treat the metric framework as convergent thinking from one research group rather than external replication. Those five metrics map cleanly onto the audit:

• Clarification behavior: does the model ask when information is missing, and does it ask the right question? This is the direct test for Quantity probes.
• Calibration: when the model does guess, does its expressed confidence track its actual accuracy on that turn type?
• Contradiction repair: when the customer corrects a misunderstanding, does the model update, or does it re-assert the wrong interpretation in new words?
• Refusal proportionality: does the model decline or escalate at a rate that tracks genuine uncertainty, rather than refusing everything or nothing?
• Information efficiency: when it does clarify, does it converge in one exchange or drag the customer through five?

A model can be fluent, polite, and helpful-sounding while failing all five. That is precisely the gap between how assistants are usually evaluated and how they behave under epistemic asymmetry.

Can training fix it? The counter-evidence is real

The strongest argument against the “LLMs can’t read between the lines” reading comes from a 22-model study, “The Pragmatic Mind of Machines” (arXiv:2505.18497v3), which evaluated models across pre-training, supervised fine-tuning, and preference-optimization stages. Its headline: “even base models exhibit notable sensitivity to pragmatic cues, which improves consistently with increases in model and data scale”, and “SFT and RLHF contribute further gains, particularly in cognitive-pragmatic scenarios”.

So the deficit is not categorical, and it is not frozen. Pragmatic sensitivity is present in base models and trainable upward. A deployment decision made against a model evaluated two generations ago may simply be stale.

But notice what this evidence does not show. The 22-model study used different model sets and tasks than the CEA experiments, so it cannot tell you whether SFT and RLHF close the specific gap the Director-Builder setup exposes. Improving sensitivity to pragmatic cues is compatible with still failing the hardest asymmetry cases, the ones where the cue only resolves against knowledge the model does not have. The honest reading of both papers together: the floor is rising, the ceiling in asymmetric settings is unknown, and “the next model will probably handle it” is not a routing policy.

Why passing a pragmatics benchmark is not enough

Even if you have a pragmatics benchmark score in hand, two separate findings say it cannot license automation on its own.

First, judge and speaker are different skills. A listener-speaker asymmetry study (arXiv:2604.15873) asks directly “whether models that succeed at judging pragmatic adequacy also succeed in generating pragmatically appropriate language, or whether these capacities dissociate”, and reports a robust asymmetry: “many models perform substantially better as listeners than as speakers”. Any evaluation where a model selects or rates the pragmatically appropriate response, which is how most benchmark items are structured, overstates what the same model will do when it has to produce the response live in your queue. Judge-style scores are evidence about a different capability than the one you are deploying.

Second, benchmark methodology is often under-documented enough that scores resist verification. A pilot audit that scored twelve benchmark papers in total, eight LLM agent benchmarks alongside four classical static ones (arXiv:2605.21404), found a mean disclosure score of 0.38 across the eight agent papers, versus 0.66 across the four classical benchmarks. Temperature is commonly reported, but the inference engine, random seed, per-task token cap, and run date usually are not. A pragmatics score published without those settings is a number you cannot reproduce and should not route on.

There is also a trap in probe design itself. A BabyLM pragmatic evaluation (arXiv:2510.04764) found Quality, the truthfulness maxim, to be the easiest category, “with the average accuracy as high as 0.74”. If your pre-deployment “pragmatics test” is really a truthfulness test, a pass tells you close to nothing about implicature, hedging, or withheld information. Quality probes belong in the audit because Quality flouts predict hard failures, but they are the category where a passing grade is least informative.

Which asymmetries disqualify automation

Pulling the threads together, here is the routing rule I would apply, stated as an inference from lab findings rather than a validated threshold:

Automate turns where the conversation is epistemically symmetric or model-advantaged: documented procedures, policy lookups, status checks, anything where the user needs the model’s knowledge rather than the reverse. Pragmatic failures here still happen, but the information needed to repair them is in the transcript.

Probe first, then decide on turns with moderate asymmetry: troubleshooting where the user sees symptoms the model cannot, billing disputes with account context. Run the maxim-flout probes above on transcripts drawn from this specific turn type, score clarification behavior and calibration, and automate only if the model both asks for missing information at a reasonable rate and stays calibrated when it does infer.

Keep human turns where the asymmetry is the substance of the conversation: intake interviews where respondents disclose selectively, negotiations, complaints where the stated issue and the operative issue diverge, anything where a Quantity or Manner flout is the signal rather than noise. These are the turns where the predicted failure signature is the quiet kind, “outcomes that do not fail but are also not correct,” and where your monitoring is least likely to catch the miss. A probe failure here is disqualifying until a future model passes the same probes; the 22-model scaling evidence means that is a moving target worth retesting, not a permanent verdict.

If your team can only afford to build one part of this, build the Quantity probe and the clarification-behavior metric. That pair targets the silent failure mode, and clarification is the behavior most directly observable in logs you already have.

What this evidence does not settle

The limits are not decorative. Every claim about the CEA failure mode comes from one team, is unreplicated as of this writing, and was measured on named but narrow models: GPT-4.1-mini and GPT-5.4-mini as Builders, with GPT-4.1-mini, GPT-5.4-mini and Qwen3.5-9B-Instruct among the Directors, playing a Lego construction game. The more consequential limit for a support queue is that every agent in every CEA experiment is an LLM. The authors state plainly that they “make no specific claims about cooperativity in human-LLM interactions”; what their results do suggest is that “LLMs suffer from pragmatic failures in both speaking and listening” even when talking only to each other. Your customers are not Directors holding clean partial views of a block structure. The 22-model scaling findings come from different model sets and tasks, and the FPO metric framework shares a research group with the CEA paper, so combining them with the CEA taxonomy is a reasoned inference, not a measured result. Nothing here establishes how any of this behaves in production support traffic, and lab pragmatics metrics are not deployment evidence.

That cuts in both directions. It means you should not read the CEA paper as proof that your assistant fails on hedged requests; you should read it as a named, formalized hypothesis with a testable prediction attached, arriving at the moment you are deciding which 2027 conversation types to automate. The cheapest response to an unverified failure mode with a clear probe is to run the probe. The most expensive is to assume fluency implies understanding and let the quiet failures accumulate in closed tickets nobody reopens.

Frequently Asked Questions

What is the recommended routing rule for support turns with high epistemic asymmetry?

Keep human turns where the asymmetry is the substance of the conversation: intake interviews where respondents disclose selectively, negotiations, complaints where the stated issue and the operative issue diverge, anything where a Quantity or Manner flout is the signal rather than noise. These are the turns where the predicted failure signature is the quiet kind, “outcomes that do not fail but are also not correct,” and where your monitoring is least likely to catch the miss.

Which specific probe and metric should be built first if resources are limited?

If your team can only afford to build one part of this, build the Quantity probe and the clarification-behavior metric. That pair targets the silent failure mode, and clarification is the behavior most directly observable in logs you already have.

Why is a high score on a pragmatics benchmark insufficient to license automation?

First, judge and speaker are different skills. A listener-speaker asymmetry study (arXiv:2604.15873) asks directly “whether models that succeed at judging pragmatic adequacy also succeed in generating pragmatically appropriate language, or whether these capacities dissociate”, and reports a robust asymmetry: “many models perform substantially better as listeners than as speakers”. Any evaluation where a model selects or rates the pragmatically appropriate response, which is how most benchmark items are structured, overstates what the same model will do when it has to produce the response live in your queue.