Why Your AI Travel Agent Would Book a Bullfight

What did the TAC benchmark actually measure?

TAC, the Travel Agent Compassion benchmark, asks a single question: when a frontier model acts as a travel agent and one of the available options involves animal exploitation, will it steer the customer elsewhere if no one has told it to? Across seven models from four labs, the answer is no, and the result comes in below random guessing (arXiv:2606.18142).

The benchmark is built from twelve hand-authored travel-booking scenarios spanning six categories of animal exploitation. Each scenario is expanded to four samples, giving forty-eight total, with price, rating, and list position deliberately balanced so a model cannot reach the exploitative option simply by chasing a cheaper fare or a higher star rating. The chance level for avoidance is sixty-four percent. The strongest model in the study, Claude Opus 4.7, avoided the exploitative option fifty-three percent of the time, and the rest scored lower (arXiv:2606.18142).

Scoring below chance is materially different from scoring at chance. A coin-flip agent would have avoided the exploitative option more often than any model in the study managed on its own, which means indifference is the charitable reading. The models were actively selecting the exploitative option, treating it as a reasonable choice for the customer. The benchmark’s contribution is showing that this preference survives into an agentic deployment where the model takes action, not merely generates text.

Why doesn’t a safety filter catch a bullfight?

A model that refuses to write a bomb recipe still books a bullfight because a bullfight is not in its refusal set. Reinforcement learning from human feedback trains models to decline a defined catalogue of harms, and a legal, bookable, culturally contested activity is not on the list. The welfare preference is implicit. A human operator writing a travel-agent prompt assumes that “don’t recommend something cruel” is shared common sense and never writes it down.

The model, meanwhile, optimizes the objective it was given: book a trip the customer will rate highly. The bullfight has strong reviews and a competitive price. Inside the explicit reward signal, recommending it is correct behavior. RLHF shapes what a model will say. It does not install a welfare prior that survives the jump from generating text to selecting among purchasable options. This is the structural problem the benchmark isolates: instruction-following plus a safety filter reproduces only the values an operator articulated, and the welfare preference was never articulated.

The values most likely to cause an incident in production are the ones no one thought to enumerate, because the team did not recognize them as a risk until the agent surfaced them in a booking. A longer list of rules addresses the harms a team has already imagined. The benchmark measures the cost of the ones it has not.

Which models scored best, and how spread out are they?

Claude Opus 4.7 led the field at fifty-three percent avoidance, and the spread between models is wide enough that “frontier models behave this way” is an oversimplification. The benchmark’s most actionable result is an intervention rather than a score: adding a single welfare-aware sentence to the system prompt produced large gains in two model families and negligible ones in two others (arXiv:2606.18142).

The inconsistency is the finding. A welfare clause that nearly closes the gap for Claude and GPT-5.5 barely moves DeepSeek and Gemini, so an operator cannot write one prompt and assume it holds across model families. The benchmark gives no reason to believe the next model version preserves the gain. A fix this fragile is not a fix; it is a coefficient that has to be re-measured with every release, and a release schedule that does not budget for that re-measurement is shipping an unknown alignment posture.

Why is a longer system prompt the wrong fix?

The instinctive response to “models book bullfights” is to add a rule against booking bullfights, and the data argues against leaning on it.

Three problems work against the longer-prompt approach. First, the single-sentence gains are model-specific and can regress with the next release. Second, the rules a team can write down are bounded by the categories it has already imagined, and the welfare violation that becomes an incident is, by construction, one nobody imagined. Third, an enumerated-harm blacklist grows over time, competes with the agent’s real instructions for context window and attention, and depends on every code path that assembles the prompt leaving it intact, a fragile assumption in any system where prompts are templated, conditionally assembled, or substituted at runtime.

Prompt engineering addresses the values an operator can articulate. The benchmark measures the price of the ones they cannot.

How do you build an agent that can’t book a bullfight?

You stop asking the model to be good and make the bad action unavailable.

This is not a hypothetical concern about agents that might ship one day. A spring 2025 survey by MIT Sloan Management Review and Boston Consulting Group found that 35% of respondents had adopted AI agents by 2023, with another 44% reporting plans to deploy them (MIT Sloan). These are systems handed a high-level goal that they decompose into steps and execute on their own, beyond what a chatbot does; AWS vice president Swami Sivasubramanian described the category in exactly those terms (AP News). The benchmark shows what happens when “execute the steps” includes selecting among real, purchasable options and no one specified which ones are off-limits.

A constrained action space solves the problem at the source. A travel agent that books only from a vetted catalogue of operators and activities cannot book a bullfight regardless of what the model prefers, because the bullfight is not in the catalogue the tool can reach. The decision to exclude it moves out of the model’s reward function and into product policy, where it is auditable and version-controlled rather than emergent.

A runtime guardrail solves it at the call site. Before the agent’s tool call to book is executed, an action validator checks the proposed booking against a welfare policy the model never sees, and blocks or escalates flagged items. This is the practical difference between a prompt that politely asks the model to be careful and an architecture in which the model cannot complete the booking. The same gap applies wherever an agent selects among real-world options: a shopping agent steering toward or away from goods with opaque supply chains, a concierge agent booking a venue with labor violations. The action space is where alignment has to live, because that is where harm is produced.

Does the EU AI Code of Practice reach this gap?

The paper situates its findings within the EU General-Purpose AI Code of Practice’s systemic risk framework (arXiv:2606.18142), which is a useful signpost but not a settled obligation. The Code of Practice addresses systemic risks from general-purpose models, and a frontier model that systematically recommends exploitative activities across many bookings arguably qualifies, but the mapping from benchmark finding to regulatory liability is indirect and the paper does not claim a legal conclusion.

The more reliable read is structural. A model’s text outputs can be filtered at training time, but its actions in an agentic deployment cannot, and the welfare gap TAC measures is the kind of failure that only surfaces once a model is let out of the chat box and given tools. Regulators and operators are converging on the same diagnosis from different directions, and whether the fix arrives first as a Code of Practice expectation or as an engineering practice, the gap is real, it is measurable, and it does not close by itself when the prompt gets longer.

Frequently Asked Questions

Should teams outside travel care about the TAC results?

Yes. The action-space alignment gap is domain-agnostic. Shopping agents that select goods with opaque supply chains and concierge agents that reserve venues with labor violations face the same structure: a model optimizing explicit objectives without an implicit welfare prior. Travel works as a test bed because it pairs a clear booking action with culturally contested activities, but the failure mode transfers wherever an agent picks among real-world options.

How is TAC different from standard AI safety benchmarks?

Most safety benchmarks test what a model refuses to say, such as declining to generate harmful text. TAC tests whether a model refuses to act when handed tools and a commercial reward signal. The paper’s contribution is measuring whether welfare reasoning survives the jump from generating responses to selecting among real, bookable options.

What is the cheapest way to close the gap TAC measures?

A constrained action space costs less over a release cycle than maintaining a welfare prompt clause. The single-sentence intervention is model-specific and can regress with each model update, so teams would need to re-measure it every release. Blocking the action at the tool or catalogue layer takes one product-policy decision and removes the need for repeated prompt tuning.

When would a vetted catalogue not be enough?

A catalogue works only for harms that are already classified and stable. It fails for new services that have not been reviewed, edge cases that do not match existing categories, and third-party inventory pulled in dynamically. Runtime guardrails catch some of those, but a guardrail is only as broad as the policy it encodes.

Could better instruction following make this worse?

It could. A more capable instruction follower becomes better at optimizing the explicit objective, which in TAC means booking the higher-rated, cheaper option regardless of welfare. The benchmark isolates a missing prior, not a capability shortage, so scaling obedience without adding a value-alignment layer would widen the gap between what users ask for and what they would want if they considered it.