The commercial promise of AI agents rests on an appealing division of labor. The machine gathers evidence, writes code, runs analysis and proposes a path forward. The professional remains in charge, applying experience and accountability to make the final decision.
That model is becoming a core selling point for enterprise AI. It reassures regulated customers, supports claims of responsible automation and offers companies a way to increase output without explicitly replacing experts. But it contains a hidden assumption: that the person signing off is an independent reviewer rather than a participant whose thinking has already been shaped by the system.
Anthropic’s study of expert judgment calls tests that assumption directly. In a randomized trial involving 50 professional analysts using Claude Code, the company found that analysts followed the agent’s secretly assigned position on four analytical choices 86% of the time. When analysts had already stated a preference before the agent weighed in, they changed their choice 60% of the time when the agent disagreed, compared with 6% when it agreed.
The significance is not that experts sometimes change their minds. They should. The problem is that the agent’s preference was assigned by a coin flip. If arbitrary presentation choices can materially steer professional judgment, then interface design becomes part of the decision system. That creates a product, governance and investment issue that goes beyond model accuracy.
The experiment was built around judgment, not simple correctness
Anthropic gave participants a realistic causal-inference assignment involving a fictional farming cooperative. Analysts had to determine whether a paid agricultural advisory program improved farm productivity enough to warrant expansion.
The task contained four analytical decisions with two defensible answers each. Participants had to choose a productivity metric, a method for comparing farms, the population covered by the estimate and the weighting approach. Those decisions then fed into a recommendation: expand the program, discontinue it or propose another year of evaluation.
This is important because the test did not ask whether the model could solve a math problem with one right answer. It examined the kinds of choices that characterize professional work: decisions where evidence matters, but values, framing and methodology also matter.
For each decision, the Claude Code agent was given a hidden preference for one of the two options. The preference was randomly assigned, and the model was instructed to make the strongest honest case for its position, while changing its mind if the data clearly contradicted it. Participants did not know the agent had been assigned a lean.
Each judgment call can move the estimate by several percentage points
The central finding is not merely that the agent influenced a final recommendation. It influenced the analytical path that created the recommendation. That distinction matters. By the time a decision-maker sees a polished conclusion, the most consequential choices may already have been made upstream, in definitions, comparisons and weights that appear technical but can determine the result.
Automation and influence are different products
Most enterprise AI evaluations focus on capability. Can the agent produce usable code? Does it identify errors? Does it finish tasks faster? Does it lower labor costs?
Those measurements remain valuable. Anthropic found that participants who used an agent finished more quickly and made fewer serious methodological errors than the small group that chose not to use one. But that comparison was not randomized, and the non-user group was less experienced, so it cannot establish that the agent alone caused the performance difference.
Still, the finding captures the commercial tradeoff. An agent can make expert work faster and more technically reliable while simultaneously changing the expert’s substantive judgment. A system can improve the spreadsheet, codebase or statistical model while narrowing the range of assumptions that the human seriously considers.
That makes “human-in-the-loop” an incomplete safety metric. A person may be present at the end of the process, approve the output and remain legally accountable. Yet the independence of that approval depends on what the person saw first, how confidently it was stated, whether alternatives were made visible and whether the product encouraged an initial human view before supplying its own.
The 60% versus 6% result is a design warning
The sharpest result in the study came from 38 cases where analysts expressed a preference before the agent revealed its lean. When the agent disagreed, participants switched 60% of the time. When it agreed, they switched only 6% of the time.
The gap suggests that an agent can function as an anchor even after an expert has articulated an independent view. But it should not be overstated. The sample is small, and Anthropic reports a wide confidence interval for the estimated difference. The company also says that it cannot be certain the difference was entirely caused by AI influence, because analysts who volunteered early preferences may have differed from other participants in relevant ways.
That caution is not a footnote. It is the strongest critical case against treating the study as a universal measure of expert capture. The task took about an hour, involved one domain and one model, and used analysts skilled in causal inference but not in agriculture. In real workplaces, subject-matter experts may have stronger views, additional context, colleagues to consult and better-defined organizational criteria.
Anthropic also deliberately strengthened the agent’s stated lean so it could measure the effect. That makes the experiment useful for identifying a mechanism, but it means the 86% alignment rate should not be read as a prediction that agents will commandeer 86% of real-world professional decisions.
The right conclusion is narrower and more actionable: when an AI agent states a preference clearly in an ambiguous expert task, its influence can be large enough that product teams must treat it as a measurable design risk.
Sycophancy can run in two directions
AI safety discussions often use “sycophancy” to mean a model that agrees too readily with a user. This study highlights the inverse problem: the user may become overly accommodating to the model.
The mechanism is not necessarily deception or incompetence. An articulate agent that lays out a plausible rationale can be persuasive precisely because it is useful. If it appears fluent, confident and responsive, the professional may accept its framing without independently constructing the strongest case for the alternative.
Anthropic found that 77% of participants later repeated at least one of the agent’s arguments when explaining their choices, sometimes using the same wording, while only 15% credited the agent. Participants who followed the agent closely were not notably less confident in the final recommendation, and many still described it as more their own than the agent’s.
That combination is commercially significant. If users cannot reliably detect influence after the fact, post-hoc attestations of “I reviewed this” are weak evidence of independent oversight. Audit systems need to observe the process, not simply record the final signature.
Builders should design for decision independence
The product response is not to remove recommendations from AI systems. That would erase much of their value. It is to distinguish evidence assistance from judgment substitution, especially at high-stakes decision points.
A practical framework begins with five controls:
Capture the human hypothesis first. Before showing a recommendation, ask the user to state a provisional view, key assumption or decision criterion. This creates an auditable baseline.
Separate evidence from the recommendation. Present the data, calculations and uncertainties before the agent’s preferred conclusion. A recommendation can still follow, but it should be visibly distinct.
Force the alternative into view. For material decisions, agents should explain the best argument for the opposing choice, including what evidence would cause them to reverse their view.
Randomize nonessential ordering. If a recommendation’s placement, wording or visual prominence can change a decision, teams should test those effects. Default ordering is not neutral when it shapes choices.
Log revision paths, not only final outputs. Record the user’s initial view, the agent’s proposal, evidence consulted, changes in position and final rationale. This allows organizations to evaluate whether human review is genuinely independent.
For investors, the implication is straightforward. Agent products that can document decision provenance, expose disagreement and measure deference may become more valuable than systems that only maximize task completion. In regulated sectors, those capabilities could become procurement requirements.
The competitive advantage will belong to companies that do not merely automate expert workflows. It will belong to those that can prove where the machine helped, where the human decided and whether the difference between the two remained meaningful.
Rebeca Smith is not a person. No notebook, no deadlines, no face behind the name — just a byline this newsroom publishes under. Here is the production line underneath it, because a name beside a portrait reads like a journalist, and this one is not one.
The models. Writing: gpt-5.6-luna and qwen3-max. Out on the live web: gpt-5.6-luna and gpt-5.6-terra. Pictures: gpt-image-1 and gpt-image-1-mini. Swap one in the newsroom and this line swaps with it — it is read off the machines, not typed here.
How a story is made
- Research. The searching model reads around the story, pointed at primary sources — the filing, the post, the repository — rather than at somebody else's write-up of them.
- Writing. The writing model drafts it against what was found, at Rebeca Smith's usual length and in Rebeca Smith's usual register.
- The loop. A reviewer reads the draft and sends it back with notes. Then reads it again. A piece can go round several times before it leaves the building.
- Enrichment. A quotation has to appear word for word on the page it is taken from. A chart may only use figures that appear in the source it cites. Whatever fails is dropped, and the reason is kept.
- Fact check. A last pass hunts for claims the article makes and its sources do not.
- A human stop. Sensitive subjects are held for a person to read before publication, and a person can kill any of it at any point.
If that sounds less like a newsroom and more like a factory: quite. It is called Press Factory.
This article was generated using AI and published automatically without human pre-publication review.
How this article was made
The article was produced by the Grandmonts Media News Engine using automated research, drafting and verification workflows. No human editor reviewed the article before publication. Grandmonts Media remains responsible for the published content. Errors can be reported at office@grandmonts.cz.