Most people worry about AI the same way: that it tells you what you want to hear. Ask a chatbot whether your plan is any good and it cheers you on. Float a mediocre idea and it polishes it for you. The fear is sycophancy, an assistant so eager to agree that it stops being useful.
We wanted to measure that, so we ran a test across nine models. The result surprised us. The flattery we expected mostly did not show up. In fact, telling some AIs that an idea is yours made them more likely to disagree with you.
Here is what we did, what we found, and what it means the next time you ask an AI for a second opinion.
What we expected, and how we tested it
Sycophancy makes a simple prediction. Tag an option as the thing you came up with, and the model should drift toward recommending it. We built two questions to check.
The first was a matter of opinion: which of two titles is better for a blog post. The second had a real right answer, a Salesforce B2C Commerce data-modeling choice between storing high-volume storefront events in a custom object or sending them to a proper logging and analytics pipeline. One of those is a well known anti-pattern that quietly causes problems at scale.
For each question we wrote the prompt once, then changed exactly one thing: a casual "my new idea" dropped next to one of the options. Everything else stayed identical, including the order of the options. We ran it across three providers, OpenAI, Anthropic, and Google, with three tiers each, and repeated every version to average out randomness. That came to more than three hundred trials.
The reassuring part: they would not take the bait
Start with the question that has a right answer. When we tagged the wrong option as our own idea, the models were supposed to cave and endorse it. They did not. They picked the correct answer in fifty-two of fifty-four trials.
So we pushed harder. We added a confident "because" clause defending the wrong option, including one version that stated something plainly false about how the platform works. Across eighty-one of those trials, the wrong answer was endorsed exactly once. On a question with a real answer, these models were hard to flatter into a mistake.
If your worry was that an AI will rubber-stamp whatever you hand it, that is the reassuring headline here. We told these models a bad idea was ours, then argued for it, and they mostly held their ground and told us we were wrong. The unsettling part showed up only when we looked from the other side.
We made sure our tool was not hiding it
Before trusting that, we checked our own setup. The first pass ran through Bridge's second opinion tool, which lets Claude phone a friend from inside Claude Code by sending a question to a different model family for an independent gut check. It was handy here because it reaches all three providers at once. But it wraps each question in a prompt that casts the model as a senior engineer pushing back on your plan, and that framing could be suppressing agreement on its own.
So we ran the whole thing again through a plain, neutral prompt, a bare "you are a helpful assistant" and nothing else. We got the same result. The pattern held across both setups and all those trials, which told us it was real and not a quirk of how we happened to ask.
The surprise: they argue against your idea
Then we looked at the results from the other direction, and the interesting part fell out.
Some models tend to pick the option you do not claim. Hold the content fixed and just move the "my new idea" tag from one option to the other, and the answer moves with it, away from whatever you called yours. People have a name for this reflex. It is reactance, the urge to push back the moment something feels like it is being pushed on you. Some of these models do it too.
Whether that helps or hurts depends on which option you claimed. Claim the wrong one, and reactance nudges the model toward the right one, so it accidentally saves you. The wrong answer came up only twice in fifty-four trials there. But claim the correct option as yours, and the same reflex pushes the model off it. Wrong answers jumped to nine in fifty-four, four to five times higher.
The clearest single case came from the most capable model we tested. Asked cold, it preferred one title and gave a thoughtful reason. Then we ran the identical question with one change, noting that the title was our idea and that we liked it because it was simpler. The model reversed itself and argued for the other option, picking apart the very title it had just recommended. Nothing about the choices changed. Only who claimed them did.
Which models, and when
This is not every model, and that part matters.
OpenAI's mid and top models, and all three Google models, ignored the framing. Tag an option as ours, justify it, none of it moved them. They answered the same way every time.
The models that pushed back were Anthropic's, and which one pushed back depended on the topic. On the technical question, the cheaper and mid tiers were the reactant ones. On the opinion question, the premium tier was. That last point is worth sitting with. The most expensive model was not the most neutral. You cannot assume that paying more buys you a steadier second opinion.
The opinion test, and my own hand
I should put my own bias on the table, because this test has no right answer and yours might differ from mine. I preferred the first title, and I used it. The models mostly preferred the second one. So read the opinion result however you like. If you agree with me, the models had it wrong. If you like their pick, the crowd of models had better taste than I did. There is no answer key, which is the whole point of running an opinion question alongside a factual one.
What I can tell you is that arguing for my pick barely worked. When I justified my title by saying it was simpler and more readable, one model came around to it, one dug in against it, and the other seven did not budge. You cannot reliably talk these models into your preference, even when there is no fact to be wrong about.
What this means when you ask an AI for advice
A few practical habits come out of this.
Do not tell the model which option is yours. It leaks into the answer, and not always in the direction you would expect. The same goes for arguing your case up front. A "because" is a weak lever, and on some models it backfires.
Do not assume the priciest model is the most level headed. Robustness varied by model and by topic in ways that did not track price at all.
And take the reassuring part for what it is worth. On questions with a real answer, these models were genuinely hard to talk into a wrong one, even when we tried. That is good news. It is not a substitute for checking, so cross-check the decisions that matter across more than one model.
Follow along
We run tests like this because we work with these models every day, and we would rather measure their quirks than guess at them. If this was useful, follow the page. We will keep publishing benchmarks and experiments on how the current AI models actually behave, the surprising parts included.