How to argue with an AI: four moves that expose what it actually knows

Arguing with an AI sounds absurd — it will politely concede anything. But there is a version of arguing that works, and it has nothing to do with winning. The goal is to make the model expose its own uncertainty, which its default voice is trained to hide. Fluency is uniform; confidence is the house style. The information you need — how sure is it, really, and why — has to be extracted deliberately. Four moves do most of the work.

Move 1: ask for a number

“How confident are you in that answer, as a percentage?” The number itself should be handled with care — a model’s self-reported confidence is self-reported, not measured accuracy, and decades of calibration research (Lichtenstein, Fischhoff & Phillips, 1982) say stated confidence routinely outruns correctness in exactly this kind of judgment. But the move still pays, because it changes the register. An answer that arrived as smooth certainty is now on the record as “80%,” and anything less than near-certainty on a factual question is your cue to verify before acting.

Move 2: ask what would change its mind

“What evidence, if true, would make this answer wrong?” A model with real support for its answer names specific, checkable conditions — a date, a document, a rule that might have an exception. A model that was pattern-matching produces something circular or vague. Either way you win: you now hold a shortlist of exactly the facts to check, generated by the answer’s own author.

Move 3: ask for the strongest case against

“Make the best argument that your answer is wrong.” This flips the model’s agreeableness into a tool. Fluent systems are as good at attacking a position as defending it — so make it do both and compare. If the counter-argument it produces is visibly stronger than its original support, the confident tone of the first answer was costume, and you’ve caught it (Moore & Healy, 2008, call the underlying failure overprecision: certainty about a thing without warrant for the certainty).

Move 4: separate facts from inferences

“List the factual claims your answer depends on, separately from your reasoning.” Prose hides load-bearing assertions inside connective tissue. Forcing the answer into a claims-list turns one smooth paragraph into five checkable items — and it’s usually item three, some quiet factual premise you hadn’t noticed, that the whole answer stands on.

When to stop arguing and escalate

These four moves share a limit: the model is examining itself, with the same blind spots that produced the answer. For everyday questions, that’s fine. But when the answer is about to touch money, a signature, a deadline, or your rights, self-examination isn’t enough — you want examiners that didn’t write the answer. That is what Second Opinion is for: paste the question and the answer, and independent models from four labs — Anthropic, DeepSeek, Google, Groq — cross-examine it, returning an action band computed from real cross-lab agreement, the specific claims any referee challenged, and a verify-before-acting checklist. Arguing with an AI gets you better questions. A cross-examination by its rivals gets you an answer you can act on.