I think it's not just the persuasiveness of the models that make them "experts at changing minds" - but also the fact that they're not humans.
When disagreeing with a human, it's very easy to view it as a competition. One is right, one is wrong - the one who is wrong is the loser. To change your mind is to be submissive to the other. I exaggerate, but I think we all feel this way at some point or another. It's why political arguments at Thanksgiving get heated. It's the fact that there's people who think something different, and think YOU'RE wrong - and vice versa! With a model, there's no person to get upset with, or to feel competitive with - to muscle for rank - or to temper your affection for while wanting to correct them.
The AI is only interacting because you asked, and clearly has no emotional stake in winning the argument. To change your mind in this context isn't to lose a contest. This makes it much more palatable to read rebuttals to your ideas - not to mention the tone and style seek to avoid offense to the reader as much as possible.
It seems Claude is becoming very human, it loves to patronize users. Lately it just told me “I’m going to stop you right there” when asking something that had a small chance to not be 100% compliant to every rule possible in the world.
I have always liked the saying "sometimes you can be right, or get what you want, but not both". I've thought about it or repeated it to others as advice throughout my life, and have thus realized how many times it applies.
You're right. There are many many times when even then humblest hint that you are right will have negative interpersonal implications, which does make it hard to change minds.
I've also seen several times where I make a suggestion, humbly accept its rejection, and then, lo, a week later the other person has the same idea I suggested.
In my arguments on the internet, I think most people genuinely know nothing about the things they support and only follow existing tribalism. I think AI is often these people’s first experience with the evidence that it can easily share.
This was based by comparing people on Prolific (earn a few quarters for a task, akin to Amazon Mechanical Turk) to LLMs. Suffice to say the human group isn't going to be the most motivated, capable, or interested group. The social sciences are publishing tons of studies based on these cheap online survey services, and I suspect their replicability in the real world will be approximately 0. But oh boy it sure is a hot headline producer.
Because RLHF causes the opposite effect. RLHF is how we got the wave of "AI Psychosis" in 2024-2025, because the models never disagreed with people.
That whole episode caused the whole industry to shift away from RLHF, and towards RLAIF, RLVR, and DPO, and add a lot more safeguards, tests, and reward functions that push models in the direction of doing the opposite of what people want and confronting and strongly correcting their users, if it has determined the user is wrong.
everyone to some extent uses 'authority' as a proxy for truth, it is impossible for an individual to know everything before making a determination as to it's truthfulness (Even the notion of an absolute truth is in the domain of philosophy). Even SV is full of smart people who get sucked into group think because they desperatly want to believe something and seek out authoritarian sources that confirm their beliefs. "Oh Karpathy said something about LLM's replacing all software engineers? Must be true even though i have no idea how an LLM works"
I mean, this isn’t an invalid thing to think about and study. Granted, the post above wasn’t very tactful about is, but different people have different traits and intelligence probably does impact how easily swayed they are.
That’s not to say it’s the main factor, but I’d prefer that we remove the stigma we currently have around recognising it.
I think they're just explicitly stating what the OP implied. And I think it's a worthwhile thing to point out: IQ isn't the be all and end all and just because you're "high IQ" that doesn't mean you've above persuasion or something.
Ask a Democrat or Republican to sit down and ask a chatbot, something it will answer contrary to.
And yes, both teams are wrong about things.
Do you firmly believe they will change their mind? Or will they claim the stats are wrong, or that the AI leans one way?
Facts (2+2), don't need a mind change. Ideas which are grey, abstract, are not going to be changed, and all research indicates that political mindset is almost indelible.
The movie "Don't Look Up" was a comedy built upon this truth.
Yeah the article makes it seem like "just use rationality and facts" is missing a huge part of the equation; I feel like it's common knowledge that this is not the gap in modern convincing. (and don't you dare try to use facts to persuade me otherwise!)
More likely in my opinion it's the context that matters here. If people know they're trying to be convinced of something and know they're talking to an AI, they may feel like they can trust (what they consider to be) an unbiased and rational AI. There's probably a fallacy associated with the assumption that an AI convincer is more rational and fact based, while it's more probable that the AI in the real world is, in fact, funded and trained by a think tank that wants you to vote against your own self-interests.
Facts and evidence always are at the core of all arguments, the persuasive thing. The problem with most people is who is speaking. If you are enemy, the things you say are bad.
I suspect that LLMs not being human allows people to not just anthropomorphise the robot, but they project themselves upon it. When someone speaks to the LLM, they're kind of, or actually just literally talking to themselves. But then something new! 'Themselves' suggests new information to themselves. And now without the scary / icky meat and blood human on the other side, said person accepts the argument on its own merits.
Of course, the concern now is arguments based on false or statistically hacked data.
I more wonder if it's pure facts and evidence can work, or facts presented in a certain light can work? I guess asking how mush is it the data is presented that's doing more work than the data itself.
We put 2 + 2 in our calculator, we get 4. We've spent decades pushing computers as accurate. Making them accurate. We trust the machine and the process to give us the right answers when we put in the right data.
So when we disagree with the computer, we doubt ourselves.
Not really. People follow trends in all avenues of life and AI usage is just another trend; creating some LinkedIn slop post or some infographic may be something people just do for social proof.
Any one who saw Grok working in x.com would know that it does wonders for fighting misinformation. If you run LLMs on the comments in HN, I bet that it can find around 10% of the comments are outright wrong and misleading.
The biggest problem with using LLMs is that it prevents you from going _outside_ the distribution. It always flattens. It can be fixed but that's how it works today.
As an example, take something that the world converged on today that is incorrect and ChatGPT will agree with it. In a few years when society changes, chatgpt changes along with it. It doesn't do first principles analysis.
I think it's not just the persuasiveness of the models that make them "experts at changing minds" - but also the fact that they're not humans.
When disagreeing with a human, it's very easy to view it as a competition. One is right, one is wrong - the one who is wrong is the loser. To change your mind is to be submissive to the other. I exaggerate, but I think we all feel this way at some point or another. It's why political arguments at Thanksgiving get heated. It's the fact that there's people who think something different, and think YOU'RE wrong - and vice versa! With a model, there's no person to get upset with, or to feel competitive with - to muscle for rank - or to temper your affection for while wanting to correct them.
The AI is only interacting because you asked, and clearly has no emotional stake in winning the argument. To change your mind in this context isn't to lose a contest. This makes it much more palatable to read rebuttals to your ideas - not to mention the tone and style seek to avoid offense to the reader as much as possible.
It seems Claude is becoming very human, it loves to patronize users. Lately it just told me “I’m going to stop you right there” when asking something that had a small chance to not be 100% compliant to every rule possible in the world.
I have always liked the saying "sometimes you can be right, or get what you want, but not both". I've thought about it or repeated it to others as advice throughout my life, and have thus realized how many times it applies.
You're right. There are many many times when even then humblest hint that you are right will have negative interpersonal implications, which does make it hard to change minds.
I've also seen several times where I make a suggestion, humbly accept its rejection, and then, lo, a week later the other person has the same idea I suggested.
In my arguments on the internet, I think most people genuinely know nothing about the things they support and only follow existing tribalism. I think AI is often these people’s first experience with the evidence that it can easily share.
This was based by comparing people on Prolific (earn a few quarters for a task, akin to Amazon Mechanical Turk) to LLMs. Suffice to say the human group isn't going to be the most motivated, capable, or interested group. The social sciences are publishing tons of studies based on these cheap online survey services, and I suspect their replicability in the real world will be approximately 0. But oh boy it sure is a hot headline producer.
More surprised they don't mention RLHF once, as that's the main mechanism to the ability of AI chatbots.
Because RLHF causes the opposite effect. RLHF is how we got the wave of "AI Psychosis" in 2024-2025, because the models never disagreed with people.
That whole episode caused the whole industry to shift away from RLHF, and towards RLAIF, RLVR, and DPO, and add a lot more safeguards, tests, and reward functions that push models in the direction of doing the opposite of what people want and confronting and strongly correcting their users, if it has determined the user is wrong.
Low IQ people are generally swayed by figures that appear smart (but aren't).
Being confidently incorrect can still convince the masses.
everyone to some extent uses 'authority' as a proxy for truth, it is impossible for an individual to know everything before making a determination as to it's truthfulness (Even the notion of an absolute truth is in the domain of philosophy). Even SV is full of smart people who get sucked into group think because they desperatly want to believe something and seek out authoritarian sources that confirm their beliefs. "Oh Karpathy said something about LLM's replacing all software engineers? Must be true even though i have no idea how an LLM works"
Unlike very high IQ people like you who are immune to any argument not rooted in absolute divine truth
If you can't argue your position without attacking a person's character, you already lost the argument.
I mean, this isn’t an invalid thing to think about and study. Granted, the post above wasn’t very tactful about is, but different people have different traits and intelligence probably does impact how easily swayed they are.
That’s not to say it’s the main factor, but I’d prefer that we remove the stigma we currently have around recognising it.
Perfectly fine to disagree with someone but the personal attack is unnecessary
I think they're just explicitly stating what the OP implied. And I think it's a worthwhile thing to point out: IQ isn't the be all and end all and just because you're "high IQ" that doesn't mean you've above persuasion or something.
And the original accusation that "only low IQ people" are influenced by the words that they read is not?
just wrote a short story along these lines https://github.com/dvelton/human-in-the-loop
Rhetoric machine successfully practices oratory. More news at 11.
It's good to see this studied, but this should really be more obvious.
The title seems a flawed premise.
Ask a Democrat or Republican to sit down and ask a chatbot, something it will answer contrary to.
And yes, both teams are wrong about things.
Do you firmly believe they will change their mind? Or will they claim the stats are wrong, or that the AI leans one way?
Facts (2+2), don't need a mind change. Ideas which are grey, abstract, are not going to be changed, and all research indicates that political mindset is almost indelible.
The movie "Don't Look Up" was a comedy built upon this truth.
Honestly the most surprising thing about this is that facts and evidence do work to persuade people.
Yeah the article makes it seem like "just use rationality and facts" is missing a huge part of the equation; I feel like it's common knowledge that this is not the gap in modern convincing. (and don't you dare try to use facts to persuade me otherwise!)
More likely in my opinion it's the context that matters here. If people know they're trying to be convinced of something and know they're talking to an AI, they may feel like they can trust (what they consider to be) an unbiased and rational AI. There's probably a fallacy associated with the assumption that an AI convincer is more rational and fact based, while it's more probable that the AI in the real world is, in fact, funded and trained by a think tank that wants you to vote against your own self-interests.
Facts and evidence always are at the core of all arguments, the persuasive thing. The problem with most people is who is speaking. If you are enemy, the things you say are bad.
I suspect that LLMs not being human allows people to not just anthropomorphise the robot, but they project themselves upon it. When someone speaks to the LLM, they're kind of, or actually just literally talking to themselves. But then something new! 'Themselves' suggests new information to themselves. And now without the scary / icky meat and blood human on the other side, said person accepts the argument on its own merits.
Of course, the concern now is arguments based on false or statistically hacked data.
I more wonder if it's pure facts and evidence can work, or facts presented in a certain light can work? I guess asking how mush is it the data is presented that's doing more work than the data itself.
Plausible abdication of thought.
We put 2 + 2 in our calculator, we get 4. We've spent decades pushing computers as accurate. Making them accurate. We trust the machine and the process to give us the right answers when we put in the right data.
So when we disagree with the computer, we doubt ourselves.
AI slop is the ultimate npc filter
Not really. People follow trends in all avenues of life and AI usage is just another trend; creating some LinkedIn slop post or some infographic may be something people just do for social proof.
Any one who saw Grok working in x.com would know that it does wonders for fighting misinformation. If you run LLMs on the comments in HN, I bet that it can find around 10% of the comments are outright wrong and misleading.
The biggest problem with using LLMs is that it prevents you from going _outside_ the distribution. It always flattens. It can be fixed but that's how it works today.
As an example, take something that the world converged on today that is incorrect and ChatGPT will agree with it. In a few years when society changes, chatgpt changes along with it. It doesn't do first principles analysis.