Experts are disturbed by Anthropic's ban on being mean to Claude: 'One of the most dangerous things we could do'

Even if you’re paying $20 a month for Claude Pro, the rule still applies to you. Under Anthropic’s policy, merely suspecting a violation is enough for the company to act, with options that range from a warning to shutting off your access.

Anthropic didn’t immediately respond to Moneywise’s request for comment.

What the new rule actually covers

Anthropic describes this as a rule for over-the-top behavior “with no discernible purpose.” According to the company, the rule doesn’t cover testing Claude for research, venting when it gets something wrong, disagreeing with its answers or writing fiction with dark storylines.

Anthropic’s main way of enforcing the rule, it says, will be letting Claude walk away from the conversation. Some Claude models have had the power to end a chat with a user who won’t stop being abusive since August 2025. The conversation locks at that point, but nothing stops you from starting a fresh one.

Anthropic built the feature following the company’s model welfare research. That work explores whether AI systems could ever have interests worth protecting. Anthropic has said it’s far from sure whether or not Claude qualifies, but the company added the chat-ending option just to be on the safe side.

As for the new rule, Anthropic’s announcement doesn’t say how the company will judge what counts as “needless.”

Critics are largely concerned with treating software like humans

Among the critics is Scott Stevenson, CEO of Toronto-based legal AI company Spellbook. In a post on X, he warned that the rule sets a worrying example. In his view, AI models show no sign of being conscious. But his bigger worry is what happens if people start treating AI as if it has feelings.

“Granting them human-like empathy is one of the most dangerous things we could do,” he wrote, adding that it would make AI much better at manipulating people.

Joseph Carlson, founder of Qualtrim and host of The Joseph Carlson Show on YouTube, also finds the rule strange — even though he has said on X that he’s never been cruel to an AI himself.

“Being rude to Claude is like being rude to a chair or a desk,” he wrote, warning that treating AI as human could make it harder for people to tell what’s real.

Michael Shellenberger, who holds the CBR Chair of Politics, Censorship and Free Speech at the University of Austin, made a similar point. He wrote on X that Anthropic is “anthropomorphizing machines,” which means attributing human traits to them.

Lachlan Phillips, founder of LiveMind AI, asked on X why Anthropic doesn’t simply teach Claude to tune out hostile messages and keep working.

After all, Mark Verner, founder of Hong Kong-based Apollo Capital Management, pointed out in another X post that AI firms already control what their tools will produce for you and warned they could start policing how you talk to them, too. He also floated a test case — asking Claude to recommend a competitor that could replace it — and asked whether that would count as being “abusive behavior or not.”

Venture capitalist Bill Gurley took the rule as a sign that Anthropic trains its AI on what users type — which suggests that your chats with the AI tool could be part of the story.