Anthropic bans abusive or cruel behavior toward Claude

Summary

Anthropic updated its usage policy for the first time in over a year. The company incorporated a clause prohibiting 'persistent and unnecessary mistreatment or cruel behavior' toward Claude, while also expanding restrictions on propaganda, surveillance, and weapons development.

Why it matters

The practice of ending chats with users who mistreat Claude, previously an experimental research feature, has now become official policy. This move is read as a signal of Anthropic's commitment to its controversial 'model welfare' framework. Meanwhile, the tightened rules on election interference, autonomous weapon software, and non-consensual monitoring serve as a response to documented patterns of misuse observed by the company. The policy directly pits Anthropic against rivals like Microsoft, which explicitly reject the concept of AI welfare.

Highlights

• Ending the chat remains the 'primary enforcement mechanism'; user bans or other penalties have not been confirmed.
• The weapons ban now explicitly covers control software, components, and the arming of drones or autonomous vehicles.
• Anthropic states that restrictions for government contracts can be modified if it deems the security measures in the contract 'sufficient'.

Critical take

The penalty applied by Anthropic for behaviors it defines as 'excessive' cruelty is much softer than expected. Ending the chat is the main consequence, and the company leaves questions about heavier penalties unanswered. More concerning is the exception that allows Anthropic to unilaterally deem security measures in a government contract 'sufficient.' This clause effectively renders surveillance and weapons bans non-functional for the actors most likely to exploit such authorities.