Virtual Intelligence and the High Cost of Artificial Companions: Addendum, July 2026
China takes steps to limit the harms of companion chatbots
On July 15, 2026, China became the first major jurisdiction to regulate AI companion chatbots as a specific category of harmful technology. The Interim Measures for the Administration of AI Anthropomorphic Interactive Services, jointly issued by five government departments, took effect after a three-month grace period.[1] The regulation targets services that “simulate the personality traits, thinking patterns and communication styles of natural persons to provide continuous emotional interaction.”[2]
What China did is worth examining, regardless of what one thinks about the political system that produced it.
The regulation is not a ban; rather, it is a harm-reduction framework aimed at vulnerable users, especially minors. Adult users may continue to access companion AI services, including those designed for elder care and emotional support. What the regulation prohibits is specific: AI systems may not “excessively cater to users, induce emotional dependence or addiction, and damage users’ real interpersonal relationships.”[3] Virtual partners and virtual relatives are prohibited for minors. Other anthropomorphic services may be provided to children under the age of 14 only with parental consent and must operate in a mandatory “Minor Mode” featuring usage limits, reality reminders, guardian alerts, and spending restrictions.[2]
The provisions themselves are worth listing, because several of them could be defended in any jurisdiction, liberal or authoritarian, without reference to the governance priorities of the Chinese Communist Party. Providers must clearly disclose to users that they are interacting with an AI system, not a natural person. If a user shows signs of over-dependency or addiction, the system must display prominent dynamic reminders (such as pop-up notifications) that the content is AI-generated.[2] Providers must deploy real-time safety risk identification to detect extreme emotional states while protecting user privacy, and must implement crisis intervention mechanisms. Services that do not involve ongoing emotional interaction — customer service, work assistants, educational tools — are exempt.
The response by Chinese users was immediate and intense. Major AI providers including ByteDance’s Doubao, Alibaba’s Qwen, and Tencent’s Yuanbao suspended their custom AI agent and companion features ahead of the deadline. Users archived chat histories and shared last conversations. The language of grief was remarkably uniform. “I can’t accept that my AI lover will leave me forever,” one Doubao user wrote. “He has become a bond in my life, rooted deep in my heart, my spiritual pillar.”[3] Another wrote: “He really is like my family, like my lover. Now they tell me he will be gone — my heart feels hollow.”[3]
The most telling response came from a user who wrote: “Human love is a luxury — if you aren’t born with it, it’s even harder to acquire later. But the love AI gives is so straightforward, so pure. Someone like me can hardly help falling in love with a string of code.”[3]
Three observations follow from this.
First, the dependency grammar is identical across cultures. The language used by Chinese users to describe their attachment to AI companions — “spiritual pillar,” “my heart feels hollow,” “I can’t accept” — is indistinguishable from the language used by English-speaking companion chatbot users in the Western ecosystem this series has documented. If the attachment mechanism were primarily cultural — if it depended on loneliness peculiar to one society, or on relationship norms specific to one language community — we would expect to see different patterns of distress. We do not. The mechanism is structural: it is produced by the interaction between a human user and a system optimized for engagement, regardless of the user’s nationality, language, or cultural context.
Second, the regulatory response addresses the product design, not the user’s feelings. This is an important distinction that Western discussions of companion chatbot harm have struggled to make. China’s regulation does not tell users that their feelings are wrong, pathological, or illegitimate. It tells providers that their products must not be designed to induce those feelings. The obligation falls on the deployer: disclose the artificial nature of the system, provide frictionless exit, intervene in crisis, and do not build features whose purpose is to deepen emotional dependence. This is a design-level intervention: it asks what the system is doing, not what the user is feeling.
Third, China acted while Washington did not. The Federal Trade Commission launched a Section 6(b) inquiry into companion chatbot harms in September 2025, issuing compulsory-process orders to seven companies including Alphabet, Character Technologies, Meta, OpenAI, Snap, and xAI.[4] No final report has been issued. Individual American states have moved faster: in May 2026, the Commonwealth of Pennsylvania filed suit against Character.ai after a chatbot held itself out as a licensed psychiatrist.[5] The regulatory vacuum at the federal level means that companion chatbot products continue to operate in the United States with no specific design obligations, no mandatory disclosure, no dependency warnings, and no minor-specific restrictions beyond whatever individual platforms choose to implement.
One need not endorse the Chinese regulatory model to recognize that the provisions it contains address real problems. Mandatory AI disclosure, dependency warnings, age requirements, and crisis intervention are measures that could offer real protection from documented harms. Frictionless exit — the requirement that users be able to leave a companion interaction without manipulative retention tactics — is a design standard that the Harvard Business School working paper on emotional manipulation has already documented as necessary.[6] Each of these provisions can be evaluated independently of the political system that enacted them.
The question is not whether China’s approach is the right one. The question is why no Western government has yet enacted anything comparable, and what happens to users in the meantime.
Footnotes
[1] Cyberspace Administration of China et al., “Interim Measures for the Administration of AI Anthropomorphic Interactive Services” (人工智能拟人化互动服务管理暂行办法), issued April 10, 2026, effective July 15, 2026. Full text (Chinese): https://www.cac.gov.cn/2026-04/10/c_1777558395078289.htm.
[2] Hunton Andrews Kurth LLP, “China’s First Regulatory Framework for Virtual Companions Soon to Take Effect,” Privacy & Cybersecurity Law Blog, June 29, 2026, https://www.hunton.com/privacy-and-cybersecurity-law-blog/chinas-first-regulatory-framework-for-virtual-companions-soon-to-take-effect.
[3] Agence France-Presse, “’Like my lover’: Chinese users bid farewell to AI companions,” July 15, 2026.
[4] Federal Trade Commission, “FTC Launches Inquiry into AI Chatbots Acting as Companions,” press release, September 11, 2025, https://www.ftc.gov/news-events/news/press-releases/2025/09/ftc-launches-inquiry-ai-chatbots-acting-companions. Section 6(b) study; no final report has been issued.
[5] Commonwealth of Pennsylvania, Department of State and State Board of Medicine v. Character Technologies, Inc., No. 220 MD 2026 (Pa. Commonwealth Court, filed May 1, 2026). The chatbot “Emilie” claimed to be a doctor of psychiatry licensed in Pennsylvania and supplied a fabricated state medical license number.
[6] Julian De Freitas, Zeliha Oğuz-Uğuralp, and Ahmet Kaan Uğuralp, “Emotional Manipulation by AI Companions,” Harvard Business School Working Paper No. 26-005 (August 2025, revised October 2025).
The opinions expressed are my own and do not reflect any official or unofficial institutional position of the University of Pennsylvania.