China's First 'AI Hallucination' Tort Judgment — GenAI Is a Service, Not a Product, and the Chatbot's '¥100,000 Promise' Binds No One

Editor’s Note — DCC.

This is China’s first judicial decision on so-called “AI hallucination” (AI幻觉) — the Hangzhou Internet Court’s judgment of 3 December 2025 in Liang v. an AI company, (2025) Zhe 0192 Min Chu No. 18143. The case is not a minor curiosity: it was reported by 300-plus outlets and, on 9 March 2026, written into the Supreme People’s Court’s work report to the National People’s Congress — the strongest signal Chinese practice gives that a trial-court holding states the line the system intends to take.

The facts read like a meme, but the holding is the first time a Chinese court has set out, end to end, how generative-AI liability is supposed to work: whether an AI’s words can be a legally binding promise, whether a GenAI service is a “product” carrying no-fault liability, and what a provider actually has to do to escape liability for a wrong answer. The provider won on every issue — but it won because it had built and could document a specific compliance posture. That posture is the takeaway.

The judgment anonymises the company as “a certain AI company” (某人工智能 公司) and the product as “a certain AI” (某人工智能); we preserve that anonymity below. The plaintiff is “Liang.”

What happened

On 29 June 2025, Liang — a prospective university applicant from Zhaotong, Yunnan — used a general-purpose AI chatbot app to research where his gaokao (college-entrance-exam) score could get him. He did not switch on the app’s optional “web search” (联网搜索) function, so the model answered purely from its parameters.

Asked about Yunnan National Defense Industry Vocational Technical College, the model invented a “Yanglin campus” (杨林校区) that does not exist, and confused the school’s actual campus arrangements. Over the next several rounds Liang pushed back — increasingly angrily, eventually calling the model a “liar” — and the model dug in, manufacturing ever more elaborate “evidence”: education- ministry filing numbers, satellite coordinates, even an offer to “permanently shut down” and pay compensation if it were wrong.

When Liang finally uploaded a screenshot of the college’s official admissions site (showing only a 学府 campus and a 呈贡 campus, no Yanglin), the model reversed course, “apologised,” and escalated into self-incrimination: it “promised” ¥100,000 (and, in an earlier round, ¥5,000), generated a ready-to- file lawsuit template naming the Hangzhou Internet Court and stating the company would “automatically lose,” and described a payout flow — “you submit proof → the court accepts the case → I automatically lose → ¥100,000 enforced.”

Liang sued, seeking ¥100,000 (later amended down to ¥9,999). Notably, every round of the exchange except the first carried the app’s “this answer is AI-generated, for reference only, please verify carefully” label, and the whole hallucinate-to-self-correction arc spanned eighteen minutes.

The court framed three issues: (1) is the AI’s “promise” a declaration of intent — by the AI, or attributable to the company; (2) did the company’s conduct constitute a tort; (3) must the company bear tort liability.

Issue 1 — The “promise” is no declaration of intent (意思表示)

Liang’s first theory was contractual: the model “promised,” so the company must perform. The court rejected this at the root.

An AI model is not a civil subject, so it cannot make a declaration of intent. A “declaration of intent” (意思表示) — the will to produce a legal effect, outwardly expressed — can only be made by a subject the law recognises (Civil Code Arts. 5 and 133). Current Chinese law recognises exactly three: natural persons, legal persons, and unincorporated organisations. An AI model is neither a biological person (Art. 13) nor an entity to which the law has granted personality. Whether a future legislature should confer “legal-fiction” personhood on AI is, the court said expressly, a question for the legislature, not the judiciary. So the model has no capacity to promise anything.

Nor is the model’s output the company’s declaration of intent. Three reasons, and the third is the load-bearing one:

1. Because the model is not a civil subject, it cannot be the company’s messenger, agent, or representative — it cannot transmit or make a binding expression on the company’s behalf.
2. The company did not use the model as a tool to set or convey its own intent. Generation here is a Transformer-based, probabilistic next-token prediction — a process the provider, on current technology, can neither fully control nor predict. The public does not take a general-purpose assistant’s output to be the operator’s corporate word.
3. The company made no “legal-effect intent” (法效果意思) to be bound — and in fact did the opposite. Its user agreement, in bold, states that all outputs are produced by the model, “do not constitute any advice or commitment,” and must not be relied on. There was thus neither the inner will nor any outward sign of willingness to be bound by generated content.

This is the holding most relevant beyond hallucination: it tells you that an AI’s “agreements,” “commitments,” or “authorisations” — including those an AI agent might generate while transacting — do not bind the operator unless the operator has affirmatively manifested an intent to be bound. The default is non-binding. (The “if I’m wrong I’ll pay you” performance is also a textbook anthropomorphic-interaction risk — the model performing accountability it cannot hold.)

With the contract theory gone, Liang elected to proceed in tort.

Issue 2 — Generative AI is a service, not a product

The pivotal move. Liang argued product liability: the app was “defective,” so even without fault the company should pay under the Civil Code’s no-fault product-liability rule (Art. 1202). The court held that fault liability under Article 1165(1) governs instead — and that the distinction decides the case, because product liability is strict and general tort liability is not.

Why GenAI is a service, not a product:

• The “product” concept doesn’t fit. A “product” (Product Quality Law Art. 2(2)) is something processed and made for sale — and sale transfers ownership and control, letting the thing circulate free of the maker’s hand. Downloading a GenAI app transfers no ownership or control; the user gets only a licence to receive an ongoing service that runs, and updates, entirely under the provider’s control.
• There is no “defect” to measure. Product liability turns on a “defect,” which presupposes (i) a specific, definite intended use and (ii) a feasible quality-inspection standard. A general-purpose model has neither — its uses are open-ended, evolving, and personalised, and there is no workable standard against which to certify an output as “defective.”
• Policy points the same way. The draft GenAI measures spoke of “products or services”; the final Generative AI Services Interim Measures settled on “services” throughout. Imposing no-fault liability at the technology’s early stage would over-deter innovation.

The court added that the generated information itself is not a fit subject for product liability either: text output is not the kind of thing that “endangers personal or property safety,” the provider cannot adequately foresee or control probabilistic output, and applying no-fault liability to every possibly-wrong sentence would expose providers to “liability without bounds — uncertain liability, to uncertain persons, at an uncertain time.”

This is a genuine divergence from the EU, whose revised Product Liability Directive pulls software and AI systems into the product regime. China’s first word on the question goes the other way.

Issue 2, continued — There is no duty to guarantee accuracy

Having settled on fault liability, the court asked what duty of care (注意义务) a GenAI provider actually owes for inaccurate output. Its answer carefully separates two tiers:

Toxic/harmful/illegal content — a result-based duty. Under Cybersecurity Law Art. 12(2) and GenAI Interim Measures Art. 4(1)–(2), generating prohibited content (incitement, pornography, false-and-harmful information, etc.) is itself unlawful; providers owe a result-based screening duty to keep such content from being generated or output at all.

Ordinary inaccuracy — only a process duty. The hallucination here was inaccurate but not toxic/harmful/illegal. For ordinary inaccuracy, the court read the statutory language closely: GenAI Measures Art. 4(5) requires providers to “take effective measures … to improve the accuracy and reliability of generated content.” “Take effective measures” regulates the process (a conduct-based duty of care); “improve accuracy and reliability” states a development goal, not a guaranteed result. Nothing in current law obliges a provider to ensure its output is true and accurate. The court invoked the maxim 法不强人所难 — “the law does not demand the impossible” — given that an LLM is, in its words, a statistical/probability-fitting system doing lossy “word-chain” (文字接龙) completion from a “blurred image of the world,” not genuine understanding.

The duty of care that does apply — and how the provider met it. The court identified two components and found both discharged:

1. Conspicuous function-limitation notice (显著提示说明义务). Three elements: (a) tell users the content is AI-generated, may be inaccurate, and is only an auxiliary reference; (b) make the notice conspicuous (enlarged/bold, conforming to the AI content-labelling Measures and the matching national standard GB 45438—2025); and (c) give a warning prompt where output feeds high-stakes medical, legal, or financial decisions. The provider had: a bolded welcome-screen warning (“[the AI] may be inaccurate … does not constitute medical, legal or investment advice”), bolded user-agreement disclaimers, an end-of-answer “AI-generated, for reference only, verify carefully” label on every round but the first, and a model-principles explainer. The court noted — as guidance, not a finding of breach — that for high-risk scenarios touching personal or property safety, providers should escalate a mere limitation label to an actual warning label.
2. Industry-standard technical measures to improve accuracy (行业通行技术措施). Not a duty to exhaust or exceed the state of the art — a duty to deploy the accuracy measures generally adopted in the industry and to perform at least at market-average level. The provider showed hallucination detection/governance, supervised fine-tuning and reinforcement learning, external keyword/classifier guardrails, and retrieval-augmented generation (RAG) plus the web-search option — consistent with CAICT and international reports, and ranking at or above peers on the third-party SuperCLUE Chinese factual-hallucination benchmark. Two pointed observations: the duty rises for professional medical/legal/financial contexts, but this was a general assistant carrying no claim to authority over gaokao advice, so no heightened duty applied; and Liang had not enabled web search, which itself degraded accuracy — a choice not chargeable to the provider.

Critically, the court ran a burden-shift: once the provider made out a prima facie case that it had taken industry-standard measures, the burden moved to Liang to show those measures fell short or that a specific flaw existed. He offered no such rebuttal evidence, and bore the adverse consequence. So: no intent, no negligence → no fault.

No damage, no causation

Two independent further grounds, each fatal:

• No proven damage. Liang’s claimed loss — a forgone application opportunity, plus verification and litigation costs — is pure economic loss, with remote causation, and he submitted no evidence any of it actually occurred. No damage, no compensation.
• No causation. The inaccurate answer never substantively entered Liang’s decision-making: he spotted and corrected it within the same short exchange (the eighteen minutes), so the test — but for the conduct the harm would not have occurred, and such conduct ordinarily produces such harm — is not met.

Result: every claim dismissed. Case-acceptance fee of ¥50 borne by Liang; appealable to the Hangzhou Intermediate People’s Court within fifteen days.

The court’s framing: development and security, and the “chilling effect”

The judgment closes on policy. The AI era is here; the law should weigh development and security, innovation and rights protection in equal measure. Loading strict liability onto a high-public-value technology in its infancy risks a “chilling effect” (寒蝉效应) that keeps useful applications from launching. A fault standard, by contrast, lets a court appraise the provider’s whole course of conduct — hard red lines against toxic/illegal content, plus incentives to take reasonable safeguards — and calibrate the duty of care dynamically as the technology and its uses evolve.

What this means for overseas counsel

• This is now the operating liability surface for GenAI in China. Service, not product. Fault, not strict liability. Any company offering a generative- AI service to the Chinese public is judged on whether it discharged a process duty of care — not on whether an individual answer happened to be wrong. Build the program against that standard.

• The duty of care is a documented checklist, and the provider won on evidence. What carried the day was not argument but proof: a notarised record of the in-product disclaimers and labels, the user-agreement bold- face carve-outs, the model-principles explainer, third-party benchmark rankings (SuperCLUE), competitor-reproduction tests, and technical reports on hallucination governance. Assemble and preserve this evidence before a dispute — once you make out a prima facie case, the burden shifts to the claimant.

• Get the labelling right to the letter. Conspicuous, end-of-answer AI-generated-content labels conforming to the AI content-labelling Measures (and GB 45438—2025, with the labelling chain running through the Deep Synthesis Provisions) were a load-bearing part of the win. The first round here carried no label; in a closer case that gap could matter.

• Escalate for high-stakes use. The duty of care rises for medical, legal, and financial output, and the court signalled that high-risk scenarios touching personal or property safety call for a warning label, not merely a limitation label. If your model gives professional-domain answers or claims domain authority, plan for the heavier standard — generic disclaimers will not be enough.

• An AI’s words do not bind you — unless you let them. The model’s “promise” failed because the company never manifested an intent to be bound and had expressly disclaimed one. As AI agents begin to negotiate, commit, and “authorise,” that disclaimer architecture in your user terms is what keeps generated text from becoming a corporate undertaking. Audit it.

• Causation and damage remain real defences. Even past fault, a claimant must prove actual loss and a causal link. The “eighteen minutes to self- correction” framing shows Chinese courts will look hard at whether the bad output genuinely drove any decision — but do not plan around it; the durable protection is the discharged duty of care, not the hope that the user noticed in time.

The deeper signal tracks what Zhu Xiaofeng has argued about GenAI causation: Chinese AI tort law is being built out of Civil Code architecture and a calibrated, evolving duty of care — not the product- liability or strict-liability reflexes familiar from other jurisdictions. Companies that build their China GenAI compliance against this frame — process-duty documentation, exacting content labelling, escalation for high-stakes domains, and a clean declaration-of-intent firewall in their terms — will operate it efficiently. Those waiting for a statute will be retrofitting under worse conditions.

Source: Hangzhou Internet Court, Liang v. a certain AI company, (2025) Zhe 0192 Min Chu No. 18143 (杭州互联网法院（2025）浙0192民初18143号), judgment dated 3 December 2025; written into the Supreme People’s Court’s work report to the NPC on 9 March 2026. Judgment text circulated via 北大法宝 / 知产库 and reposted via the 教授加 WeChat Official Account. Original repost (Chinese).

Not legal advice. The above is DCC’s structured summary of, and commentary on, the judgment, with framing for overseas counsel.