When Machines Make Precedent: The Supreme Court Confronts AI Hallucinations

AUTHOR: Md. Ammaar is Founding member of the FairLex, and serving as the PR head for the platform.

Introduction

On 2 July 2026, the Supreme Court of India, in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. & Anr., delivered a judgment that will likely be remembered less for the insolvency dispute it formally resolved than for the institutional alarm it sounded. A Bench of Justices Pamidighantam Sri Narasimha and Alok Aradhe set aside the orders of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT) after discovering that the very foundation of the Tribunal's reasoning rested on case law that simply did not exist but created by an artificial intelligence tool and accepted without a moment's verification. The case is a sobering marker of how generative AI has quietly begun to seep into the machinery of adjudication, and of the judiciary's determination to draw a hard line before the seepage becomes a flood.

The Factual Matrix

The appellant, a suspended director of Essel Infraprojects Ltd. (EIL), challenged the admission of a Section 7 application under the Insolvency and Bankruptcy Code, 2016, filed by Jammu and Kashmir Bank against EIL as corporate guarantor for loans extended to Pan India Utilities Distribution Company Ltd. The NCLT had admitted the application, finding debt and default established, and had rejected the appellant's argument that a corporate restructuring within the Essel group and a later sanction letter had extinguished the guarantee. The NCLAT upheld that view.

What should have been a fairly routine appeal on the interpretation of a guarantee clause took an unusual turn when senior counsel for the appellant pointed out that the very judgments the NCLT had relied upon to reject the appellant's contentions — cited verbatim by the NCLAT in its own order — were either wholly fictitious or misattributed. On independent verification, the Supreme Court found that of the six authorities cited, several carried case numbers that corresponded to entirely different judgments, while others, though correctly cited, contained paragraph excerpts that appeared nowhere in the actual reported decisions. In effect, the Tribunal had built its reasoning on a scaffold of invented law.

The Court's Core Holding

The Supreme Court's response was unambiguous. It held that a decision founded, even partly, on fabricated or hallucinated material cannot stand as a decision in the eyes of the law. The Bench went further than mere factual correction: it declared that even a single trace of such material entering the decision-making process is sufficient to vitiate the entire adjudication, since it strikes at the sanctity and integrity of the judicial process itself. This is a significant doctrinal statement. It does not ask whether the fake precedent was outcome-determinative — that is, whether the case would have been decided differently absent the fabricated citations. Instead, it treats the mere presence of hallucinated material in the reasoning as a structural defect that automatically vitiates the order, regardless of its bearing on the conclusion.

This is a deliberately strict standard, and understandably so. Once courts begin parsing "how much" hallucinated material is tolerable, or attempting to isolate its causal contribution to the final decision, the exercise becomes speculative and invites exactly the kind of erosion of trust the Court sought to prevent.

The Chain of Responsibility

Equally instructive is how the Court apportioned institutional accountability. The Bank's own affidavit indicated that its counsel had not cited the fabricated judgments at the bar; the Tribunal appeared to have generated them through what the Court described as its own research. This detail matters, because it locates the failure not with a litigant or an advocate cutting corners, but with the adjudicating body itself — the very authority meant to be the last line of defence against unreliable material. Compounding this, the appellate tribunal, despite an additional layer of judicial scrutiny, failed to catch the fabrication and simply reproduced the tainted reasoning in its own order.

The Court used this to make a broader point about the fragility of the citation-trust system on which judicial administration depends. Courts and tribunals routinely rely on the good faith of counsel and, increasingly, on their own research tools, without independently verifying every authority placed before them. That system, built for an era when fabricating a citation required real effort, is simply not calibrated for a world where a plausible-sounding but wholly invented judgment can be produced in seconds.

Directions Going Forward

The judgment does not stop at diagnosis; it prescribes a governance response. The Court declared a policy of zero tolerance for the citation or reliance upon AI-generated precedents without verification, and clarified that professional misconduct attaches to advocates who cite unverified material, while a judge relying on such material commits a serious lapse. Importantly, the Court was careful to state that this censure is directed at the presentation and reliance on fabricated material as if it were genuine precedent, not at the legitimate use of AI as a tool in legal practice or adjudication. It also directed the Bar Council of India to constitute a committee to examine the issue of members of the bar placing such material before courts, and to formulate guiding principles and disciplinary consequences.

Notably, the Court refrained from prescribing detailed operational rules itself, recognizing that the deeper solution lies in public policy and enforceable regulation rather than in judicial pronouncement alone. This restraint is doctrinally sound: courts are better positioned to declare standards of validity and misconduct than to design the granular verification protocols that regulators and the profession must ultimately build.

Broader Significance

Beyond its facts, the judgment situates itself within a growing global conversation about AI hallucination infiltrating legal process — the Court draws attention to a UK regulatory approval of an AI-driven law firm subject to safeguards against hallucination, and to an English High Court matter where a firm's reliance on an AI-fabricated statutory quote drew judicial censure. The comparative references suggest the Indian judiciary is positioning itself deliberately within an emerging international discourse on AI accountability in legal practice, rather than treating this as an isolated domestic anomaly.

For practitioners, the immediate lesson is unambiguous: every citation placed before a court, whether sourced from traditional research or an AI tool, must be independently verified against the original law report before it is relied upon. For tribunals and quasi-judicial bodies increasingly using AI-assisted research, the judgment imposes an affirmative duty of verification that cannot be delegated to the tool itself. The human-in-the-loop principle the Court articulates is not a suggestion but, after this ruling, a condition of the validity of the decision.

Conclusion

Pooja Ramesh Singh is unlikely to be the last word on AI's relationship with adjudication, but it is an important first word from India's apex court. By insisting on absolute human control over the reasoning process, while acknowledging AI's undeniable utility, the Court has attempted to draw a workable line: technology may assist the search for law, but it may never be permitted to author it. Whether tribunals across the country absorb this lesson before the next fabricated citation slips through will be the real test of the judgment impact.