Supreme Court says AI-generated hallucinated precedents ‘catastrophic’ to judicial process, sets aside NCLT order
The top court also directed the Bar Council of India to constitute a committee to examine the issue and formulate appropriate norms
360° Perspective Analysis
Deep-dive into Geography, Polity, Economy, History, Environment & Social dimensions — AI-powered, on-demand
Context
The has set aside an order by the (NCLT) after discovering the tribunal relied on non-existent, AI-generated case laws—a phenomenon known as 'hallucination'. The apex court emphasized a 'zero-tolerance' approach to the use of fake precedents, warning that it compromises the integrity of adjudication. The court also directed the to frame norms regarding AI usage in the legal profession and referenced its own .
UPSC Perspectives
Polity
This judgment directly intersects with the concept of the Rule of Law, which forms part of the Basic Structure of the Constitution. The common law system heavily relies on the Doctrine of Stare Decisis (the legal principle of determining points in litigation according to precedent). When judicial or quasi-judicial bodies like the rely on 'hallucinated' (fabricated) precedents generated by Artificial Intelligence, the fundamental predictability and legality of the justice system are subverted. The , exercising its appellate and supervisory jurisdiction, has recognized that the unquestioned adoption of technological outputs undermines judicial independence and the essential human reasoning required for adjudication. The court's reference to the indicates an ongoing institutional effort to regulate technology, ensuring AI remains strictly an assistive tool rather than supplanting the core adjudicatory functions of a judge. Aspirants must prepare for questions on the balance between technological integration in courts (e-Courts project) and maintaining judicial integrity.
Governance
From a governance standpoint, this incident highlights the significant risks associated with the rapid adoption of emerging technologies by statutory bodies and tribunals. The (IBC) relies on swift and accurate decision-making by the to resolve corporate distress efficiently. The revelation that the NCLT conducted 'own research' yielding fake precedents points to a systemic vulnerability where the pressure for speedy disposal might lead to over-reliance on unverified AI tools. The Supreme Court's directive to the (BCI) to formulate norms represents a crucial regulatory intervention. The BCI, as the apex statutory body regulating legal education and professional standards under the Advocates Act, is now tasked with creating ethical guidelines and disciplinary mechanisms for technology use. This underscores the need for capacity building and digital literacy within the legal ecosystem—both for the Bar (lawyers) and the Bench (judges)—to ensure meaningful human oversight is maintained at every stage of the dispute resolution process.
Ethics
The Supreme Court's observations offer a profound critique of human reliance on machines, warning against the 'habit' of delegating 'thinking' to Artificial Intelligence. This touches upon the core ethical concept of human agency and accountability in public service. The court argues that the capacity for reasoning is uniquely human, shaped through deliberate training, and cannot be ethically relinquished to algorithms. The comparison of AI hallucinations to a 'catastrophic' gas leak emphasizes the invisible yet profound damage caused when integrity and probity in decision-making are compromised. The draft regulations' prohibition on using AI in sentencing or bail decisions highlights the ethical consensus that matters concerning human liberty require empathy, moral judgment, and contextual understanding—qualities absent in AI. This case serves as a vital case study for GS Paper 4 on the ethical dimensions of technology in governance, demonstrating how the pursuit of efficiency must never override the fundamental requirements of fairness, transparency, and truth in public administration.