Supreme Court asks government to consider plea flagging ‘high-risk’ AI in public welfare
The petition sought a direction to the Centre to disclose all existing and proposed high-risk AI systems used by Union ministries/agencies, including their legal basis, vendors, purpose, data use, safeguards, grievance mechanisms and audit status
360° Perspective Analysis
Deep-dive into Geography, Polity, Economy, History, Environment & Social dimensions — AI-powered, on-demand
Context
The has directed the Union Government to consider a petition seeking a statutory framework to regulate the use of 'high-risk' Artificial Intelligence (AI) and Machine Learning (ML) in public welfare schemes. The petition argues that automated decision-making systems impacting fundamental rights—such as access to food, health, pensions, and subsidies—must operate within constitutional bounds, emphasizing the need for meaningful human oversight and legal safeguards to prevent technological arbitrariness.
UPSC Perspectives
Polity
This issue directly intersects with fundamental rights, particularly [Article 14] (Right to Equality) and [Article 21] (Right to Life and Personal Liberty). The core argument is that 'high-risk' AI used in public welfare, if left unregulated, could lead to algorithmic bias or exclusion errors, violating citizens' rights to essential services. If an automated system denies a legitimate beneficiary a ration card or pension without explanation or human recourse, it constitutes arbitrary state action. The petition seeks to 'constitutionalise innovation,' arguing that technology affecting fundamental rights cannot operate in a legal vacuum. The 's hesitation, noting its lack of technical expertise, highlights the separation of powers and the court's traditional deference to the executive in complex policy matters, yet the plea underscores that protecting fundamental rights from technological infringement is a core judicial function.
Governance
The integration of AI/ML into governance, often termed e-governance or algorithmic governance, presents a dual-edged sword. While it enhances efficiency through fraud detection, de-duplication, and optimized resource distribution (e.g., -linked authentication), the absence of a comprehensive statutory framework creates significant risks of 'black box' decision-making. The demand for a 'complete affidavit' detailing vendor information, data categories, safeguards, and grievance mechanisms addresses the crucial governance principles of transparency and accountability. Without clear guidelines, citizens lack a mechanism to challenge automated decisions, leading to potential denial of service. The UPSC frequently examines the balance between administrative efficiency and citizen-centric governance; this case perfectly illustrates the need for a legal architecture that mandates human-in-the-loop oversight for high-stakes administrative decisions to prevent technological marginalization.
Science & Technology
The petition differentiates between general AI and 'high-risk' AI, aligning with global regulatory trends like the 's , which categorizes AI systems based on their potential to harm fundamental rights. High-risk systems in the Indian context include those used for biometric surveillance, predictive policing, and automated welfare distribution. The lack of an indigenous regulatory framework for responsible AI leaves a significant legislative gap. The UPSC syllabus explicitly covers 'awareness in the fields of IT, Space, Computers, robotics, nano-technology, bio-technology.' Questions can focus on the ethical implications of AI, the necessity of an AI regulation policy in India, and how to mitigate algorithmic bias in machine learning models deployed by public agencies. The focus is shifting from merely adopting technology to establishing the legal and ethical guardrails required for its deployment in sensitive domains.