https://www.financialexpress.com/opinion/ai-as-decision-maker-consent-not-enough/4322980
Beyond Consent: Why the DPDP Act Leaves Gaps in AI Governance and Algorithmic Decisions
By Harsh Walia, Partner, Khaitan & Co
With the advent of new-age technologies and AI, the role of personal data within organisations is changing. Historically, data was collected to support human or manual decision-making. Increasingly, it is being used to train systems that generate recommendations, predictions, and, in some cases, decisions. From detecting fraud and assessing creditworthiness to screening job applicants, AI is moving from an assistive to a decision-making tool.
This raises an important question: is obtaining consent to process personal data sufficient when the real impact on individuals arises from the decisions generated from that data? The Digital Personal Data Protection (DPDP) Act, 2023, appears, at first glance, to offer a straightforward answer. The Act is built largely on consent-based lawful processing of personal data and lays emphasis on such consent and principles of transparency. Data principals must be informed about the purposes for which their data is being processed and retain rights on access, correction, erasure, nomination, grievance redress, etc.
Yet AI challenges that assumption in a fundamental way. A customer applying for a loan may consent to the use of personal data. A job applicant may consent to processing of information submitted during recruitment, etc. In each case, the consent is sought for collection and processing of the personal data. It doesn’t necessarily address how an AI system will interpret the data, draw inferences, and how the inferences influence the eventual decision.
An AI decision may be entirely based on lawful consent and still yield results that are inaccurate, biased, or difficult to explain due to an absence of manual interference. Consent, by itself, provides little protection against such risks because the issue lies not in the use of data but the operation of the decision-making AI system.
This is increasingly reflected in Indian and global regulatory discourses. The EU’s General Data Protection Regulation (GDPR) has regulatory protections on certain forms of automated decision-making and profiling. NITI Aayog’s Principles for Responsible AI: A Background discussed how AI systems can generate risks relating to exclusion, discrimination, and lack of accountability, particularly in sectors where algorithmic outputs influence consequential decisions. It also focused on governance steps that could mitigate these risks such as explainability, accountability structures, audit mechanisms, and human oversight. In many respects, the recommendations foreshadow safeguards that are increasingly being discussed for high-impact AI systems. More importantly, they reinforce a recurring theme in India’s AI policy discourse: lawful access to data, while important, is insufficient on its own to ensure trustworthy AI outcomes.
NITI Aayog’s National Strategy for Artificial Intelligence too recognised that AI raises challenges beyond traditional privacy concerns (collection, processing) including for automated decision-making. It benchmarked privacy and data protection laws against international standards like the GDPR and referred to France’s right to explanation for administrative algorithmic decisions, while observing India’s privacy framework needed to evolve with understanding AI-related risks. Although such initiatives aren’t legally binding, they signal an expectation that organisations should be able to justify not only their use of data, but also the outcomes from it.
The Supreme Court released the Draft Regulations for Use of AI in Courts, 2026, giving an important indication of how sectors in India are starting to approach AI governance especially in the context of automated decision-making. The draft regulations define “algorithmic decision-making” as the use of algorithmic outputs to inform, recommend, or arrive at a decision affecting a person or process. They provide that AI systems used in court must remain assistive in nature. Most critically, they prohibit reaching judicial outcomes solely through algorithmic decision-making.
Human oversight, explainability, accountability, auditability, and data minimisation figure prominently in the draft rules. Similar principles can be interpreted in NITI Aayog’s Responsible AI framework. These developments suggest an emerging consensus that where AI systems materially affect rights, opportunities, or outcomes, governance measures must extend beyond consent and privacy notices.
The government enacted the DPDP Act, India’s first-ever comprehensive data protection law, to provide a foundational framework for data privacy. But it does not explicitly regulate automated decision-making. It doesn’t contain a right to explanation, right to human review of AI-generated decisions, or curbs comparable to those in foreign jurisdictions. But several aspects of it may indirectly influence how organisations approach AI governance, especially considering there may not be a standalone law for AI regulation as yet.
The DPDP Act is built on the principle that personal data should only be processed for specified purposes. This may sit uneasily with AI development practices that encourage the collection and retention of large data sets in anticipation of future use cases. As organisations seek to build increasingly sophisticated models, questions may arise about whether all categories of personal data being collected are necessary for the stated purpose of processing.
Similarly, the rights available to data principals under the DPDP Act may become especially relevant in AI-driven scenarios. They can seek correction of inaccurate data and access redress mechanisms; but such rights may become hard to operationalise when organisations can’t identify the data points influencing a particular outcome or explain how a decision was reached. As AI systems become more complex, businesses may find that effective governance requires a level of explainability beyond what the DPDP Act demands.
This is where governance measures for responsible AI become increasingly relevant. Human review mechanisms can be a safeguard against erroneous automated outcomes. Similarly, clear explanations of significant decisions can enhance transparency and consumer trust. The absence of a dedicated AI law in India should therefore not be viewed as a reason to defer AI governance efforts. If anything, the DPDP Act, read alongside India’s evolving AI policy framework, suggests businesses may be expected to demonstrate accountability for automated decisions.
Co-authored with Vanshika Lal, Associate, Khaitan & Co
Disclaimer: The views expressed are the author’s own and do not reflect the official policy or position of Financial Express.
This article was first uploaded on August twenty-two, twenty twenty-six, at forty-four minutes past twelve in the am.
© The Indian Express (P) Ltd
aniltikotekar4sme.com 