Is AI Manipulation Always Wrong?

Artificial intelligence (AI) is becoming remarkably good at shaping human behaviour. It recommends what we read, curates what we watch, predicts what we buy and increasingly influences how we learn, work and make decisions. Much of the public conversation has settled on what appears to be a straightforward ethical concern: AI is manipulating us. A recent guest lecture at the EthicsLab by Dr Anass Sedrati approached this concern from a different perspective. Rather than focusing primarily on manipulation through misinformation and informational influence, the lecture considered whether contemporary generative AI systems are increasingly designed around human vulnerabilities, aligning themselves with existing reward pathways, patterns of attention and behavioural habits. Thousands of recent lawsuits in 2026 reflect the same suspicions, highlighting in particular allegations that systems are intentionally designed to develop addictive behaviours among young users (Jeyaretnam, 2026; Hays et al., 2026).

This raises concerns extending beyond misinformation towards three key concepts:

1. Epistemic dependence, defined by Hardwig (1985) as the condition of having to rely on the knowledge, expertise, and testimony of other actors, originally people, but now extendable to AI, to form one's own rational beliefs.
2. Normative surrender, coined and criticised by Elkind (1988), who observed how realist political scientists gave up on ethical standards, laws or moral principles when noticing that they are difficult to enforce or because powerful actors frequently violate them. In this context, individuals may abdicate their cognitive responsibility for moral deliberation, succumbing instead to the algorithmic convenience of deferring value-based judgments to highly persuasive artificial agents.
3. The gradual erosion of independent judgement and mental integrity, discussed during the lecture as the psychological process marked by a slow decline in a person's own senses, memories, and judgement, leading them to doubt their own perception of events and defer to an external authority, such as AI, to define objective truth.

At first glance, the ethical conclusion appears almost self-evident. If AI manipulates us, then manipulation must simply be wrong. The intuition is compelling because manipulation seems fundamentally opposed to autonomy. We generally understand ourselves as authors of our own decisions, capable of weighing reasons, reflecting upon alternatives and choosing freely. An AI system that quietly learns our preferences, anticipates our responses and exploits predictable features of human psychology appears to interrupt that process. Rather than responding to our choices, it begins to shape them. Recent discussions of AI governance reinforce this concern. Li et al. (2026) argue that debates about trustworthy AI have focused predominantly on accuracy while overlooking subtler epistemic harms. An AI system need not produce false information to become ethically problematic. Even accurate systems can cultivate excessive trust, encourage dependence and gradually reshape how individuals form beliefs and exercise judgement. The ethical concern, in other words, is not merely what AI tells us, but what prolonged interaction with AI does to our capacity to think for ourselves.

Yet philosophy has long resisted conclusions that appear too obvious. Human life has never unfolded independently of influence. Parents intentionally cultivate the character of their children long before those children possess the capacity for critical reflection. Teachers influence how students understand history, science and morality. Physicians persuade patients to adopt healthier lifestyles. Public health campaigns are designed to change behaviour. Even philosophers hope their arguments will persuade readers to abandon one position in favour of another. If every attempt to shape another person's beliefs or actions counts as manipulation, then influence is not an ethical anomaly introduced by AI. It is a defining feature of social existence. The apparent moral clarity surrounding AI therefore begins to dissolve because the problem cannot simply be that behaviour is influenced. Human flourishing has always depended upon forms of influence that educate, guide, persuade and encourage, an idea that is central to EthicsLab co-director Francis Nyamnjoh's (2017) concept of incompleteness.

This observation forces a different question. Perhaps manipulation is not inherently wrong. Perhaps what matters is the manner in which influence operates, and what ideas are being shared. Kant (1785/1996) argued that persons deserve moral respect because they possess the capacity for rational self-government. To manipulate another person is objectionable not because their behaviour changes, but because they are treated merely as a means to another's ends rather than as an agent capable of determining those ends for themselves. Contemporary scholarship on technological manipulation develops a similar concern. Susser et al. (2019) argue that digital technologies introduce a distinctive form of influence precisely because they increasingly operate beneath the threshold of conscious reflection. Rather than engaging our reasons, they exploit cognitive biases, emotional responses and behavioural tendencies, often without our awareness. Manipulation, on this account, becomes ethically troubling because it bypasses rather than engages our capacity for reflective judgement.

Even this distinction, however, remains unstable. Consider an AI system that reminds an elderly patient to take medication, encourages healthier eating, helps an individual living with depression maintain therapeutic routines or supports someone attempting to overcome addiction. These systems are also explicitly designed to influence behaviour. Their success depends upon doing so. It would nevertheless seem illogical to describe such technologies as morally equivalent to algorithms designed to maximise advertising revenue by exploiting loneliness, outrage, or compulsive engagement. Both influence human behaviour, yet they do so within profoundly different moral relationships. One appears directed towards supporting goals individuals already recognise as their own. The other risks substituting commercial or institutional objectives for those individuals' considered interests. The ethical distinction therefore cannot lie in influence itself. It appears to lie in the relationship between influence and agency.

Agency, however, is itself a more complicated concept than contemporary discussions of AI sometimes acknowledge. Human beings do not become autonomous by escaping influence. We become autonomous through it. Our language, values, habits, aspirations and moral commitments emerge through families, schools, communities, cultures, and institutions. Foucault's (1975/1977) analyses of power remind us that human subjects are never formed outside networks of influence. We are continually shaped by the practices, discourses and institutions through which we come to understand ourselves. From this perspective, the aspiration to eliminate influence altogether becomes both philosophically implausible and socially undesirable. The question is not whether influence exists, but what kinds of influence enable people to become reflective agents and what kinds quietly diminish that possibility.

Perhaps this is where AI introduces something genuinely novel. Unlike teachers, clinicians, friends, or family members, AI systems increasingly operate through unprecedented asymmetries of knowledge and computational power. They continuously gather data, observe behaviour, personalise interactions across millions of users simultaneously, optimise persuasive strategies through constant feedback, and adapt in ways that remain largely invisible to those they influence (Matz et al., 2024). Their capacity to shape behaviour does not arise simply from their capacity for persuasion or because they have more resources than any single person would ever have. It arises because they transform influence into an ongoing computational process, one capable of identifying precisely which forms of persuasion succeed under particular psychological conditions. What distinguishes AI is therefore not that it influences human behaviour, but that influence itself becomes personalised, scalable, adaptive, undetectable and increasingly opaque.

Seen in this light, the concern with independent judgement and mental integrity begins to take on broader philosophical significance. The central ethical question is no longer whether AI manipulates us, since influence has always been inseparable from human life. Nor is it simply whether AI systems are accurate or truthful. Rather, the question concerns whether increasingly intelligent systems preserve the conditions under which people remain capable of recognising influence, questioning it and deciding whether it deserves their endorsement. Mental integrity is not threatened because AI changes our behaviour. It is threatened when those behavioural changes occur without leaving space for reflection, resistance or genuine authorship over the decisions we ultimately make.

The challenge posed by AI manipulation is therefore less about manipulation itself than about the future of human agency. If influence is an unavoidable feature of social life, then eliminating manipulation altogether is neither possible nor desirable. Education, healthcare, friendship, and democratic politics would all become suspect under such a standard. The more demanding task is to distinguish forms of influence that cultivate independent judgement from those that quietly replace it. AI forces us to ask whether technological systems remain tools that support human deliberation or whether they are becoming environments within which deliberation itself is increasingly engineered. Perhaps that is the deeper lesson emerging from the lecture and the reflections developed here. The future of AI ethics will depend not simply on protecting autonomy in the abstract, but on preserving the fragile conditions under which human beings continue to recognise themselves as the authors of their own judgement, avoiding a "cognitive surrender".

References

Elkind, J. B. (1988). Normative surrender. Michigan Yearbook of International Legal Studies, 9, 263. https://repository.law.umich.edu/mjil/vol9/iss1/9

Foucault, M. (1977). Discipline and punish: The birth of the prison (A. Sheridan, Trans.). Pantheon Books. (Original work published 1975). https://books.google.co.za/books/about/Discipline_and_Punish.html?id=o9cPAQAAMAAJ&redir_esc=y

Hardwig, J. (1985). Epistemic dependence. The Journal of Philosophy, 82(7), 335–349. https://doi.org/10.2307/2026523

Hays, K., Saad, N., & Morris, R. (2026, March 27). Campaigners welcome Meta and YouTube's defeat in landmark social media addiction trial. BBC News. https://www.bbc.com/news/articles/c747x7gz249o

Jeyaretnam, M. (2026, August 11). Tech firms to face thousands of lawsuits over social media addiction. Time. https://time.com/article/2026/08/11/tech-social-media-addiction-tiktok-meta-snap-google-lawsuits/

Kant, I. (1996). Groundwork of the metaphysics of morals (M. Gregor, Trans.). Cambridge University Press. (Original work published 1785). https://doi.org/10.1017/CBO9780511919978

Li, Z., Yi, W., & Chen, J. (2026). Accuracy paradox: Addressing epistemic, manipulative, and societal risks of hallucination in AI governance. Computer Law & Security Review, 61, 106311. https://doi.org/10.1016/j.clsr.2026.106311

Matz, S. C., Teeny, J. D., Vaid, S. S., et al. (2024). The potential of generative AI for personalized persuasion at scale. Sci Rep, 14, 4692. https://doi.org/10.1038/s41598-024-53755-0

Nyamnjoh, F. B. (2017). Incompleteness: Frontier Africa and the currency of conviviality. Journal of Asian and African Studies, 52(3), 253–270. https://doi.org/10.1177/0021909615580867

Susser, D., Roessler, B., & Nissenbaum, H. (2019). Online manipulation: Hidden influences in a digital world. Georgetown Law Technology Review, 4, 1–45. http://dx.doi.org/10.2139/ssrn.3306006