Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion. The current definitions of AI persuasion are unclear and related harms are insufficiently studied. Existing harm mitigation approaches prioritise harms from the outcome of persuasion over harms from the process of persuasion. In this paper, we lay the groundwork for the systematic study of AI persuasion. We first put forward definitions of persuasive generative AI. We distinguish between rationally persuasive generative AI, which relies on providing relevant facts, sound reasoning, or other forms of trustworthy evidence, and manipulative generative AI, which relies on taking advantage of cognitive biases and heuristics or misrepresenting information. We also put forward a map of harms from AI persuasion, including definitions and examples of economic, physical, environmental, psychological, sociocultural, political, privacy, and autonomy harm. We then introduce a map of mechanisms that contribute to harmful persuasion. Lastly, we provide an overview of approaches that can be used to mitigate against process harms of persuasion, including prompt engineering for manipulation classification and red teaming. Future work will operationalise these mitigations and study the interaction between different types of mechanisms of persuasion.

As regular readers know, I've been writing quite a bit over the last year about the opportunities and challenges associated with bringing advances in data and digital to bear on the discovery and development of impactful new medicines. I've been struck by the potential of many of these powerful approaches, tools, and techniques, but underwhelmed by the drooling that's often accompanied them. An important theme of this column has been that despite what seems like exceptional potential, the impact of data science and digital on drug discovery and development to date has been conspicuously limited. This may reflect the extravagant expectations around big data, which has become viewed as a self-evident religion (preached by managerialist consultants), rather than as a potentially useful tool that must rigorously prove itself in context, as I recently discussed. I've also examined the impact of cultural factors (and how the culture of data science differs from that of pharma), here; the challenge of AI black boxes, here; and the importance of understanding the difference between invention and implementation (here).