Over the last two weeks, we’ve been discussing the importance of trust and corrosive nature of pervasive distrust in democratic societies. As we have already mentioned, in 2024 around 2 billion people across the United States, Mexico, South America, the UK, the EU, South Africa and India will be voting in national elections. Needless to say, this makes for unusually fertile ground for the sowing of political disinformation and epistemic chaos.
But combine this with the fact that it has never been easier to generate “synthetic content” — from images and audio to video — and the likelihood of voter confusion, widespread deception and outright fraud becomes too dangerous to ignore. (Indeed, we’ve already seen examples of the nefarious uses to which this technology can be put in the US, the UK and eastern Europe.) Over the last 18 months, enormously powerful generative AI tools have been placed in the hands of anyone who wants them; as a consequence, the internet and our social media feeds have been inundated with wholly or partially synthetic content.
While this has undoubtedly been cause for concern among lawmakers, academics and some in the media, the dangers posed by “deepfakes” seemed to raise little public concern — until AI-generated “non-consensual intimate imagery” of Taylor Swift began circulating on X.
There are two discrete issues at play here.
The first is what the ubiquity of synthetic or AI-generated content is doing to the possibility of a shared epistemic reality whereby we deliberate, disagree and decide on the conditions of our common life. The weaponisation of propaganda and disinformation to gain political advantage for oneself or to undermine one’s opponents and enemies is, of course, nothing new. And there are good reasons to greet predictions of an impending “epistemic apocalypse” with a degree of suspicion. However, given the way social media platforms privilege speed and scale in the dissemination of content, and the fact that they amplify content that is divisive, emotive and highly partisan, there is also good reason to suspect that the effects of the widespread circulation of “deepfakes” in an already fractious online ecosystem could be devastating.
Second, quite apart from the harm that “deepfakes” might do to our democratic societies, is there a further violation of the dignity or moral status of individuals — especially prominent women — when their appearance and voice are converted into raw materials for the AI-generated realisation of someone else’s debased fantasy?
Guest: Uri Gal is a Professor of Business Information Systems at the University of Sydney. His research focuses on the organisational and ethical aspects of digital technologies.