Inside the marketplace powering bespoke AI deepfakes of real women

New research details how Civitai lets users buy and sell tools to fine-tune deepfakes the company says are banned. Civitai--an online marketplace for buying and selling AI-generated content, backed by the venture capital firm Andreessen Horowitz--is letting users buy custom instruction files for generating celebrity deepfakes. Some of these files were specifically designed to make pornographic images banned by the site, a new analysis has found. The study, from researchers at Stanford and Indiana University, looked at people's requests for content on the site, called "bounties." The researchers found that between mid-2023 and the end of 2024, most bounties asked for animated content--but a significant portion were for deepfakes of real people, and 90% of these deepfake requests targeted women. The debate around deepfakes, as illustrated by the recent backlash to explicit images on the X-owned chatbot Grok, has revolved around what platforms should do to block such content.