San Francisco goes after websites that make AI deepfake nudes of women and girls

After AI-generated nude images of high school girls upended a community in southern Spain, a juvenile court this summer sentenced 15 of their classmates to a year of probation.

Nearly a year after AI-generated nude images of high school girls upended a community in southern Spain, a juvenile court this summer sentenced 15 of their classmates to a year of probation.

But the artificial intelligence tool used to create the harmful deepfakes is still easily accessible on the internet, promising to “undress any photo" uploaded to the website within seconds.

Now a new effort to shut down the app and others like it is being pursued in California, where San Francisco this week filed a first-of-its-kind lawsuit that experts say could set a precedent but will also face many hurdles.

“The proliferation of these images has exploited a shocking number of women and girls across the globe,” said David Chiu, the elected city attorney of San Francisco who brought the case against a group of widely visited websites based in Estonia, Serbia, the United Kingdom and elsewhere.

(Unravel the complexities of our digital world on The Interface podcast, where business leaders and scientists share insights that shape tomorrow’s innovation. The Interface is also available on YouTube, Apple Podcasts and Spotify.)

“These images are used to bully, humiliate and threaten women and girls," he said in an interview with The Associated Press. "And the impact on the victims has been devastating on their reputation, mental health, loss of autonomy, and in some instances, causing some to become suicidal.”

The lawsuit brought on behalf of the people of California alleges that the services broke numerous state laws against fraudulent business practices, non-consensual pornography and the sexual abuse of children. But it can be hard to determine who runs the apps, which are unavailable in phone app stores but still easily found on the internet.