A teenager in Australia was arrested for creating and sharing AI-generated nude images of over 50 female students at their school.
Bacchus Marsh Grammar School in Victoria reported that images of their students were manipulated with artificial intelligence to create highly offensive and explicit content. These altered images were then shared on social media platforms like Instagram and Snapchat.
Per the report, the manipulated images originated from the social media accounts of senior high school students. On Friday, June 7, authorities received reports that several explicit images were sent to someone in the Melton area through an online platform.
In response, a teenager was apprehended by the police in connection with the circulation of these images online. He has since been released, and further investigations are ongoing. School principal Andrew Neal condemned the incident as “appalling.”
“It is something that strikes to the heart of students, particularly girls growing up at this age. They should be able to learn and go about their business without this kind of nonsense,” he told ABC News.
Principal Neal emphasised that the school is providing support services to the affected students. The school is also actively collaborating with investigators to ensure the complete removal of the manipulated images from social media platforms.
The Rise of Non-Consensual AI Imagery and Its Impact
While the nature of these images suggests a possible connection to someone within the school community, Principal Neal assured that the school and authorities are exploring all possibilities thoroughly.
One mother recalled picking up her 16-year-old daughter from a sleepover only to find her in a state of extreme distress. The daughter, visibly shaken, had become physically ill upon encountering the disturbingly graphic manipulated images. The teen was “very upset, and she was throwing up,” her mother told ABC Radio Melbourne.
“I mean they are children. The photos were mutilated and so graphic. I almost threw up when I saw it,” she said. “Fifty girls is a lot. It is really disturbing. How can we reassure them that once measures are in place, it won’t happen again?”
Victorian premier Jacinta Allan criticised the incident and emphasised women and girls deserve respect. “My thoughts are with the young women of Bacchus Marsh Grammar and their families. There is no place for this disgraceful and misogynistic conduct in Victoria,” Ms Allan said.
Further condemning this behaviour, she argued, “Women and girls deserve respect in class, online and everywhere else in our community, which is why we have made laws against this behaviour, and we are teaching respectful relationships in schools to stop violence before it starts.”
The Dangers Of AI
Despite the potential of AI-generated images, there are concerning applications. Malicious actors have misused this technology to create deepfakes, which are manipulated images or videos that can be highly realistic and often target women, including celebrities.
In a 2018 incident, a face-swap app was used to create non-consensual pornography featuring celebrities. The legal implications of AI-generated content are still evolving. In 2017, an incident involving an AI-generated video featuring Gal Gadot, the “Wonder Woman” star, raised concerns.
This highlights the potential for creating deepfakes of celebrities without their consent, which may be illegal in certain jurisdictions.