Increased use of artificial intelligence has opened a Pandora’s box that includes new concerns regarding Child Sexual Exploitation Material (CSEM). A study has found that more than a third of its teen participants reported non-consensual sexual images of them have been made using AI.
Results of this study from George Mason University were recently published in the PLOS One journal. It included a survey of 557 English speaking individuals aged 13 to 18 from the U.S.
“A significant percentage of adolescents in this national survey were found to engage in the use of [generative AI] GenAI applications to create sexualized images, including nudification applications,” said the study. “This represents a potentially significant source of CSEM creation and distribution, and shows that, to some degree, risky engagement with GenAI is widespread in this population.”
In addition to more than a third (36.3%) of the teens reported having a non-consensual image created, nearly a third (33.2%) reported having received at least one non-consensual AI-generated sexualized image. These images were created using GenAI programs or nudificiation software and both teen girls and boys were victimized.
“Use of nudification tools was widespread, with 55.3% (308) of participants reporting having created and 54.4% (303) having received at least one image,” said the publication.
Concern about nudification software led lawmakers in Minnesota to pass a ban on the technology that goes into effect on Aug. 1. According to FOX 9, it was the first ban on the technology passed in the U.S.
“Anyone can take a non-explicit photo of someone, upload it to a platform that offers an AI nudification feature and create a non-consensual, sexually explicit image or video of the person,” said an article posted by the Minnesota House of Representatives this April. “Those images and videos often end up on the dark web, shared on social media or exchanged for money.”
Grok, the AI tool offered through Elon Musk-owned social media platform X, has been identified as a source of sexualized images, according to The New York Times. Its Center for Countering Digital Hate found that Grok shared at least 1.8 million sexualized images of women as of Jan. 22 of this year.
Last week, Wired reported that its investigation into Grok revealed “dozens of ‘nudified’ deepfake images and videos on Grok’s website, including nonconsensual depictions of celebrities and at least one prominent U.S. politician.” Wired said that it appears Grok is still being used to create and host non-consensual sexual images even after Musk’s AI firm xAI said it “would introduce restrictions to stop the creation of potentially harmful sexualized deepfakes.”
Results of the George Mason study said nudification software stood out as a particular concern.
“In terms of victimization, the usage of and sharing of content from nudification tools was higher than that of general GenAI creation tools, portending a higher degree of direct victimization as nudification tools generally involve a known/direct child victim,” said study.
According to the research, “male participants had higher degrees of regular GenAI sexual image creation and distribution, both consensual and non-consensual,” and usage was similar among all ages in the 13 to 17 range. While most of the activities surveyed represent the creation of CSEM, a violation of U.S. federal law, the study also noted that “many of the actions were consensual and involving peers,” and that much of the usage appeared to be “exploratory rather than habitual” and thus indicate “an opportunity for positive intervention.”
“Policymakers need to consider if these activities, which may be considered part of normal sexual exploration, warrant specific legal exceptions beyond current discretionary prosecution,” the study said.
Going forward, a number of steps were suggested to mitigate harm, starting with education on use of these tools before age 13. Better controls on usage of nudification software and GenAI applications to detect CSEM are also needed, the study said.
“While calling for application makers to provide better controls to limit their use by minors is necessary, many of the applications are gray market tools and age restriction controls are not likely to be fully successful unless coupled with an education strategy,” the study noted. Furthermore, it said that the extensive use and normalization of GenAI should be considered when developing educational interventions.
More research into the subject – including the long-term impacts of creating sexualized images – is also warranted, the study added. Use of these tools by adults to create CSEM should also be studied.
“Additional training for law enforcement on handling both offenders and victims, as well as digital forensics specialists on the identification of GenAI images, is needed,” the study said. “Finally, decision makers need to consider the prevalence and nature of these use cases in developing new legislation related to GenAI CSEM.”
