The possibilities of creating credible images offered by Generative Artificial Intelligence have become a tool for abuse in South Korea, where it was discovered an extensive network of Telegram groups in which thousands of men uncontrollably share fake nudes of female and male students from 500 colleges and universities.
An investigation by journalist Ko Narin, published in the Hankyoreh newspaper, discovered dozens of chat groups in which photos of women they knew were exchanged and the use of software to convert them into pornographic in a matter of seconds.
It had all started at the beginning of August, when the Police announced that they had managed to close two Telegram channels where they exchanged pornography deepfake (that is, realistic erotic images created with Generative Artificial Intelligence) of students from the two main universities in the country.
After reading the report, Ko was convinced that there must be more. What he never imagined is that That was simply the tip of a sinister iceberg.where not only were college students targeted but there were rooms dedicated to specific high schools and even middle schools.
“Every minute, photographs are being published of girls that a user knows and who ask to be made the object of a deepfake porno“, the journalist told the BBC.
It is difficult to know how far this practice goes, since only public or easily accessible groups could be investigated. There are others that are more restrictive.
While the largest group he found has two thousand members, It is difficult to know how far this practice goes.since only public or easily accessible groups could be investigated.
There are, however, private rooms or rooms with strict entry conditions that sometimes have innocent names such as “friends of friends rooms” or less subtle, such as “humiliation rooms.”
An app in the spotlight
When consulted by the press, Telegram responded that its moderators “proactively monitor public parts of the app using AI tools to remove millions of pieces of content every day that violate the terms of service.” However, it is clear that these actions are not enough.
The truth is that the app is not going through its best moment: its CEO was arrested a few days ago in France because Justice considers him responsible for allowing illegal activity. South Korean authorities assured that they are considering a similar path while promising to impose severe punishments.
However, we are in complex terrain, since severe restrictions to Telegram groups can harm freedom of expression and even limit its use in territories where other forms of democratic communication are not possible, as became clear with Nicolás Maduro’s attack against WhatsApp in Venezuela.
Pornography made with deepfakesIn addition, it has its own complexities, since the legislation is not always up to date. In our country, for example, in August of last year a student from the Faculty of Architecture of San Juan was taken to court by fifteen women for sell fake nudes of his colleaguesbut he could not be criminally charged because there was no figure for his actions.