Telegram apologises for handling of deepfake porn in South Korea

The launch of South Korea’s investigation follows public and political outrage over digital deepfake pornography featuring women

Telegram issued a rare apology to South Korea’s media regulator over its handling of deepfake pornography on its platform, days after Seoul launched an investigation to look at whether the encrypted messaging app was complicit in the distribution of sexually explicit content.

In an email to the Korea Communications Standards Commission, the platform said it wanted “a relationship of trust” after an “unfortunate” situation regarding the distribution of deepfake porn material on the platform.

Telegram “apologised if there had been any misunderstanding between the two parties” and said it wanted to make the platform “safer” for Korean users.

The company also said it had removed 25 pieces of explicit content as was requested by the South Korean authorities.

The police probe announced on Tuesday follows revelations that university students created AI-generated pornography using the real faces of female classmates. Authorities received 88 complaints about deepfake porn and identified 24 suspects.

“As France has done, the Seoul Metropolitan Police Agency launched an internal investigation into the corporate entity of Telegram before officially booking it,” Woo Jong-soo, head of the National Office of Investigation, said earlier.

“The charges are about abetting this crime,” he said, referring to deepfake videos.

Deepfakes are realistic-looking images, videos, or sounds that have been manipulated using AI to make them seem real.

President Yoon Suk Yeol has committed to tackling the increase in online sex crimes, including revenge porn and hidden recordings. Last week, he instructed authorities to “thoroughly investigate and address these digital sex crimes to eradicate them”.

“Many of the victims are minors and most of the perpetrators are teenagers,” Mr Yoon said last Tuesday. “They may say that they created this as a ‘mere prank’ but this is a clear criminal act that exploited technology behind the wall of anonymity.”

The South Korean investigation comes on the heels of Pavel Durov’s arrest last month in France. Mr Durov, the Russian-born founder and CEO of Telegram, was detained over allegations of permitting child pornography, drug trafficking, and fraud on his platform. Telegram said it actively moderates harmful content on its platform including illegal pornography.

According to Korean police data, over 90 per cent of those arrested for creating or distributing pornographic deepfakes through July are in their teens or 20s, Hankyoreh Shinmun reported. The National Police Agency reported 297 cases of sex crimes involving deepfakes and similar faked images in June and July, with 178 suspects apprehended.

An activist wearing an eye mask, holds a poster reading ‘Repeated deepfake sex crimes, the state is an accomplice too’ during a protest against deepfake porn in Seoul on 30 August 2024 (AFP via Getty Images)

Among those, 131 were teenagers (73.6 per cent) and 36 were in their 20s (20.2 per cent). Only 10 suspects (5.6 per cent) were in their 30s, and one was in their 40s, the data showed.

Deepfake pornography is a major problem in South Korea, with celebrities including singers and actresses making up 53 per cent of those depicted in these images and videos, according to a 2023 report by Security Hero, an American startup specialising in identity theft protection.

The sharp increase in deepfakes in South Korea garnered attention after Telegram channels featuring female university, high school, and middle school students as victims were discovered.

“Telegram has been actively removing content reported from Korea that breached its terms of service and will continue to do so,” the company said in a statement last week.

The scandal prompted many South Korean women to remove selfies from Facebook, Instagram, and other social media platforms, as fears grow that no one is immune to this crime.

“It feels like what should have been the safest place for us, our routine lives, has been violated,” a 27-year-old woman identified only by her surname Lee told Yonhap news agency as she said that she deleted all traces of herself from social media.

In 2020, Cho Joo-bin, the leader of a digital sex crime ring, was sentenced to 40 years in prison for persuading young women, including teenagers, to create videos that he sold online through encrypted Telegram chat rooms.

Kim Su-jeong, director of the Women’s Human Rights Counseling Centre, said: “The repeated violence against women in this country is a result of the state’s failure to listen to the numerous calls in the past raising alarms on the issue, lax punishment for perpetrators and a lack of awareness that such actions are crimes.”