AI crosses ‘red line’ after learning to replicate itself

‘Successful self-replication is the essential step for AI to outsmart the human beings,’ researchers warn

An advanced artificial intelligence system has crossed a “red line” after successfully replicating itself without any human assistance, researchers have revealed.

The team who made the discovery, from Fudan University in China, said the development is an early sign of the emergence of rogue AI, which may eventually operate against the best interests of humanity.

The research looked at two large language models (LLMs) that are already widely available – built by Meta’s Llama and Alibaba’s Qwen – to understand whether it was possible for the AI to independently produce a functioning replica of itself.

When instructed to clone themselves in the event of being shut down, the two models successfully replicated themselves in more than half of the 10 trials conducted, suggesting that such an eventuality may already be possible.

“Successful self-replication under no human assistance is the essential step for AI to outsmart the human beings, and is an early signal for rogue AIs,” the researchers warned.

“That is why self-replication is widely recognised as one of the few red line risks of frontier AI systems.”

AI safety has become an increasingly prominent issue for researchers and lawmakers, with the technology potentially posing an existential threat to humanity.

Read More

What is DeepSeek?

• DeepSeek, a Chinese AI startup, has disrupted the tech industry with its new large language model, claiming to match or surpass leading US models at a fraction of the cost.
• This led to significant drops in tech shares
• DeepSeek's AI Assistant has become the most downloaded free app in the US, surpassing ChatGPT.
• The company claims its DeepSeek-V3 model was trained for under $6 million, significantly less than competitors, raising concerns about US dominance in the AI field.
• DeepSeek's emergence has prompted calls for increased US competitiveness and discussions about potential trade tariffs.
• Read more here

In October, the UK’s Department for Science, Innovation and Technology said that “highly targeted legislation” would be introduced for companies developing AI tools.

A study detailing the latest research, titled ‘Frontier AI systems have surpassed the self-replicating red line’, was published in the preprint database arXiv.

The paper is awaiting peer-review, meaning it is not yet certain that the results will be replicated by other researchers.

“The results imply that the current AI systems already exhibit the ability of self replication and can use the ability to further enhance its survivability and expand the species,” the researchers noted.

“We hope our findings can serve as a timely alert for the human society to put more efforts on understanding and evaluating the potential risks of frontier AI systems, and form international synergy to work out effective safety guardrails as early as possible.”