AI code hallucinations increase the risk of attack “confusion of the package”

Computer code generated by AI It is performed with references to non -existent third -party libraries, creating a golden possibility of attacking a supply chain that poisoned legal programs with malicious packages that can steal data, plant the background and carry out other malicious actions, newly published research shows.

The study, which uses 16 of the most widely used large language models to generate 576,000 code samples, found that 440,000 of the dependency dependent addicts were “hallucinated”, which means that they do not exist. Open code models hallucinate the most, with 21 percent of dependencies associated with non-existent libraries. Addiction is a basic code component that a separate piece of code requires to work properly. Addictions save developers of the difficulty of rewriting code and are an essential part of the modern software supply chain.

Packet hallucinations

These non -existent dependencies pose a threat to the software supply chain by sharpening the so -called attack attacks. These attacks work by causing a software package to access the wrong dependence of the components, for example by posting a malicious package and giving the same name as the legal but short -seal of the version. The software, which depends on the package, will in some cases choose the malicious version, not the legal, because the first seems more recent.

Also known as the confusion of the package, this form of attack was first demonstrated In 2021, in the operation of proof of the concept that performs a fake code in networks belonging to some of the largest companies on the planet, Apple, Microsoft and Tesla. This is a type of technique used in attacks by a software chain that aims to poison the software at its very source in an attempt to infect all users down the chain.

“After the attacker publishes a package under the hallucinated name containing some malicious code, they rely on the model that suggests that name of unsuspecting users,” Joseph Popklen, Texas University of Doctor of San Antonio. Student and leading researcher, Ars told ARS by email. “If the user trusts the LLM exit and installs the package without checking it carefully, the attacker’s payable load hidden in a malicious package will be executed in the user of the user.”

In AI hallucinations, it occurs when LLM produces outputs that are actually incorrect, meaningless or completely unrelated to the task that has been assigned. Hallucinations have long dock. LLM because they worsen their usefulness and reliability and have proven to be difficult to predict and remove. In a Paper Scheduled to be introduced at the security symposium of the Usenix 2025, they called the phenomenon “the Hallucination Package”.

For the study, the researchers conducted 30 tests, 16 in the Python and 14 programming language in JavaScript, which generated 19,200 code samples for a total of 576,000 code samples. Of the 2.23 million packages contained in these samples, 440 445 or 19.7 percent, packages that do not exist indicated. Among these 440 445 package hallucinations, 205 474 had unique package names.

One of the things that make the package hallucinations potentially useful in the attack attacks is that 43 percent of the hallucinations of the package are repeated over 10 requests. “In addition,” the researchers wrote, “58 percent of the time, a hallucinated package is repeated more than once in 10 reps, indicating that the greater part of the hallucinations are not just random errors, but a recurrent phenomenon that persists in multiple iterations.