AI generated code found to produce predictable weaknesses

AI-generated code introduces an average of 15 confirmed vulnerabilities per codebase, research published by Secure Code Warrior indicates.

An evaluation of 1760 complete codebases generated by 16 popular AI models found that on average, 4.3 of these vulnerabilities are considered severe.

The analysis identified 86 unique Common Weakness Enumeration (CWE) vulnerabilities, with the most common CWEs involving logging failures, injection vulnerabilities, insecure design and broken access control. The most prevalent was insertion of sensitive information into log files.

The research forms part of the SCW AI Trust Index, a benchmark for AI coding security based on a methodology created by RMIT University and then extended by the company.

The index found that AI-generated coding risk can be predictable by model and framework, Secure Code Warrior said. Each model has a distinct security footprint and produces a repeatable mix of vulnerability categories, and no single AI model consistently produces the most secure code, the company added. Security outcomes are purportedly also independent of model costs.

“Every AI model we tested leaves a predictable, repeatable pattern of security gaps and weaknesses. Developers are also predictable in that they aren’t going to abandon their preferred model over a security score,” Secure Code Warrior CEO Pieter Danhieux said. “The SCW AI Trust Index was purpose-built to help CISOs and security leaders manage the models already in use; there is no identified ‘winner’, but this data provides the crucial insights needed to truly manage AI tools safely, with consideration to those inherent security gaps, and allow the right guardrails and developer learning pathways to be brought to life in a modernised security program.”

Image credit: iStock.com/Jacob Wackerhausen