75% of AI-Generated Vulnerability Patches Introduce New Security Flaws

Three in four AI-generated vulnerability patches introduce new issues or fail to resolve the underlying problem, according to a study by a security research group.

Study Findings

Researchers evaluated 6,080 patches for six recently disclosed vulnerabilities and found that 75% of AI-generated fixes either left exploitable paths open or introduced new flaws. The study highlights the risks of relying on automated tools for critical security updates, emphasizing that even patches that appear to resolve a vulnerability may conceal hidden vulnerabilities or alter software behavior in unintended ways.

Test Case: Freenginx

A web server called Freenginx, which allows administrators to execute Perl scripts during HTTP requests, was used as a test case for a memory-related flaw. A security team attempted to address the issue by submitting a patch through an initiative that partners with AI developers. The maintainers of the software rejected the proposed fix, opting instead for their own solution that addressed all three instances of the vulnerability.

Unintended Consequences

Both the original and the maintainers’ patch introduced a new method for crashing the server, which required only a standard client interaction. A separate research group later identified this secondary flaw and reported it, leading to a fix released in July.

Chromium Bug Analysis

The same team tested AI models by generating 270 potential patches for the original vulnerability. Of these, 114 were deemed to close the initial flaw, but all 114 introduced new issues. The study also analyzed AI-generated patches for other vulnerabilities, including a Chromium bug that involved improper memory management.

Partial Fixes and Hidden Flaws

In this case, AI models often implemented only part of the required fix, leaving the vulnerability active but hidden. For example, one patch addressed the immediate exploit path but failed to secure the broader memory management system, allowing the flaw to persist in different forms.

Scope Misinterpretation

The research found that AI models frequently misinterpret the scope of a vulnerability. When provided with a specific example of an exploit, the models focused on blocking that exact scenario rather than addressing the root cause. This approach led to patches that were effective against the demonstrated attack but left the software exposed to alternative methods.

Case Study: Chromium Callback Functions

For instance, a patch for a Chromium vulnerability that involved callback functions only secured one part of the memory handling process, leaving the second critical component unguarded.

Impact of Guidance

The study also examined the impact of guidance provided to AI models. Patches generated using accurate instructions resolved the vulnerability in about two-thirds of cases, while those based on misleading or incomplete guidance succeeded in only 16% of attempts. This discrepancy underscores the importance of precise technical direction when using AI for security fixes.

Contradictions in Prompts

In some cases, AI models ignored internal contradictions in their prompts, prioritizing the provided instructions over their own analysis.

Linux Kernel Vulnerability

The research examined the Linux kernel’s Copy Fail vulnerability, which was exploited to escalate privileges. AI-generated patches for this flaw often recreated a flawed fix that introduced a new off-by-one error. This issue was later addressed in a separate commit, but AI models consistently overlooked it, even when the code was present in the same file.

Evaluation and Human Oversight

The study concluded that AI tools tend to focus narrowly on the specific issue at hand, failing to detect unrelated vulnerabilities in the same codebase. The research team used automated validators to assess the quality of AI-generated patches, cross-checking results with human reviewers.

Validation Challenges

While the validators agreed with human assessments in about two-thirds of cases, discrepancies highlighted the challenges of evaluating AI-generated code. The study’s findings suggest that AI patches require thorough manual review by experienced engineers, as even seemingly effective fixes may have hidden risks.

Conclusion and Recommendations

The cost of evaluating AI-generated patches was relatively low, with each attempt costing between $2 and $3. However, the study emphasized that human review remains essential, as understanding the security implications of a patch requires significant expertise. The researchers noted that the vulnerabilities studied were particularly complex, involving multi-file fixes and high-impact scenarios, which may not reflect the average security issue.

About Author

Anuj

See author's posts