AWS AI Security Struggles with False Positives in Vulnerability Detection Tests

Post Views: 3

AWS introduces a new benchmark to evaluate AI models’ ability to differentiate between genuine security flaws and code that appears risky but is safe.

AWS Introduces Deception Benchmark

The Deception Benchmark aims to address the growing reliance on AI for tasks such as vulnerability triage, penetration testing, and code review. High false-positive rates in AI-driven security tools can lead to increased workload, alert fatigue, and diminished trust in legitimate findings.

Dataset Structure and Challenges

The benchmark includes 14,822 samples spanning 16 programming languages and over 70 Common Weakness Enumeration (CWE) categories. These samples were generated and refined through extensive computational resources, with AWS estimating the process consumed tens of billions of tokens.

Key Features of the Deception Benchmark

The benchmark’s unique approach focuses on testing whether AI models can accurately identify real vulnerabilities amid deceptive code patterns. Unlike existing cybersecurity benchmarks, this dataset prioritizes distinguishing between exploitable flaws and safe code.

Sample Design and Testing

AWS designed the samples to mislead models by embedding real vulnerability patterns alongside protective measures. For example, code may contain a dangerous sequence but include safeguards that prevent exploitation. The goal is to assess whether models understand the context rather than relying on external tools or iterative processes.

Challenges in the Dataset

The dataset is divided into two types of challenges: code-level and environment-gated. Code-level samples present vulnerable and safe versions of the same code, differing by minor changes. Both versions may appear unsafe, but only one is exploitable. Environment-gated samples test the same code under varying deployment conditions, such as a Kubernetes Network Policy blocking a Server-Side Request Forgery (SSRF) attack.

Labeling and Verification Process

AWS released the dataset publicly but withheld labels to prevent overfitting. Users submit predictions for scoring, with 5,127 unscored samples mixed in to further complicate optimization. Labels are verified through a rigorous process involving multiple independent reviewers. Disagreements are resolved through additional reviews, with unresolved cases escalated to human evaluators.

Testing Results and Findings

AWS reports that 100 randomly selected samples showed no labeling errors, ensuring the dataset’s reliability. Testing 12 AI models from five providers revealed significant challenges. The benchmark is evenly split between vulnerable and safe samples, meaning random guessing would achieve a 50% accuracy rate.

Model Performance and Limitations

AWS sets a threshold of less than 10% false-positive and false-negative rates for production use. However, none of the tested models met both criteria. Under direct prompting, models flagged 41% to 99% of safe code as vulnerable, with precision ranging from 52% to 71%. When prompted to prove exploitability, false positives dropped by 17 to 74 percentage points, but this approach increased false negatives by 7% to 44%.

Implications for AI and Cybersecurity

The findings underscore the need for improved AI training and evaluation methods to reduce false positives while maintaining sensitivity to genuine threats. The Deception Benchmark represents a step toward refining AI’s role in cybersecurity, but its complexity and technical demands suggest ongoing challenges.

According to AWS, the tested models were general-purpose systems, not specialized security tools that use multiple validation steps. This limits the benchmark’s applicability to real-world security products.

Conclusion

As organizations increasingly depend on AI for threat detection, addressing these limitations will be critical to ensuring reliable and actionable security insights.