Adversarial Testing Solutions for Enterprise Artificial Intelligence Systems

Post Views: 1

Enterprise AI Systems Face Enhanced Threat Landscape with ASAPP’s New Capability

In a move aimed at bolstering the security of enterprise-grade artificial intelligence (AI) systems, ASAPP has introduced Continuous Red Teaming, a novel capability that embeds adversarial AI testing within its model evaluation framework.

What is Continuous Red Teaming?

Continuous Red Teaming leverages Promptfoo, an AI security platform that detects and addresses vulnerabilities in AI systems during development.

According to ASAPP, “This approach ensures that AI systems are not only evaluated for their accuracy but also for their robustness against potential threats.”

How Does Continuous Red Teaming Work?

Core Model Integrity Control: Continually testing against various forms of adversarial attacks, including many-shot attacks, character-level obfuscations, and system override attempts.
Data Privacy: Evaluating the knowledge base security of retrieved data, testing against indirect prompt injection via poisoned documents, knowledge base exfiltration, and cross-session PII leakage.
Agentic Operational Security: Validating that AI agents cannot be manipulated into unauthorized data access, used as proxies for internal network reconnaissance, or expose underlying system architecture through tool-calling exploitation.

Risk Benchmarking Framework

Continuous Red Teaming employs a risk benchmarking framework that tracks the Attack Success Rate (ASR) for each model update. This metric is crucial for organizations seeking assurance that their AI systems are secure.

According to Khash Kiani, CISO of ASAPP, “Trust has to be earned through specific metrics, a clear approach, and a regular reporting system to share with their boards.”

Conclusion

ASPP’s commitment to continuous red teaming demonstrates its dedication to providing a safety architecture that automatically identifies and fixes weaknesses in real-time, setting a new standard for enterprise AI security.