Hack The Box: Evaluating AI Agents for Enterprise Cybersecurity

Post Views: 12

Hack The Box has launched AI Range Enterprise Edition, expanding its platform for evaluating the effectiveness of AI security agents to enterprise security teams.

AI Range Enterprise Edition

This solution allows organizations to verify if their AI systems can fulfill assigned cybersecurity responsibilities and make data-driven decisions about integrating agents into operational workflows. Many enterprises deploy AI tools based on initial performance metrics without subjecting them to continuous real-world testing. As AI models, data inputs, and threat landscapes evolve, the reliability of these agents can fluctuate. The platform, initially introduced in December 2025 as AI Range, provides controlled environments to test AI security agents. It has since undergone rigorous testing by AI research labs, government entities, and large-scale cloud providers.

Enhanced Evaluation Framework

AI Range Enterprise Edition enhances this capability by offering a role-specific evaluation framework for organizations to assess their AI systems. The company’s Cyber Workforce Development strategy combines human and AI capabilities through new innovations. Agentic Worker Competence enables organizations to validate if AI agents meet predefined cybersecurity role requirements under realistic conditions. This process generates recurring, evidence-based assessments of an agent’s ability to perform specific tasks. Agentic Operator Competence Scoring introduces a proprietary system to evaluate whether cybersecurity professionals can critically assess AI outputs, intervene when necessary, and ensure accuracy.

Key Roles and Future Plans

Initial roles include AI-augmented penetration testing and SOC analyst functions, with additional cybersecurity positions planned for future release. “AI is increasingly handling critical cybersecurity tasks, requiring confidence in both agent performance and human oversight,” stated Haris Pylarinos, CEO of Hack The Box. “Leaders must ensure agents can execute assigned duties while also having skilled personnel to monitor, validate, and correct their work. Cyber readiness depends on aligning human expertise with AI capabilities.”

“AI is increasingly handling critical cybersecurity tasks, requiring confidence in both agent performance and human oversight,” stated Haris Pylarinos, CEO of Hack The Box. “Leaders must ensure agents can execute assigned duties while also having skilled personnel to monitor, validate, and correct their work. Cyber readiness depends on aligning human expertise with AI capabilities.”

Continuous Evaluation of AI Agents

Agentic Worker Competence is designed to demonstrate whether AI agents meet expected performance standards for specific roles under dynamic conditions. AI Range Enterprise Edition delivers role-based metrics, pass/fail outcomes for individual environments, and historical performance tracking. The platform regularly updates its testing environments to reflect emerging vulnerabilities and attack techniques while maintaining assessment consistency. Organizations can re-evaluate agents as models, software, data, or operational contexts change, enabling them to track performance improvements, declines, or stability.

“Connect the agent to a role and let it operate,” said Gerasimos Marketos, Chief Product Officer at Hack The Box. “AI Range monitors how the agent handles tasks independently, identifying strengths and weaknesses. Security leaders can use these insights to determine which responsibilities an agent is prepared for and where human intervention remains critical.”

Developing Human Judgment to Manage AI

As AI systems become integral to cybersecurity operations, professionals face increased responsibility for directing their use, verifying outcomes, and intervening when required. HTB’s AI-augmented roles embed these responsibilities into existing workflows. The AI-augmented SOC Analyst role places practitioners in realistic scenarios where they use AI to analyze alerts, investigate anomalies, and assess threat indicators. Agentic Operator Competence Scoring evaluates the decision-making skills of cybersecurity professionals through hands-on exercises. It measures whether practitioners can identify errors in AI reasoning, understand the rationale behind recommendations, and determine when to override automated decisions. The system also assesses when to allow AI-driven processes to proceed without unnecessary interference.

Conclusion

The platform’s focus on continuous validation ensures organizations can adapt to evolving threats and technological advancements while maintaining operational integrity. By combining rigorous AI testing with human oversight frameworks, Hack The Box aims to establish a balanced approach to integrating AI into enterprise cybersecurity strategies.