Humans as the Real AI Risk: Accountability Over Regulation to Keep Big Tech Honest

Post Views: 8

Current discussions around artificial intelligence often focus on autonomous systems exceeding human control. While these long-term concerns remain relevant, they overshadow a pressing issue for security professionals today: individuals now wield tools of immense power developed at unprecedented speed, with insufficient oversight of their use.

The Immediate Threat of Human-Driven AI

The primary threat does not stem from an AI model independently targeting critical infrastructure. Instead, the immediate danger lies in state-sponsored actors or cybercriminals leveraging AI to enhance reconnaissance efforts, phishing campaigns, and exploit development. Organizations also face risks when deploying AI agents to production environments without restricted permissions or monitoring mechanisms. In both scenarios, human decisions enable these vulnerabilities.

Regulatory Challenges

Regulatory frameworks struggle to keep pace with AI advancements. By the time legislation is finalized, technological landscapes have already evolved, creating gaps that bad actors exploit. Financial accountability offers a more effective solution. Executives respond swiftly when risks directly impact their financial stability or employment.

Historical Parallels in Accountability

Historical parallels in aviation and automotive industries highlight the effectiveness of accountability measures. Safety improvements in these sectors resulted from certification processes, thorough investigations, and liability structures that imposed financial consequences on organizations and leaders.

The Case for Financial Accountability

The Boeing 737 MAX incidents demonstrated how such exposure transforms safety from a vague principle into a mandatory engineering requirement. Software development, however, has largely avoided similar accountability. Vendors often shield themselves through “AS-IS” licensing agreements, and U.S. courts have traditionally classified software as a service rather than a product.

Legal Shifts and AI Liability

This arrangement was acceptable when software flaws caused minor disruptions, but it becomes untenable as AI systems autonomously interact with financial networks and critical infrastructure. Recent legal shifts indicate a move toward greater responsibility. The EU’s revised Product Liability Directive now includes AI, and the 2023 U.S. National Cybersecurity Strategy advocates transferring liability to software developers.

U.S. courts have begun permitting product liability claims against AI companies. Despite these developments, the industry’s ability to adapt remains uncertain.

Five Key Steps for AI Accountability

Five key steps could establish meaningful AI accountability. First, eliminate blanket “AS-IS” disclaimers for commercial AI systems, ensuring vendors and users cannot entirely evade responsibility for foreseeable harm. Second, adopt a negligence standard rather than demanding perfection. While no model can be entirely secure, developers must address known risks.

Accountability in Deployment and Insurance

Third, hold deployers accountable for their decisions, such as granting AI agents unrestricted access to critical systems. Fourth, leverage insurance mechanisms to price risk, as cyber insurers have done with multi-factor authentication. Finally, ensure accountability is enforceable through tamper-evident logs and mechanisms to revoke AI agent access.

Challenges with Open-Weight Models

Open-weight models complicate this framework, as vendor control diminishes once models are publicly released, shifting responsibility to deployers. For security leaders, the challenge involves managing human-driven AI deployments rather than hypothetical rogue systems.

Recommendations for Security Leaders

SOC teams must treat AI agents as privileged entities, limiting their access, logging actions rigorously, and requiring approvals for high-impact operations. Executives and boards must also understand the evolving accountability landscape. While accountability cannot eliminate all AI misuse, it ensures those making critical decisions face consequences when failures occur.