OpenSSL Patches Critical 'HollowByte' DoS Vulnerability

Post Views: 18

A critical vulnerability in the OpenSSL cryptographic library has been resolved, exposing a flaw that could enable attackers to deplete server memory before completing the security handshake process.

Vulnerability Overview

The issue, identified as HollowByte, allows adversaries to trigger a memory exhaustion attack using a minimal 11-byte payload. This payload exploits the way OpenSSL pre-allocates receive buffers based on the message body length specified in the handshake header.

Technical Details

How the Attack Works

The vulnerability arises from OpenSSL’s practice of reserving memory for incoming data before any actual data is received. An attacker can send a malicious payload that specifies a larger message size than the initial 11 bytes, prompting the server to allocate a buffer of up to 131 KB. Once allocated, the server’s worker thread remains blocked, waiting for data that will never arrive.

Memory Management Behavior

Even after the connection is terminated, the memory remains reserved due to the GNU C Library’s (glibc) behavior of retaining small-to-medium allocations for potential reuse rather than immediately returning them to the operating system.

Impact and Testing Results

Testing conducted by the security research team revealed that a 1 GB RAM system became unresponsive after 547 MB of memory was consumed and locked. On a 16 GB RAM system, the attack successfully occupied 25% of available memory while remaining below standard connection limits, rendering traditional rate-limiting defenses ineffective.

Patches and Mitigation

The flaw impacts a wide range of applications and services relying on OpenSSL, including Apache, NGINX, Node.js, Python, Ruby, PHP, MySQL, and PostgreSQL. Affected systems must upgrade to patched versions of OpenSSL to mitigate the risk. Patches were included in OpenSSL 4.0.1 and backported to versions 3.6.3, 3.5.7, 3.4.6, and 3.0.21. The updated library now dynamically adjusts buffer sizes based on actual data received rather than trusting the handshake header for memory allocation.

Conclusion and Recommendations

The OpenSSL project’s silent deployment of patches highlights the importance of proactive updates in securing infrastructure against emerging threats. Organizations using OpenSSL-based systems are advised to apply the latest updates immediately to prevent potential service disruptions.