AWS Multi-Year Routing Control Plane Overhaul Ensures Network Uptime

Post Views: 10

AWS undertook a multi-year initiative to overhaul its routing control plane without disrupting network operations.

Introduction

AWS undertook a multi-year initiative to overhaul its routing control plane without disrupting network operations. The company’s border network, which handles every API request, CloudFront content delivery, and Route 53 domain query, now operates on a completely redesigned routing system. This infrastructure spans 39 global regions, 123 availability zones, over 750 points of presence across six continents, and connects with more than 5,000 external networks. The routing control plane serves as the core mechanism for directing traffic, maintaining an up-to-date map of network destinations and optimal paths.

Original Network Challenges

AWS compares this function to air traffic control, where continuous position updates, route calculations, and broadcasted instructions ensure safe and efficient traffic flow. However, when network changes occur—such as link failures or new connections—the system must reconcile conflicting information through a process called convergence. During this period, inconsistent routing data can lead to packet loss, delayed transmissions, or traffic loops. Critical applications like financial transactions, telemedicine, and live streaming require near-instantaneous consistency to avoid service disruptions. The original network architecture faced challenges due to its incremental growth.

The Redesign: Three Key Innovations

1. Unidirectional Route Propagation

Over time, separate control planes emerged in different regions, each operating independently yet interconnected. While these localized systems functioned reliably, their lack of synchronization created inconsistencies. For example, one segment might still route traffic through a decommissioned path while another had updated its configuration. This discrepancy could result in routing loops where packets circulate indefinitely or traffic targeting non-existent routes. AWS determined that increasing coordination between these systems would compound complexity, as each additional component required more overhead to maintain alignment.

2. Tunneling for Isolation

To address this, engineers developed a unified control plane that provides a single, consistent network view to all devices. This approach reduces convergence time and limits the impact of individual changes. The redesign involved three key innovations. First, route propagation was made unidirectional. Traditional routers both receive and advertise routing information, creating potential for outdated data. AWS separated this process: collection nodes gather local routes and propagate them outward, while distribution nodes receive remote routes and deliver them to local devices. This ensures each device receives routes directly from their source.

3. Centralized Control Plane

Second, tunneling was implemented to isolate traffic from outdated routing states. Original packets are encapsulated in a secondary header containing routing instructions, allowing intermediate nodes to follow the designated path without inspecting the payload. This shields traffic from stale information. Third, the entire network transitioned to a single, centralized control plane. This unified system replaces fragmented architectures, creating a cohesive framework for routing decisions.

Implementation and Results

The implementation required careful execution across thousands of devices, including internet-facing routers, backbone infrastructure, Direct Connect endpoints, availability zone nodes, and edge services like CloudFront and Route 53. AWS maintained service availability throughout by deploying a validation system that tested configuration changes before deployment. The new system processes significantly more routing computations than its predecessor, enabling convergence speeds to improve by up to 96% on certain network fabrics. Customers experienced reduced retry rates and more stable latency, while engineers benefit from a unified architecture for troubleshooting. The entire border network now operates on this redesigned infrastructure.