SlimeMold: Scaling Distributed Hardware Load Balancers via a Unified Giant Connection Table

In cloud data centers, stateful load balancers (LBs) are crucial for scaling applications and ensuring availability. Hardware load balancers (HLBs) implemented on programmable switches are increasingly adopted for their high throughput and low latency. However, existing approaches to scaling HLBs largely rely on VIP-based partitioning and ECMP-driven scale-out. Such methods do not adequately accommodate traffic dynamics or HLB-node heterogeneity, and are prone to per-connection consistency (PCC) violations.

To address these issues, we present SlimeMold, a scale-out HLB architecture that provides a global and unified Giant Connection Table (GCT) across a multi-node HLB deployment. SlimeMold partitions the GCT into fine-grained segments and assigns segments to nodes in proportion to their capacities (CPS and connection-table size). To make this practical, we design low-overhead node lookup, efficient rebalancing with minimal state movement, and fast fault tolerance with low performance impact. We implement SlimeMold on the Broadcom BCM56788 SmartToR switch. Results from a real testbed demonstrate near-linear scalability up to four logical nodes and fast failover with minimal performance degradation. Large-scale simulations show that SlimeMold achieves over 98% resource utilization despite node heterogeneity and dynamic traffic.