What is the impact of limited bandwidth on federated learning systems?

Limited bandwidth significantly impacts federated learning systems by creating communication bottlenecks, reducing model quality, and introducing practical constraints. Federated learning relies on frequent exchanges of model updates (e.g., gradients or weights) between edge devices and a central server. When bandwidth is constrained, these updates take longer to transmit, slowing the overall training process. For example, a model with millions of parameters, like ResNet-50, might require sending 100MB of data per client per round. On a slow connection, this could take minutes instead of seconds, delaying synchronization and increasing total training time. Devices on cellular networks or in remote areas may struggle to participate effectively, creating imbalances in the system and reducing the pool of usable data.

Bandwidth limitations can also degrade model performance. To reduce data size, developers often compress updates via techniques like quantization (e.g., converting 32-bit floats to 8-bit integers) or pruning (removing less critical parameters). However, aggressive compression risks losing fine-grained information, leading to slower convergence or lower accuracy. For instance, over-quantizing gradients might prevent the model from adjusting properly to subtle patterns in the data. Similarly, limiting update frequency to save bandwidth—such as syncing every 10 rounds instead of every round—can cause the global model to overshoot local optima, resulting in unstable training. These trade-offs force developers to choose between communication efficiency and model effectiveness.

Practical workarounds exist but add complexity. Asynchronous communication lets slower clients send updates without holding up others, but this can introduce stale gradients that misalign the global model. Smaller architectures (e.g., MobileNet instead of ResNet) reduce per-update data size but may sacrifice accuracy for tasks requiring deeper networks. Developers might also prioritize critical updates—for example, only transmitting parameters that changed significantly—or schedule training during off-peak hours. These strategies require careful tuning: a poorly configured asynchronous system might waste compute resources, while overly aggressive model pruning could nullify the benefits of federated learning. Balancing bandwidth constraints with functional requirements remains a key challenge in real-world deployments.