What are the common architectures used in federated learning systems?

Federated learning systems are designed to train machine learning models across decentralized devices or servers without centralizing data. Three common architectures are centralized, decentralized (peer-to-peer), and hybrid setups. Each addresses different scalability, privacy, and communication needs, and the choice depends on factors like network constraints, data distribution, and trust models.

In a centralized architecture, a central server coordinates the training process. Clients (e.g., mobile devices or edge nodes) download a global model, train it locally on their data, and send model updates (e.g., gradients) back to the server. The server aggregates these updates (e.g., using Federated Averaging) to improve the global model. This approach is straightforward to implement and widely used—for example, in smartphone keyboard suggestions where user data stays on-device. However, the central server becomes a single point of failure and a potential communication bottleneck, especially with many clients. Security risks also arise if the server is compromised.

Decentralized architectures eliminate the central server by allowing clients to communicate directly. Each device trains a local model and shares updates with peers, often using consensus algorithms or gossip protocols to synchronize parameters. This is useful in scenarios where no trusted central entity exists, such as cross-organization collaborations (e.g., hospitals sharing medical insights without exposing patient data). However, coordination becomes complex: clients must manage communication schedules, handle network dropouts, and resolve conflicts from inconsistent updates. For instance, IoT sensor networks might use this approach to avoid reliance on a central hub, but synchronization overhead can limit scalability.

Hybrid architectures combine elements of both approaches. For example, a central server might handle initial model distribution and high-level coordination, while subsets of clients form peer groups to share updates locally. This balances efficiency and fault tolerance. A real-world use case could involve smart factories: a central server trains a base model for defect detection, while individual factories fine-tune it locally and share only anonymized insights with peers. Hybrid setups offer flexibility but require careful design to avoid duplicating the drawbacks of pure centralized or decentralized systems. Developers must weigh trade-offs like communication costs, privacy guarantees, and the complexity of managing heterogeneous devices.