One-shot Federated Learning combined dataset distillation and ensemble modeling

What is it about?

This paper introduces OFedED, a novel framework for one-shot federated learning (FL). Unlike traditional FL's multi-round exchanges, OFedED combines dataset distillation—compressing each client's local data into tiny synthetic data that capture key features via layer-wise matching—and server-side model ensembling. Clients train local models, distill data using a global ensemble for guidance, and upload these synthetic data. The server aggregates them, generates more data with a weighted ensemble (to handle class imbalances) and class-center loss (to align features), then trains a global model close to centralized performance. It includes theoretical proofs bounding the gap to ideal centralized training under mild assumptions.

Featured Image

Photo by Mayank Sehgal on Unsplash

Photo by Mayank Sehgal on Unsplash

Why is it important?

One-shot FL addresses real-world bottlenecks in standard FL, like high communication overhead and security risks from repeated data exchanges, which limit scalability in edge devices (e.g., mobiles, IoT). OFedED achieves near-centralized accuracy while handling data heterogeneity and ensuring differential privacy via noise addition. This enables efficient, robust AI training in privacy-sensitive scenarios like healthcare or finance, without needing raw data sharing, and scales to diverse architectures, making collaborative ML more practical and secure.

Perspectives