The problem
Federated learning papers are mostly evaluated in simulation on one server, so real-device costs (memory, energy, dropped connections) are unknown. Frameworks differ in how easy they are to deploy and how much they cost on small devices; there is no simple, reproducible comparison.
Who needs it
Companies choosing a federated learning framework for products; researchers who need realistic baselines.
Idea of the approach
- Pick 3 tasks from the LEAF benchmark (image, text, sensor).
- Run them with Flower and FedML on 5–10 Raspberry Pis / phones and in simulation.
- Measure training time, energy, memory, communication and accuracy; inject device drop-outs.
- Publish scripts and results as an open repository.
What you will build and deliver
- Benchmark scripts, results and a public results page.
- A paper (benchmark track).
Skills needed
Python, Linux, PyTorch; careful experimentation.
Research contribution & where to publish
Contribution: real-device, energy-aware comparison of FL frameworks. Venues: IEEE IISWC, MLSys workshops, ACM SEC, Future Generation Computer Systems.
Key references — start reading here
- S. Caldas, S. M. K. Duddu, P. Wu, T. Li, J. Konečný, H. B. McMahan, V. Smith, A. Talwalkar. LEAF: A Benchmark for Federated Settings. arXiv, 2018.
- D. J. Beutel, T. Topal, A. Mathur, X. Qiu, J. Fernandez-Marques, Y. Gao, et al.. Flower: A Friendly Federated Learning Research Framework. arXiv, 2020.
- C. He, S. Li, J. So, X. Zeng, M. Zhang, H. Wang, et al.. FedML: A Research Library and Benchmark for Federated Machine Learning. arXiv, 2020.
- B. McMahan, E. Moore, D. Ramage, S. Hampson, B. Agüera y Arcas. Communication-Efficient Learning of Deep Networks from Decentralized Data. AISTATS, 2017.
Interested?
Send the topic title and the courses you have taken.
Contact Dr. Hamza Djigal