Abstract
In Federated Learning, a global model is constructed by aggregating models that have been trained locally on distributed clients. Data is generated on client devices as well, but does not ever leave them. Thus, Federated Learning heavily reduces the amount of data that needs to be sent over the network. Examples of client devices include smartphones and connected vehicles, which highlights the practical relevance of this approach to machine learning.
We compare three algorithms for Federated Learning. Their performance is evaluated against a centralised approach where data resides on the server.
The algorithms covered are Federated Averaging, Federated Stochastic Variance Reduced Gradient (FSVRG), and CO-OP. They are benchmarked with the MNIST dataset, using both IID and non-IID partitionings of the data. Our results show that, among the three federated algorithms, FedAvg achieves the highest accuracy, regardless of whether IID or non-IID partitionings are used. Our comparison between FedAvg and centralised learning shows that they are practically equivalent when IID data is used. However, the centralised approach outperforms FedAvg with non-IID partitioned data. For practical use, we recommend FedAvg over FSVRG. Lastly, we highlight practical benefits of using an asynchronous algorithm, such as CO-OP.