Handling Data Heterogeneity in Federated Learning with Global Data Distribution

C. Nagaraju, Mrinmay Sen, C. Mohan

2023

Abstract

Federated learning, a different direction of distributed optimization, is very much important when there are re- strictions of data sharing due to privacy and communication overhead. In federated learning, instead of sharing raw data, information from different sources are gathered in terms of model parameters or gradients of local loss functions and these information is fused in such way that we can find the optima of average of all the local loss functions (global objective). Exiting analyses on federated learning show that federated optimization gets slow convergence when data distribution across all the clients or sources are not homogeneous. Heterogeneous data distribution in federated learning causes objective inconsistency which means global model converges to a another stationary point which is not same as the optima of the global objective which results in poor per- formance of the global model. In this paper, we propose a federated Learning(FL) algorithm in heterogeneous data distribution. To handle data heterogeneity during collaborative training, we generate data in local clients with the help of a globally trained Gaussian Mixture Models(GMM). We update each local model with the help of both original and generated local data and then perform the similar operations of the most popular algorithm called FedAvg. We compare our proposed method with exiting FedAvg and FedProx algorithms with CIFAR10 and FashionMNIST Non-IID data. Our experimental results show that our proposed method performs better than the exiting FedAvg and FedProx algorithm in terms of training loss, test loss and test accuracy in heterogeneous system.

Paper Citation

in Harvard Style

Nagaraju C., Sen M. and Mohan C. (2023). Handling Data Heterogeneity in Federated Learning with Global Data Distribution. In Proceedings of the 3rd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE; ISBN 978-989-758-642-2, SciTePress, pages 121-125. DOI: 10.5220/0011955400003497

in Bibtex Style

@conference{improve23,
author={C. Nagaraju and Mrinmay Sen and C. Mohan},
title={Handling Data Heterogeneity in Federated Learning with Global Data Distribution},
booktitle={Proceedings of the 3rd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE},
year={2023},
pages={121-125},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011955400003497},
isbn={978-989-758-642-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 3rd International Conference on Image Processing and Vision Engineering - Volume 1: IMPROVE
TI - Handling Data Heterogeneity in Federated Learning with Global Data Distribution
SN - 978-989-758-642-2
AU - Nagaraju C.
AU - Sen M.
AU - Mohan C.
PY - 2023
SP - 121
EP - 125
DO - 10.5220/0011955400003497
PB - SciTePress