FR4.R7.2

SignSGD-FV: Communication-Efficient Distributed Learning through Heterogeneous Edges

Chanho Park, POSTECH, Korea (South); H. Vincent Poor, Princeton University, United States; Namyoon Lee, Korea University, Korea (South)

Session:
Distributed Learning

Track:
15: Distributed and Federated Learning

Location:
VIP

Presentation Time:
Fri, 12 Jul, 16:45 - 17:05

Session Chair:
Randall Berry, Northwestern University
Abstract
This paper presents signSGD with \textit{federated voting} (signSGD-FV), a communication-efficient distributed learning algorithm with heterogeneous edge workers. The FV aggregation leverages the log-likelihood ratio (LLR) weight assigned to each worker, and performs weighted majority voting aggregation by interpreting the conventional signSGD with majority voting (signSGD-MV) algorithm in a coding-theoretical approach. The LLR weights are estimated based on the aggregation results determined by the sign votes of workers, which shows the essence of federated voting. Our theoretical analyses and the experimental results on real-world datasets demonstrate the superiority of signSGD-FV for both communication efficiency and learning performance when the workers employ different sizes of mini-batches.
Resources