Executive Summary : | This project aims to advance the problem of distributed parameter estimation and its applications in distributed machine learning. In many cases, data is generated and collected in a distributed fashion, but inference is made in a central unit or server. Communication links form the bottleneck, and it is generally not feasible to share the data with the central server. Privacy concerns also prevent data sharing. This has led to the development of federated learning, where users use their private data to train local models, which are then shared with the server who aggregates the local models to create a global model. The problem of estimating the global model can be modeled as communication-efficient distributed estimation of the mean of high-dimensional vectors. The project will focus on developing improved algorithms for this problem and applying them to federated learning algorithms. It assumes a framework with n clients/users, with the i'th user having a sample x(i). The goal is to study this problem under two different settings: random samples (each x(i) is drawn independently from a distribution from a parametric family) and worst-case samples (each x(i) can be adversarially chosen as a function of the protocol). The goal is to design a protocol that allows the server to estimate the mean of the distribution in Case 1 or the empirical mean of the samples in Case 2 as reliably as possible. The broad scope of the problem is to understand fundamental tradeoffs between communication and reliability of the estimate and develop robust algorithms for statistical inference in constrained settings. |