Federated learning into true individual alignment

My next order of business is developing a concrete understanding of federated learning (and, if it comes along with it, distributed training, etc.). This is an important problem for me because I actually do not believe in a single gigantic “aligned” model.

I reason the following steps to achieve a good understanding:

• gain the intuition behind federated learning,
• know the unique problems FL has and what the SOTA solution to them are.
• know for sure whether it can (and if it does, how to) help true individual alignment of AI models.

Some seminal papers:

• Federated Optimization, 2016
• Federated Learning, 2016
• Improving Communication Efficiency of FL, 2017

Importantly, one may think about LoRA as one of the tangential directions.

Then I can start reviewing the papers under this direction, such as:

• Federated Online and Bandit Convex Optimization