1 minute read

We have 1 paper accepted by CVPR 2025! This paper focuses on the federated learning and contributes to the traditional federated learning with pre-trained models. This paper extends the weight-invariance in the analytic continual learning and proposes the analytic federated learning, which provides closed-formed solutions to federated learning with invariance to data partitioning. Also, this paper achieves 150x-200x faster training speed when compared with existing methods. Detailed information is listed below.

AFL: A Single-Round Analytic Approach for Federated Learning with Pre-trained Models

Authors: Run He, Kai Tong, Di Fang, Han Sun, Ziqian Zeng, Haoran Li, Tianyi Chen, Huiping Zhuang

Abstract:

In this paper, we introduce analytic federated learning (AFL), a new training paradigm that brings analytical (i.e., closed-form) solutions to the federated learning (FL) with pre-trained models. Our AFL draws inspiration from analytic learning—a gradient-free technique that trains neural networks with analytical solutions in one epoch. In the local client training stage, the AFL facilitates a one-epoch training, eliminating the necessity for multi-epoch updates. In the aggregation stage, we derive an absolute aggregation (AA) law. This AA law allows a single-round aggregation, reducing heavy communication overhead and achieving fast convergence by removing the need for multiple aggregation rounds. More importantly, the AFL exhibits a property that invariance to data partitioning, meaning that regardless of how the full dataset is distributed among clients, the aggregated result remains identical. This could spawn various potentials, such as data heterogeneity invariance and client-number invariance. We conduct experiments across various FL settings including extremely non-IID ones, and scenarios with a large number of clients (e.g., >1000). In all these settings, our AFL constantly performs competitively while existing FL techniques encounter various obstacles.

Paradigm of proposed analytic continual learning

Paradigm of proposed analytic continual learning

Paper: https://arxiv.org/abs/2405.16240

Code: https://github.com/ZHUANGHP/Analytic-federated-learning

Updated:

You may also enjoy

We have 3 papers accepted by ICML 2025!

3 minute read

We have three papers are accepted by the 42nd International Conference on Machine Learning (ICML’2025). Two papers contribute to the continual learning, addr...

We have 4 papers accepted by ACL 2025!

4 minute read

We have four papers are accepted by the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025). Three papers have made contributions...