Communication Dans Un Congrès Année : 2025

In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting

Résumé

We analyze a distributed algorithm to compute a low-rank matrix factorization on $N$ clients, each holding a local dataset $\mathbf{S}^i \in \mathbb{R}^{n_i \times d}$, mathematically, we seek to solve $min_{\mathbf{U}^i \in \mathbb{R}^{n_i\times r}, \mathbf{V}\in \mathbb{R}^{d \times r} } \frac{1}{2} \sum_{i=1}^N \|\mathbf{S}^i - \mathbf{U}^i \mathbf{V}^\top\|^2_{\text{F}}$. Considering a power initialization of $\mathbf{V}$, we rewrite the previous smooth non-convex problem into a smooth strongly-convex problem that we solve using a parallel Nesterov gradient descent potentially requiring a single step of communication at the initialization step. For any client $i$ in $\{1, \dots, N\}$, we obtain a global $\mathbf{V}$ in $\mathbb{R}^{d \times r}$ common to all clients and a local variable $\mathbf{U}^i$ in $\mathbb{R}^{n_i \times r}$. We provide a linear rate of convergence of the excess loss which depends on $\sigma_{\max} / \sigma_{r}$, where $\sigma_{r}$ is the $r^{\mathrm{th}}$ singular value of the concatenation $\mathbf{S}$ of the matrices $(\mathbf{S}^i)_{i=1}^N$. This result improves the rates of convergence given in the literature, which depend on $\sigma_{\max}^2 / \sigma_{\min}^2$. We provide an upper bound on the Frobenius-norm error of reconstruction under the power initialization strategy. We complete our analysis with experiments on both synthetic and real data.
Fichier principal
Vignette du fichier
2024_matrix_factorisation_AAAI.pdf (733.95 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04893622 , version 1 (17-01-2025)

Licence

Identifiants

  • HAL Id : hal-04893622 , version 1

Citer

Constantin Philippenko, Kevin Scaman, Laurent Massoulié. In-depth Analysis of Low-rank Matrix Factorisation in a Federated Setting. AAAI 2025 - 39th Annual AAAI Conference on Artificial Intelligence, Feb 2025, Philadelphia, United States. ⟨hal-04893622⟩
0 Consultations
0 Téléchargements

Partager

More