Automatic Performance Estimation for Decentralized Optimization

We present a methodology to automatically compute worst-case performance bounds for a large class of first-order decentralized optimization algorithms. These algorithms aim at minimizing the average of local functions that are distributed across a network of agents. They typically combine local comp...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on automatic control 2023-12, Vol.68 (12), p.1-15
Main Authors: Colla, Sebastien, Hendrickx, Julien M.
Format: Article
Language:English
Subjects:
Algorithms
Communication networks
Consensus
Convergence
Distributed optimization
Estimation
Interpolation
Methodology
Optimization
Optimization algorithms
Optimization methods
Performance estimation problem
Rates of convergence
Robustness
Symmetric matrices
Tuning
Worst-case analysis
Citations: Items that this one cites
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:We present a methodology to automatically compute worst-case performance bounds for a large class of first-order decentralized optimization algorithms. These algorithms aim at minimizing the average of local functions that are distributed across a network of agents. They typically combine local computations and consensus steps. Our methodology is based on the approach of Performance Estimation Problem (PEP), which allows computing the worst-case performance and a worst-case instance of first-order optimization algorithms by solving an SDP. We propose two ways of representing consensus steps in PEPs, which allow writing and solving PEPs for decentralized optimization. The first formulation is exact but specific to a given averaging matrix. The second formulation is a relaxation but provides guarantees valid over an entire class of averaging matrices, characterized by their spectral range. This formulation often allows recovering a posteriori the worst possible averaging matrix for the given algorithm. We apply our methodology to three different decentralized methods. For each of them, we obtain numerically tight worst-case performance bounds that significantly improve on the existing ones, as well as insights about the parameters tuning and the worst communication networks.
ISSN:0018-9286
1558-2523
DOI:10.1109/TAC.2023.3251902