Loading…
Optimal Diagonal Preconditioning
A New Practical Algorithm Enables Optimal Preconditioning A classic and important question in optimization and numerical methods is how to find diagonal preconditioners to maximally reduce the condition number of any matrix with full rank. Until recently, few practical methods could handle large spa...
Saved in:
| Published in: | Operations research 2024-03 |
|---|---|
| Main Authors: | , , , , |
| Format: | Article |
| Language: | English |
| Subjects: | |
| Citations: | Items that this one cites |
| Online Access: | Get full text |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | A New Practical Algorithm Enables Optimal Preconditioning
A classic and important question in optimization and numerical methods is how to find diagonal preconditioners to maximally reduce the condition number of any matrix with full rank. Until recently, few practical methods could handle large sparse matrices. In “Optimal Diagonal Preconditioning,” the authors show that this problem can be modeled using quasiconvex optimization and semidefinite programming. Leveraging these insights, they develop algorithms to efficiently find optimal diagonal preconditioners for large sparse systems. They find that although heuristic diagonal preconditioners are popular in practice, their performance at reducing condition numbers could have a significant gap to optimal diagonal preconditioners. This work provides theoretical foundation for future works on optimal preconditioning as well as practical implementations that could be used to build more sophisticated software for optimal preconditioning at scale. A main advantage of the framework in “Optimal Diagonal Preconditioning” is its potential to be scaled up to handle even larger matrices, which is an exciting direction for numerical optimization.
Preconditioning has long been a staple technique in optimization, often applied to reduce the condition number of a matrix and speed up the convergence of algorithms. Although there are many popular preconditioning techniques in practice, most lack guarantees on reductions in condition number, and the degree to which we can improve over existing heuristic preconditioners remains an important question. In this paper, we study the problem of optimal diagonal preconditioning that achieves maximal reduction in the condition number of any full-rank matrix by scaling its rows and/or columns with positive numbers. We first reformulate the problem as a quasiconvex optimization problem and provide a simple algorithm based on bisection. Then, we develop an interior point algorithm with
O
(
log
(
1
/
ϵ
)
)
iteration complexity, where each iteration consists of a Newton update based on the Nesterov-Todd direction. Next, we specialize in one-sided optimal diagonal preconditioning problems and demonstrate that they can be formulated as standard dual semidefinite program (SDP) problems. We then develop efficient customized solvers for the SDP approach and study the empirical performance of our optimal diagonal preconditioning procedures through extensive experiments. Our findings suggest t |
|---|---|
| ISSN: | 0030-364X 1526-5463 |
| DOI: | 10.1287/opre.2022.0592 |