Max-Affine Regression: Parameter Estimation for Gaussian Designs

Max-affine regression refers to a model where the unknown regression function is modeled as a maximum of k unknown affine functions for a fixed k \geq 1 . This generalizes linear regression and (real) phase retrieval, and is closely related to convex regression. We study this problem in the high-...

Full description

Saved in:
Bibliographic Details
Published in:IEEE transactions on information theory 2022-03, Vol.68 (3), p.1851-1885
Main Authors: Ghosh, Avishek, Pananjady, Ashwin, Guntuboyina, Adityanand, Ramchandran, Kannan
Format: Article
Language:English
Subjects:
Algorithms
alternating minimization
Computational modeling
Convex functions
dimension reduction
Estimation
iterative optimization
Mathematical models
Max-affine regression
Minimax technique
Minimization
Modeling
Optimization
Parameter estimation
Phase retrieval
Regression models
Regularization
Search algorithms
Sharpness
Spectral methods
Statistical analysis
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Max-affine regression refers to a model where the unknown regression function is modeled as a maximum of k unknown affine functions for a fixed k \geq 1 . This generalizes linear regression and (real) phase retrieval, and is closely related to convex regression. We study this problem in the high-dimensional setting assuming that k is a fixed constant, and focus on the estimation of the unknown coefficients of the affine functions underlying the model. We analyze a natural alternating minimization (AM) algorithm for the non-convex least squares objective when the design is Gaussian. We show that the AM algorithm, when initialized suitably, converges with high probability and at a geometric rate to a small ball around the optimal coefficients. In order to initialize the algorithm, we propose and analyze a combination of a spectral method and a search algorithm in a low-dimensional space, which may be of independent interest. The final rate that we obtain is near-parametric and minimax optimal (up to a polylogarithmic factor) as a function of the dimension, sample size, and noise variance. In that sense, our approach should be viewed as a direct and implementable method of enforcing regularization to alleviate the curse of dimensionality in problems of the convex regression type. Numerical experiments illustrate the sharpness of our bounds in the various problem parameters.
ISSN:0018-9448
1557-9654
DOI:10.1109/TIT.2021.3130717