CHARACTERIZATION OF ALL SOLUTIONS FOR UNDERSAMPLED UNCORRELATED LINEAR DISCRIMINANT ANALYSIS PROBLEMS

In this paper the uncorrelated linear discriminant analysis (ULDA) for undersampled problems is studied. The main contributions of the present work include the following: (i) all solutions of the optimization problem used for establishing the ULDA are parameterized explicitly; (ii) the optimal solut...

Full description

Saved in:
Bibliographic Details
Published in:SIAM journal on matrix analysis and applications 2011-07, Vol.32 (3), p.820-844
Main Authors: DELIN CHU, SIONG THYE GOH, HUNG, Y. S
Format: Article
Language:English
Subjects:
Algebra
Algorithms
Applied mathematics
Calculus of variations and optimal control
Discriminant analysis
Exact sciences and technology
Experiments
Hierarchies
Linear and multilinear algebra, matrix theory
Mathematical analysis
Mathematics
Multivariate analysis
Numerical analysis
Numerical analysis. Scientific computation
Numerical linear algebra
Optimization
Probability and statistics
Sciences and techniques of general use
Statistics
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In this paper the uncorrelated linear discriminant analysis (ULDA) for undersampled problems is studied. The main contributions of the present work include the following: (i) all solutions of the optimization problem used for establishing the ULDA are parameterized explicitly; (ii) the optimal solutions among all solutions of the corresponding optimization problem are characterized in terms of both the ratio of between-class distance to within-class distance and the maximum likelihood classification, and it is proved that these optimal solutions are exactly the solutions of the corresponding optimization problem with minimum Frobenius norm, also minimum nuclear norm; these properties provide a good mathematical justification for preferring the minimum-norm transformation over other possible solutions as the optimal transformation in ULDA; (iii) explicit necessary and sufficient conditions are provided to ensure that these minimal solutions lead to a larger ratio of between-class distance to within-class distance, thereby achieving larger discrimination in the reduced subspace than that in the original data space, and our numerical experiments show that these necessary and sufficient conditions hold true generally. Furthermore, a new and fast ULDA algorithm is developed, which is eigendecomposition-free and SVD-free, and its effectiveness is demonstrated by some real-world data sets. [PUBLICATION ABSTRACT]
ISSN:0895-4798
1095-7162
DOI:10.1137/100792007