A new class of stochastic EM algorithms. Escaping local maxima and handling intractable sampling

The expectation–maximization (EM) algorithm is a powerful computational technique for maximum likelihood estimation in incomplete data models. When the expectation step cannot be performed in closed form, a stochastic approximation of EM (SAEM) can be used. The convergence of the SAEM toward critica...

Full description

Saved in:
Bibliographic Details
Published in:Computational statistics & data analysis 2021-07, Vol.159, p.107159, Article 107159
Main Authors: Allassonnière, Stéphanie, Chevallier, Juliette
Format: Article
Language:English
Subjects:
Computer Science
EM-like algorithm
Machine Learning
Mathematics
Methodology
Modeling and Simulation
Statistics
Statistics Theory
Stochastic approximation
Stochastic optimization
Tempered distribution
Theoretical convergence
Citations: Items that this one cites
Items that cite this one
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The expectation–maximization (EM) algorithm is a powerful computational technique for maximum likelihood estimation in incomplete data models. When the expectation step cannot be performed in closed form, a stochastic approximation of EM (SAEM) can be used. The convergence of the SAEM toward critical points of the observed likelihood has been proved and its numerical efficiency has been demonstrated. However, sampling from the posterior distribution may be intractable or have a high computational cost. Moreover, despite appealing features, the limit position of this algorithm can strongly depend on its starting one. Sampling from an approximation of the distribution in the expectation phase of the SAEM allows coping with these two issues. This new procedure is referred to as approximated-SAEM and is proved to converge toward critical points of the observed likelihood. Experiments on synthetic and real data highlight the performance of this algorithm in comparison to the SAEM and the EM when feasible.
ISSN:0167-9473
1872-7352
DOI:10.1016/j.csda.2020.107159