Publication
Online EM with weight-based forgetting
Journal Article (2015)
Journal
Neural Computation
Pages
1142–1157
Volume
27
Number
5
Doc link
http://dx.doi.org/10.1162/NECO_a_00723
File
Authors
-
Celaya Llover, Enric
-
Agostini, Alejandro
Abstract
In the on-line version of the EM algorithm introduced by Sato and Ishii (2000), a time-dependent discount factor is introduced for forgetting the effect of the old posterior values obtained with an earlier, inaccurate estimator. In their approach, forgetting is uniformly applied to the estimators of each mixture component depending exclusively on time, irrespective of the weight attributed to each unit for the observed sample. This causes an excessive forgetting in the less frequently sampled regions. To address this problem we propose a modification of the algorithm that involves a weight-dependent forgetting, different for each mixture component, in which old observations are forgotten according to the actual weight of the new samples used to replace older values. A comparison of the time-dependent versus the weight-dependent approach shows that the last one improves the accuracy of the approximation and exhibits a much greater stability.
Categories
learning (artificial intelligence), stochastic programming.
Scientific reference
E. Celaya and A. Agostini. Online EM with weight-based forgetting. Neural Computation, 27(5): 1142–1157, 2015.
Follow us!