[mlpack-git] [mlpack] hmm_train failed to converge (#496)

[Ryan Curtin]

Hi there,

Sorry for the slow response.  It took me a while to dig into this one.  I believe that what was happening here was that some particular Gaussian from one of the GMMs that make up your HMM emission distribution had a covariance matrix which wasn't positive definite (or, positive definite enough for chol()).  Previously, this issue had been handled by the file `src/mlpack/methods/gmm/positive_definite_constraint.hpp` [1], which, at the end of each EM algorithm iteration during GMM training, would add small amounts to the diagonal until the determinant was greater than 0 (and thus the matrix would be positive definite).

However, what seemed to actually be occurring was that the determinant was indeed greater than 0, but because of floating point errors (I theorize) the call to `arma::chol()` would fail.  So, in e08a8ff I committed a better strategy for checking positive definiteness: instead of using `arma::det()` and forcing the determinant to be greater than some small value like 1e-50, we just call `arma::chol()` to make sure it converges, and if not, add to the diagonal.  This guarantees that when the `PositiveDefiniteConstraint` is applied, the covariance matrix can be decomposed with `arma::chol()`, and therefore the problem you're having, you shouldn't have anymore (assuming you update to git master).

As a side note, I am about to rename all of the executables to be prefixed with `mlpack_`, so `hmm_train` will become `mlpack_hmm_train`, as per #229, in the next couple days or so, so if you update and rebuild mlpack, you may notice that the names have changed.  But no compatibility breakage or anything, so there shouldn't be any issues -- just giving you a heads up. :)

If my big long explanation turns out to have been wrong or my fix didn't actually fix your issue, feel free to reopen... (I really hope it is a fix, though, it took me a while to dig to the bottom of that one.) :)

[1] https://github.com/mlpack/mlpack/blob/master/src/mlpack/methods/gmm/positive_definite_constraint.hpp

---
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/issues/496#issuecomment-165892114
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20151218/09090c33/attachment-0001.html>