[mlpack] Fwd: Degenerate cases and other GMM problems

[John Demme]

Thanks for your help, Ryan!

I'm testing out the latest now. Seems to fix at least one problem. It's
looking like fixing the other may just be a matter of adding trials until
it finds something reasonable, though I'm not sure. I'll be looking at it
closer tomorrow.

~John


On Tue, Apr 9, 2013 at 2:52 PM, Ryan Curtin <gth671b at mail.gatech.edu> wrote:

> On Tue, Apr 09, 2013 at 02:14:09PM -0400, John Demme wrote:
> > (missed reply-all a second ago)
> >
> > That patch is somewhat similar to a mod I tried. Mine helped sometimes,
> but
> > not in all situations. Both problems still occur with r14883:
> >
> > $ ./gmm -i gmm_obs0.csv -g 5
> > [DEBUG] Compiled with debugging symbols.
> > [DEBUG] Iterations: 42
> > [DEBUG] EMFit::Estimate(): initial clustering log-likelihood: 1.4249e+06
> >
> > error: inv(): matrix appears to be singular
> >
> > terminate called after throwing an instance of 'std::runtime_error'
> >   what():
> > Aborted (core dumped)
>
> I have solved one of these, but I may not get to the second problem by
> the end of the day.  r14884 commits the fix.
>
> The issue was not that the entire covariance matrix was zero but instead
> that only one element on the diagonal was zero -- which makes it
> non-invertible.  This could happen when the points for a particular
> cluster have no variance in one dimension (in your case, it was the
> fifth dimension).  So instead of checking if the sum of the covariance
> diagonal is zero, now it checks each element and adds a perturbation
> accordingly.
>
> The perturbation is small (1e-50) so it does not affect the results of
> the phi() function to get the probability of a point.  At the same time,
> it's large enough to make the matrix invertible.
>
> When I have a solution for the second one, I'll let you know.
>
> > On Tue, Apr 9, 2013 at 2:06 PM, Ryan Curtin <gth671b at mail.gatech.edu>
> wrote:
> > > (r148883)
>
> Oops, one too many 8s there.  I was thinking that number looked a little
> large...
>
> Thanks for pointing these issues out.  :)
>
> Ryan
>
> --
> Ryan Curtin       | "Do I sound like I'm ordering a pizza?"
> ryan at igglybob.com |   - John McClane
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20130409/edc9cfb8/attachment.html>