[mlpack] Discussion about a new Idea

[Ryan Curtin]

On Wed, Apr 24, 2013 at 09:47:27AM -0700, Salman Javaid wrote:
> Hello Ryan:
>                Currently from what I know of the ml-pack documentation, EM
> algorithm has not been implemented. Is there a possibility that we can
> discuss implementing it for ml-pack?

Hello Salman,

The EM algorithm is implemented for Gaussian mixture models (GMMs) as
EMFit.  How do you plan on implementing it?  
itself to an implementation kind of like...

template<typename FunctionType>
class EM;

where the FunctionType is the probability distribution of the model type
we are optimizing.  But taking the expectation of the log-likelihood is,
in general, something that has to be done differently for each possible
model (from an implementation standpoint).  Meaning that your
FunctionType would have to have some function that calculates the
expectation of the log-likelihood, and then it will also have to have
some function which does the maximization step.

So then your EM::Train() function (or whatever you call it) ends up
looking something like this:

EM::Train(FunctionType& f)
{
  arma::mat coordinates = f.GetInitialPoint();
  while(!converged)
  {
    f.Expectation(coordinates);
    f.Maximize(coordinates);
    converged = f.IsConverged();
  }
}

which is so simple it's not actually even worth implementing.

It would be better to implement a particular application of the EM
algorithm instead of just the generalized EM algorithm, which would be
too generalized to be useful (unless you know something I don't!).

Ryan

-- 
Ryan Curtin       | "Don't fight it son. Confess quickly! If you hold out too
ryan at igglybob.com | long you could jeopardize your credit rating."  - Guard