[mlpack-svn] [MLPACK] #294: constraints on the covariance matrix in GMM/EMFit

[MLPACK Trac]

#294: constraints on the covariance matrix in GMM/EMFit
-----------------------------------------------+----------------------------
  Reporter:  hanslovsky                        |        Owner:  rcurtin     
      Type:  enhancement                       |       Status:  closed      
  Priority:  major                             |    Milestone:  mlpack 1.0.7
 Component:  mlpack                            |   Resolution:  fixed       
  Keywords:  GMM EMFit covariance constraints  |     Blocking:              
Blocked By:                                    |  
-----------------------------------------------+----------------------------
Changes (by rcurtin):

  * status:  accepted => closed
  * resolution:  => fixed


Comment:

 Ok, I have implemented all of these features.  EMFit<> now accepts a
 template parameter that can apply constraints to the covariance matrix
 after estimation.  So, after each covariance re-estimation step, the
 ApplyConstraint() function of the CovarianceConstraintPolicy class is
 called.  There are a number of choices for this policy class that are
 found in src/mlpack/methods/gmm/:

  * NoConstraint (no_constraint.hpp): doesn't modify anything
  * PositiveDefiniteConstraint (positive_definite_constraint.hpp): makes
 sure det(cov) >= 1e-50 so that the matrix is invertible
  * DiagonalConstraint (diagonal_constraint.hpp): forces non-diagonal
 elements of the covariance matrix to zero
  * EigenvalueRatioConstraint (eigenvalue_ratio_constraint.hpp): forces
 eigenvalues to adhere to a predefined list of ratios

 So, in practice, here is an example of how you would use
 EigenvalueRatioConstraint:

 {{{
 arma::vec ratios; // Fill this somehow...
 EigenvalueRatioConstraint erc(ratios);

 // Now create the EMFit object.
 // The key is that we're passing the instantiated constraint object as the
 fourth parameter.
 EMFit<KMeans<>, EigenvalueRatioConstraint> em(300, 1e-10, KMeans(), erc);

 // Now create the GMM object with the instantiated EMFit object.
 GMM<EMFit<KMeans<>, EigenvalueRatioConstraint> gmm(gaussians,
 dimensionality, em);

 // Now you can run the training, etc...
 }}}

 I actually think using DiagonalConstraint yields the same results as if
 you re-derive the EM algorithm for the case of diagonal covariance, but it
 isn't as fast because it's calculating the complete covariance matrix and
 then setting most of those elements to zero.

 The only option given in the 'gmm' command-line executable is still the
 option to not check for a positive-definite covariance matrix (i.e. use
 NoConstraint instead of PositiveDefiniteConstraint), because adding too
 many more options makes it rather complex.

 Anyway, I'm about to release 1.0.7, so you can either wait for that or
 just download trunk from svn now.  Your feedback is appreciated, if you
 find the features useful. :)

-- 
Ticket URL: <http://trac.research.cc.gatech.edu/fastlab/ticket/294#comment:3>
MLPACK <www.fast-lab.org>
MLPACK is an intuitive, fast, and scalable C++ machine learning library developed at Georgia Tech.