[mlpack-svn] [MLPACK] #254: NCA does not find good solutions

[MLPACK Trac]

#254: NCA does not find good solutions
---------------------+------------------------------------------------------
 Reporter:  rcurtin  |        Owner:                                            
     Type:  defect   |       Status:  new                                       
 Priority:  major    |    Milestone:  mlpack 1.0.4                              
Component:  mlpack   |     Keywords:  nca, cgd, solution, convergence, nonconvex
 Blocking:           |   Blocked By:                                            
---------------------+------------------------------------------------------
 Currently NCA uses the L-BFGS optimizer, but, the NCA objective function
 is potentially nonconvex and thus the L-BFGS optimizer may not find a good
 solution.

 Here is an excerpt of results comparing a couple of methods:

 {{{

 cardiotoc       train mean      train variance        runtime (mean)
 test mean      test variance

 baseline        40.423%         0.015178%             -----
 35.031%          0.065745%
 lmnn            71.969%         0.085596%               15.228s
 66.050%          0.097625%
 nca (mlpack)    38.985%         0.019756%              186.37s
 33.103%          0.050762%
 nca (matlab)
 rca
 fdsa
 m-h mcmc        66.176%         0.062989%             2892.6s
 55.392%          0.10958%
 m-h mcmc diag   74.637%         0.036214%              826.43s
 66.176%          0.16475%
 m-h diag (10k)  77.863%         0.032059%             3005.9s
 69.843%          0.07890%
 m-h diag (25k)  80.040%         0.018136%             7401.9s
 72.038%          0.05013%
 gibbs mcmc
 }}}

 Almost certainly it's getting stuck in local minima.  Therefore, we need
 to use a different optimizer...

-- 
Ticket URL: <http://trac.research.cc.gatech.edu/fastlab/ticket/254>
MLPACK <www.fast-lab.org>
MLPACK is an intuitive, fast, and scalable C++ machine learning library developed by the FASTLAB at Georgia Tech under Dr. Alex Gray.