[mlpack] sparse coding test examples in mlpack

[Ryan Curtin]

On Wed, Jun 10, 2015 at 05:00:48PM -0700, Jianyu Huang wrote:
> Hi Ryan & Nishant,
> 
> Thank you for illuminating these points.
> 
> 1. Thanks for that. It helps!
> 3. OK, so I guess there are some corner cases which mlpack cannot handle
> with. Is it related to numerically stability, especially for some
> ill-conditioned matrix input?

It might be easier to figure out what is going on here if you can
pinpoint the exact operation that is failing.  Like I said, I didn't
write the sparse coding code, so I'm maybe not the best person to help
you with that issue.

> 4. I still insist on that mlpack doesn't implement feature-sign search
> algorithm.
> In sparse_coding_impl.hpp file:
> template<typename DictionaryInitializer>
> void SparseCoding<DictionaryInitializer>::OptimizeCode()
> only invokes lars.Regress function. Looking through lars.Regress()
> function, I felt it is a traditional Cholesky-based implementation of the
> LARS-Lasso algorithm, instead of Feature-sign algorithm in Honglak Lee's
> "Efficient sparse coding algorithms" > (NIPS 2006) paper. Correct me if I
> am wrong.

Yes, I spent some time with the paper, and I believe that you are
correct.  I've committed a comment in sparse_coding.hpp which points
this out (4156e3f).

> 5.
> How can I use parallel BLAS when I configured and built mlpack? I only see
> the configuration page for CMake here:
> http://www.mlpack.org/doxygen.php?doc=build.html
> So I don't know how to add cflag options like "mkl=parallel" etc.

mlpack's CMake configuration should replicate your configuration of
Armadillo.  So whatever you have set in armadillo_bits/config.hpp will
be the configuration that is used.  If you have ARMA_USE_WRAPPER set,
then mlpack will link against libarmadillo.so, and then if you built
Armadillo against MKL, then mlpack will be using MKL too.

If you don't have ARMA_USE_WRAPPER set, it may be a bit more difficult
to get MKL working.  In order of preference, mlpack's CMake will use
MKL, ACML, OpenBLAS, ATLAS, then LAPACK/BLAS.  This is the same as for
Armadillo's CMake configuration.  When you use CMake to configure
mlpack, do you get output on what Armadillo libraries are linked
against?  It should be something like this:

-- Armadillo libraries: /usr/lib/libarmadillo.so

Anyway, sorry for the slow response -- I was out of town late last week
and couldn't get to this.

-- 
Ryan Curtin    | "Hold still."
ryan at ratml.org |   - Mr. Blonde