[mlpack-git] [mlpack] implement command line programs of softmaxRegression (#466)

Ryan Curtin notifications at github.com
Mon Nov 2 15:39:33 EST 2015


> I would like to, but which optimizer is suit for SoftmaxRegressionFunction?

Just L-BFGS and SGD for now, but more optimizers may be implemented later.

> ... if you think this is acceptable, I would prefer auto in this case; If you think explicit type is better, I would use explicit types(already change to explicit type)

I see and understand your reasoning, but I disagree with Herb Sutter's reasoning about clarity.  I think the `auto x = type{ statement }` syntax is really clunky.  But without descending into a long discussion about opinions, I think the reasonable thing to do here is to say that you can use `auto` if it's your preference (as long as it's clear what the type actually is), even though I'd personally do it differently; I don't think it's reasonable to make any kind of "no auto" rule for mlpack.

> could we make Prediction as const member function?Because it never changed any data member. 

Absolutely, I see no reason why not.

> Another suggestion is about performance, maybe cache the local variables of Predict function(like probabilities and hypothesis) can speed up the process, especially when you need to use the same model to predict a lot of input data(this one need test, maybe it is not worth to do this kind of optimization). But this would make the Predict functions become non thread safe.

The `Predict()` function already works in batch---it takes an `arma::mat&` as input.  If you want to classify many points at once, then just pass a big `arma::mat&` instead of `arma::vec&`s one at a time.  We have to be careful with caching: although you can get speedup from caching, you don't want the class to ever be in a state where `Predict()` can give incorrect results if the user does something unexpected, like changing the model parameters between calls to `Predict()`.

> I do not familiar with gpgpu yet(studying), but quite interesting in gpgpu. If mlpack has a plan(in future) to provide CUDA support, I would like to participate too:)

There's not a plan in place at the moment, but the corporate powers that loosely control what I do want to eventually see GPU support.  I personally think that CUDA code is really ugly and difficult to work with, but I have yet to find adequate time to do a good investigation of alternatives.  I'll let you know when I get around to working on it (it could be years), or, if you start working on it, feel free to open some issues/pull requests---but expect a lot of discussion because my huge priority is that the GPU code is easy to maintain, easy to understand, and easy to work with (and the CUDA code I have seen is not any of those things).

One interesting thing is NVBLAS, which is an implementation of BLAS that will send the computation to the GPU if the library thinks the GPU will be faster.  I have yet to play with it, but you could use Armadillo+NVBLAS under mlpack and you might get some great speedup, especially for algorithms that use a lot of linear algebra (like maybe LARS).

---
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/pull/466#issuecomment-153149669
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20151102/b442d91e/attachment.html>


More information about the mlpack-git mailing list