[mlpack] Parallel Stochastic Potimazation Methods
Ryan Curtin
ryan at ratml.org
Thu Mar 17 09:43:49 EDT 2016
On Wed, Mar 16, 2016 at 08:50:06PM +0200, pvachill at ece.auth.gr wrote:
> Hello!
>
> I have implemented a naive parallel edition of the serial sgd implementation
> that already exists in mlpack build, using pthreads. My goal is to make
> HOGWILD! algorithm. I send my code and an example using the test_function.
> Before i continue implementing HOGWILD! algorithm i would like you to
> mention if i am in the correct way.I would also appreciate if someone could
> send me a sparse dataset as mentioned at this paper:
> http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf.
Hi Axilleas,
Thanks for the code. Could you please use OpenMP instead of pthreads?
I think that this problem is solvable with OpenMP, and OpenMP gives much
cleaner and easier to read code.
> From what i have already understand, i could make some suggestions which are
> still a bit of fuzzy in my mind:
> 1)Create stepSize an abstract class that would have different
> implementations for different approaches
This would be a good idea; you could add a "StepStrategy" template
parameter, and then the user could choose something like a constant step
size, a decreasing step size, and so forth.
> 2)Change Optimize member function to template function for different
> approaches
> 3)Create functions that don't use datasets as an abstract class and a second
> abstract class for those who use datasets.
> 4)Also, change sgd to abstract sgd by the same reasoning.
Can you explain more of what you mean by these suggestions? I'm not
sure I understand them.
You can find large sparse datasets at the MovieLens website:
http://grouplens.org/datasets/movielens/
But there is a little more work that you'll have to do to test Hogwild!
in the manner suggested in the paper; you'll have to write a function
class that implements the objective function for matrix decomposition,
and then use that function with your parallel SGD class to test.
Let me know if I can clarify anything.
Thanks!
Ryan
--
Ryan Curtin | "Gentlemen, you can't fight in here! This is the
ryan at ratml.org | War Room!" - President Muffley
More information about the mlpack
mailing list