[mlpack] Parallel Stochastic Potimazation Methods
pvachill at ece.auth.gr
pvachill at ece.auth.gr
Wed Mar 16 14:50:06 EDT 2016
Hello!
I have implemented a naive parallel edition of the serial sgd
implementation that already exists in mlpack build, using pthreads. My
goal is to make HOGWILD! algorithm. I send my code and an example
using the test_function. Before i continue implementing HOGWILD!
algorithm i would like you to mention if i am in the correct way.I
would also appreciate if someone could send me a sparse dataset as
mentioned at this paper:
http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf.
From what i have already understand, i could make some suggestions
which are still a bit of fuzzy in my mind:
1)Create stepSize an abstract class that would have different
implementations for different approaches
2)Change Optimize member function to template function for different
approaches
3)Create functions that don't use datasets as an abstract class and a
second abstract class for those who use datasets.
4)Also, change sgd to abstract sgd by the same reasoning.
Thanks,
Axilleas
Quoting Ryan Curtin <ryan at ratml.org>:
> On Fri, Mar 11, 2016 at 07:59:08PM +0200, pvachill at ece.auth.gr wrote:
>>
>> Hi
>>
>> I am a 5th year undergraduate Electrical Engineer and Computer Engineer
>> student at Aristotle University. My name is Axilleas Pasias and i am from
>> Greece. I have background in Pthreads, OpenMP, MPI and Cuda.I also have some
>> knowledge in convex optimization. I would like you to recommend me some
>> papers about Parallel stochastic optimization methods to get in touch with
>> the idea.
>
> Hi Axilleas,
>
> Here are some interesting papers.
>
> http://papers.nips.cc/paper/4390-hogwild-a-lock-free-approach-to-parallelizing-stochastic-gradient-descent.pdf
> http://papers.nips.cc/paper/4006-parallelized-stochastic-gradient-descent.pdf
> http://leon.bottou.org/publications/pdf/compstat-2010.pdf
>
> Note that for mlpack, the desired scenario is a multi-core single
> machine, not a distributed system.
>
> I hope that this is helpful. Let me know if I can clarify anything.
>
> Thanks,
>
> Ryan
>
> --
> Ryan Curtin | "And they say there is no fate, but there is: it's
> ryan at ratml.org | what you create." - Minister
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Parallel_SGD_An_Attempt.tar.gz
Type: application/x-compressed-tar
Size: 50488 bytes
Desc: not available
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20160316/6665edf4/attachment-0001.bin>
More information about the mlpack
mailing list