[mlpack] add rbm to mlpack

Marcus Edel marcus.edel at fu-berlin.de
Wed Jan 21 13:52:51 EST 2015 

Hello Shangtong,

> Do you mean you have implemented these neural networks or just some preparing
work?

Basically I've implemented the structure with extensibility in mind to cover a
bunch of different architectures and techniques; Which is helpful. If we can
provide an algorithm that is highly modular, this opens up possibilities for
other researchers to try modifying the algorithm slightly with ease. For
instance, take a look at the basic neuron layer code in
src/mlpack/methods/ann/layer/ and note that one of the template parameters is a
class which defines how to transform the units (ActivationFunction). If a
researcher wanted to play around with different activation function for basic
layers, then they can implement their own without having to deal with the
structure or training of the net at all. The same applies to the way how the
units are connected or how to optimize the weights, etc.

Anyway right now there isn't any executable for the networks I've mentioned in
the first mail. You need to write some lines of glue code to get results. You
can take a look at the feedforward test to get an impression on how to use the
code.

> It seems I don’t find code of implementation of NN in mlpack in github.


The code may be found at:

https://github.com/mlpack/mlpack/tree/master/src/mlpack/methods/ann

> I think I can learn much from your code.


Like the rest rest of mlpack, the code uses a lot of template-based optimization
techniques so it could be a little bit difficult at the beginning to get the
first gist. The ann code also makes use of variadic templates which allows us to
write templates that take a variable number of arguments which doesn't make
things better. Let me know if you have any problems or questions. Maybe I should
write a small tutorial or something like that.

> And have you implemented RBM ? Can I start with implementing rbm ?


Currently the code lacks an RBM implementation. I think an RBM implementation is
a great start. I've talked with Udit about neural network in another thread
(https://mailman.cc.gatech.edu/pipermail/mlpack/2015-January/000553.html). He is
also interested in contributing an RBM. Maybe we can find a way which works for
both of you so that we can avoid code duplication. Udit, I cc'ed you in case you
have any advice on this.

If you have more questions, feel free to ask.

Thanks,
Marcus



> On 21 Jan 2015, at 16:27, Shangtong Zhang <zhangshangtong.cpp at qq.com> wrote:
> 
> Hi Marcus,
> 
> Thanks for your reply. 
> And sorry for my late reply, I’m tring to get familiar with mlpack these days.
> 
> >Recently I've committed some code to support neural networks 
> Do you mean you have implemented these neural networks or just some preparing work? 
> It seems I don’t find code of implementation of NN in mlpack in github.
> I think I can learn much from your code.
> 
> And have you implemented RBM ? Can I start with implementing rbm ?
> 
> Best 
> Regards
> 
> Shangtong Zhang,
> Third Year Undergraduate,
> School of Computer Science,
> Fudan University, China.
> 
>                              原始邮件                             
> 发件人: Marcus Edel<marcus.edel at fu-berlin.de>
> 收件人: Shangtong Zhang<zhangshangtong.cpp at qq.com>
> 抄送: mlpack<mlpack at cc.gatech.edu>
> 发送时间: 2015年1月5日(周一) 03:22
> 主题: Re: [mlpack] add rbm to mlpack
> 
> Hi Shangtong,
> 
> Thanks for your interest. mlpack is open source and everyone is welcome to
> contribute. :)
> 
> Recently I've committed some code to support neural networks (feed forward
> neural networks, recurrent neural networks with a bunch of different layers
> including LSTM). It's a first a temp to cover recent work. Anyway I'm highly
> interested in projects that are going in that direction. I think RBM's are an
> interesting idea to start, if you like we can discuss about some details. Over
> the mailing list or in irc either way is fine with us.
> 
> I think it's a good idea to get familiar with mlpack in the meantime. The best
> way to get started is to download mlpack and compile it from source, then use it
> for some simple machine learning tasks. The tutorials might prove helpful:
> 
> http://www.mlpack.org/tutorial.html <http://www.mlpack.org/tutorial.html>
> 
> Once you've got a basic feel for mlpack programs and source, you can take a look
> at the list of open tickets you might find something interesting:
> 
> https://github.com/mlpack/mlpack/issues <https://github.com/mlpack/mlpack/issues>
> 
> Also, there is the mlpack IRC channel (#mlpack on freenode), which could be a
> useful resource for real-time help. The logs are here:
> 
> http://www.mlpack.org/irc/ <http://www.mlpack.org/irc/>
> 
> Also, be aware that Google hasn't selected orgs yet (The application phase isn't
> even open! I guess they open the application phase in about a month). We've
> participated in the past years, but this is no guarantee they'll select us
> again.
> 
> Let me know if I can help out further.
> 
> Marcus
> 
> 
> 
>> On 03 Jan 2015, at 13:21, Shangtong Zhang <zhangshangtong.cpp at qq.com <mailto:zhangshangtong.cpp at qq.com>> wrote:
>> 
>> Hi,
>> 
>> I’m a third year undergraduate majoring in Computer Science in Fudan University, China.
>> I want to get involved in GSoC 2015 with mlpack. I completed GSoC 2014 with Xapian.
>> 
>> I’m interested in machine learning, and in this field, matlab is popular, but many still want a tool written C++.
>> Neural Network has been getting more and more popular in recent years and have made great success 
>> in various fields. But it seems I don’t find support for NN in mlpack. So is it appropriate that we make NN supported 
>> in GSoC 2015 ? If this task is a little big, can we just start up with support for Restricted Boltzmann Machines(RBM) ?
>> 
>> Thanks for your reading.
>> 
>> Best Regards
>> 
>> 
>> 
>> Shangtong Zhang,
>> Third Year Undergraduate,
>> School of Computer Science,
>> Fudan University, China.
>> _______________________________________________
>> mlpack mailing list
>> mlpack at cc.gatech.edu <mailto:mlpack at cc.gatech.edu>
>> https://mailman.cc.gatech.edu/mailman/listinfo/mlpack <https://mailman.cc.gatech.edu/mailman/listinfo/mlpack>
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20150121/2a716bf2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5136 bytes
Desc: not available
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20150121/2a716bf2/attachment-0001.bin>

More information about the mlpack mailing list