[mlpack] contribute to CNN

Fri Jan 30 23:02:13 EST 2015

HiMarcus,

After reading your implementation of ffnn, I want to contribute to Convolutional NN.
Have you implemeted CNN now? If not, can I start with CNN ?
I think CNN is also in the scope of your framework of ffnn,
I just need to implement ConvolutionalConnectionType and ConvolutionalKernel.
Is it appropriate ?
I’m looking forward to any feedback.

Thanks.

Shangtong Zhang,
Third Year Undergraduate,
School of Computer Science,
Fudan University, China.

原始邮件
发件人:Marcus Edelmarcus.edel at fu-berlin.de
收件人:Shangtong Zhangzhangshangtong.cpp at qq.com
抄送:mlpackmlpack at cc.gatech.edu; Udit Saxenasaxena.udit at gmail.com
发送时间:2015年1月22日(周四) 02:52
主题:Re: [mlpack] add rbm to mlpack

Hello Shangtong,

Do you mean you have implemented these neural networks or just some preparing
work?

Basically I've implemented the structure with extensibility in mind to cover a
bunch of different architectures and techniques; Which is helpful. If we can
provide an algorithm that is highly modular, this opens up possibilities for
other researchers to try modifying the algorithm slightly with ease. For
instance, take a look at the basic neuron layer code in
src/mlpack/methods/ann/layer/ and note that one of the template parameters is a
class which defines how to transform the units (ActivationFunction). If a
researcher wanted to play around with different activation function for basic
layers, then they can implement their own without having to deal with the
structure or training of the net at all. The same applies to the way how the
units are connected or how to optimize the weights, etc.

Anyway right now there isn't any executable for the networks I've mentioned in
the first mail. You need to write some lines of glue code to get results. You
can take a look at the feedforward test to get an impression on how to use the
code.

It seems I don’t find code of implementation of NN in mlpack in github.

The code may be found at:

https://github.com/mlpack/mlpack/tree/master/src/mlpack/methods/ann

I think I can learn much from your code.

Like the rest rest of mlpack, the code uses a lot of template-based optimization
techniques so it could be a little bit difficult at the beginning to get the
first gist. The ann code also makes use of variadic templates which allows us to
write templates that take a variable number of arguments which doesn't make
things better. Let me know if you have any problems or questions. Maybe I should
write a small tutorial or something like that.

And have you implemented RBM ? Can I start with implementing rbm ?

Currently the code lacks an RBM implementation. I think an RBM implementation is
a great start. I've talked with Udit about neural network in another thread
(https://mailman.cc.gatech.edu/pipermail/mlpack/2015-January/000553.html). He is
also interested in contributing an RBM. Maybe we can find a way which works for
both of you so that we can avoid code duplication. Udit, I cc'ed you in case you
have any advice on this.

If you have more questions, feel free to ask.

Thanks,
Marcus

On 21 Jan 2015, at 16:27, Shangtong Zhang zhangshangtong.cpp at qq.com wrote:

Hi Marcus,

Thanks for your reply.
And sorry for my late reply, I’m tring to get familiar with mlpack these days.

Recently I've committed some code to support neural networks
Do you mean you have implementedthese neural networksor just some preparing work?
It seems I don’t find code of implementation of NN in mlpack in github.
I think I can learn much from your code.

And have you implemented RBM ? Can I start with implementing rbm ?

Best
Regards

Shangtong Zhang,
Third Year Undergraduate,
School of Computer Science,
Fudan University, China.

原始邮件
发件人:Marcus Edelmarcus.edel at fu-berlin.de
收件人:Shangtong Zhangzhangshangtong.cpp at qq.com
抄送:mlpackmlpack at cc.gatech.edu
发送时间:2015年1月5日(周一) 03:22
主题:Re: [mlpack] add rbm to mlpack

Hi Shangtong,

Thanks for your interest. mlpack is open source and everyone is welcome to
contribute. :)

Recently I've committed some code to support neural networks (feed forward
neural networks, recurrent neural networks with a bunch of different layers
including LSTM). It's a first a temp to cover recent work. Anyway I'm highly
interested in projects that are going in that direction. I think RBM's are an
interesting idea to start, if you like we can discuss about some details. Over
the mailing list or in irc either way is fine with us.

I think it's a good idea to get familiar with mlpack in the meantime. The best
way to get started is to download mlpack and compile it from source, then use it
for some simple machine learning tasks. The tutorials might prove helpful:

http://www.mlpack.org/tutorial.html

Once you've got a basic feel for mlpack programs and source, you can take a look
at the list of open tickets you might find something interesting:

https://github.com/mlpack/mlpack/issues

Also, there is the mlpack IRC channel (#mlpack on freenode), which could be a
useful resource for real-time help. The logs are here:

http://www.mlpack.org/irc/

Also, be aware that Google hasn't selected orgs yet (The application phase isn't
even open! I guess they open the application phase in about a month). We've
participated in the past years, but this is no guarantee they'll select us
again.

Let me know if I can help out further.

Marcus

On 03 Jan 2015, at 13:21, Shangtong Zhang zhangshangtong.cpp at qq.com wrote:

Hi,

I’m a third year undergraduate majoring in Computer Science in Fudan University, China.
I want to get involved in GSoC 2015 with mlpack. I completed GSoC 2014 with Xapian.

I’m interested in machine learning, and in this field, matlab is popular, but many still want a tool written C++.
Neural Network has been getting more and more popular in recent years and have made great success
in various fields. But it seems I don’t find support for NN in mlpack. So is it appropriate that we make NN supported
in GSoC 2015 ? If this task is a little big, can we just start up with support for Restricted Boltzmann Machines(RBM) ?

Thanks for your reading.

Best Regards

Shangtong Zhang,
Third Year Undergraduate,
School of Computer Science,
Fudan University, China.
_______________________________________________
mlpack mailing list
mlpack at cc.gatech.edu
https://mailman.cc.gatech.edu/mailman/listinfo/mlpack
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20150131/a36b1fb7/attachment-0001.html>