[mlpack] disadvantages of using tuple as framework for neural network

[Marcus Edel]

Hello,

> Using a base class for all ConnectionType classes would mean virtual
> functions, which in this context would incur non-negligible slowdown
> (since the ConnectionType's functions will be used so much).


This is absolutely right, using inheritance would results in a measurable
slowdown, which is non-negligible. I couldn't think of another solution as using
tuples in combination with variadic templates to support different layers. Which
doesn't mean there isn't a better solution.

>> But it seems I can't generate it automatically
>> or I need some very complicate c++ technique to do this.
>> http://stackoverflow.com/questions/28410697/c-convert-vector-to-tuple <http://stackoverflow.com/questions/28410697/c-convert-vector-to-tuple>
I think Barry's solution is quite neat, I've done something similar in the RNN
and FFN class. Originally there was the plan to write a 'BuildNetwork' function
which simply creates a network with k units and n layer (layer with the same
type), but I couldn't find the time to write such a function. However, the problem
with the idea is that often we need a different layer type between two layer. Maybe
it makes more sense to write a function which generates a block of layers, like
you asked at stackoverflo. What do you think?

>> This means there are 6 * 16 = 96 connections,
>> then I need to instantiates 96 class of ConnectionType and store them in a tuple.
>> so I need to get a tuple like c1, c2, c3, ………, c96, it’s too large that I can’t write it manually.

I'm not sure about the number of connection. Maybe it is a better idea to write
a new layer type and a new connection. Which will reduce the problem to 5 layer.

Thank you for according time and effort into the code,

Marcus



> On 09 Feb 2015, at 17:36, Ryan Curtin <ryan at ratml.org> wrote:
> 
> On Mon, Feb 09, 2015 at 10:17:39PM +0800, Shangtong Zhang wrote:
>> Hi,
>> 
>> 
>> I implement cnn based on current framework of NN.
>> I meet some problem when I try to test my cnn on mnist dataset.
>> To do classification on mnist dataset, I try to realize LeNet5,
>> but LeNet5 has so many connections.
>> In one layer, there are 6 feature maps, and in next layer there are 16 feature maps.
>> And an implementation in theano connects these two layers use full connection.
>> This means there are 6 * 16 = 96 connections,
>> then I need to instantiates 96 class of ConnectionType and store them in a tuple.
>> so I need to get a tuple like c1, c2, c3, ………, c96, it’s too large that I can’t write it manually.
>> But it seems I can't generate it automatically
>> or I need some very complicate c++ technique to do this.
>> http://stackoverflow.com/questions/28410697/c-convert-vector-to-tuple
>> 
>> So I think tuple isn't appropriate for this work.
>> I suggest that we use vector to replace tuple and have a base class for all ConnectionType class.
>> In this way, ConnectionTraits may also be not used.
> 
> Using a base class for all ConnectionType classes would mean virtual
> functions, which in this context would incur non-negligible slowdown
> (since the ConnectionType's functions will be used so much).
> 
> Although I agree that making a std::tuple<> with 96 objects in it is
> unwieldy, we should be careful to consider the drawbacks of inheritance
> in a library that focuses on speed.
> 
> I've barely looked at the code in methods/ann/, so I certainly don't
> have any better ideas, but I at least wanted to point out this
> perspective.
> 
> -- 
> Ryan Curtin    | "You got to stick with your principles."
> ryan at ratml.org <mailto:ryan at ratml.org> |   - Harry Waters

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20150209/e20adaa8/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 5136 bytes
Desc: not available
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack/attachments/20150209/e20adaa8/attachment-0001.bin>