[mlpack] GSOC 2014: Introduction

[Ryan Curtin]

On Tue, Feb 25, 2014 at 11:04:05PM +0530, Udit Saxena wrote:
> Hi,
> 
> This was regarding the idea of implementing Adaboost.
> 
> I have started looking up a few papers on Adaboost implementations. most of
> which involve Schapire and Freund.
> 
> I was going through the list of methods implemented by mlpack, and believe
> the some weak learners have to be implemented also.
> I imagine the list of tasks would be something similar to :
> 
>    - implementing a few weak learners:
>       - Alternating decision trees
>       - C4.5/C5: note C5 also includes boosting options
>       - something simple like weighted linear least squares
>       - some controlled version of random forests ( unlikely, this one)
>    - the basic adaboost algorithm is quite susceptible to noise and
>    outliers, and a good goal would be to focus on "gentle adaboost"
>    - also, the adaboost.m1, .m2, are also a good goal for implementing
>    multiclass classification.
> 
> So as you can see,I'd welcome suggestions for variants of weak learners, as
> most of mine are boosting decision tree based. I am reading a paper on this
> too.

Some other weak learners could be simple neural networks or maybe even
perceptrons.  mlpack already implements the Naive Bayes classifier so
that could be used also.  Can you post a link to the paper you are
reading?

> Also there are wide variety of adaboost algorithms based on extensions:
> logitboost, mpboost, icsiboost. I guess we will be coming up with one of
> our own, specific to mlpack, but just to post a few ideas.

Ideally the mlpack implementation will be flexible enough to extend to
any of those algorithms through template parameters and flexible
abstractions.  For instance, see how the NeighborSearch class extends to
any type of tree, and how the optimizers (L_BFGS, SGD) extend to any
optimizable function.

> Who might be a potential mentor for this project/idea ?

Marcus Edel was the one who suggested it, but I will definitely be
helping out regardless because I find it interesting.  :)

> Going through last year's list, I am also interested in packaging mlpack in
> debian and ubuntu. I think it could be clubbed with this idea for a
> summer's worth of coding.

I'm hoping to have the Debian packaging finished by then, but if not,
you are right that it could be paired with your idea for a good project.

Feel free to ask any further questions if necessary.

Thanks,

Ryan

-- 
Ryan Curtin    | "...wildcat... ...wild... cat... ... ...pow...
ryan at ratml.org | ...wildcat... I'm going to go."  - Eli Cash