<div dir="ltr">The quantization i am referring here is "<span style="line-height:19.84px"><font face="monospace, monospace">converting a continuous range of values into a finite range of discreet values" like 10-20 one group.</font></span><div><font face="monospace, monospace"><span style="line-height:19.84px">Thanks for the the suggestion for including more file formats.</span></font></div><div><font face="monospace, monospace"><span style="line-height:19.84px">I'll mail you structure of how API will work in some time.</span></font></div><div><font face="monospace, monospace"><span style="line-height:19.84px">Wanted to ask if we can discuss on mailing list after GSOC starts accepting proposal?</span></font></div><div><font face="monospace, monospace"><span style="line-height:19.84px"><br></span></font><div><span style="line-height:19.84px"><font face="monospace, monospace">I proposed the second idea because i thought the first idea is little bit easy than other as also mentioned by difficulty level on the ideas page.so less preference can be given to people trying to work on command line/c++ API idea.</font></span></div><div><span style="line-height:19.84px"><font face="monospace, monospace">As the slots organizations get is fixed,i am confused which project to work on and that's why i am working on both.</font></span></div></div><div><span style="line-height:19.84px"><font face="monospace, monospace">Please guide me in this.</font></span></div><div><span style="line-height:19.84px"><font face="monospace, monospace">Have a Nice Day</font></span></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Regards,<div>Nirmal Singhania</div><div>B.tech III Yr</div></div></div></div></div></div></div></div>
<br><div class="gmail_quote">On Mon, Mar 14, 2016 at 7:28 PM, Ryan Curtin <span dir="ltr"><<a href="mailto:ryan@ratml.org" target="_blank">ryan@ratml.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Mon, Mar 14, 2016 at 05:53:15AM +0530, nirmal singhania wrote:<br>
> Hello,<br>
<br>
Hi Nirmal,<br>
<br>
There is no need to send your email multiple times. Everyone on the<br>
list received it the first time.<br>
<span class=""><br>
> Preprocessing Modules can include-<br>
> 1)checking a dataset for loading problems and printing errors<br>
> 2)Standardization module(mean removal and variance scaling) using z-score<br>
> 3)Scaling features to range(min-max)<br>
> 4)Handling Missing values/na<br>
> This can be done by removing the entire rows/columns containing<br>
> missing values.<br>
> or imputing the missing values using given data<br>
> 5)Scaling data with outliers<br>
> 6)converting categorical features into binary features<br>
><br>
> 7)Normalization of data(Not required for every ML algorithm but it doesn't<br>
> hurt if applied)<br>
> 8)splitting a dataset into a training and test set<br>
><br>
> Other features we can consider adding<br>
> 1)Handling Class Imbalance(Smote(Synthetic Minority Over-Sampling<br>
> Technique),Oversampling and Undersampling)<br>
> 2)Quantlization of Numerical Attributes<br>
<br>
</span>Do you mean quantization of categorical attributes here?<br>
<span class=""><br>
> A C++ API will will developed which will serve the purpose of<br>
> pre-processing data before using any ML-pack algorithm on it.<br>
> A command line interface will also be developed through which user can<br>
> check for problems and apply pre-processing methods on data set.<br>
> Command line and C++ API will intially support csv and arff files and<br>
> support for other formats may be added later.<br>
> There will be a option to save the pre-processed data set.<br>
> Optional-One Extra feature which can added is converted pre-processed arff<br>
> to csv and vice-versa.<br>
><br>
> Since Data handling and pre-processing will be crucial and common<br>
> step,Extensive documentation will be created using Doxygen on<br>
> 1)How to use various Methods Present in C++ API<br>
> 2)How to Handle and Pre-Process data using command line<br>
><br>
> Sample Programs and Tutorials on various data handling steps will also be<br>
> created using some open datasets.<br>
><br>
><br>
</span><span class="">> I want to ask how much information about each of the above steps i should<br>
> give in my proposal to make it a good proposal.<br>
<br>
</span>I like the ideas you've proposed here. When you put your proposal<br>
together, though, please spend some time detailing what the proposed C++<br>
API will be (and we can go back and forth on this if necessary). I<br>
think maybe the design guidelines would be helpful here:<br>
<br>
<a href="https://github.com/mlpack/mlpack/wiki/DesignGuidelines" rel="noreferrer" target="_blank">https://github.com/mlpack/mlpack/wiki/DesignGuidelines</a><br>
<br>
A couple other thoughts:<br>
<br>
* Don't worry about writing an imputer. A colleague of mine and I are<br>
planning on adding this support in the next few months. Detecting<br>
NaNs and missing values in a dataset is a good idea though.<br>
<br>
* We should try and support all of the file formats that Armadillo<br>
supports, instead of just CSV and ARFF. It would be good to provide<br>
a tool that can work with any dataset a user might otherwise use with<br>
mlpack.<br>
<br>
I hope this is helpful. Please let me know if I can clarify anything.<br>
<span class=""><br>
> 2)Implementing Decision trees and other algorithms in ml-pack<br>
> I've have understood the decision stump implementation done by Udit Saxena<br>
> for adaboost and would like to add more "weak learner" adaboost some of<br>
> which are already implemented in ml-pack and some which will be implemented<br>
> by me.<br>
> since Decison stumps are basically 1-level decision tree i would like to<br>
> continue on the Udit Saxena's work and implement full fledged decison trees<br>
> like ID3,C4.5,C5.0,CART.<br>
> I also looked at the code for DET(Density Estimation Trees) and would like<br>
> to borrow tree construction ideas from it.<br>
><br>
> Also will try to implement NB-Tree(Naive Bayes Tree) and<br>
> CI-Tree(Conditional Inference Tree) which are very useful in some tasks.<br>
> I have some knowledge about above mentioned methods and am currently going<br>
> through literature for more information and implementation.<br>
><br>
> All the above points about documentation,tutorial also apply here.<br>
> As in this,we are adding new algorithm to ml-pack library<br>
> Testing of implemented algorithms will be an important phase of this<br>
> project.<br>
> Also as everyone knows ml-pack is known for its fast speed and scalability.<br>
> We will benchmark it against similar methods available in<br>
> scikit-learn,weka,R and Shogun machine learning toolkit<br>
> and the results will provided via interactive and charts.<br>
> The automatic benchmarking system by Marcus Edel and Anand Soni during GSOC<br>
> will be used for benchmarkinghttps://<a href="http://github.com/zoq/benchmarks" rel="noreferrer" target="_blank">github.com/zoq/benchmarks</a><br>
<br>
</span>I think that you should focus on just one of these two ideas; it's hard<br>
to write two good proposals. Again the same advice applies for this<br>
proposal: make sure to spend some time designing the API and mentioning<br>
what it will be in your proposal.<br>
<br>
Thanks,<br>
<br>
Ryan<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Ryan Curtin | "I just ran out of it, you see."<br>
<a href="mailto:ryan@ratml.org">ryan@ratml.org</a> | - Howard Beale<br>
</font></span></blockquote></div><br></div>