[mlpack-git] [mlpack/mlpack] Fix mapping issue (#660)

stereomatchingkiss notifications at github.com
Thu Jun 2 00:53:54 EDT 2016


Unfortunately, looks like we better try to optimized the speed of CSV loading. I do not think re-read the file is a good idea since IO manipulation is very time consuming, fetch from main memory took about 100 nanosec, read 1MB sequentially from memory took 250,000 nanosec(([This link give a measurement](http://norvig.com/21-days.html))). 

I think, if we want to make file reading faster, we could try our best to minimize the time of IO manipulation(trading memory for speed), next step is speed up the processing speed of string. 

I would generate a file around 50MB to do the test, write a new class to parsed the CSV file. Test the performance with different solutions, including boost::spirit and multithread. spirit is very fast, but the compile time error messages are crazy, I will write down details comments if we decide to use spirit to boost speed up.

---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/pull/660#issuecomment-223195166
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20160601/1541e6de/attachment.html>


More information about the mlpack-git mailing list