<blockquote>
<p>then just pass a big arma::mat& instead of arma::vec&s one at a time</p>
</blockquote>
<p>why arma::vec& but not arma::Row?</p>
<blockquote>
<p>Just L-BFGS and SGD for now, but more optimizers may be implemented later.</p>
</blockquote>
<p>To work with SGD, I have to implement three more functions</p>
<pre><code>size_t NumFunctions() const ;
double Evaluate(const arma::mat& parameters, const size_t i);
double Gradient(const arma::mat &data, const arma::mat& parameters, const size_t i);
</code></pre>
<p>I do not familiar with SGD, correctly me if I am wrong</p>
<p>It is same as mini-batch, but the batch size of sgd always equal to 1</p>
<p>I impement mini-batch before, the principals are</p>
<pre><code>//begin and end are random number
Evaluate(data.submat(0, begin, data.n_rows-1, end), parameters);
Gradient(data.submat(0, begin, data.n_rows-1, end), parameters);
</code></pre>
<p>The Evaluate and Gradient functions still do the same thing, but this time I only update the cost and gradient base on random batch of data. If this is SGD, begin will always same as end.</p>
<blockquote>
<p>I personally think that CUDA code is really ugly and difficult to work with</p>
</blockquote>
<p>We could rely on the huge libraries developed by NVDIA and other open source community. Like mshadow(<a href="https://github.com/dmlc/mshadow">https://github.com/dmlc/mshadow</a>) and Thrust</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br>Reply to this email directly or <a href="https://github.com/mlpack/mlpack/pull/466#issuecomment-153624485">view it on GitHub</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AJ4bFLqw8vE86714JMadie9XV9elLnfuks5pCbEAgaJpZM4GX7Ld.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/mlpack/mlpack/pull/466#issuecomment-153624485"></link>
<meta itemprop="name" content="View Pull Request"></meta>
</div>
<meta itemprop="description" content="View this Pull Request on GitHub"></meta>
</div>