[mlpack-git] [mlpack] Weird Condition in SGD (#461)

Ryan Curtin notifications at github.com
Wed Oct 14 11:58:55 EDT 2015

Hi there,

If the step size is too large, the optimization may jump back and forth in a valley, taking progressively larger steps, until the overall objective diverges to NaN.  That's the justification for the warning.  In various situations where I've used SGD (that led to this comment being added), reducing the step size was usually a solution to the problem that caused convergence to a reasonable objective value.

I hope this is helpful; let me know if I should clarify further.


Reply to this email directly or view it on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20151014/f4a4d497/attachment.html>

More information about the mlpack-git mailing list