[mlpack-git] [mlpack] Weird Condition in SGD (#461)

[Ryan Curtin]

Hi there,

If the step size is too large, the optimization may jump back and forth in a valley, taking progressively larger steps, until the overall objective diverges to NaN.  That's the justification for the warning.  In various situations where I've used SGD (that led to this comment being added), reducing the step size was usually a solution to the problem that caused convergence to a reasonable objective value.

I hope this is helpful; let me know if I should clarify further.

Thanks,
Ryan

---
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/issues/461#issuecomment-148098264
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20151014/f4a4d497/attachment.html>