<p>Hi there,</p>

<p>If the step size is too large, the optimization may jump back and forth in a valley, taking progressively larger steps, until the overall objective diverges to NaN.  That's the justification for the warning.  In various situations where I've used SGD (that led to this comment being added), reducing the step size was usually a solution to the problem that caused convergence to a reasonable objective value.</p>

<p>I hope this is helpful; let me know if I should clarify further.</p>

<p>Thanks,<br>
Ryan</p>

<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">&mdash;<br>Reply to this email directly or <a href="https://github.com/mlpack/mlpack/issues/461#issuecomment-148098264">view it on GitHub</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AJ4bFApLj8U8QD1T_1y8cYk_FSLeyvqpks5o7nM_gaJpZM4GNy_s.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
  <link itemprop="url" href="https://github.com/mlpack/mlpack/issues/461#issuecomment-148098264"></link>
  <meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>