<p>In <a href="https://github.com/mlpack/mlpack/pull/749#discussion_r76078481">src/mlpack/methods/lsh/objectivefunction.hpp</a>:</p>
<pre style='color:#555'>> +
> + double gammaChain =
> + - 2.0 * alpha * std::pow(k, beta) * std::log(x) * std::pow(x, gamma);
> +
> + // 3x1 column vector (in matrix form).
> + gradient(0, 0) += error * alphaChain;
> + gradient(1, 0) += error * betaChain;
> + gradient(2, 0) += error * gammaChain;
> + }
> +
> + // Return the average of each gradient after the summation is complete.
> + gradient(0, 0) /= ((double) M);
> + gradient(1, 0) /= ((double) M);
> + gradient(2, 0) /= ((double) M);
> +}
> +
</pre>
<p>In <code>objectivefunction</code>, I initialize the point randomly. I'm not sure if that also plays a role, since I don't seed the random generator.</p>
<p>Try using</p>
<pre><code>$ bin/mlpack_lshmodel -r iris.csv -p 0.5 -v
</code></pre>
<p>and</p>
<pre><code>bin/mlpack_lshmodel -r iris.csv -p 0.6 -v
</code></pre>
<p>The first should converge to some real values, while the second should "converge" to NaN after 10000 iterations (which I imagine is the default maximum number of iterations).<br>
Both the cost function and the gradients include logarithms, so I suspect we pass negative values somewhere in there. I'll investigate that and let you know.</p>
<p>I think that's also a bug in L_BFGS - shouldn't the iterations stop if objective is NaN?</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="https://github.com/mlpack/mlpack/pull/749/files/57c9d5e634d7d3d7e2ca1618353fe37d9e23b34a#r76078481">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AJ4bFKB8FHLJLFR-V6TFFk7NP-dtGyoFks5qjGIPgaJpZM4JczVR">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AJ4bFK22WAAYnwMW0NafPcOdYardMb4hks5qjGIPgaJpZM4JczVR.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/mlpack/mlpack/pull/749/files/57c9d5e634d7d3d7e2ca1618353fe37d9e23b34a#r76078481"></link>
<meta itemprop="name" content="View Pull Request"></meta>
</div>
<meta itemprop="description" content="View this Pull Request on GitHub"></meta>
</div>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/mlpack/mlpack","title":"mlpack/mlpack","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/mlpack/mlpack"}},"updates":{"snippets":[{"icon":"PERSON","message":"@mentekid in #749: In `objectivefunction`, I initialize the point randomly. I'm not sure if that also plays a role, since I don't seed the random generator.\r\n\r\nTry using\r\n```\r\n$ bin/mlpack_lshmodel -r iris.csv -p 0.5 -v\r\n```\r\nand\r\n```\r\nbin/mlpack_lshmodel -r iris.csv -p 0.6 -v\r\n```\r\nThe first should converge to some real values, while the second should \"converge\" to NaN after 10000 iterations (which I imagine is the default maximum number of iterations).\r\nBoth the cost function and the gradients include logarithms, so I suspect we pass negative values somewhere in there. I'll investigate that and let you know.\r\n\r\nI think that's also a bug in L_BFGS - shouldn't the iterations stop if objective is NaN?"}],"action":{"name":"View Pull Request","url":"https://github.com/mlpack/mlpack/pull/749/files/57c9d5e634d7d3d7e2ca1618353fe37d9e23b34a#r76078481"}}}</script>