<p>Okay; I understand the changes you've made then. Thanks for the explanation. One thing I overlooked, though---you've split out the line search termination conditions in order to give better failure messages; but on those failures, it no longer returns false, it just breaks. Is there a particular reason for this? I think that this could cause <code>L_BFGS</code> to iterate forever with no improvement, although the relative improvement termination condition should catch that. Which then implies that there's no need for <code>LineSearch()</code> to return anything at all or check whether or not it failed. If you agree with that line of reasoning, then after I merge it I'll make <code>LineSearch()</code> <code>void</code> and remove the checks.</p>
<blockquote>
<p>In fact, if you email sam burer asking about these heuristics, he will tell you he can't even quite piece them together unless he spends quite a bit of time looking at his code.</p>
</blockquote>
<p>Hah, I have one of those emails too! :)</p>
<p>He suggested that he avoided the R=0 problem by using the determinant of R^T R, and linked to a Mathematical Programming paper titled "Local Minima and Convergence in Low-Rank Semidefinite Programming":<br>
<a href="http://www.researchgate.net/publication/225703502_Local_Minima_and_Convergence_in_Low-Rank_Semidefinite_Programming/file/d912f508148a86b469.pdf">http://www.researchgate.net/publication/225703502_Local_Minima_and_Convergence_in_Low-Rank_Semidefinite_Programming/file/d912f508148a86b469.pdf</a></p>
<p>and also suggested a (slightly) more recent paper:<br>
<a href="http://link.springer.com/article/10.1007%2Fs10898-008-9328-4">http://link.springer.com/article/10.1007%2Fs10898-008-9328-4</a></p>
<p>It has been a very long time since I have done so much as open those papers, though...</p>
<p>I do think a more robust LRSDP would definitely be paperworthy and of high interest. It would also allow mlpack to provide faster implementations of SDP-based algorithms that actually work. (This is better than the old rejected code that was fast but didn't work...) A dual solver would definitely help us know when a restart was necessary, but I don't know if this would cause the entire algorithm to be slower overall than just a regular SDP solver. Either way, that is certainly an interesting avenue to investigate.</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br>Reply to this email directly or <a href="https://github.com/mlpack/mlpack/issues/370#issuecomment-70039134">view it on GitHub</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AJ4bFGHLWb8vBBxA-2HW34RV4XbUvoQDks5nhzxqgaJpZM4DNsex.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="https://github.com/mlpack/mlpack/issues/370#issuecomment-70039134"></link>
<meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>