[mlpack-git] [mlpack] WIP: Refactor LRSDP to support more general linear constraints (#580)

[Ryan Curtin]

Hey Evan,

Sorry for the slow response.  Things have been busy and I haven't had a chance to look further into this until now.

I wonder, should we even be including an `Evaluate()` and a `Gradient()` function in these constraint types?  Or instead should we simply be allowing access to the A matrix (however it should be represented) and the b vector, and allow the optimizers to do what they need with those?  (This is closer to how the current code works, but the current code has a lot of template specializations that I'd love to do away with, favoring just one non-specialized implementation instead.)

This is simple for the sparse and dense constraints, which can just return A as types `arma::sp_mat` and `arma::mat`, respectively, but the linear constraint is a bit harder because we want to represent A as (aa').  We could possibly use the Armadillo type that represents two vectors to be multiplied for this... it'd look something like `arma::glue<arma::Col<eT>, arma::Col<eT>, arma::op_multiply>`.  If we return that type for the A matrix and then use that in a larger Armadillo expression, I think we can reasonably expect that the template metaprogramming that evaluates Armadillo expressions will do a decent job.

Is what I'm saying making any sense?  Please let me know if not.  It has been a while since I have thought about SDPs.  My concern though is that ideally we don't want to include to much optimizer-specific logic in the constraints, and it seems like in this case that's what's happening with `EvaluateLR()` and `GradientLR()`; if we were to come up with another technique for solving SDPs, like for instance with block diagonal representations instead of low-rank, we would probably need an `EvaluateBlockDiagonal()` and `GradientBlockDiagonal()` in each constraint, which I'd like to avoid.


---
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/pull/580#issuecomment-200369662
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20160323/e3d99fff/attachment.html>