[mlpack-git] [mlpack/mlpack] Automatic bindings step 2: matrix input/output parameters (#775)

Ryan Curtin notifications at github.com
Fri Sep 2 18:11:01 EDT 2016

Here I have completely refactored the CLI code in order to support matrix-type parameters.  So now I can write

PARAM_MATRIX("input", "Input matrix.", "i");

and then in my code,

// Either works:
arma::mat& matrix = CLI::GetParam<arma::mat>("input");
arma::mat matrixmove = std::move(CLI::GetParam<arma::mat>("input"));

And the user specifies the matrix on the command line as `--input_file` (and the documentation says that too).

There are a couple variations, like `PARAM_UMATRIX_IN` (which is `arma::Mat<size_t>`), and `PARAM_TMATRIX_IN` (which is a non-transposed-on-load matrix), and `PARAM_MATRIX_OUT` (an output matrix).  Output matrices are saved when the program terminates, and input matrices are loaded when `GetParam<>()` is called for the first time with that argument name.

The PR also contains a few bugfixes for CLI and vastly expands the test suite that we had for CLI.

In most (but not all cases) I have been able to adapt `PARAM_STRING_IN` parameters to `PARAM_MATRIX_IN` parameters and preserve reverse compatibility, but there are a few cases where I've broken reverse compatibility so technically this means this patch can't happen until mlpack 3.0.0.  I think maybe it is worth having an mlpack-3.x branch, and then switch master to that branch when 3.0.0 is released.  Alternately we release mlpack 2.1.0 very soon (like next week) with the features we have now, and then merge in the mlpack 3 features and break reverse compatibility in the git master branch (I am fine with that).

Lastly, the long-term: now that CLI loads matrices automatically, I should be next able to modify the command-line programs in such a way that they can be generated as bindings for other languages too.  Since I can extract the parameters directly from CLI, the programs can be agnostic to how they are being used.  So I will probably first play with generating a Python binding for PCA (since it does not save any models); then go from there.

I'll let this PR sit for a few weeks (since I am not in a hurry here) and then if there are no comments I'll go ahead and merge it and choose arbitrarily whether to make a new branch or to just release 2.1.0 and move on. :)
You can view, comment on, or merge this pull request online at:


-- Commit Summary --

  * First pass at PARAM_MATRIX; checking in to work from another system.
  * Fix style.
  * Handle ignoreInput correctly for Log::Fatal.
  * Remove extra macros; fix transpose.
  * Handle matrix parameters correctly.
  * Add tests for matrix options.  Refactor CLI tests.
  * Don't provide a backtrace if we are ignoring input.
  * Significant refactoring for simplicity.
  * Overhaul tests; add some new tests for parameter types.
  * Fix duplicate alias.
  * Add UMATRIX option type for labels.
  * Set default value in Add() instead of GetParam().
  * Bugfixes for destructor.
  * Print empty strings for empty matrix parameters.
  * Add GetUnmappedParam() so that we can get the string names of matrix parameters.
  * Add macros for transposed matrices.
  * Add tests for GetUnmappedParam().
  * Refactor executables to handle matrix parameters.
  * Merge branch 'master' into bindings
  * Check the actual name first and only fall back on aliases if needed.
  * Refactor programs to use matrix parameters.

-- File Changes --

    M src/mlpack/core/util/CMakeLists.txt (1)
    M src/mlpack/core/util/cli.cpp (745)
    M src/mlpack/core/util/cli.hpp (224)
    M src/mlpack/core/util/cli_impl.hpp (235)
    M src/mlpack/core/util/option.hpp (36)
    D src/mlpack/core/util/option_impl.hpp (54)
    M src/mlpack/core/util/param.hpp (314)
    M src/mlpack/core/util/prefixedoutstream.hpp (2)
    M src/mlpack/core/util/prefixedoutstream_impl.hpp (14)
    M src/mlpack/methods/adaboost/adaboost_main.cpp (56)
    M src/mlpack/methods/cf/cf.cpp (2)
    M src/mlpack/methods/cf/cf.hpp (2)
    M src/mlpack/methods/cf/cf_main.cpp (63)
    M src/mlpack/methods/decision_stump/decision_stump_main.cpp (45)
    M src/mlpack/methods/det/det_main.cpp (81)
    M src/mlpack/methods/emst/emst_main.cpp (21)
    M src/mlpack/methods/fastmks/fastmks_main.cpp (55)
    M src/mlpack/methods/gmm/gmm_generate_main.cpp (17)
    M src/mlpack/methods/gmm/gmm_probability_main.cpp (24)
    M src/mlpack/methods/gmm/gmm_train_main.cpp (7)
    M src/mlpack/methods/hmm/hmm_generate_main.cpp (20)
    M src/mlpack/methods/hmm/hmm_loglik_main.cpp (6)
    M src/mlpack/methods/hmm/hmm_train_main.cpp (2)
    M src/mlpack/methods/hmm/hmm_viterbi_main.cpp (17)
    M src/mlpack/methods/hoeffding_trees/hoeffding_tree_main.cpp (113)
    M src/mlpack/methods/kernel_pca/kernel_pca_main.cpp (23)
    M src/mlpack/methods/kmeans/kmeans_main.cpp (64)
    M src/mlpack/methods/lars/lars_main.cpp (71)
    M src/mlpack/methods/linear_regression/linear_regression_main.cpp (80)
    M src/mlpack/methods/local_coordinate_coding/local_coordinate_coding_main.cpp (51)
    M src/mlpack/methods/logistic_regression/logistic_regression_main.cpp (89)
    M src/mlpack/methods/lsh/lsh_main.cpp (79)
    M src/mlpack/methods/mean_shift/mean_shift_main.cpp (68)
    M src/mlpack/methods/mvu/mvu_main.cpp (13)
    M src/mlpack/methods/naive_bayes/nbc_main.cpp (50)
    M src/mlpack/methods/nca/nca_main.cpp (30)
    M src/mlpack/methods/neighbor_search/kfn_main.cpp (73)
    M src/mlpack/methods/neighbor_search/knn_main.cpp (73)
    M src/mlpack/methods/nmf/nmf_main.cpp (22)
    M src/mlpack/methods/pca/pca_main.cpp (15)
    M src/mlpack/methods/perceptron/perceptron_main.cpp (66)
    M src/mlpack/methods/preprocess/preprocess_binarize_main.cpp (17)
    M src/mlpack/methods/preprocess/preprocess_describe_main.cpp (6)
    M src/mlpack/methods/preprocess/preprocess_split_main.cpp (69)
    M src/mlpack/methods/radical/radical_main.cpp (57)
    M src/mlpack/methods/range_search/range_search_main.cpp (31)
    M src/mlpack/methods/rann/krann_main.cpp (49)
    M src/mlpack/methods/rmva/rmva_main.cpp (54)
    M src/mlpack/methods/softmax_regression/softmax_regression_main.cpp (105)
    M src/mlpack/methods/sparse_coding/sparse_coding_main.cpp (52)
    M src/mlpack/tests/cli_test.cpp (391)

-- Patch Links --


You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20160902/a15e085a/attachment.html>

More information about the mlpack-git mailing list