[mlpack-git] [mlpack] improve speed of SparseAutoencoder and make it more flexible (#451)

[stereomatchingkiss]

The Error function of the performance function need the network parameters,because the SparseCost need to access the Weights and RhoCap

The FFN take the copy of the network but not the reference of it, so it is no use to use the pointer to point to the weights and RhoCap of the network

      //in this case, input can get from the network too
      double Error(const DataType &target, const DataType &error, const Network &network)
      {
        // Calculate squared L2-norms of w1 and w2.
        const double wL2SquaredNorm =
            arma::accu(std::get<0>(network).Weights() % std::get<0>(network).Weights()) +
            arma::accu(std::get<3>(network).Weights() % std::get<3>(network).Weights());    
  
        const double sumOfSquaresError =
            0.5 * arma::accu(error % error) / target.n_cols;
        const double weightDecay = 0.5 * lambda * wL2SquaredNorm;
        const double klDivergence =
            beta * arma::accu(rho * arma::trunc_log(rho / std::get<0>(network).RhoCap()) + (1 - rho) *
                              arma::trunc_log((1 - rho) / (1 - std::get<0>(network).RhoCap())));    
    
        // The cost is the sum of the terms calculated above.
        return sumOfSquaresError + weightDecay + klDivergence;
      }

If you agree with it, I would make the necessary changes.

---
Reply to this email directly or view it on GitHub:
https://github.com/mlpack/mlpack/pull/451#issuecomment-164266642
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.cc.gatech.edu/pipermail/mlpack-git/attachments/20151213/1141d4bd/attachment.html>