[mlpack-svn] [MLPACK] #298: Enhancement of linear regression library

MLPACK Trac trac at coffeetalk-1.cc.gatech.edu
Wed Sep 25 15:29:22 EDT 2013


#298: Enhancement of linear regression library
--------------------------------+-------------------------------------------
  Reporter:  sumedhghaisas      |        Owner:  rcurtin     
      Type:  enhancement        |       Status:  accepted    
  Priority:  major              |    Milestone:  mlpack 1.0.7
 Component:  mlpack             |   Resolution:              
  Keywords:  linear_regression  |     Blocking:              
Blocked By:                     |  
--------------------------------+-------------------------------------------

Comment (by rcurtin):

 Ok, that looks good for a test for ComputeCost().  I've commited r15835,
 r15836, and r15837 where I added your implementation of ComputeCost()
 although I renamed it to ComputeError() to be more in line with the terms
 used elsewhere in mlpack.  I modified it minorly to avoid a matrix copy of
 the predictors matrix.

 I found that the dataset you gave can be predicted by linear regression
 perfectly, so it should have an error of 0.  The test checks that
 ComputeCost() returns a number smaller than 1e-25 (since floating point
 error can mean that it won't be exactly 0).

 I also added another dataset that linear regression can't predict
 perfectly:

 {{{
 [[  0  16 ]   [[ 0 ]
  [  1   8 ]    [ 2 ]
  [  2   4 ]    [ 4 ]
  [  4   2 ]    [ 3 ]
  [  8   1 ]    [ 8 ]
  [ 16   0 ]    [ 8 ]]
 }}}

 I hand-calculated the error in Octave and it gives 1.189500337, so the
 test makes sure ComputeError() returns a value close to that.

 These datasets could be used to check that ridge regression is successful;
 however, the best test for ridge regression will be a matrix where lambda
 *must* be added to the diagonal for the matrix to be invertible.  If you
 can provide a matrix like this, and then the value of lambda that makes it
 invertible, I'm happy to write the test and integrate the last bit of
 support.  Then, I will also write a test that runs ridge regression for
 some lambda on some test dataset, and make sure the results are similar to
 that given by MATLAB's implementation of ridge regression.

-- 
Ticket URL: <http://trac.research.cc.gatech.edu/fastlab/ticket/298#comment:11>
MLPACK <www.fast-lab.org>
MLPACK is an intuitive, fast, and scalable C++ machine learning library developed at Georgia Tech.


More information about the mlpack-svn mailing list