[mlpack-svn] [MLPACK] #298: Enhancement of linear regression library
MLPACK Trac
trac at coffeetalk-1.cc.gatech.edu
Wed Sep 25 15:29:22 EDT 2013
#298: Enhancement of linear regression library
--------------------------------+-------------------------------------------
Reporter: sumedhghaisas | Owner: rcurtin
Type: enhancement | Status: accepted
Priority: major | Milestone: mlpack 1.0.7
Component: mlpack | Resolution:
Keywords: linear_regression | Blocking:
Blocked By: |
--------------------------------+-------------------------------------------
Comment (by rcurtin):
Ok, that looks good for a test for ComputeCost(). I've commited r15835,
r15836, and r15837 where I added your implementation of ComputeCost()
although I renamed it to ComputeError() to be more in line with the terms
used elsewhere in mlpack. I modified it minorly to avoid a matrix copy of
the predictors matrix.
I found that the dataset you gave can be predicted by linear regression
perfectly, so it should have an error of 0. The test checks that
ComputeCost() returns a number smaller than 1e-25 (since floating point
error can mean that it won't be exactly 0).
I also added another dataset that linear regression can't predict
perfectly:
{{{
[[ 0 16 ] [[ 0 ]
[ 1 8 ] [ 2 ]
[ 2 4 ] [ 4 ]
[ 4 2 ] [ 3 ]
[ 8 1 ] [ 8 ]
[ 16 0 ] [ 8 ]]
}}}
I hand-calculated the error in Octave and it gives 1.189500337, so the
test makes sure ComputeError() returns a value close to that.
These datasets could be used to check that ridge regression is successful;
however, the best test for ridge regression will be a matrix where lambda
*must* be added to the diagonal for the matrix to be invertible. If you
can provide a matrix like this, and then the value of lambda that makes it
invertible, I'm happy to write the test and integrate the last bit of
support. Then, I will also write a test that runs ridge regression for
some lambda on some test dataset, and make sure the results are similar to
that given by MATLAB's implementation of ridge regression.
--
Ticket URL: <http://trac.research.cc.gatech.edu/fastlab/ticket/298#comment:11>
MLPACK <www.fast-lab.org>
MLPACK is an intuitive, fast, and scalable C++ machine learning library developed at Georgia Tech.
More information about the mlpack-svn
mailing list