public class OLSMultipleLinearRegression extends AbstractMultipleLinearRegression
Implements ordinary least squares (OLS) to estimate the parameters of a multiple linear regression model.
The regression coefficients, b, satisfy the normal equations:
XT X b = XT y
To solve the normal equations, this implementation uses QR decomposition of the X matrix. (See
QRDecomposition for details on the decomposition algorithm.) The X matrix, also known as the
design matrix, has rows corresponding to sample observations and columns corresponding to independent
variables. When the model is estimated using an intercept term (i.e. when isNoIntercept is
false as it is by default), the X matrix includes an initial column identically equal to 1. We solve the
normal equations as follows:
XTX b = XT y
(QR)T (QR) b = (QR)Ty
RT (QTQ) R b = RT QT y
RT R b = RT QT y
(RT)-1 RT R b = (RT)-1 RT QT y
R b = QT y
Given Q and R, the last equation is solved by back-substitution.
| Constructor and Description |
|---|
OLSMultipleLinearRegression() |
| Modifier and Type | Method and Description |
|---|---|
double |
calculateAdjustedRSquared()
Returns the adjusted R-squared statistic, defined by the formula
R2adj = 1 - [SSR (n - 1)] / [SSTO (n - p)]
where SSR is the
sum of squared residuals, SSTO is the
total sum of squares, n is the number of observations and p is the number
of parameters estimated (including the intercept). |
protected RealVector |
calculateBeta()
Calculates the regression coefficients using OLS.
|
protected RealMatrix |
calculateBetaVariance()
Calculates the variance-covariance matrix of the regression parameters.
|
RealMatrix |
calculateHat()
Compute the "hat" matrix.
|
double |
calculateResidualSumOfSquares()
Returns the sum of squared residuals.
|
double |
calculateRSquared()
Returns the R-Squared statistic, defined by the formula
R2 = 1 - SSR / SSTO
where SSR is the
sum of squared residuals and SSTO is the
total sum of squares |
double |
calculateTotalSumOfSquares()
Returns the sum of squared deviations of Y from its mean.
|
void |
newSampleData(double[] y,
double[][] x)
Loads model x and y sample data, overriding any previous sample.
|
void |
newSampleData(double[] data,
int nobs,
int nvars)
Loads model x and y sample data from a flat input array, overriding any previous sample.
|
protected void |
newXSampleData(double[][] x)
Loads new x sample data, overriding any previous data.
|
calculateErrorVariance, calculateResiduals, calculateYVariance, estimateErrorVariance, estimateRegressandVariance, estimateRegressionParameters, estimateRegressionParametersStandardErrors, estimateRegressionParametersVariance, estimateRegressionStandardError, estimateResiduals, getX, getY, isNoIntercept, newYSampleData, setNoIntercept, validateCovarianceData, validateSampleDatapublic void newSampleData(double[] y,
double[][] x)
y - the [n,1] array representing the y samplex - the [n,k] array representing the x sampleMathIllegalArgumentException - if the x and y array data are not
compatible for the regressionpublic void newSampleData(double[] data,
int nobs,
int nvars)
Loads model x and y sample data from a flat input array, overriding any previous sample.
Assumes that rows are concatenated with y values first in each row. For example, an input data array
containing the sequence of values (1, 2, 3, 4, 5, 6, 7, 8, 9) with nobs = 3 and
nvars = 2 creates a regression dataset with two independent variables, as below:
y x[0] x[1] -------------- 1 2 3 4 5 6 7 8 9
Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an
intercept term. If AbstractMultipleLinearRegression.isNoIntercept() is true, the X matrix will be created without an initial
column of "1"s; otherwise this column will be added.
Throws IllegalArgumentException if any of the following preconditions fail:
data cannot be nulldata.length = nobs * (nvars + 1)nobs > nvarsThis implementation computes and caches the QR decomposition of the X matrix.
newSampleData in class AbstractMultipleLinearRegressiondata - input data arraynobs - number of observations (rows)nvars - number of independent variables (columns, not counting y)public RealMatrix calculateHat()
Compute the "hat" matrix.
The hat matrix is defined in terms of the design matrix X by X(XTX)-1XT
The implementation here uses the QR decomposition to compute the hat matrix as Q IpQT where Ip is the p-dimensional identity matrix augmented by 0's. This computational formula is from "The Hat Matrix in Regression and ANOVA", David C. Hoaglin and Roy E. Welsch, The American Statistician, Vol. 32, No. 1 (Feb., 1978), pp. 17-22.
Data for the model must have been successfully loaded using one of the newSampleData methods before
invoking this method; otherwise a NullPointerException will be thrown.
public double calculateTotalSumOfSquares()
Returns the sum of squared deviations of Y from its mean.
If the model has no intercept term, 0 is used for the mean of Y - i.e., what is returned is the sum
of the squared Y values.
The value returned by this method is the SSTO value used in the R-squared
computation.
MathIllegalArgumentException - if the sample has not been set or does
not contain at least 3 observationsAbstractMultipleLinearRegression.isNoIntercept()public double calculateResidualSumOfSquares()
public double calculateRSquared()
R2 = 1 - SSR / SSTOwhere SSR is the
sum of squared residuals and SSTO is the
total sum of squaresMathIllegalArgumentException - if the sample has not been set or does
not contain at least 3 observationspublic double calculateAdjustedRSquared()
Returns the adjusted R-squared statistic, defined by the formula
R2adj = 1 - [SSR (n - 1)] / [SSTO (n - p)]where SSR is the
sum of squared residuals, SSTO is the
total sum of squares, n is the number of observations and p is the number
of parameters estimated (including the intercept).
If the regression is estimated without an intercept term, what is returned is
1 - (1 - calculateRSquared()) * (n / (n - p))
MathIllegalArgumentException - if the sample has not been set or does
not contain at least 3 observationsAbstractMultipleLinearRegression.isNoIntercept()protected void newXSampleData(double[][] x)
Loads new x sample data, overriding any previous data.
The inputx array should have one row for each sample
observation, with columns corresponding to independent variables.
For example, if
x = new double[][] {{1, 2}, {3, 4}, {5, 6}}
then setXSampleData(x) results in a model with two independent
variables and 3 observations:
x[0] x[1]
----------
1 2
3 4
5 6
Note that there is no need to add an initial unitary column (column of 1's) when specifying a model including an intercept term.
This implementation computes and caches the QR decomposition of the X matrix once it is successfully loaded.
newXSampleData in class AbstractMultipleLinearRegressionx - the rectangular array representing the x sampleprotected RealVector calculateBeta()
Data for the model must have been successfully loaded using one of the newSampleData methods before
invoking this method; otherwise a NullPointerException will be thrown.
calculateBeta in class AbstractMultipleLinearRegressionprotected RealMatrix calculateBetaVariance()
Calculates the variance-covariance matrix of the regression parameters.
Var(b) = (XTX)-1
Uses QR decomposition to reduce (XTX)-1 to (RTR)-1, with only the top p rows of R included, where p = the length of the beta vector.
Data for the model must have been successfully loaded using one of the newSampleData methods before
invoking this method; otherwise a NullPointerException will be thrown.
calculateBetaVariance in class AbstractMultipleLinearRegressionCopyright © 2020 CNES. All rights reserved.