I am quite surprised that a variant of linear regression has been proposed for a challenge, whereas an estimation via ordinary least squares regression has not, despite the fact the this is arguably the most widely used method in applied economics, biology, psychology, and social sciences!
For details, check out the Wikipedia page on OLS. To keep things concise, suppose one has a model:
where all right-hand side X variables—regressors—are linearly independent and are assumed to be exogenous (or at least uncorrelated) to the model error U. Then you solve the problem
on a sample of size n, given observations
The OLS solution to this problem as a vector looks like
where Y is the first column of the input matrix and X is a matrix made of a column of ones and remaining columns. This solution can be obtained via many numerical methods (matrix inversion, QR decomposition, Cholesky decomposition etc.), so pick your favourite!
Of course, econometricians prefer slightly different notation, but let’s just ignore them.
Non other than Gauss himself is watching you from the skies, so do not disappoint one of the greatest mathematicians of all times and write the shortest code possible.
Given the observations in a matrix form as shown above, estimate the coefficients of the linear regression model via OLS.
A matrix of values. The first column is always
Y, ..., Y[n], the second column is
X1, ..., X1[n], the next one is
X2, ..., X2[n] etc. The column of ones is not given (as in real datasets), but you have to add it first in order to estimate
beta0 (as in real models).
NB. In statistics, regressing on a constant is widely used. This means that the model is
Y = b0 + U, and the OLS estimate of
b0 is the sample average of
Y. In this case, the input is just a matrix with one column,
Y, and it is regressed on a column of ones.
You can safely assume that the variables are not exactly collinear, so the matrices above are invertible. In case your language cannot invert matrices with condition numbers larger than a certain threshold, state it explicitly, and provide a unique return value (that cannot be confused with output in case of success) denoting that the system seems to be computationally singular (
S or any other unambiguous characters).
OLS estimates of
beta_0, ..., beta_k from a linear regression model in an unambiguous format or an indication that your language could not solve the system.
- I/O formats are flexible. A matrix can be several lines of space-delimited numbers separated by newlines, or an array of row vectors, or an array of column vectors etc.
- This is code-golf, so shortest answer in bytes wins.
- Built-in functions are allowed as long as they are tweaked to produce a solution given the input matrix as an argument. That is, the two-byte answer
lmin R is not a valid answer because it expects a different kind of input.
- Standard rules apply for your answer, so you are allowed to use STDIN/STDOUT, functions/method with the proper parameters and return-type, full programs.
- Default loopholes are forbidden.
[3,1]. Explanation: X = [[1,1,1], [1,2,3]], X'X = [[3,6],[6,14]], inverse(X'X) = [[2.333,-1],[-1,0.5]], X'Y = [15, 32], and inverse(X'X)⋅X'Y=[3, 1]
S(any code for a computationally singular system).
Bonus points (to your karma, not to your byte count) if your code can solve very ill-conditioned (quasi-multicollinear) problems (e. g. if you throw in an extra zero in the decimal part of
X2 in Test Case 3) with high precision.
For fun: you can implement estimation of standard errors (White’s sandwich form or the simplified homoskedastic version) or any other kind of post-estimation diagnostics to amuse yourself if your language has built-in tools for that (I am looking at you, R, Python and Julia users). In other words, if your language allows to do cool stuff with regression objects in few bytes, you can show it!