MATL, 18 15 1510 bytes
llZ(GlZ)Y\
Basically a straight translation fromworks the same as my Octave program. Thanks for -5 bytes @LuisMendo! This improvement was achieved by inputting the ones directly into the input instead of deleting a column and concatenating another one as well as some reordering of the steps.
1Z)t0^G4LZ)hwY\
Explanation
1Z)ll push extracttwo theones firstfor column
later use in tthe next line
Z( duplicate first column
implicitly take the 0^input and plug in all ONES in to the FIRST generatecolumn a
vector of ones of appropriateGl length
G4LZ) push input again, push geta theone remainingfor columnsuse fromin the input
next line
Z) h get the concatenateFIRST itcolumn withof the vector ofinput onesmatrix
Y\ wY\ multiply firstthis column with the pseudo inverse of the matrix wefrom justthe constructedbeginnign