We introduce three different ways to make regressions run faster.

We first introduce sparse solvers and show how to run regressions on sparse matrices via scikit-learn and the fastreg libraries.

We then lay out the Frisch-Waugh-Lovell theorem and the alternating projections algorithm and show how to speed it up on the CPU (via numba) and on the GPU (via JAX) as implemented in the pyfixest library.

Finally, we demonstrate how to drastically speed up regression estimation by first preprocessing the data in duckdb and then fitting a regression via weighted least squares in memory.

References:

• fastreg: https://github.com/iamlemec/fastreg
• scikit-learn: https://github.com/scikit-learn/scikit-learn
• pyfixest: https://github.com/py-econometrics/pyfixest
• duckreg: https://github.com/py-econometrics/duckreg