ridge parameter

[Gabriella Miniero]

Hi Forum! I have a simple question for you that could help me a lot: is there any stata command that I can use in order to choose the best "k" parameter for my ridge regression?

[Nick Cox]

This is the third thread you have started on this in a brief period:

https://www.statalist.org/forums/for...ion-choosing-k

https://www.statalist.org/forums/for...ng-k-parameter

so that raises questions on different levels.

We do have advice on what to do -- and what not to do -- if you do not get an answer quickly. See #1 at https://www.statalist.org/forums/help#adviceextras which advises against several posts of the same or similar short questions. Far from getting you more attention, the net result may be less.

That said, I know why I didn't answer -- I have never used ridge regression and don't know much about it.

ridgereg is from SSC as you are asked to explain (FAQ Advice #12)

net describe ridgereg, from(http://fmwww.bc.edu/RePEc/bocode/r)

The author has posted here, but not for some while.

My impression is that ridge regression is not much practised by Stata people. It is not supported by StataCorp and my guess is that there are now much better ways to select predictors, such as lasso (LASSO).

None of that rules out a good answer to your question, but please keep posts on one question in one thread.

[Rich Goldstein]

actually, ridge regression is supported by StataCorp; here is a quote from the manual entry for -lasso-:

Ridge regression is a special case of the elastic net. When = 0 and > 0, (2) reduces to the objective function for the ridge-regression estimator.

there may be discussion within the manual that would help answer your question but I am not a fan of ridge regression and cannot help

[Gabriella Miniero]

Thank you Rich!

[Nick Cox]

Rich Goldstein Thanks for the correction.

[Achim Ahrens]

RIdge is also supported by lassopack (ssc install lassopack). And there are indeed valid reasons for using Ridge and cases where you might want to prefer it over the lasso; e.g. when sparsity assumption doesn't hold. That said, it is not a predictor selection method like the lasso.

lassopack supports penalty selection for the ridge using cross-validation (cvlasso) and information criteria (lasso2).

[Nick Cox]

Ho hum. I'd regard most regression modelling as being about selection of predictors although there is certainly a difference between commands that purport to automate choice and those that require independent decisions by researchers. Beyond that, selection of predictors also encounters very different willingness to change the model mix and/or consider other choices e.g. of transformations or interaction terms.

[Achim Ahrens]

It's not always about variable selection. If the aim of your analysis is prediction, you are not necessarily concerned with variable selection. In that case, your aim is not to interpret which predictors enter the model, but to optimise prediction performance. And a ridge-like model that has non-zero coefficients for all predictors might be better than a lasso-like sparse model with a only few predictors.

To give another example: say you want to estimate an optimal instrument based on a large number of "potential" instruments. You could use the lasso (see "ivlasso", Belloni et al, 2012, Econometrica), but the lasso might not be the best choice for the first stage if you don't have sparsity (e.g. many weak instruments). The ridge can be an alternative (see https://doi.org/10.1016/j.jeconom.2014.04.022).

Not saying that Ridge is the best method for anything specifically, but I certainly wouldn't discard it; even though it has naturally nothing to do with variable selection.

[Nick Cox]

Indeed, I agree that there can be different goals and styles to regression. #7 was a little tongue in cheek. Also, selection in my view includes the limiting case that the predictors you use are those you started with.