Dealing with heteroskedasticity in non-linear logit and probit models

[Robbie maris]

Hi everyone, as a pre-cursor, I know parts of my enquiry have been answered in other forum posts. However, I am hoping that this question brings together some of those posts and people might be able to provide some overarching advice on the use of non-linear regression models for binary outcome variables!

I want to run a series of non-linear (probably logit) regression models to predict binary outcome variables. I have been reading about how to deal with heteroskedasticity in the errors and it has become clear that I shouldn't be using "robust" standard errors as I would under a linear model (because if there is heteroskedasticity, the parameters are also inconsistent).

However, others have made note that heteroskedasticity is inherent and to be expected in the MLE of logit and other non-linear regression models (and we shouldn't be concerned about this).

Given those pieces of advice, what are people's recommendations on running logit regression models?

i.e.

Should I be accounting for heteroskedasticity somehow (and if so, how)?

Are there key diagnostics for these non-linear models that will show how appropriate the model is (and how unbiased/consistent the estimators are)?

Could I use a linear probability model (LPM) instead if the model doesn't give significant out-of-bounds predictions? I know this raises other issues too about the best way to model binary outcome variables!

Thanks in advance for the help!

[Maxence Morlet]

"Could I use a linear probability model (LPM) instead if the model doesn't give significant out-of-bounds predictions? I know this raises other issues too about the best way to model binary outcome variables!"

--> Is your regressor of interest normally distributed? If so, LPM will recover the average partial effect (Stoker, 1986, Chuhuui et al., 2022), which is what you're interested in. If not, you'd want to run both logit and LPM.


"Should I be accounting for heteroskedasticity somehow (and if so, how)?"

You should read Gourieroux et al. (1984). Are your data panel? That is quite important to know to give you a good recommendation; if yes, run a "pooled" model in order to avoid assuming serial independence in your outcome variable across time.

The main point is: use robust standard errors to account for "residual" misspecification (and cluster them in panel data settings to account for dependence), but as long as you've specified the marginal probability equation correctly, you'll get consistent pseudo-maximum likelihood estimates. However, if you get the marginal probability equation wrong (e.g. omit a relevant regressor, did not specify a quadratic relationship but only a linear one, etc.), estimates are inconsistent.

[Maxence Morlet]

I forgot to mention; if you have panel data, the incidental parameters problem may arise (Neyman and Scott, 1948) and be of substantial magnitude. In that case, for instance xtlogit, fe which uses conditional logit, and "logit", which uses max likelihood with "brute force" will yield different estimates.

Correlated random effects would then become an option, in combination with logit or probit...

[Robbie maris]

Thanks Maxence - that is very helpful!

My regressors of interest are also binary (mostly binary treatment group membership variables), so would that suggest using both an LPM and logit is the best course of action?

Or, are we able to say the regressor of interest is approximately normal (using the normal approximation of the binomial distribution when N*P and N*(1-P) > 5) and thus stick with the LPM to recover average partial effects?

Thanks again for your help!

[Maxence Morlet]

Interesting question - economists would say that they would want to see both, and would want the results to be qualitatively similar.

I am not sure what line of business you're in, but you may want to run both just to be on the safe side. How confident are you that you've correctly specified the P(Y=1 | X) (marginal equation)?

And are your data panel?

If so, how many time periods and how many Ns?

[Robbie maris]

Thanks Maxence.

I am in the field of economics, so that is helpful advice. My data is not panel and has around 650 observations.

For the modelling I'm doing at the moment, I am reasonably confident that the equation is well specified. I am interested in the marginal effects of three treatments that were randomly assigned and we have a rich set of controls that account for other relevant factors in our context.

If you have any further advice, that would be awesome. Otherwise, thanks for your help - I'm thinking the best strategy is start with a LPM and use logit as a robustness check!

[Maxence Morlet]

"My data is not panel and has around 650 observations."

Interesting, the IPP will thus not apply here, unless you have fixed-effects variables with not many observations per fixed-effect.

" I am reasonably confident that the equation is well specified. I am interested in the marginal effects of three treatments that were randomly assigned and we have a rich set of controls that account for other relevant factors in our context."

That definitely speaks in favour of logit with robust standard errors.

I agree with you though, if you obtain similar results with both methods, great. If not, choose logit.

Don't forget to take marginal effects when you use logit, because the coefficients you see are not directly marginal effects, although they have the same sign as the marginal effect.