Issue about xtpoisson & xtnbreg panel data regression with not concave iteration

[Chris Jiao]

Hi Statalisters,

I am working on a project by using counting regression model of panel data. The data is in panel data form with panel variable (Company Name) and time variable (Year). Hausman test suggests a fixed-effect regression that fits in better. After detecting over-dispersion, I select negative binomial regression (xtnbreg) as my main regression model. However, the question is that the iteration can not reach a concave optimization result, which results in infinite loops.

I have read several related posts but still feel confused about the problem. If anyone could give me some suggestions based on the following alternatives, I will be more than grateful.

1. Use Poisson (xtpoisson) regression to substitute NB regression (xtnbreg), but it may result in an over-dispersion problem.

2. Use multilevel mixed-effects regression (menbreg) to substitute NB regression (xtnbreg), but as the manual book suggests, it may only work for the case -xtnbreg, re-. Because Hausman test already suggests fixed effects in my model, this alternative may not work well.

3. Add some tricks in NB regression. Some manual books suggest using some tricky commands to modify the optimization algorithm without losing the generality. For example, we can assign -difficult-, -technique()-, and -iterate()- in the regression. I have doubts about this because (1) The result usually keeps not concave after adding in some tricks; (2) I don't find enough theories to support the validity of these commands, like if I assign a fixed iteration number using -iterate()-, how many iterations should I assign?

An example is here

asdoc xtnbreg $y1list $x1list $controls i.industry i.fyear, fe dec(4) difficult technique(bhhh) iterate(50) replace

Hope anyone can kindly help me figure this out.
Thanks in advance

[Joao Santos Silva]

Dear Chris Jiao,

If you do not nee to estimate probabilities of events (e.g., probability of zero given x), just use Poisson with FE because it is fully robust to overdispersion. Actually, I do not think you can say you have overdispersion in your data because what matters are the conditional mean and variance.

Best wishes,

Joao

[Chris Jiao]

Dear Chris Jiao,

If you do not nee to estimate probabilities of events (e.g., probability of zero given x), just use Poisson with FE because it is fully robust to overdispersion. Actually, I do not think you can say you have overdispersion in your data because what matters are the conditional mean and variance.

Best wishes,

Joao

Hi Joao,

See your points. One concern for me is that the Poisson regression assumes equi-dispersion with equivalent mean and variance. When I test the overdispersion parameter alpha, I already test a positive alpha that documents a overdispersion in my data. Why is it possible to say the Poisson FE is still robust in this way? If possible, could you please provide me with some reference?

Many thanks.

[Jeff Wooldridge]

Chris: The Poisson FE requires nothing about the variance-mean relationship. It also allows any kind of serial correlation, unlike xtnbreg. See my 1999 Journal of Econometrics paper or my 2020 textbook.

I’d now go as far as stating FE NBREG should never be used. It’s a weird model and has no known robustness properties.

[Chris Jiao]

Jeff Wooldridge

Thanks Jeff. I shall refer your 1999 papers and textbook for more info.

[Chris Jiao]

Chris: The Poisson FE requires nothing about the variance-mean relationship. It also allows any kind of serial correlation, unlike xtnbreg. See my 1999 Journal of Econometrics paper or my 2020 textbook.

I’d now go as far as stating FE NBREG should never be used. It’s a weird model and has no known robustness properties.

Jeff:

May I have the name of your paper that documents the invalidity of -nbreg, fe-?

Thanks in advance.

[Joao Santos Silva]

Dear Chris Jiao

On the weird properties of the FE NBREG, see:

Guimarães, Paulo (2008). "The fixed effects negative binomial model revisited," Economics Letters, vol. 99(1), pages 63-66.

Best wishes,

Joao

[Jeff Wooldridge]

As an addendum to Joao's helpful reference, in my 1999 Journal of Econometrics paper, "Distribution-Free Estimation of Some Nonlinear Panel Data Models," I pointed out that the FE NBREG model degenerates to one that depends only on a single heterogeneity -- not two, as originally claimed by Hausman, Hall, and Griliches -- and the nature of overdispersion is unappealing. Paulo showed further that the FE NBREG approach does not even eliminate time-constant variables. (Paulo also claims, incorrectly, that in the first edition of my MIT Press book I claim the FE NBREG is a "true" FE estimator. In fact, I don't discuss the estimator at all.)

[Vladimir Sobota]

Dear Jeff Wooldridge

thanks for this interesting discussion.

I’d now go as far as stating FE NBREG should never be used. It’s a weird model and has no known robustness properties.

Does this extend to 'xtnbreg, fe'? I first thought that 'xtnbreg, fe' was a good fit for my purpose, but after reading this discussion, I get the feeling that I should go with 'xtpoisson, fe'. I initially chose xtnbreg based on overdispersion (variance is more than 10 times the mean).

Many thanks in advance.
Vladimir