Likelihood ratio test for the presence of random effects and robust errors

[Alfonso Sánchez-Peñalver]

Hi everyone. I have just had this thought while doing several probit estimations with and without random effects and testing for the presence of those effects using a likelihood ratio test on the boundary method (since I'm testing the significance of the variance). Let's keep this simple for the purpose of discussion. Consider we estimate a binary probit model with and without a random constant for some categorical variable. Now consider we estimate each specification (with and without a random constant) with and without robust variance-covariance matrix to ensure that we can do proper inference on the estimate of the coefficients just in case there is heteroskedasticity. The estimations with robust standard errors will change the standard error on the estimate of the variance in the models. However the likelihood of the models are unchanged because the coefficients are the same and thus a probit with robust errors will yield the same likelihood than one without. Thus the likelihood ratio test statistic is unchanged whether we use robust standard errors or not in our estimations. This doesn't sound reasonable does it?

[Carlos Avellaneda]

Correct me if I'm wrong, but fitting models with vce(robust) and then performing a likelihood ratio test implies that you are violating one of the assumptions of the test. Thus the test may not be longer valid.

[Alfonso Sánchez-Peñalver]

Hola Carlos,

do you mean to say that we cannot preform a likelihood ratio test of linear restrictions of the parameters between two probit models estimated with robust standard errors? Can you please expand on this, because I have never heard about that, and then I wonder how, then, you are supposed to test restrictions in fractional response models which have to be estimated with the robust variance-covariance matrix.

Thanks,

Alfonso.

[Carlos Avellaneda]

Hi Alfonso,

Stata's manual specifies the following:

Specifying the vce(robust) option similarly would indicate that you are worried about the valid specification of the model, so you would not use lrtest

Also, when you try to use the lrtest command with a model with the vce(robust) option, then Stata gives the following warning:
LR test likely invalid for models with robust vce
Here is an example to compare:

sysuse auto
probit foreign weight mpg displacement gear_ratio length
est sto model1
probit foreign weight mpg
lrtest model1
probit foreign weight mpg displacement gear_ratio length, vce(robust)
est sto mdel2
probit foreign weight mpg, vce(robust)
lrtest model2

You can force Stata to calculate the likelihood ratio test with the force option, but without a guarantee of the validity or interpretation of the test. Perhaps a Wald-type test is more suitable in that case.
Besides that, you should also notice that the likelihood ratio test also assumes that the test statistic is approximately distributed as chi-squared, and this assumption is not true when you estimate a random effects model. Stata also warns you about this:

webuse bangladesh
meprobit c_use urban age child* || district:
est sto model1
meprobit c_use urban age child*
lrtest model1

As you can see in the output, the reported test is conservative precisely because of the distribution assumption.
Hope this helps!
Best regards,
Carlos

[Alfonso Sánchez-Peñalver]

Hi again, thanks for pointing this out. It would be of value if there was some reference to the literature or some explanation of why estimating the models with robust standard errors violates the assumptions of the test. Such estimation is simply to get consistent estimates of the standard errors in order to do inference on the coefficients, not necessarily because the model is miss-specified somehow else. Since the test is on the boundary, as you pointed out, I was doing the tests by hand, which is why I didn't see lrtest not producing any results, since I wasn't using it. It still is baffling then as to how can you then test restrictions on a fractional response model.

Again, thanks for pointing this out.

Alfonso.

[Sandip Agarwal]

Hi,

Thanks Alfonso and Carlos for this. I too have been looking for this. It is still not known to me what assumptions of LR test are violated when conducted under robust standard errors. Can anyone elaborate?

Best,
Sandip

[Andrew Musau]

I would not say that assumptions of LR test are violated, but what is in question is the validity of the test for models with robust vce, as indicated by the warning message from Stata

LR test likely invalid for models with robust vce
r(498);

Alfonso's initial post in #1 summarizes the issue

the likelihood ratio test statistic is unchanged whether we use robust standard errors or not in our estimations

This implies that the likelihood for models with robust vce does not reflect the fact that individual observations are not independent (which is what we assume with clustering). Specifying robust vce in Stata is exactly clustering. Borrowing from Carlos' example in #4, notice that these two commands are equivalent

probit foreign weight mpg, vce(robust)
probit foreign weight mpg, cluster(make)

So in essence, the likelihood that Stata outputs under robust vce does not account for the fact that observations are not independent, and likely invalidates any subsequent LR test. See this Stata FAQ by William Sribney for alternatives to the LR test with robust vce

http://www.stata.com/support/faqs/st...od-ratio-test/