Compare models with robust standard errors using an F-test

[Riccardo Valboni]

Dear all,

I am trying to compare two models I ran with robust standard errors using an F-test. However, when I use the ftest command as suggested by Richard Williams
(https://www3.nd.edu/~rwilliam/stats2/l51.pdf) I get an error saying Robust vce not allowed. Does anyone know if there is an alternative method/command I could employ here?

Thank you in advance, Riccardo

[Steve Samuels]

Please show use your code and your results as FAQ #12 asks. Also, you are asked to document the source of contributed commands like ftest.

[Eric de Souza]

The ftest command supposes that the residuals are homoscedastic. If you are simply testing the nullity of two or more coefficients, just use test.

[Riccardo Valboni]

Hi Steven and Eric,

Thank you both for your answers. I think I found a solution to my own question, which I'd like to illustrate below. First, Steve, to respond to your suggestion, the code I used was similar to this

SSC install ftest
help ftest

reg y var1-var12, robust
estimates store mod_1

reg y var1-var13, robust
estimates store mod_2

ftest mod_1 mod_2

The author of this command is Maarten L. Buis, and the general description in the help file says ftest can be used to "compare two nested models using an F-test". Unfortunately, as Eric pointed out, the F-test assumes the homoskedasticity of the residuals. Thus, the command generated the error I reported above when I tried to test the two models.

Hence, I found my solution with nestreg

nestreg:

reg y var1-var12 (var13), robust

This seemed to produce what I was looking for. Many thanks again for the help.
Riccardo

[Eric de Souza]

You could just as well have done:

 reg y var1-var13, robust
test var13

[Riccardo Valboni]

Hi Eric, in this case you're right. But can you do the same when you add a block of variables instead of a single one?

[Clareta Treger]

Dear all,
I looked at this thread and I have a follow up question.

I wanted to run an F-test to compare a nested model without interaction terms to a model with interaction terms.
I entered the following code:

local int var1#var2
reg supported i.var1 i.var2 `int' i.var3 i.var4 i.var5 i.var6 , cl(rid1) baselevel
test `int'

where var1, var3-6 are categorical variables
and var2 is a dummy variable

The reg command worked, but the test command produced the following error:
coer#dum_party not found

Do you know what the problem might be?

Thank you
Clareta

[Joro Kolev]

Try

reg supported i.var1 i.var2 i.var1#i.var2 i.var3 i.var4 i.var5 i.var6 , cl(rid1) baselevel

testparm i.var1#i.var2

[Jeff Wooldridge]

Why not just use the robust t statistic on the interaction? The F test gives less information.

[Joro Kolev]

Professor Wooldridge, you mean it gives less information because you lose the sign? I.e., the t-test is a signed test, and the F-test is not?

Otherwise if Original Poster has stated correctly that "var1, var3-6 are categorical variables," then it is a multiple restrictions test and t-test would not do. And the term var1#var2 in OP's local should be specified as var1#i.var2 or i.var1#i.var2 as I have done.

Why not just use the robust t statistic on the interaction? The F test gives less information.

[Jeff Wooldridge]

Joro: I agree with you generally, of course. But as you note, for a single restriction a t test tells us all we need.