Comparing R-squared of Different Models

[Viana Junior]

Hi guys,

I'm going crazy with a simple stuff.
Basically, I just need to compare if the difference of two R-squared (from two different models) are statistically significant or not.
In a general way, my models are:

MODEL 1 (from 2010 to 2013)
reg y x1 x2 x3 i.industry

MODEL 2 (from 2014 to 2016)
reg y x1 x2 x3 i.industry


And, then, compare the R-squared from Model 1 and 2 in order to know if they are statistically different or not.
I'm reading for 3 hours about Cramer (1987), Lien and Vuong (1987), Vuong (1989), etc.
But all the previous posts on the subject are pretty confused.

Please, does anyone could help me?
Thanks.

[Jeff Wooldridge]

Is this a panel data set on industries? That is, do you follow the same set of industries from 2010 through 2016? If so, then I have several suggestions. At a minimum, define a dummy variable equal to one for the second set of of years. Then, include this dummy variable and interact it with x1 x2 x3. Estimate using all years and test whether the interactions are significant. I would not test d2 by itself; it is expected there will be changes over time in almost any response variable not explained by covariates. (If you tell us what y, x1, and so on are, I could give a better opinion.)

In fact, I would say put in a full set of year dummies. And if it is a panel of industries, use the xtreg command. The following assumes the year variable is never missing.

xtset industry year
gen d2 = (year == 2014) | (year == 2015) | (year == 2016)
xtreg y x1 x2 x3 i.year i.d2#c.x1 i.d2#c.x2 i.d2#c.x2, fe vce(cluster industry)
test 1.d2#c.x1 1.d2#c.x2 1.d2#c.x3

If you only want a single intercept shift, replace i.year with i.d2. I wouldn't include it in the test command, but if your hypothesis is that there is no change at all in the regression function then you can include it.

JW

[Phil Bromiley]

I think Jeff reinterpreted your question in a way that makes more sense econometrically. We often compare parameter values which is what his code does. We don't often compare explained variances. Some folks are not keen on explained variance for various reasons (including that it is manipulable by transformations that don't chance the underlying model). As Nick once posted, "adjusted R-squares are a lousy basis to compare two models, even of the same kind. They leave out too much information."

Since the square root for r-squared is a correlation, you might be able to use cortest. See https://www.statalist.org/forums/for...elation-levels
https://ideas.repec.org/c/boc/bocode/s407302.html

https://www.stata.com/statalist/arch.../msg00149.html
Suggests a bootstrap approach.

You might also consider
https://www.albany.edu/faculty/kreth...Parameters.pdf

[Jeff Wooldridge]

I did reinterpret the question. I don't see how once can conclude of the models are "statistically different" without actually comparing the estimated parameters. One could get similar R-squareds but the parameters could be quite different. It seems to me that Viana wants to do a kind of Chow test with panel data, over two different periods.