T-test for coefficients across multiple regressions

[Laura Müller]

Hi everyone

I would like to test if two coefficients are significantly different from each other. I know the ttest function in stata but it does not work in case the coefficients are coming from different regressions (as far as I know).

In fact, I run twice the same regression but with different subsamples. Let's assume DV is the dependent variable and IV stands for the independent variable. C is the condition which splits the sample into two subsamples.

The regressions look as follows ("a" and "b" are just added to easier differentiate it):
(a) areg DVa IV1a IV2a IV3a ... IV11a IV12a i.year, absorb(company) cluster(company), if C==1
(b) areg DVb IV1b IV2b IV3b ... IV11b IV12b i.year, absorb(company) cluster(company), if C==0

I want to test if the coefficient of the first regression _b[IV2a] is significantly different from the coefficient of the second regression _b[IV2b].
(For the sake of completeness:IV2 is an interaction term where one of the variables is a dummy variable. I have panel data and cluster by company)

My intention was to test if 0=_b[IV2a] - _b[IV2b]. But this does not work as I would need to run both regressions at the same time that both coefficients are in the memory of stata.

Have anyone got an idea what I could do instead?

Thanks a lot for your help,
Laura

[Rich Goldstein]

try the -suest- command (or, possibly, -sureg-)

[Laura Müller]

Thanks for your answer. Unfortunately, I get the following error message: "areg is not supported by suest". Is there another way? To run the regression -sureg- would result in different coefficients.

[Rich Goldstein]

I would then use -qui regress- with the full set of dummies (unless you need to see certain results from the regression; in that case I would do the -areg- and then -qui regress-

[Laura Müller]

What do you mean with "the full set of dummies"? So far, I do not see how I can use the -qui- command to solve the problem. May I ask you to explain it a bit further? Thanks!

[Rich Goldstein]

-areg- absorbs a set of dummies; that is what I was referring to; the use of "quietly" was only in case you did not want to see a, possibly, voluminous output; -suest- will work with -regress- so I am suggesting you base your tests on -regress- rather than -areg-

[Roman Mostazir]

Laura, what Rich essentially suggesting at #4 is to run a normal regression with all your variables i.e. including 'company'. The idea of 'areg' command is to hide the coefficients of 'company' because you don't want to see them as perhaps they are large number of dummies. In order to be able to use 'suest' command, you need to run normal regression as 'suest' is not supported after 'areg'. If you don't mind seeing large outputs just type:

 reg DV iv1 iv2 companies if c==1

or if you prefer not to see the coefficient type:

qui reg DV iv1 iv2 companies if c==1

. The mock solution to your problem can be solved as follows:

reg dv iv1 iv2 if c==0
estimates store cequalzero

reg dv iv1 iv2 if c==1
estimates store cequalone

suest cequalzero cequalone

test [cequalzero_mean]iv1-[cequalone_mean]iv1 = 0

test [cequalzero_mean]iv2-[cequalone_mean]iv2 = 0

This should provide you a chi2 test of difference between the coefficients.

[Laura Müller]

Hi Roman and Rich

Many thanks for your answers.

Now I see how it would work. The only thing I still do not understand is how I can transform the areg regression to a normal regression in a way that the coefficients remain the same. How can I include these options: i.year, absorb(company) cluster(company)? I have ten years and about 100 different companies.

Or does it not matter that the coefficients are differently in the normal regression and the -areg- command when testing for the difference? This would mean that I can get the coefficient values from the -areg- command and the significants of the difference of these coefficients by doing a normal regression plus chi2 test?

Thanks,
Laura

[Sarah Edgington]

It would help if you showed the actual commands you used and the output. (Put these in code tags by selecting the underlined A in the post editor and then the code tag which looks like #).

Make sure that when you add companies to your regression that you're putting it in as indicator variables. For example

reg outcome var1 var2 i.year i.company, cluster(company)

[Roman Mostazir]

You don't need to transform anything. 'reg' and 'areg' performs same thing, the later suppresses the coefficients of certain dummy variable while the variable is still in the model. Try this:

reg DVa IV1a IV2a IV3a ... IV11a IV12a i.year company if C==0, cluster(company)
estimate store c_zero

reg DVb IV1b IV2b IV3b ... IV11b IV12b i.year company if C==1, cluster(company)
estimate store c_one

suest czero cone

test [czero_mean]IV2a -  [cone_mean]IV2b = 0
test [czero_mean]IV3a -  [cone_mean]IV3b = 0

and so on

Assuming company is numerical coded therefore I added it as a continuous variable to minimise the output (avoiding 99 dummies). If you want to see (unlikely as you wanted 'areg') the coefficients for all companies, use 'i.company' which will give you 99 rows of coefficients only for company.

[Sarah Edgington]

Including company as a continuous variable in the regression rather than using indicator variables is estimating a different model than the model estimated with areg. This will almost certainly result in different coefficients for the other variables since it's a different model.

[Laura Müller]

These are the codes I used before, attached you will find all outputs:

areg DV IV1 IV2 IV3 IV4 IV5 IV6 IV7 IV8 IV9 IV10 IV11 IV12 i.year, absorb(company) cluster(company), if C==1

areg DV IV1 IV2 IV3 IV4 IV5 IV6 IV7 IV8 IV9 IV10 IV11 IV12 i.year, absorb(company) cluster(company), if C==0

These are the regressions for using the -suest- command (cluster(company) is not allowed for using the -suest- command):

reg DV IV1 IV2 IV3 IV4 IV5 IV6 IV7 IV8 IV9 IV10 IV11 IV12 i.year i.company if C==1

estimate store C1

reg DV IV1 IV2 IV3 IV4 IV5 IV6 IV7 IV8 IV9 IV10 IV11 IV12 i.year i.company if C==0

estimate store C0

I use the cluster(company) option here as it does not work for the above regression:

suest C1 C0, cluster(company)

The coefficients are similiar, thanks a lot for your great help. But the p-values are not the same in all outputs. Is there somewhere a mistake? In case that this is normal, which p-value is relevant to me?

[Roman Mostazir]

Laura, the difference in p' values between areg and reg output is coming from the calculation of robust standard errors. your areg model is with robust s.e. while the reg model is not. However, I don't see why you should run both. Just run 'reg' models as you have already done and then run 'suest' which you have acomplished too. What you have not done yet is to compare the coefficients of IV1 from C1 model with IV1 from C0 model after suest. This can be done, as I can see your output:

test [C1_mean]IV1 - [C0_MEAN] IV1 = 0

There might be issues with the use of 'cluster' option this way if you have a survey data with multi-level structure and has intra-class correlation. I don't wish to get to that issue and leaving it with you. Googling and learning the concepts should guide you best.

best,

[Neyla Tad]

Dear all,
how can I compare coefficients from two different IV regressions? Suest does not work after ivreg.

[Katherine Adams]

Hello,

I want to test if the difference between the coefficients of two separate regressions is statistically significant.

In particular, I regress a consumption variable, “cons”, on a post-treatment/post-interaction dummy, “treat”; I use location-by-hour and hour-by-day FE, I cluster SE at location-by-hour. I want to know if there is a statistically significant difference between coefficients of my “treat” variable for peak consumption hours (peak==1) versus off-peak hours (peak==0).

I code as follows:

reghdfe cons treat if peak==1, absorb (i.location#i.hour i.hour#i.calday) vce(cluster i.location#i.hour)
est store peak1
reghdfe cons treat if peak==0, absorb (i.location#i.hour i.hour#i.calday) vce(cluster i.location#i.hour)
est store peak0

suest peak1 peak0

test [peak1_mean]treat=[peak0_mean]treat

Stata stopped at the “suest” command: I got the error that “peak1 was estimated with cluster(location#hour); re-estimate without the cluster() option, and specify the cluster() option with suest, r(332)”.

I tried to do that by regressing without clustered SE and coding “suest peak1 peak0, vce(cluster i.location#i.hour)”.
In that case, Stata said that for the “suest peak1 peak0, vce(cluster i.location#i.hour)” factor-variable and time-series operators/interactions not allowed, r(101).

Then, I tried regressing without clustering again, but I coded “suest peak1 peak1” after.
Stat said that “unable to generate scores for model peak1; suest requires that predict allow the score option, r(322)”.


Do you know how I can test the statistical significance in my case?

Thank you.