Interpreting and testing significance of quadratic interaction term

[Ferdinand Berr]

Hello!

I'm currently working with the replication data and Stata code of the paper "Temperature Shocks and Economic Growth: Evidence from the Last Half Century." by Dell, Jones and Olken (2012).

Here is an excerpt from the data:

* Example generated by -dataex-. To install: ssc install dataex
clear
input int year str24 country float(g wtem) byte(initxtilegdp1 initxtilegdp2) float(wtem_initxtilegdp1 wtem_initxtilegdp2)
1979 "United Arab Emirates"   9.536743  27.24524 0 1 0  27.24524
1980 "United Arab Emirates"  13.610554 27.591845 0 1 0 27.591845
1981 "United Arab Emirates"  .05979538  27.17235 0 1 0  27.17235
1982 "United Arab Emirates" -14.233494 26.075714 0 1 0 26.075714
1983 "United Arab Emirates" -11.116982   26.1611 0 1 0   26.1611
1984 "United Arab Emirates" -1.0873795 25.826963 0 1 0 25.826963
1985 "United Arab Emirates" -15.190983 25.842203 0 1 0 25.842203
1986 "United Arab Emirates"   -23.5939  25.92592 0 1 0  25.92592
1987 "United Arab Emirates"   .4487038 26.229404 0 1 0 26.229404
1988 "United Arab Emirates"  -8.002472  26.76122 0 1 0  26.76122
1989 "United Arab Emirates"   7.117462  26.13744 0 1 0  26.13744
end

g is growth in GDP, wtem is temperature, initxtilegdp1 and initxtilegdp2 are dummies with 1 indicating poor country, wtem_initxtilegdp1 and wtem_initxtilegdp2 are interaction terms of temperature with the poor country dummy.

The paper uses a linear specification of temperature and interacts temperature with the poor country dummy. The original code for the most basic specification looks like this:

cgmreg g wtem wtem_initxtilegdp1 RY* i.cc_num, cluster(parent_num rynum)

My problem:

I want to include a quadratic specification of temperature to the model, as I expect non-linear effects of temperature on growth. However, I want to preserve the heterogeneous effects of temperature on rich/poor countries. After looking at some Stata threads, I specified this using factor variable notation as follows:

cgmreg g i.initxtilegdp1##c.wtem##c.wtem RY* i.cc_num, cluster(parent_num rynum)

First of all - is this correct?

And secondly - how do I test the significance of those effects now? Not showing all the fixed effects and the constant, my results look like this:

Number of obs     =    4924
                                                 Num clusvars      =    2
                                                 Num combinations  =    3

                                                 G(parent_num)     =    126
                                                 G(rynum)          =    324

---------------------------------------------------------------------------------------
                    g |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
----------------------+----------------------------------------------------------------
      1.initxtilegdp1 |   17.00579   14.92628     1.14   0.255    -12.24919    46.26076
                 wtem |   .1410002   .5102136     0.28   0.782        -.859       1.141
                      |
 initxtilegdp1#c.wtem |
                   1  |   .4265483   1.240141     0.34   0.731    -2.004084    2.857181
                      |
        c.wtem#c.wtem |   .0045207   .0204894     0.22   0.825    -.0356378    .0446793
                      |
 initxtilegdp1#c.wtem#|
               c.wtem |
                   1  |  -.0537301   .0337138    -1.59   0.111    -.1198079    .0123477

I tried this for example, but to be honest I don't completely understand what it means yet:

testparm i.initxtilegdp1##c.wtem##c.wtem

 ( 1)  1.initxtilegdp1 = 0
 ( 2)  wtem = 0
 ( 3)  1.initxtilegdp1#c.wtem = 0
 ( 4)  c.wtem#c.wtem = 0
 ( 5)  1.initxtilegdp1#c.wtem#c.wtem = 0

           chi2(  5) =   16.48
         Prob > chi2 =    0.0056

I understand I would accordingly reject the hypothesis that all of the above coefficients would be 0 simultaneously, but I'm not sure whether that is the hypothesis I'm interested in. Especially as (1) 1.initxtilegdp1 = 0 would imply that country X is rich.

Basically I'm interested in the hypotheses: (1) Does temperature have some significant effect on growth? (2) If so, is the effect only significant for either poor or rich countries? (3) Is the effect of temperature on growth non-linear (as in the quadratic specification)?

Thank you for any input on how to test significance and interpret marginal effects in this case with a quadratic interaction!



References:

Dell, Melissa, Benjamin F. Jones, and Benjamin A. Olken. 2012. "Temperature Shocks and Economic Growth: Evidence from the Last Half Century." American Economic Journal: Macroeconomics, 4 (3): 66-95.

[Clyde Schechter]

Basically I'm interested in the hypotheses: (1) Does temperature have

some

significant effect on growth? (2) If so, is the effect only significant for either poor or rich countries? (3) Is the effect of temperature on growth non-linear (as in the quadratic specification)?

// HYPOTHESIS 1
test c.wtem 1.initxtilegdp1#c.wtem c.wtem#c. wtem 1.initxtilegdp1#c.wtem#c.wtem

// HYPOTHESES 2 (BUT READ MY COMMENTS BELOW)
margins initxtilegdp1, dydx(wtem) // AVERAGE MARGINAL EFFECT OF TEMPERATURE, BY RICH VS POOR

// HYPOTHESIS 3
test c.wtem#c.wtem 1.initxtilegdp1#c.wtem#c.wtem

Concerning hypotheses 2: the code shown will give you separate estimates of the average marginal effects of temperature in rich and poor countries. You may, however, be interested in the marginal effects of temperature in rich and poor countries at particular levels of temperature. If so, add an -at()- option to the -margins- command to specify those particular levels.

IMPORTANT: the difference between statistically significant and not statistically significant is not, itself, statistically significant. Statements like: there was a significant effect in poor countries but not in rich countries are meaningless and misleading. They are a form of statistical malpractice. What you might want to do is quantify the difference in average marginal effect of temperature between rich and poor countries--which can be done by adding the -pwcompare(cieffects)- option to the -margins- command.

Finally, the American Statistical Association has recommended that the concept of statistical significance be abandoned. See https://www.tandfonline.com/doi/full...5.2019.1583913 for the "executive summary" and https://www.tandfonline.com/toc/utas20/73/sup1 for all 43 supporting articles. Or https://www.nature.com/articles/d41586-019-00857-9 for the tl;dr. I suggest you focus on quanitfying effect estimates and their precision instead of testing hypotheses.

By the way, what is cgmreg? I have never heard of it; it is not part of official Stata, nor does -search cgmreg- turn up anything.

[Ferdinand Berr]

Thanks a lot for the helpful reply, I will try to understand the code provided by you.

Thank you also for the helpful input on my hypotheses! I specified those here trying to formulate what I am interested in, but you're definitely right. Probably it's more concise and correct to just say I'm interested in studying the exact nature of temperature effects on growth, specifically looking at non-linear and heterogeneous effects through my specification here.
But I'm definitely interested in looking at the difference in average marginal effect between rich/poor you mention!

cgmreg, as far as I currently understand, employs a Fixed Effects Panel Model with 2-way error term clustering according to Cameron,Gelbach and Miller (2011). It is the method used in the Dell, Jones and Olken (2012) paper I'm trying to replicate/play with!


References:

Cameron, A. Colin, Jonah B. Gelbach, and Douglas L. Miller. 2011. “Robust Inference with Multiway Clustering.” Journal of Business and Economic Statistics 29 (2): 238–49.

[Clyde Schechter]

Thank you.