Testing difference in slopes of margins in threeway interaction

[Lotte Dijkstra]

I am running a model that includes a threeway interactions of age3 year and stedcbin. Age3 and stedcbin are binary. Year ranges from 2015 to 2021. The outcome is continuous, log-transformed and standardized. The dataset had over 500.000 observations. I want to test whether the increase in the effect of age3 over the included years is steeper for one category of stedcbin compared to another eg if the slope on the marginsplot is steeper. See my code and plot below. How do I test this?

Thank you for your advice!

CODE

regress ln_kostggz2_std sex age3##i.year##stedcbin [aweight=AANTAL_VER],cl(code3)
margins, dydx(age3) at(year=(2015/2021) stedcbin=(0/1))
marginsplot,recast(line) title (Association between mental health care cost and age group - by year and urbanicity) ytitle(Regression coefficient) xtitle(year)

[Andrew Musau]

help test

But you need to post the margins results to be accessible to test.

margins, dydx(age3) at(year=(2015/2021) stedcbin=(0/1)) post

[Lotte Dijkstra]

Thank you for the advice! I think were I went wrong was trying to test on the regress instead of the margins

[Leonardo Guizzetti]

I think you would rather test the coefficients from your interaction, rather than the estimated marginal means, which are a consequence of the model. The margins command is still helpful to show some aspect of the interaction in a more meaningful and digestible format.

testparm i.age3#i.year#i.stedcbin

I think something like the above is what you want, and it is testing the coefficient for the 3-way interaction specifically.

[Lotte Dijkstra]

Hi Leonardo,

Thank you for the advice. What you suggest is what I initially did, but I think that doesn't test the hypothesis of whether the two slopes are different. The p for that test was <0.0001, but I think that i simply based on the fact that there is in fact an interaction between age3 and stedcbin, (as evidenced by the plots), but it doesn't say anything about the slopes right?

Best,
Lotte

[Andrew Musau]

While what Leonardo says in #4 is true, he neglects to identify the correspondence between the margins output and the coefficients on the three-way interaction terms. Whether you use test on the margins output or on the coefficients on the interaction terms, you will get the same results. But it is more convenient to directly test the coefficients on the interaction terms. Here is the correspondence illustrated by means of an example.

webuse grunfeld, clear
keep if year<=1940
set seed 12182021
gen group= runiformint(0,1)
regress invest c.mvalue##i.year##group kstock
margins, dydx(c.mvalue) at(year=(1935/1940) group=(0 1)) post
*FOR EXAMPLE
di (_b[mvalue:4._at]-_b[mvalue:2._at])- (_b[mvalue:3._at]-_b[mvalue:1._at])

Res.:

. regress invest c.mvalue##i.year##group kstock

      Source |       SS           df       MS      Number of obs   =        60
-------------+----------------------------------   F(24, 35)       =      9.09
       Model |  854154.192        24   35589.758   Prob > F        =    0.0000
    Residual |  137038.144        35  3915.37555   R-squared       =    0.8617
-------------+----------------------------------   Adj R-squared   =    0.7669
       Total |  991192.336        59  16799.8701   Root MSE        =    62.573

-------------------------------------------------------------------------------------
             invest | Coefficient  Std. err.      t    P>|t|     [95% conf. interval]
--------------------+----------------------------------------------------------------
             mvalue |   .1045759   .0242648     4.31   0.000     .0553157     .153836
                    |
               year |
              1936  |   21.12691   51.31687     0.41   0.683    -83.05187    125.3057
              1937  |  -1.750276   53.74839    -0.03   0.974    -110.8653    107.3648
              1938  |  -4.928619   53.69891    -0.09   0.927    -113.9432     104.086
              1939  |   21.57041   74.53032     0.29   0.774    -129.7342     172.875
              1940  |  -11.05939   62.03709    -0.18   0.860    -137.0014    114.8826
                    |
      year#c.mvalue |
              1936  |  -.0229673   .0294509    -0.78   0.441    -.0827558    .0368211
              1937  |    -.03975   .0284259    -1.40   0.171    -.0974578    .0179577
              1938  |  -.0037927   .0372664    -0.10   0.920    -.0794476    .0718621
              1939  |  -.0326181   .0323109    -1.01   0.320    -.0982127    .0329765
              1940  |  -.0061121   .0444711    -0.14   0.891    -.0963933    .0841691
                    |
            1.group |   3.222769   89.68751     0.04   0.972    -178.8526    185.2981
                    |
     group#c.mvalue |
                 1  |  -.0160323   .2951572    -0.05   0.957    -.6152333    .5831687
                    |
         year#group |
            1936 1  |  -11.24198    114.383    -0.10   0.922    -243.4519    220.9679
            1937 1  |  -18.51496   104.2758    -0.18   0.860    -230.2061    193.1762
            1938 1  |   33.68572   105.0204     0.32   0.750    -179.5171    246.8885
            1939 1  |  -6.586384   116.1593    -0.06   0.955    -242.4024    229.2296
            1940 1  |     2.9365   109.6213     0.03   0.979    -219.6065    225.4795
                    |
year#group#c.mvalue |
            1936 1  |  -.0305349   .3532365    -0.09   0.932    -.7476431    .6865734
            1937 1  |   .1303172   .2969072     0.44   0.663    -.4724364    .7330709
            1938 1  |  -.0820868   .2986707    -0.27   0.785    -.6884206    .5242471
            1939 1  |  -.0450828   .2982605    -0.15   0.881    -.6505838    .5604181
            1940 1  |   .0143383   .2973849     0.05   0.962    -.5893852    .6180618
                    |
             kstock |   .1176607   .1109336     1.06   0.296    -.1075463    .3428678
              _cons |  -8.195448   36.28771    -0.23   0.823    -81.86342    65.47252
-------------------------------------------------------------------------------------

.
. margins, dydx(c.mvalue) at(year=(1935/1940) group=(0 1)) post

Average marginal effects                                    Number of obs = 60
Model VCE: OLS

Expression: Linear prediction, predict()
dy/dx wrt:  mvalue
1._at:  year  = 1935
        group =    0
2._at:  year  = 1935
        group =    1
3._at:  year  = 1936
        group =    0
4._at:  year  = 1936
        group =    1
5._at:  year  = 1937
        group =    0
6._at:  year  = 1937
        group =    1
7._at:  year  = 1938
        group =    0
8._at:  year  = 1938
        group =    1
9._at:  year  = 1939
        group =    0
10._at: year  = 1939
        group =    1
11._at: year  = 1940
        group =    0
12._at: year  = 1940
        group =    1

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
mvalue       |
         _at |
          1  |   .1045759   .0242648     4.31   0.000     .0553157     .153836
          2  |   .0885436   .2942532     0.30   0.765    -.5088222    .6859094
          3  |   .0816086   .0166883     4.89   0.000     .0477294    .1154877
          4  |   .0350414   .1943652     0.18   0.858     -.359541    .4296238
          5  |   .0648259   .0146786     4.42   0.000     .0350267     .094625
          6  |   .1791108   .0283581     6.32   0.000     .1215409    .2366808
          7  |   .1007831   .0277509     3.63   0.001     .0444459    .1571204
          8  |   .0026641   .0385479     0.07   0.945    -.0755923    .0809204
          9  |   .0719578   .0216532     3.32   0.002     .0279996     .115916
         10  |   .0108427   .0343042     0.32   0.754    -.0587986     .080484
         11  |   .0984638   .0366289     2.69   0.011     .0241032    .1728244
         12  |   .0967698   .0156539     6.18   0.000     .0649906     .128549
------------------------------------------------------------------------------

.
. *FOR EXAMPLE

.
. di (_b[mvalue:4._at]-_b[mvalue:2._at])- (_b[mvalue:3._at]-_b[mvalue:1._at])
-.03053485

Asking whether the slopes are the same is equivalent to asking whether the difference in estimated margins of the groups at each year is equal to 0. The coefficients on the three way interactions are differences as shown above, and this will be exactly what you test following Leonardo's advice.

[Lotte Dijkstra]

Thank you for this clarification Andrew! Just to clarify I tried the following syntax in my data to see if this also checked out for my data:

regress ln_kostggz2_std sex age3##i.year##stedcbin [aweight=AANTAL_VER],cl(code3)
margins, dydx(age3) at(year=(2015/2021) stedcbin=(0/1)) post
di (_b[age3:1._at]-_b[age3:13._at]) - (_b[age3:2._at]-_b[age3:14._at])

The only thing is I got the error "equation age3 not found", did I do something wrong here? Or is it to do with 'age3' being binary?

[Andrew Musau]

Can you show your exact Stata output? Perhaps some variables are omitted in the original regression. If all 3 variables are categorical, you can have

regress ln_kostggz2_std sex i.age3##i.year##i.stedcbin [aweight=AANTAL_VER],cl(code3)
margins age3#year#stedcbin, post

But for an indicator, whether you consider it continuous or categorical does not matter in this instance. So you should get equivalent results with

regress ln_kostggz2_std sex c.age3##i.year##i.stedcbin [aweight=AANTAL_VER],cl(code3)
margins,  dydx(age3) at(year=(2015/2021) stedcbin=(0 1)) post

[Lotte Dijkstra]

Hi Andrew,

Please find my full output for this below. Thank you for your help!

Best, Lotte

. regress ln_kostggz2_std sex age3##i.year##stedcbin [aweight=AANTAL_VER],cl(code3)
(sum of wgt is 73,868,937)

Linear regression                               Number of obs     =    515,051
                                                F(28, 795)        =     488.68
                                                Prob > F          =     0.0000
                                                R-squared         =     0.1338
                                                Root MSE          =     .74352

                                       (Std. err. adjusted for 796 clusters in code3)
-------------------------------------------------------------------------------------
                    |               Robust
    ln_kostggz2_std | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
--------------------+----------------------------------------------------------------
                sex |   .2028535   .0074433    27.25   0.000     .1882426    .2174643
                    |
               age3 |
       18-34 years  |   .3186267   .0079569    40.04   0.000     .3030078    .3342457
                    |
               year |
              2016  |   .0123499   .0066711     1.85   0.065    -.0007451     .025445
              2017  |   .0485235   .0070076     6.92   0.000     .0347679    .0622792
              2018  |   .0970791   .0073055    13.29   0.000     .0827388    .1114193
              2019  |   .1317372   .0091919    14.33   0.000      .113694    .1497804
              2020  |   .1621379   .0087133    18.61   0.000     .1450342    .1792416
              2021  |  -.1353133   .0093257   -14.51   0.000    -.1536193   -.1170074
                    |
          age3#year |
  18-34 years#2016  |  -.0061934   .0081212    -0.76   0.446     -.022135    .0097482
  18-34 years#2017  |   .0205243     .00883     2.32   0.020     .0031914    .0378572
  18-34 years#2018  |   .0570288   .0086577     6.59   0.000     .0400341    .0740235
  18-34 years#2019  |    .103066   .0092178    11.18   0.000     .0849718    .1211602
  18-34 years#2020  |   .1271401   .0093979    13.53   0.000     .1086924    .1455879
  18-34 years#2021  |   .1245072   .0098753    12.61   0.000     .1051223     .143892
                    |
         1.stedcbin |   .5749673   .0428748    13.41   0.000     .4908061    .6591286
                    |
      age3#stedcbin |
     18-34 years#1  |  -.3362293   .0349081    -9.63   0.000    -.4047522   -.2677065
                    |
      year#stedcbin |
            2016 1  |  -.0220281   .0151449    -1.45   0.146    -.0517569    .0077007
            2017 1  |  -.0179284   .0125512    -1.43   0.154    -.0425658     .006709
            2018 1  |   -.017175   .0143134    -1.20   0.231    -.0452715    .0109216
            2019 1  |   -.036581   .0189934    -1.93   0.054    -.0738641    .0007021
            2020 1  |  -.0254067   .0189114    -1.34   0.180    -.0625289    .0117155
            2021 1  |  -.0706841   .0175651    -4.02   0.000    -.1051635   -.0362048
                    |
 age3#year#stedcbin |
18-34 years#2016#1  |   .0504916   .0191415     2.64   0.009     .0129177    .0880655
18-34 years#2017#1  |   .0484959   .0211667     2.29   0.022     .0069466    .0900452
18-34 years#2018#1  |    .045628   .0220061     2.07   0.038      .002431     .088825
18-34 years#2019#1  |    .022792   .0226616     1.01   0.315    -.0216917    .0672757
18-34 years#2020#1  |   .0739235   .0217149     3.40   0.001     .0312981    .1165489
18-34 years#2021#1  |   .1384408   .0245835     5.63   0.000     .0901845     .186697
                    |
              _cons |  -.2280229     .02504    -9.11   0.000    -.2771753   -.1788705
-------------------------------------------------------------------------------------

. 
. margins, dydx(age3) at(year=(2015/2021) stedcbin=(0/1)) post

Average marginal effects                               Number of obs = 515,051
Model VCE: Robust

Expression: Linear prediction, predict()
dy/dx wrt:  1.age3
1._at:  year     = 2015
        stedcbin =    0
2._at:  year     = 2015
        stedcbin =    1
3._at:  year     = 2016
        stedcbin =    0
4._at:  year     = 2016
        stedcbin =    1
5._at:  year     = 2017
        stedcbin =    0
6._at:  year     = 2017
        stedcbin =    1
7._at:  year     = 2018
        stedcbin =    0
8._at:  year     = 2018
        stedcbin =    1
9._at:  year     = 2019
        stedcbin =    0
10._at: year     = 2019
        stedcbin =    1
11._at: year     = 2020
        stedcbin =    0
12._at: year     = 2020
        stedcbin =    1
13._at: year     = 2021
        stedcbin =    0
14._at: year     = 2021
        stedcbin =    1

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
0.age3       |  (base outcome)
-------------+----------------------------------------------------------------
1.age3       |
         _at |
          1  |   .3186267   .0079569    40.04   0.000     .3030078    .3342457
          2  |  -.0176026   .0340139    -0.52   0.605    -.0843704    .0491651
          3  |   .3124333   .0082843    37.71   0.000     .2961716     .328695
          4  |   .0266955    .029344     0.91   0.363    -.0309053    .0842963
          5  |   .3391511   .0084925    39.94   0.000     .3224806    .3558215
          6  |   .0514176   .0297385     1.73   0.084    -.0069576    .1097929
          7  |   .3756555   .0082919    45.30   0.000     .3593788    .3919322
          8  |   .0850542   .0310219     2.74   0.006     .0241597    .1459487
          9  |   .4216927   .0085569    49.28   0.000      .404896    .4384895
         10  |   .1082554   .0296224     3.65   0.000     .0501081    .1664027
         11  |   .4457669   .0087109    51.17   0.000     .4286678     .462866
         12  |    .183461   .0281839     6.51   0.000     .1281374    .2387846
         13  |   .4431339   .0088643    49.99   0.000     .4257337    .4605341
         14  |   .2453453   .0256296     9.57   0.000     .1950356    .2956551
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. 
. test _b[1._at] - _b[13._at] = _b[2._at] - _b[14._at]

 ( 1)  [0b.age3]1bno._at - [0b.age3]2o._at - [0b.age3]13o._at + [0b.age3]14o._at = 0
       Constraint 1 dropped

       F(  0,   795) =       .
            Prob > F =         .

di (_b[age3:1._at]-_b[age3:13._at]) - (_b[age3:2._at]-_b[age3:14._at])
equation age3 not found
r(111);

[Andrew Musau]

Your coefficients should be e.g.,

 
 _b[1.age3:1._at]

Run

margins, dydx(age3) at(year=(2015/2021) stedcbin=(0/1)) post coeflegend

to see exactly how they are named.

[Lotte Dijkstra]

Thank you Andrew, I will have another go!