Help on Interpretation of Interaction Results

[amish dugar]

Hi
I am running a model which has a continuous dependent variable (How much earnings management is done by a firm) and two main effects independent variables (1. Dummy of Director having professional expertise, and 2. Stake of promoters in the firm which is a continuous variable but converted into categorical using bins) along with other control variables. The main effect model returns significant negative coefficients for both the effects meaning: having professional expertise and higher stake of promoters in firm reduce earnings management in firms. Then I generate an interaction variable between the two main effects and run regression analysis on revised model. The results remain significantly negative for the main effects as earlier, however, the interaction effect is significantly positive. How to interpret this result?

Regards

Amish

[Clyde Schechter]

It is easier to discuss this with a concrete example than in the abstract. Since you don't show us any of your output, let's use an example from the auto.dta that comes with Stata.

. sysuse auto, clear
(1978 Automobile Data)

. 
. gen byte domestic = !foreign

. xtile mpg_tertile = mpg, nq(3)

. 
. regress price i.domestic##i.mpg_tertile

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(5, 68)        =      4.45
       Model |   156625171         5  31325034.1   Prob > F        =    0.0014
    Residual |   478440226        68  7035885.67   R-squared       =    0.2466
-------------+----------------------------------   Adj R-squared   =    0.1912
       Total |   635065396        73  8699525.97   Root MSE        =    2652.5

--------------------------------------------------------------------------------------
               price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------------+----------------------------------------------------------------
          1.domestic |  -1643.282    1314.15    -1.25   0.215    -4265.627    979.0636
                     |
         mpg_tertile |
                  2  |    -2840.8   1677.604    -1.69   0.095    -6188.406    506.8061
                  3  |  -4085.183   1411.914    -2.89   0.005    -6902.614   -1267.753
                     |
domestic#mpg_tertile |
                1 2  |   490.2187   1872.025     0.26   0.794    -3245.349    4225.786
                1 3  |   851.5924   1718.412     0.50   0.622    -2577.445     4280.63
                     |
               _cons |     9258.6   1186.245     7.80   0.000     6891.485    11625.71
--------------------------------------------------------------------------------------

This is analogous to your situation: domestic is a dichotomous variable, mpg_tertile is a binned version of a continuous variable. (Digression: binning a continuous variable to use as a regression predictor is a bad idea, and I encourage you to reconsider using the original continuous variable instead, but I won't go on here about the reasons way.) So domestic plays the role of director having professional experience, and mpg_tertile plays the role of your binned promoters' stake variable.

The main effects of domestic and mpg_tertile are all negative, but the interaction terms have positive coefficients.

When using an interaction model you have to remember that the "main" effects do not mean what they mean in the corresponding model without the interaction term. For example, the effect of "domestic" is not to reduce price by 1643.282. In fact, in an interaction model, there is no such thing as the effect of domestic. There are several different effects of domestic, depending on the value of mpg_tertile. The number 1643.282 is the effect of domestic on price in those cars that fall in tertile 1 of mpg. Similarly, -2840.8 is not the effect of being in tertile 2 of mpg on price; there is no such effect. Rather, there are two different effects of being in tertile 2, one corresponding to foreign cars (domestic = 0), which is -2840.8, and the other corresponding to domestic cars (domestic = 1) which is -2840.8 + 490.2187. That last number is, of course, the 1domestic#2.mpg_tertile interaction coefficient. So the fact that that interaction coefficient is positive implies that the effect on price of being in tertile 2 of mpg is larger (i.e. less negative) for domestic cars (domestic = 1) than it is for foreign cars. Similar calculations and states can be said for tertile 3.

Probably the simplest way to understand the results of an interaction model is to not dwell too long on the coefficient table, but rather to use the -margins- command.

. margins domestic#mpg_tertile

Adjusted predictions                            Number of obs     =         74
Model VCE    : OLS

Expression   : Linear prediction, predict()

--------------------------------------------------------------------------------------
                     |            Delta-method
                     |     Margin   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------------+----------------------------------------------------------------
domestic#mpg_tertile |
                0 1  |     9258.6   1186.245     7.80   0.000     6891.485    11625.71
                0 2  |     6417.8   1186.245     5.41   0.000     4050.685    8784.915
                0 3  |   5173.417   765.7178     6.76   0.000     3645.451    6701.383
                1 1  |   7615.318   565.5201    13.47   0.000      6486.84    8743.796
                1 2  |   5264.737   608.5308     8.65   0.000     4050.432    6479.041
                1 3  |   4381.727   799.7662     5.48   0.000     2785.819    5977.636
--------------------------------------------------------------------------------------

shows us the predicted value of price in each combination of domestic = 0 or 1 and mpg_tertile = 1, 2, or 3.

. margins mpg_tertile, dydx(domestic)

Conditional marginal effects                    Number of obs     =         74
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 1.domestic

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.domestic   |
 mpg_tertile |
          1  |  -1643.282    1314.15    -1.25   0.215    -4265.627    979.0636
          2  |  -1153.063   1333.224    -0.86   0.390    -3813.471    1507.344
          3  |  -791.6894   1107.226    -0.72   0.477    -3001.125    1417.746
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

shows us the marginal effect of domestic origin on price in each of the three mpg tertiles.

And, finally,

. margins domestic, dydx(mpg_tertile)

Conditional marginal effects                    Number of obs     =         74
Model VCE    : OLS

Expression   : Linear prediction, predict()
dy/dx w.r.t. : 2.mpg_tertile 3.mpg_tertile

--------------------------------------------------------------------------------
               |            Delta-method
               |      dy/dx   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------+----------------------------------------------------------------
2.mpg_tertile  |
      domestic |
            0  |    -2840.8   1677.604    -1.69   0.095    -6188.406    506.8061
            1  |  -2350.581   830.7363    -2.83   0.006     -4008.29   -692.8729
---------------+----------------------------------------------------------------
3.mpg_tertile  |
      domestic |
            0  |  -4085.183   1411.914    -2.89   0.005    -6902.614   -1267.753
            1  |  -3233.591   979.5095    -3.30   0.002    -5188.172    -1279.01
--------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

shows us the marginal effects of being in either tertile 2 or tertile 3, relative to the base tertile 1, in domestic and foreign cars.

Often even more helpful than these tables are graphs of the -margins- results. Those can be obtained, in each case, by running -marginsplot- immediately after the -margins- command. Those pictures are typically worth more than 1,000 words.

[amish dugar]

Dear Clyde

Thanks a lot for your elaborate reply. I will be switching back to the continuous varibale.

Regards

Amish

[amish dugar]

Dear Clyde

I have now remodeled my question by keeping continuous variable as it is. I am also attaching the output for your reference. Here ABS_DACC is the dependent variable Absolute Discretionary Accruals which indicate earnings management by a firm (higher figure implies more hanky panky accounting by firm) . The first dependent variable (D_ACC) is Dummy of Presence of a Director with expertise. The second independent variable (PROMHOLD) is % of Promoter Holding in a firm and INT5 indicates the interaction variable. All other variables in model are control variables.

How should I interpret the results of interaction term? Does the result mean that with a firm having combination of a director with expertise and increased promoter holding, will resulting in higher earnings management?

[Carlo Lazzaro]

Amish:
as Clyde effectively pointed out, there's a proper way to create interactions (and categorical variables, too) via -fvvarlist- which, in turn, gives you access to two very useful Stata built-in commands, such as -margins- and -marginsplot- (which Clyde often points the poster to).
Hence, why sticking with the inefficient approach of creating interactions "by hand"?
Please also note that, from your code and outcome, it is very hard (for me, at any rate) to disentangle interactions from conditional main effects.

[Clyde Schechter]

Carlo is absolutely right. Revised your regression command using factor variable notation, and then run the -margins- commands, and you will have everything you need to understand what your model tells you. See -help fvvarlist- for details.

regress ABS_DACC i.D_ACC##c.PROMHOLD // and the other variables 
margins D_ACC#PROMHOLD
margins D_ACC, dydx(PROMHOLD)
marginsplot

Then select a set of interesting values of PROMHOLD. (If none are more interesting than others, select a bunch of values spanning the range of observed values of PROMHOLD). For the sake of illustration, let's say that the interesting values for PROMHOLD are 10, 25, 50, 75, and 90. Then run

margins, dydx(D_ACC) at (PROMHOLD = (10 25 50 75 90))
marginsplot

[Leon Schmidt]

Dear all,

I have a related question to interpreting interaction results. I post it here since it fits in quite well.

If my dependent variable is in logs and there are no interactions I would interpret the coefficient on x1 (which is in levels) the following way: An increase in x1 by one unit is associated with an increase in the dependent variable of beta_1 * 100 percent.

Assume I have the following model: log(y) = b0 + b1*x1 + b2*x2 + b3*x1*x2 (simplifying here a bit)

I want to calculate the marginal effect of increasing x1 by one unit when x2 = 5. Would the following calculation be correct (given y is in log): (b1 + 5 * b3) * 100 percent?

Thank you very much!

All the best
Leon

[Hugh Colaco]

Dear Clyde,

Based on your first regression above, I expect that one can also conclude that, given the size of the interaction coefficients, the effect on price of being in tertile 3 of mpg is larger (i.e. less negative) than tertile 2 for domestic cars (domestic = 1) than it is for foreign cars. Can you please confirm this?

Thanks,

Hugh

[Clyde Schechter]

Re #7.

I want to calculate the marginal effect of increasing x1 by one unit when x2 = 5. Would the following calculation be correct (given y is in log): (b1 + 5 * b3) * 100 percent?

Well, no, the marginal effect isn't calculated anything like this. But your formula is the correct one for the semi-elasticity.

[Clyde Schechter]

given the size of the interaction coefficients, the effect on price of being in tertile 3 of mpg is larger (i.e. less negative) than tertile 2 for domestic cars (domestic = 1) than it is for foreign cars. Can you please confirm this?

I can neither confirm it nor dispute it, as I don't understand it.

[Leon Schmidt]

Thank you very much Clyde! Then I got the names confused. The semi-elasticity is indeed what I wanted.

Would the same calculation and interpretation hold if x1 is in levels (e.g. tariffs, running from 0 to 100) and x2 in logs (e.g. log of labor-productivity)? So that I can say something like "A one percentage point increase in tariffs is associated with a ... percent increase in (log) employment when log(productivity) is ....".

Thank you!

[Clyde Schechter]

Yes.

Perhaps this is a good time for me to also remind you that this 100*coefficient percent calculation, no matter what context it is applied in, is, in fact, only an approximation. It works well so long as the coefficient is close to zero. The farther the coefficient is from zero, the worse the approximation, and it quickly gets pretty bad. The algebra is like this: you increase x1 by 1, so log y goes up by b (where b is the coefficient. So if the original value is y0, we have log y = b + log y0 (where y0 is the original value of y). Exponentiating both sides of the equation, y = exp(b)*y0. So y/y0 = exp(b). Now, the Taylor's series of exp(b) begins as 1 + b + b^2/2! + b^3/3! + ...If b is small enough that b^2 and higher powers of b can be ignored, then 1+b is a good approximation to exp(b). So y/y0 is approximately 1 + b, or in terms of percent, y has increased by 100*b percent. But this approximation is clearly only as good as the negligibility of b^2 (and higher powers). So basically for b up to about 0.10 this is reasonable (as b^2 is then 0.01, and higher powers are even smaller). But it deteriorates rapidly from that point on. (The same thing happens when b is negative: it's the magnitude of b that counts.)

[Leon Schmidt]

Thank you very much for your response and explanation Clyde! I knew that it is only an approximation but it is good to see the underlying mathematics again, thank you!

[Karl Zou]

It is easier to discuss this with a concrete example than in the abstract. Since you don't show us any of your output, let's use an example from the auto.dta that comes with Stata.

. sysuse auto, clear
(1978 Automobile Data)

.
. gen byte domestic = !foreign

. xtile mpg_tertile = mpg, nq(3)

.
. regress price i.domestic##i.mpg_tertile

Source | SS df MS Number of obs = 74
-------------+---------------------------------- F(5, 68) = 4.45
Model | 156625171 5 31325034.1 Prob > F = 0.0014
Residual | 478440226 68 7035885.67 R-squared = 0.2466
-------------+---------------------------------- Adj R-squared = 0.1912
Total | 635065396 73 8699525.97 Root MSE = 2652.5

--------------------------------------------------------------------------------------
price | Coef. Std. Err. t P>|t| [95% Conf. Interval]
---------------------+----------------------------------------------------------------
1.domestic | -1643.282 1314.15 -1.25 0.215 -4265.627 979.0636
|
mpg_tertile |
2 | -2840.8 1677.604 -1.69 0.095 -6188.406 506.8061
3 | -4085.183 1411.914 -2.89 0.005 -6902.614 -1267.753
|
domestic#mpg_tertile |
1 2 | 490.2187 1872.025 0.26 0.794 -3245.349 4225.786
1 3 | 851.5924 1718.412 0.50 0.622 -2577.445 4280.63
|
_cons | 9258.6 1186.245 7.80 0.000 6891.485 11625.71
--------------------------------------------------------------------------------------

This is analogous to your situation: domestic is a dichotomous variable, mpg_tertile is a binned version of a continuous variable. (Digression: binning a continuous variable to use as a regression predictor is a bad idea, and I encourage you to reconsider using the original continuous variable instead, but I won't go on here about the reasons way.) So domestic plays the role of director having professional experience, and mpg_tertile plays the role of your binned promoters' stake variable.

The main effects of domestic and mpg_tertile are all negative, but the interaction terms have positive coefficients.

When using an interaction model you have to remember that the "main" effects do not mean what they mean in the corresponding model without the interaction term. For example, the effect of "domestic" is not to reduce price by 1643.282. In fact, in an interaction model, there is no such thing as the effect of domestic. There are several different effects of domestic, depending on the value of mpg_tertile. The number 1643.282 is the effect of domestic on price in those cars that fall in tertile 1 of mpg. Similarly, -2840.8 is not the effect of being in tertile 2 of mpg on price; there is no such effect. Rather, there are two different effects of being in tertile 2, one corresponding to foreign cars (domestic = 0), which is -2840.8, and the other corresponding to domestic cars (domestic = 1) which is -2840.8 + 490.2187. That last number is, of course, the 1domestic#2.mpg_tertile interaction coefficient. So the fact that that interaction coefficient is positive implies that the effect on price of being in tertile 2 of mpg is larger (i.e. less negative) for domestic cars (domestic = 1) than it is for foreign cars. Similar calculations and states can be said for tertile 3.

Probably the simplest way to understand the results of an interaction model is to not dwell too long on the coefficient table, but rather to use the -margins- command.

. margins domestic#mpg_tertile

Adjusted predictions Number of obs = 74
Model VCE : OLS

Expression : Linear prediction, predict()

--------------------------------------------------------------------------------------
| Delta-method
| Margin Std. Err. t P>|t| [95% Conf. Interval]
---------------------+----------------------------------------------------------------
domestic#mpg_tertile |
0 1 | 9258.6 1186.245 7.80 0.000 6891.485 11625.71
0 2 | 6417.8 1186.245 5.41 0.000 4050.685 8784.915
0 3 | 5173.417 765.7178 6.76 0.000 3645.451 6701.383
1 1 | 7615.318 565.5201 13.47 0.000 6486.84 8743.796
1 2 | 5264.737 608.5308 8.65 0.000 4050.432 6479.041
1 3 | 4381.727 799.7662 5.48 0.000 2785.819 5977.636
--------------------------------------------------------------------------------------

shows us the predicted value of price in each combination of domestic = 0 or 1 and mpg_tertile = 1, 2, or 3.

. margins mpg_tertile, dydx(domestic)

Conditional marginal effects Number of obs = 74
Model VCE : OLS

Expression : Linear prediction, predict()
dy/dx w.r.t. : 1.domestic

------------------------------------------------------------------------------
| Delta-method
| dy/dx Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.domestic |
mpg_tertile |
1 | -1643.282 1314.15 -1.25 0.215 -4265.627 979.0636
2 | -1153.063 1333.224 -0.86 0.390 -3813.471 1507.344
3 | -791.6894 1107.226 -0.72 0.477 -3001.125 1417.746
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

shows us the marginal effect of domestic origin on price in each of the three mpg tertiles.

And, finally,

. margins domestic, dydx(mpg_tertile)

Conditional marginal effects Number of obs = 74
Model VCE : OLS

Expression : Linear prediction, predict()
dy/dx w.r.t. : 2.mpg_tertile 3.mpg_tertile

--------------------------------------------------------------------------------
| Delta-method
| dy/dx Std. Err. t P>|t| [95% Conf. Interval]
---------------+----------------------------------------------------------------
2.mpg_tertile |
domestic |
0 | -2840.8 1677.604 -1.69 0.095 -6188.406 506.8061
1 | -2350.581 830.7363 -2.83 0.006 -4008.29 -692.8729
---------------+----------------------------------------------------------------
3.mpg_tertile |
domestic |
0 | -4085.183 1411.914 -2.89 0.005 -6902.614 -1267.753
1 | -3233.591 979.5095 -3.30 0.002 -5188.172 -1279.01
--------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

shows us the marginal effects of being in either tertile 2 or tertile 3, relative to the base tertile 1, in domestic and foreign cars.

Often even more helpful than these tables are graphs of the -margins- results. Those can be obtained, in each case, by running -marginsplot- immediately after the -margins- command. Those pictures are typically worth more than 1,000 words.

When we have some coefficients are not significant in the interaction of two categorical variables. For example, in this case, the coefficients of "domestic#mpg" are not significant. Do we still interpret these two? How do we deal with the insignificance?

[Liza Vieira]

Dear all,
I have a dataset of 361615 observations. Each observation is a scientific publication. For each publication, I have its impact (avg_prc_f, which varies between [0 and 1]), the performance of the scientific system of the country of interest (EIS, a categorical variable, 1= modest, 2=moderate, 3=strong and 4=leader), a variable that indicates the type of collaboration that we can find in the publication (col_type, a categorical variable, 0 if there is no collaboration, 1 if there is a domestic collaboration, 2 if the collaborating country as performance (of the scientific system) equal to 1, 3 if the collaborating country as performance (of the scientific system) equal to 2, 4 if the collaborating country as performance (of the scientific system) equal to 3, and 5 if the collaborating country as performance (of the scientific system) equal to 4), and a set of control variables.

I used the fracreg command in Stata 16 as follows:

 fracreg probit avg_prc_f i.col_type pg nr na open_access Q1 i.col_type#i.EIS , vce (robust)

------------------------------------------------------------------------------
             |               Robust
   avg_prc_f |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
    col_type |
          1  |    .199999   .0165987    12.05   0.000     .1674662    .2325318
          2  |   .4427582   .0402287    11.01   0.000     .3639114     .521605
          3  |   .3396305   .0226657    14.98   0.000     .2952066    .3840544
          4  |   .3559337   .0220889    16.11   0.000     .3126403    .3992272
          5  |   .3606123   .0357351    10.09   0.000     .2905728    .4306518
             |
          pg |   .0046436   .0003196    14.53   0.000     .0040172    .0052699
          nr |   .0069635   .0000791    88.08   0.000     .0068085    .0071184
          na |   .0131863   .0034628     3.81   0.000     .0063992    .0199733
 open_access |   .0416439   .0026592    15.66   0.000      .036432    .0468558
          Q1 |     .42836   .0032704   130.98   0.000     .4219502    .4347698
             |
col_type#EIS |
        0 2  |   .0430817   .0117703     3.66   0.000     .0200124    .0661509
        0 3  |   .3440224   .0110171    31.23   0.000     .3224292    .3656156
        0 4  |   .4702615   .0137467    34.21   0.000     .4433184    .4972046
        1 2  |    .250351   .0078601    31.85   0.000     .2349456    .2657565
        1 3  |   .2578381   .0069532    37.08   0.000     .2442101    .2714662
        1 4  |   .3785887   .0076558    49.45   0.000     .3635837    .3935937
        2 2  |   -.089323   .0399929    -2.23   0.026    -.1677075   -.0109384
        2 3  |  -.0920803   .0399287    -2.31   0.021    -.1703391   -.0138215
        2 4  |  -.1180841   .0486326    -2.43   0.015    -.2134022    -.022766
        3 2  |   .1927859   .0164949    11.69   0.000     .1604565    .2251152
        3 3  |   .1694955   .0161466    10.50   0.000     .1378488    .2011422
        3 4  |   .2032986   .0181129    11.22   0.000     .1677981    .2387992
        4 2  |   .1893402   .0162322    11.66   0.000     .1575257    .2211547
        4 3  |    .173772   .0158526    10.96   0.000     .1427015    .2048425
        4 4  |   .2251476   .0165404    13.61   0.000     .1927289    .2575663
        5 2  |   .2362311    .033792     6.99   0.000          .17    .3024621
        5 3  |   .2287774   .0329348     6.95   0.000     .1642264    .2933284
        5 4  |   .2774569   .0334763     8.29   0.000     .2118445    .3430693
             |
       _cons |  -1.017575   .0116981   -86.99   0.000    -1.040503   -.9946476
------------------------------------------------------------------------------

Then, I determined the margins as follows:

margins col_type, dydx(EIS)

Average marginal effects                        Number of obs     =    361,615
Model VCE    : Robust

Expression   : Conditional mean of avg_prc_f, predict()
dy/dx w.r.t. : 2.EIS 3.EIS 4.EIS

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
1.EIS        |  (base outcome)
-------------+----------------------------------------------------------------
2.EIS        |
    col_type |
          0  |    .015087   .0041047     3.68   0.000      .007042     .023132
          1  |   .0942865   .0030041    31.39   0.000     .0883986    .1001744
          2  |   -.033898   .0151881    -2.23   0.026    -.0636662   -.0041299
          3  |   .0731548    .006246    11.71   0.000      .060913    .0853966
          4  |   .0718536   .0061566    11.67   0.000     .0597868    .0839204
          5  |   .0895387   .0127999     7.00   0.000     .0644513    .1146261
-------------+----------------------------------------------------------------
3.EIS        |
    col_type |
          0  |   .1258313    .003894    32.31   0.000     .1181992    .1334634
          1  |   .0971316   .0026469    36.70   0.000     .0919438    .1023194
          2  |  -.0349417    .015164    -2.30   0.021    -.0646626   -.0052207
          3  |   .0643266   .0061143    10.52   0.000     .0523428    .0763105
          4  |   .0659599   .0060106    10.97   0.000     .0541792    .0777405
          5  |   .0867339   .0124798     6.95   0.000     .0622739    .1111938
-------------+----------------------------------------------------------------
4.EIS        |
    col_type |
          0  |   .1737418   .0050014    34.74   0.000     .1639392    .1835443
          1  |   .1428853    .002906    49.17   0.000     .1371897    .1485809
          2  |  -.0447705   .0184292    -2.43   0.015    -.0808911     -.00865
          3  |   .0771346   .0068569    11.25   0.000     .0636954    .0905738
          4  |   .0853759   .0062705    13.62   0.000      .073086    .0976657
          5  |   .1049967   .0126758     8.28   0.000     .0801526    .1298409
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

How should I interpret the results of margins in this case?

For 2.EIS and col_type=0

Do the results mean that the impact of the publication when there is no collaboration increase by 0.015087 when we change from a country with a scientific system of performance equal to 1 to a country with a performance equal to 2?

For 2.EIS and col_type=1

Do the results mean that the impact of the publication when there is a domestic collaboration increase by 0.0942865 when we change from a country with a scientific system of performance equal to 1 ( and domestic collaboration) to a country with a performance equal to 2?

Thanks in advance,

Liza Vieira