Difference-in-Difference Interpretation and Understanding

[Liz Bree]

Hi,

I am a beginner in statistics and stata (14.0) and try to do and understand the following analysis:

I am conducting an difference-in-difference analysis with panel data. The Treatment Group consists of firms with companies who have a male CEO in the pre-treatment period and a female CEO in the post period. The control group consists of firms with male CEO in the pre period, a change from male to male at the time of treatment and thus also male CEO in the post period.
Consequently my treatment is the change in CEO from male to female.
Because treatment takes place in different years, I included the variable which I called "timetrend" which has a value of 0 at the time of treatment, negative integers for pre period and positive integers for post-treatment period.

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte number str48 companyname str14 country int(nacerev2code year) byte(timetrend period group) float(leverage changeinleverage tang ptob roe size stage _diff)
1 "Deutsche Post AG"      "Germany"     5320 2009  -5 0 1  .6989025  -.27010855  .21682996          2       8.05 17.348087   2.70805 0
1 "Deutsche Post AG"      "Germany"     5320 2010  -4 0 1   .650414  -.04848848  .23284854       1.45       27.2 17.424345  2.772589 0
1 "Deutsche Post AG"      "Germany"     5320 2011  -3 0 1  .6447105 -.005703561  .16395503       1.31      10.81 17.433296  2.833213 0
1 "Deutsche Post AG"      "Germany"     5320 2012  -2 0 1  .5877402  -.05697034  .19163695       1.68      14.44 17.307888  2.890372 0
1 "Deutsche Post AG"      "Germany"     5320 2013  -1 0 1  .6522491    .0645089  .19015887       3.26      22.16 17.346302  2.944439 0
1 "Deutsche Post AG"      "Germany"     5320 2014   0 0 1  .6609532  .008704134   .1970871       3.51      21.55 17.377323  2.995732 0
1 "Deutsche Post AG"      "Germany"     5320 2015   1 1 1   .623811 -.037142225   .2007711       2.85      15.09 17.395216 3.0445225 1
1 "Deutsche Post AG"      "Germany"     5320 2016   2 1 1    .63437  .010558965   .2030419        3.4      23.86 17.401886 3.0910425 1
1 "Deutsche Post AG"      "Germany"     5320 2017   3 1 1  .6059434 -.028426517  .20905674       3.85      22.87  17.41008  3.135494 1
2 "Deutsche Lufthansa AG" "Germany"     5110 2009  -3 0 1   .709519   .07796273    .374656        .88      -1.77 17.087244 4.4308167 0
2 "Deutsche Lufthansa AG" "Germany"     5110 2010  -2 0 1  .6607533   -.0487658   .3668962        .91      15.78 17.190979 4.4426513 0
2 "Deutsche Lufthansa AG" "Germany"     5110 2011  -1 0 1  .6623195  .001566215   .3773061        .53       -.16 17.149427  4.454347 0
2 "Deutsche Lufthansa AG" "Germany"     5110 2012   0 0 1  .6559472 -.006372246   .3749587         .8      12.23 17.161018  4.465908 0
2 "Deutsche Lufthansa AG" "Germany"     5110 2013   1 1 1  .7225327   .06658555   .3768719       1.18       5.78  17.16408  4.477337 1
2 "Deutsche Lufthansa AG" "Germany"     5110 2014   2 1 1  .8045897   .08205701   .3838485       1.62        1.1  17.18229 4.4886365 1
2 "Deutsche Lufthansa AG" "Germany"     5110 2015   3 1 1  .7517645  -.05282527   .3813303       1.17      34.88 17.257914 4.4998097 1
2 "Deutsche Lufthansa AG" "Germany"     5110 2016   4 1 1  .7043551  -.04740939   .3722724        .82      27.69 17.320587 4.5108595 1
2 "Deutsche Lufthansa AG" "Germany"     5110 2017   5 1 1  .6642029  -.04015225   .3666326       1.53      28.56 17.363518 4.5217886 1
3 "Danone"                "France"      1051 2009  -6 0 1  .4712593   -.1826623   .1507505       1.98      12.43 17.083252 4.7095304 0
3 "Danone"                "France"      1051 2010  -5 0 1 .54233587   .07107653   .1674351       2.43      16.63 17.127804 4.7184987 0
3 "Danone"                "France"      1051 2011  -4 0 1 .46205485    -.080281   .1666166       2.41      14.02 17.138206  4.727388 0
3 "Danone"                "France"      1051 2012  -3 0 1  .5159612   .05390639  .16392255       2.43      13.77 17.177378 4.7361984 0
3 "Danone"                "France"      1051 2013  -2 0 1  .6163863   .10042502  .16706175       2.87      12.43 17.223982  4.744932 0
3 "Danone"                "France"      1051 2014  -1 0 1 .59046185 -.025924416  .18953854       2.79         10 17.246622   4.75359 0
3 "Danone"                "France"      1051 2015   0 0 1  .5747394 -.015722452  .20161635       3.04      10.55 17.275291  4.762174 0
3 "Danone"                "France"      1051 2016   1 1 1  .6770943   .10235497  .15982074       2.83      13.38  17.57945  4.770685 1
3 "Danone"                "France"      1051 2017   2 1 1  .6414282  -.03566611  .17063136       3.05      17.77 17.589329  4.779123 1
4 "Heineken NV"           "Netherlands" 1105 2009  -6 0 1  .6967413 -.063050255  .28996417       3.04      20.73  16.79201  4.919981 0
4 "Heineken NV"           "Netherlands" 1105 2010  -5 0 1 .56629777  -.13044353  .28600717       2.06      18.44 17.078213  4.927254 0
4 "Heineken NV"           "Netherlands" 1105 2011  -4 0 1  .5915246  .025226817  .28746724        2.1      14.51 17.098412  4.934474 0
4 "Heineken NV"           "Netherlands" 1105 2012  -3 0 1  .6203728  .028848203  .25057137       2.48      27.48 17.382647  4.941642 0
4 "Heineken NV"           "Netherlands" 1105 2013  -2 0 1  .6095473 -.010825484  .25266346       2.48      11.79 17.306824   4.94876 0
4 "Heineken NV"           "Netherlands" 1105 2014  -1 0 1  .5922384 -.017308857  .25161532       2.73      12.73 17.346828  4.955827 0
4 "Heineken NV"           "Netherlands" 1105 2015   0 0 1 .58579427 -.006444144  .25039768       3.32      14.59 17.419811  4.962845 0
4 "Heineken NV"           "Netherlands" 1105 2016   1 1 1  .6171665    .0313722   .2434834       3.07       11.5 17.461222  4.969813 1
4 "Heineken NV"           "Netherlands" 1105 2017   2 1 1   .629406   .01223955  .25420904       3.72      14.57 17.511019  4.976734 1
5 "Air Liquide"           "France"      2011 2009  -6 0 1 .56413263  -.04685115   .3413025       2.88      17.15 16.824924 4.6821313 0
5 "Air Liquide"           "France"      2011 2010  -5 0 1 .53005415  -.03407847   .3448009       3.01      17.03 16.917011  4.691348 0
5 "Air Liquide"           "France"      2011 2011  -4 0 1  .5203441  -.00971009   .3488402       2.77      16.45 16.986586 4.7004805 0
5 "Air Liquide"           "France"      2011 2012  -3 0 1 .51694435 -.003399723   .3539053        2.9      16.12 17.019821 4.7095304 0
5 "Air Liquide"           "France"      2011 2013  -2 0 1 .50290537 -.014038997   .3606485       3.02      15.76 17.026068 4.7184987 0
5 "Air Liquide"           "France"      2011 2014  -1 0 1  .4933977 -.009507664   .3642355       3.07      15.03 17.091877  4.727388 0
5 "Air Liquide"           "France"      2011 2015   0 0 1  .4970567  .003659021   .3636908       2.88      14.67 17.172625 4.7361984 0
5 "Air Liquide"           "France"      2011 2016   1 1 1  .5603392   .06328249   .3222896       2.45      12.65 17.598242  4.744932 1
5 "Air Liquide"           "France"      2011 2017   2 1 1   .546536 -.013803134   .3215936       2.75      13.31  17.52343   4.75359 1
end

This is an extract of my dataset.

So I did the difference in difference analysis with "leverage" as my dependent variable. Leverage is determined by (debt of the firm/(debt of the firm + equity of the firm)). I chose to take the following years into account: two years before treatment (that is timetrend=-2) ans two years after treatment (that is timetrend=2).
The code and the output look like this:

diff lev if timetrend==2 | timetrend==-2, t(group) p(period) cov(tang ptob size roe stage)
DIFFERENCE-IN-DIFFERENCES WITH COVARIATES

DIFFERENCE-IN-DIFFERENCES ESTIMATION RESULTS
Number of observations in the DIFF-IN-DIFF: 172
            Before         After    
   Control: 43             43          86
   Treated: 43             43          86
            86             86
--------------------------------------------------------
 Outcome var.   | lever~e | S. Err. |   |t|   |  P>|t|
----------------+---------+---------+---------+---------
Before          |         |         |         |
   Control      | 0.272   |         |         |
   Treated      | 0.254   |         |         |
   Diff (T-C)   | -0.018  | 0.048   | -0.38   | 0.708
After           |         |         |         |
   Control      | 0.252   |         |         |
   Treated      | 0.190   |         |         |
   Diff (T-C)   | -0.062  | 0.047   | 1.30    | 0.194
                |         |         |         |
Diff-in-Diff    | -0.044  | 0.067   | 0.65    | 0.516
--------------------------------------------------------
R-square:    0.10
* Means and Standard Errors are estimated by linear regression
**Inference: *** p<0.01; ** p<0.05; * p<0.1

I am unsure about following aspects:
- am I right to say that the change in CEO from male to female results in -4.4% lower leverage than for comparable companies with a male CEO? Is the value in percentage points?
- what does the standard error of the dff-in-diff value say?
- what other tests would you recommend to do after conducting the diff-in-diff to verify the result? I am very confused how I should move on....

I would appreciate any help!!

Best wishes,

Liz

[Gonzalo Etchart]

[Clyde Schechter]

- am I right to say that the change in CEO from male to female results in -4.4% lower leverage than for comparable companies with a male CEO? Is the value in percentage points?

You have defined the variable leverage as debt of the firm/(debt of the firm + equity of the firm), which is a proportion (ranges from 0 to 1) not a percent (which would range from 0 to 100). So you conclusion would be that this proportion goes down by 0.044 after switching to a female CEO. Since your outcome variable is not calculated in percent units, it would be confusing to refer to a change in it as percentage points. It would be like doing an analysis where the outcome variable is measured in US dollars and then referring to an effect on it denominated in Yen. It wouldn't, strictly speaking, be wrong. But it could confuse your audience. Better to keep the units consistent throughout.

If you want to work in the percentage metric, you can: redefine your leverage variable to be 100 times its current value. Then the DID result will come out as -4.4, and you would then be correct in referring to that as a decrease of 4.4 percentage points.

The standard error is just like the standard error of any other regression coefficient. It is an estimate of the standard deviation of the sampling distribution of the coefficient. So, in your particular context, it says that if you were to draw repeated random samples from the population that gave rise to this sample and conduct your analysis in each sample, the estimated difference-in-difference estimator would vary from one sample to another, and the standard deviation of the distribution of those DID estimators would be around 0.067.

- what other tests would you recommend to do after conducting the diff-in-diff to verify the result?

Well, the first thing that strikes me is that you made the somewhat unusual choice of analyzing only the periods 2 years before and 2 years after the conversion. Is there a science-based reason for that choice? It is more common to include all time periods in the analysis. If there is not a good science-based reason for looking only at 2 years before and after, then the first additional analyses I would do is to look at what happens using all time periods, and using selected intervals other than 2 years.

You will also want to examine the parallel trends assumption, that the time-course of the leverage outcome is similar in the treatment and control groups before the intervention. So you would calculate the average leverage in the treatment and control group at each negative value of your timetrend variable. Then you would graph them and examine whether the plots are a) parallel and b) reasonably close together.

Given that your DID estimate came out so small, I don't think there would be much point in doing the other kinds of tests that are often done after a DID analysis.

[amalie larsen]

Hi,

I just came across this. I found it interesting to see the parallel assumptions graph. I did this regression for my RWA, Z score, Provisions. And I have Time (0 before 2015)(1 after 2015) I years 2013-2021. And Treated (1 countries with this policy) (0 countries without this policy). DID is my interaction term Time*Treated.

Here are my results. I wanted to ask the same as Liz, are my DID decreasing 18,765% in RWA?and increasing in Log Z score by 4,4 %? (Yes im aware the result are missing in Provisions) And that it is only my log Z score that are statistically significant and so I can conclude that the Z score is increasing in countries with this policy implementation?

	(1)	(2)	(3)
VARIABLES	RWA	Provisions	LogZscore
Time	13.189	100.787***	-0.060***
	(10.917)	(20.665)	(0.013)
Treated	-1.113	-36.977	-0.319***
	(7.996)	(24.094)	(0.015)
DID	-18.765		0.044***
	(12.948)	(25.721)	(0.016)
Capitalization	5.097*	2.659***	0.022***
	(2.702)	(0.510)	(0.001)
Deposits	-0.000*	0.004***	-0.000***
	(0.000)	(0.000)	(0.000)
logGDP	-6.369	-136.451***	0.007
	(4.174)	(7.606)	(0.005)
Inflation	0.010	0.290***	0.000***
	(0.019)	(0.031)	(0.000)
HHI	-61.884*	-130.871***	0.578***
	(37.403)	(36.019)	(0.036)
Liquidity	-0.651***	-5.701***	-0.001***
	(0.248)	(0.353)	(0.000)
Size	0.643	0.909	0.022***
	(1.132)	(6.351)	(0.002)
Constant	61.668***	871.551***	0.419***
	(8.538)	(68.221)	(0.041)
Observations	25,068	25,068	25,068
R-squared	0.003	0.580	0.185

Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1

Kind regards
Amalie

[Clyde Schechter]

The DID estimate of the effect of treatment on logZscore is 0.044, which corresponds, approximately, to an effect of increasing Z by 4.4%. It is not possible to conclude from this that Z is increasing over time (nor that it isn't). It is only possible to conclude that, on average, Z is approximately 4.4% higher in treated than untreated countries, all else equal. How it is evolving over time is not estimated in this model.

Concerning RWA, unless it is the logarithm of some other underlying variable of interest, there is no basis for inferring any kind of percentage change from this model. Instead, RWA is, on average, 18.8 units (whatever the units of RWA itself are) lower in the presence of treatment. If RWA is itself measured in %, then one would say that the treatment effect is a decrease of 18.8 percentage points (not percent).