[Neelakanda Krishna]

Dear George Ford
Treatment, as you rightly guessed is a law that got enacted in 2017 (strengthening the creditor rights). However, the outcome variable which I plotted is a book measure, leverage, defined as total debt/total assets. The treatment dummy is an artificial one that I created based on asset size classification. To reduce the transition of firms I considered total assets of firms till 2015 only, to rule any information-based changes induced by firms in the year 2016. So I classified firm-years into treatment if their assets fall in the bottom tercile and control in the top tercile based on the assets till 2016. Hence a firm that fall in the bottom tercile will be treated ones in post reform period also even if its assets got reduced drastically.

bysort co_code (year): egen mean_tang=mean( nppe_ta_w) if inrange(year,2011,2015) //nppe_ta_w is the proxy for tangibility

bysort co_code (year): egen max_mean_tang=max( mean_tang )

xtile tercile=max_mean_tang, nq(3)

gen treat=1 if tercile==3
replace treat=0 if tercile==1
univar nppe_ta_w,by(treat)

 univar nppe_ta_w,by(treat)

-> treat=0 
                                        -------------- Quantiles --------------
Variable       n     Mean     S.D.      Min      .25      Mdn      .75      Max
-------------------------------------------------------------------------------
nppe_ta_w    3300     0.08     0.07     0.00     0.02     0.06     0.11     0.76
-------------------------------------------------------------------------------

-> treat=1 
                                        -------------- Quantiles --------------
Variable       n     Mean     S.D.      Min      .25      Mdn      .75      Max
-------------------------------------------------------------------------------
nppe_ta_w    3299     0.47     0.15     0.00     0.37     0.46     0.56     0.79
-------------------------------------------------------------------------------

-> treat=. 
                                        -------------- Quantiles --------------
Variable       n     Mean     S.D.      Min      .25      Mdn      .75      Max
-------------------------------------------------------------------------------
nppe_ta_w    4557     0.25     0.14     0.00     0.17     0.24     0.32     0.79

-------------------------------------------------------------------------------

Do I make sense?

[George Ford]

Have you sniffed for outliers? That's a huge shift in 2015 unlike anything else in the sample that appears to be shifting the mean going forward. Something is amuck in the data.

[Neelakanda Krishna]

Dear George Ford
I have winsorized variables at 1 and 99. However I shall observe data more clearly and post after considerable study. Thank you for time and effort. I shall revert shortly

[George Ford]

Get histograms for each group and for each year. See if something shows up.

[Neelakanda Krishna]

Thanks for the suggestion George Ford. I shall try them

[Chinmay Korgaonkar]

Hello everyone,

I am testing for parallel trends assumption for a difference-in-differences class paper. I have only one treated and one control group (designated as industry). The treatment started at the start of the fiscal year 2016-2017 (written as 2017 in data). The pre-treatment years: 2011 to 2016. Post treatment: 2017 to 2019. It is a pooled cross sectional microdata of firms in two industries.

(1) Following Clyde Schechter , I ran the following OLS

 reg log(outcome_variable) c.year##i.industry if  year< 2017, robust

The interaction term coefficient is not statistically significant, suggesting that trends were parallel.

(2) I used the standard code following from stata manual

 gen time = 0
replace time = 1 if year >= 2017
gen treated = 0
replace treated = 1 if industry == 'treated group'
gen did = time * treated
didregress (log(outcome_variable) log(control_var1) log(control_var2)) (did), group(industry) time(year)
estat ptrends

However, estat results suggest that parallel trends assumption does not hold.

I am not sure which one to choose. I am attaching the two way graph for the variable.

Thank you!

[Clyde Schechter]

This, in part, falls into the general category of "I ran two different statistical tests and one came out significant and the other did not." This only appears to be a problem because of the inappropriate, but widespread, tendency to think of statistical significance as a logical construct denoting "presence" or "absence" of a difference. But it does not. It is an arbitrary imposition of a threshold on an inherently continuous statistic, the p-value. And as in any other situation where one imposes an arbitrary threshold on an inherently continuous statistic, you discard information, add noise, and make a mess out of things. On top of that, the p-value itself is not a measure of the size of the difference. It is a mashup of that with outcome measure variability and sample size.

Your graph shows trends that, to my eye, also look parallel. But apart from the graph, you don't provide a lot of information to guide interpretation. It would be very helpful to see the actual interaction coefficients in both of those tests. They may be the same--with the difference in p-value arising from -didregress- using clustered standard errors, whereas your -regress- command used non-clustered -robust- errors. Or they may be slightly different if there are differences in sample selection for the two approaches. Sample size for these tests is also relevant. If your sample is very large, differences that are too small to be of any practical importance (and also too small to detect by eye) may be designated as statistically significant.

To resolve the apparent conflict, I suggest a pragmatic approach. After all, the parallel trends assumption is, in reality, almost never exactly met. It would be an extraordinary coincidence if the rate of change in the outcome over time in two different groups were exactly equal. Even in a randomized study, there will be some sampling error. And in observational data, a complete absence of confounding by measured or unmeasured factors is almost never achieved. So a null-hypothesis significance test of the parallel trends assumption really makes no sense--we know a priori that the null hypothesis is false, so a test relying on a statistic that is conditioned on it can't really be very helpful. Rather, I would focus on the magnitude of the pre-intervention difference in trend in the treatment groups and assess whether it is large enough to be important in the real world. Yes this is a subjective judgment and requires expertise in the subject matter. It often corresponds to the visual assessment of the graph, but not invariably so.

[Chinmay Korgaonkar]

Dear Dr. Clyde Schechter , Thank you very much for your detailed insights. Those are very helpful.
I am sharing the output of both tests for your reference. I will include such results in future posts.
Thank you once again.

*ols

    
 . regress sales_log c.year##i.sector [pweight=mult] if  year< 2017, robust
(sum of wgt is 49,147.85246381)

Linear regression                    Number of obs    =     16,436
                                                F(3, 16432)       =      59.23
                                                Prob > F           =     0.0000
                                                R-squared         =     0.0234
                                                Root MSE          =     1.4645

-------------------------------------------------------------------------------
              |               Robust
sales_log | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
--------------+----------------------------------------------------------------
         year |   .0116959   .0103872     1.13   0.260    -.0086641    .0320559
    15.industry |   12.56344    36.6089     0.34   0.731    -59.19398    84.32085
              |
industry#c.year |
          15  |   -.006477   .0181834    -0.36   0.722    -.0421184    .0291645
              |
        _cons |  -19.77848   20.91297    -0.95   0.344    -60.77018    21.21322
-------------------------------------------------------------------------------

. margins sector, dydx(year)

Average marginal effects                                Number of obs = 16,436
Model VCE: Robust

Expression: Linear prediction, predict()
dy/dx wrt:  year

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   std. err.      t    P>|t|     [95% conf. interval]
-------------+----------------------------------------------------------------
year         |
  industry |
         14  |   .0116959   .0103872     1.13   0.260    -.0086641    .0320559
         15  |          .  (not estimable)
------------------------------------------------------------------------------

* didregress

. didregress (sales_log export_earnings_log domestic_demand_log) (did) ///
>     [pweight=mult], group(industry) time(year)
(sum of wgt is 66,419.35949187)

Number of groups and treatment time

Time variable: year
Control:       did = 0
Treatment:     did = 1
-----------------------------------
             |   Control  Treatment
-------------+---------------------
Group        |
    industry |         1          1
-------------+---------------------
Time         |
     Minimum |      2011       2017
     Maximum |      2011       2017
-----------------------------------

Difference-in-differences regression                    Number of obs = 23,324
Data type: Repeated cross-sectional

                                                 (Std. err. adjusted for 2 clusters in industry)
----------------------------------------------------------------------------------------------
                             |               Robust
           sales_log | Coefficient  std. err.      t    P>|t|     [95% conf. interval]
-----------------------------+----------------------------------------------------------------
ATET                         |
                         did |
                   (1 vs 0)  |  -.0309762   .0002102  -147.34   0.004    -.0336474   -.0283049
----------------------------------------------------------------------------------------------
Note: ATET estimate adjusted for covariates, group effects, and time effects.


. estat ptrends

Parallel-trends test (pretreatment time period)
H0: Linear trends are parallel

 F(1, 1) = 36183.37
Prob > F =   0.0033