Diff in Diff with categorical treatment

[Vats Prith]

Hello, I am very new here. Joined few minutes back but have been an active silent reader of the forum though. Before anything, I wish to thank all the members who have over the years, clarified doubts and suggested solutions for a variety of stata issues and made life easy for a lot us.
I will learn over the course of next few posts on how to post effectively and sorry if this first post comes across as rather crude and not up to the standards. But I'll try.

I am trying to a difference in difference model, I have an unbalanced panel from 2005q1 to 2016q4 with panel id = firm_id . The policy change went into effect from 2010q3. For the treatment threshold, all the firms that had asset_size >=9.0 after 2010q3 are affected. I want to run regressions of the form
y_it = firm-fixed-effects + quarter-fixed-effects + beta*(i.asset_size_category * post_cut_off) where asset_size_category takes the values = 9 if asset_size is between 9 & 9.999, = 10 if asset_size is between 10 & 10.999 all the way up to 14. (my dataset contains asset sizes from 1 to 15). Accordingly, I run

xtreg y post_cut_off##asset_size_category i.qdate, fe vce(cluster firm_id)


This gives me the stata output
xtreg y post_cut_off##ib10.total_asset_cat i.qdate l.y,fe vce(cluster idrssd)
note: 227.qdate omitted because of collinearity

Fixed-effects (within) regression Number of obs = 1,620
Group variable: idrssd Number of groups = 133

R-sq: Obs per group:
within = 0.4093 min = 1
between = 0.8942 avg = 12.2
overall = 0.7330 max = 47

F(57,132) = 42.08
corr(u_i, Xb) = 0.6746 Prob > F = 0.0000

(Std. Err. adjusted for 133 clusters in idrssd)
----------------------------------------------------------------------------------------------
| Robust
y| Coef. Std. Err. t P>|t| [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
1.post_cut_off | 8.065365 2.641259 3.05 0.003 2.840693 13.29004
|
total_asset_cat |
9 | 3.295907 1.317305 2.50 0.014 .6901476 5.901666
11 | 2.230045 1.528917 1.46 0.147 -.7943037 5.254394
12 | 2.263994 1.606081 1.41 0.161 -.9129932 5.440981
13 | 3.097159 2.474964 1.25 0.213 -1.798565 7.992883
14 | .4577359 1.892715 0.24 0.809 -3.286241 4.201713
|
post_cut_off#total_asset_cat |
1 9 | -3.38928 1.395232 -2.43 0.016 -6.149186 -.6293728
1 11 | -1.987856 1.803966 -1.10 0.272 -5.55628 1.580567
1 12 | -2.217762 1.854352 -1.20 0.234 -5.885853 1.45033
1 13 | -3.550415 2.505106 -1.42 0.159 -8.505762 1.404932
1 14 | -.6033758 1.953452 -0.31 0.758 -4.467496 3.260745
|
qdate |
182 | -.3856326 .6518373 -0.59 0.555 -1.675031 .903766
183 | -.2372812 .8511808 -0.28 0.781 -1.921001 1.446439
184 | 1.517389 2.030336 0.75 0.456 -2.498817 5.533595
185 | 1.238413 1.576169 0.79 0.433 -1.879404 4.35623
186 | 1.776161 1.782161 1.00 0.321 -1.749129 5.301451
187 | 3.411604 2.889696 1.18 0.240 -2.3045 9.127708
188 | 3.28036 2.011233 1.63 0.105 -.6980573 7.258778
189 | 2.460916 2.067131 1.19 0.236 -1.628073 6.549904
190 | 2.825048 2.094666 1.35 0.180 -1.318408 6.968503
191 | 4.629275 2.771521 1.67 0.097 -.8530669 10.11162
192 | 4.063019 4.087774 0.99 0.322 -4.023003 12.14904
193 | 4.2805 2.721774 1.57 0.118 -1.103438 9.664437
194 | 3.43001 2.603581 1.32 0.190 -1.72013 8.58015
195 | 9.433271 3.526246 2.68 0.008 2.458007 16.40853
196 | 16.24624 5.132718 3.17 0.002 6.093215 26.39926
197 | 8.455518 3.065737 2.76 0.007 2.391187 14.51985
198 | 9.882101 3.724985 2.65 0.009 2.513713 17.25049
199 | 12.69582 3.563849 3.56 0.001 5.646175 19.74547
200 | 1.867825 3.95118 0.47 0.637 -5.948 9.68365
201 | 8.240729 3.556003 2.32 0.022 1.206604 15.27485
202 | .8703124 1.075442 0.81 0.420 -1.257017 2.997642
203 | 1.608618 1.221838 1.32 0.190 -.8082977 4.025533
204 | 1.30814 1.097128 1.19 0.235 -.8620869 3.478367
205 | 1.554051 1.13313 1.37 0.173 -.687393 3.795495
206 | .7793318 1.057971 0.74 0.463 -1.31344 2.872104
207 | 1.147796 1.035874 1.11 0.270 -.9012662 3.196857
208 | .6734256 1.331553 0.51 0.614 -1.960518 3.307369
209 | .1621666 .868289 0.19 0.852 -1.555395 1.879728
210 | 1.378823 .9769813 1.41 0.161 -.5537424 3.311389
211 | 1.271435 .7697856 1.65 0.101 -.2512772 2.794147
212 | 2.198235 1.288721 1.71 0.090 -.3509823 4.747452
213 | 2.296725 1.226117 1.87 0.063 -.1286548 4.722105
214 | 1.794132 .7569187 2.37 0.019 .2968724 3.291392
215 | 2.532105 1.134156 2.23 0.027 .2886327 4.775578
216 | 2.486962 .9344627 2.66 0.009 .6385025 4.335422
217 | 2.107282 .8519065 2.47 0.015 .4221268 3.792437
218 | 2.604755 .9912652 2.63 0.010 .6439342 4.565575
219 | 2.258858 .6362073 3.55 0.001 1.000377 3.517339
220 | 1.659708 .8535926 1.94 0.054 -.0287828 3.348198
221 | 1.912985 1.169457 1.64 0.104 -.4003174 4.226288
222 | 1.595876 .6107158 2.61 0.010 .3878193 2.803932
223 | 2.688839 1.185928 2.27 0.025 .3429557 5.034723
224 | .5796222 1.211211 0.48 0.633 -1.816273 2.975517
225 | 1.336593 .615738 2.17 0.032 .1186025 2.554584
226 | .5550953 .4122213 1.35 0.180 -.2603192 1.37051
227 | 0 (omitted)
|
y |
L1. | .5327861 .0476364 11.18 0.000 .4385566 .6270155
|
_cons | 19.05293 2.840656 6.71 0.000 13.43383 24.67203#

Two questions related to this:
1) Is this the correct way to test treatment affect across different treatment groups ( where groups (buckets) here are defined according to the asset size)
2) how to interpret the results (given the presence of _cons and omitted qdate 227)

[Clyde Schechter]

Well, it appears that you have no observations in your data with asset size < 9; so every observation is treated (at some level) and you cannot possibly analyze treatment effects with this data. At best you can do some kind of "dose response" analysis contrasting the 9-9.99 group with the 10-10.99 group, etc. For the latter purpose, your approach looks viable.

That said, is it really reasonable to break them into categories this way? You are implying that two entities with asset sizes of 9.0 and 9.99 are, in effect, identical, but the 9.99 is radically different from one with asset size 10.0. Can you justify that? Would it not make more sense to treat asset size as a continuous variable? The least bad thing that can be said for treating inherently continuous variables as categories in this kind of analysis is that it throws away information and adds noise to the study. Not to mention possible bias and opens the door to manipulating the results depending on where the cutpoints are set. If something discontinuous truly happens as asset size crosses those boundaries, then fine. But otherwise, give serious thought to treating asset size as continuous.

The omitted qdate 227 is expected. Ordinarily, one value of qdate is omitted as the reference category, as with any categorical variable. However, you also have the post_cut_off variable which is defined as a particular subset of all those dates. Consequently it is colinear with the remaining date indicators. So either you lose one more date indicator or you lose the cutoff variable. For purposes of ease of understanding, it is better to lose the additional date variable. You are still adequately adjusting for any shocks to the outcome in that particular quarter, as the information is still collectively carried by all of the date variables and the cutoff.

I'm not sure what your question about the presence of _cons is.

In terms of interpreting the results, interaction models are much more easily understood from the output of the -margins- command than from the regression output itself. So try

margins post_cut_off##total_asset_cat
margins total_asset_cat, dydx(post_cut_off)
margins post_cut_off, dydx(total_asset_cat)

For an introduction to the -margins- command, I recommend Richard Williams' excellent Stata Journal article at http://www.stata-journal.com/sjpdf.h...iclenum=st0260, which covers the basics in an especially clear and compelling way. Then, you can go on to read the manual chapter on -margins- to learn more, including advanced features and other uses.

[Vats Prith]

Dr. Clyde much appreciate your help at this point. I could also do with diff in diff sense and you, over time have been quite a good resource. I should have been clearer. My control group is those below the 9 bn threshold. It is a part of my dataset. Control group 1 to 9 billion asset size firms. Treated group 9 to 15 billion size. Treatment is based on asset size and instead of using a dichotomous treatment indicator such as treated = greater than 9 billion I am using buckets of 1 billion over the threshold

[Clyde Schechter]

Well, I'm glad to hear that at least you have a control group. But why is there no sign of them in your regression output? The categories represented in the regression output are 9, 11, 12, 13, and 14. All of those fall in the treatment category. I'd like to think that the controls are the omitted category (which is how it's usually done), but you have designated 10, which is also a treatment group, as the reference category. So where did the controls disappear to?

I understand you are using the buckets of 1 billion over the threshold. I won't rehash the arguments I've already made in #2, but I still think using buckets is a bad idea.

[Vats Prith]

My regulation of interest passed in 2010q4. It was applicable to firms having assets sizes greater than 10 bn. In the pre and post regulation eras, the cumulative distribution of the assets changed around the threshold over which the regulation was applicable. There is evidence of bunching. The regulation was a bright line threshold around $10 billion and I am interested in the in the indirect effect of this regulation also on the firms who bunch under $10 bn. So I am saying that treatment by threshold buckets. How would you interpret this output.

xtreg  y post_cut_off#ib10.total_asset_cat i.qdate l.y,fe vce(cluster idrssd)
note: 1.post_cut_off#14.total_asset_cat omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =      1,620
Group variable: idrssd                          Number of groups  =        133

R-sq:                                           Obs per group:
     within  = 0.4093                                         min =          1
     between = 0.8942                                         avg =       12.2
     overall = 0.7330                                         max =         47

                                                F(57,132)         =      42.08
corr(u_i, Xb)  = 0.6746                         Prob > F          =     0.0000

                                               (Std. Err. adjusted for 133 clusters in idrssd)
----------------------------------------------------------------------------------------------
                             |               Robust
                             y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
post_cut_off#total_asset_cat |
                       0  9  |   3.295907   1.317305     2.50   0.014     .6901476    5.901666
                       0 11  |   2.230045   1.528917     1.46   0.147    -.7943037    5.254394
                       0 12  |   2.263994   1.606081     1.41   0.161    -.9129932    5.440981
                       0 13  |   3.097159   2.474964     1.25   0.213    -1.798565    7.992883
                       0 14  |   .4577359   1.892715     0.24   0.809    -3.286241    4.201713
                       1  9  |   .0522673    1.05133     0.05   0.960    -2.027368    2.131902
                       1 10  |   .1456399   .8772392     0.17   0.868    -1.589626    1.880906
                       1 11  |   .3878286   .8200579     0.47   0.637    -1.234327    2.009984
                       1 12  |   .1918721   .8916446     0.22   0.830    -1.571889    1.955633
                       1 13  |  -.3076158   .6672534    -0.46   0.646    -1.627509    1.012277
                       1 14  |          0  (omitted)
                             |
                       qdate |
                        182  |  -.3856326   .6518373    -0.59   0.555    -1.675031     .903766
                        183  |  -.2372812   .8511808    -0.28   0.781    -1.921001    1.446439
                        184  |   1.517389   2.030336     0.75   0.456    -2.498817    5.533595
                        185  |   1.238413   1.576169     0.79   0.433    -1.879404     4.35623
                        186  |   1.776161   1.782161     1.00   0.321    -1.749129    5.301451
                        187  |   3.411604   2.889696     1.18   0.240      -2.3045    9.127708
                        188  |    3.28036   2.011233     1.63   0.105    -.6980573    7.258778
                        189  |   2.460916   2.067131     1.19   0.236    -1.628073    6.549904
                        190  |   2.825048   2.094666     1.35   0.180    -1.318408    6.968503
                        191  |   4.629275   2.771521     1.67   0.097    -.8530669    10.11162
                        192  |   4.063019   4.087774     0.99   0.322    -4.023003    12.14904
                        193  |     4.2805   2.721774     1.57   0.118    -1.103438    9.664437
                        194  |    3.43001   2.603581     1.32   0.190     -1.72013     8.58015
                        195  |   9.433271   3.526246     2.68   0.008     2.458007    16.40853
                        196  |   16.24624   5.132718     3.17   0.002     6.093215    26.39926
                        197  |   8.455518   3.065737     2.76   0.007     2.391187    14.51985
                        198  |   9.882101   3.724985     2.65   0.009     2.513713    17.25049
                        199  |   12.69582   3.563849     3.56   0.001     5.646175    19.74547
                        200  |   1.867825    3.95118     0.47   0.637       -5.948     9.68365
                        201  |   8.240729   3.556003     2.32   0.022     1.206604    15.27485
                        202  |   8.790038   2.749551     3.20   0.002     3.351154    14.22892
                        203  |   9.528343   2.702801     3.53   0.001     4.181936    14.87475
                        204  |   9.227865   2.849732     3.24   0.002     3.590814    14.86492
                        205  |   9.473776   2.725762     3.48   0.001     4.081949     14.8656
                        206  |   8.699057   2.723673     3.19   0.002     3.311362    14.08675
                        207  |   9.067521   2.648174     3.42   0.001     3.829172    14.30587
                        208  |   8.593151   2.422908     3.55   0.001       3.8004     13.3859
                        209  |   8.081892   2.469327     3.27   0.001      3.19732    12.96646
                        210  |   9.298548   2.578075     3.61   0.000     4.198861    14.39824
                        211  |    9.19116   2.511938     3.66   0.000     4.222297    14.16002
                        212  |   10.11796   2.737544     3.70   0.000     4.702828    15.53309
                        213  |   10.21645   2.676009     3.82   0.000      4.92304    15.50986
                        214  |   9.713857   2.576013     3.77   0.000     4.618249    14.80947
                        215  |   10.45183   2.624484     3.98   0.000     5.260341    15.64332
                        216  |   10.40669   2.639608     3.94   0.000     5.185282    15.62809
                        217  |   10.02701   2.633361     3.81   0.000     4.817958    15.23606
                        218  |   10.52448   2.660106     3.96   0.000     5.262528    15.78643
                        219  |   10.17858   2.567348     3.96   0.000     5.100115    15.25705
                        220  |   9.579433   2.684978     3.57   0.001      4.26828    14.89059
                        221  |    9.83271   2.694706     3.65   0.000     4.502316     15.1631
                        222  |   9.515601   2.595945     3.67   0.000     4.380564    14.65064
                        223  |   10.60856   2.858906     3.71   0.000     4.953366    16.26376
                        224  |   8.499347   2.826269     3.01   0.003     2.908708    14.08999
                        225  |   9.256318   2.627529     3.52   0.001     4.058806    14.45383
                        226  |    8.47482   2.592395     3.27   0.001     3.346807    13.60283
                        227  |   7.919725    2.56316     3.09   0.002     2.849542    12.98991
                             |
                    y |
                         L1. |   .5327861   .0476364    11.18   0.000     .4385566    .6270155
                             |
                       _cons |   19.05293   2.840656     6.71   0.000     13.43383    24.67203
-----------------------------+----------------------------------------------------------------
                     sigma_u |  10.175375
                     sigma_e |  8.5311931
                         rho |  .58721952   (fraction of variance due to u_i)
----------------------------------------------------------------------------------------------

. 
end of do-file

Vs.

 xtreg  y088 post_cut_off##ib10.total_asset_cat i.qdate l.y088,fe vce(cluster idrssd)
note: 227.qdate omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =      1,620
Group variable: idrssd                          Number of groups  =        133

R-sq:                                           Obs per group:
     within  = 0.4093                                         min =          1
     between = 0.8942                                         avg =       12.2
     overall = 0.7330                                         max =         47

                                                F(57,132)         =      42.08
corr(u_i, Xb)  = 0.6746                         Prob > F          =     0.0000

                                               (Std. Err. adjusted for 133 clusters in idrssd)
----------------------------------------------------------------------------------------------
                             |               Robust
                    y088 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
              1.post_cut_off |   8.065365   2.641259     3.05   0.003     2.840693    13.29004
                             |
             total_asset_cat |
                          9  |   3.295907   1.317305     2.50   0.014     .6901476    5.901666
                         11  |   2.230045   1.528917     1.46   0.147    -.7943037    5.254394
                         12  |   2.263994   1.606081     1.41   0.161    -.9129932    5.440981
                         13  |   3.097159   2.474964     1.25   0.213    -1.798565    7.992883
                         14  |   .4577359   1.892715     0.24   0.809    -3.286241    4.201713
                             |
post_cut_off#total_asset_cat |
                       1  9  |   -3.38928   1.395232    -2.43   0.016    -6.149186   -.6293728
                       1 11  |  -1.987856   1.803966    -1.10   0.272     -5.55628    1.580567
                       1 12  |  -2.217762   1.854352    -1.20   0.234    -5.885853     1.45033
                       1 13  |  -3.550415   2.505106    -1.42   0.159    -8.505762    1.404932
                       1 14  |  -.6033758   1.953452    -0.31   0.758    -4.467496    3.260745
                             |
                       qdate |
                        182  |  -.3856326   .6518373    -0.59   0.555    -1.675031     .903766
                        183  |  -.2372812   .8511808    -0.28   0.781    -1.921001    1.446439
                        184  |   1.517389   2.030336     0.75   0.456    -2.498817    5.533595
                        185  |   1.238413   1.576169     0.79   0.433    -1.879404     4.35623
                        186  |   1.776161   1.782161     1.00   0.321    -1.749129    5.301451
                        187  |   3.411604   2.889696     1.18   0.240      -2.3045    9.127708
                        188  |    3.28036   2.011233     1.63   0.105    -.6980573    7.258778
                        189  |   2.460916   2.067131     1.19   0.236    -1.628073    6.549904
                        190  |   2.825048   2.094666     1.35   0.180    -1.318408    6.968503
                        191  |   4.629275   2.771521     1.67   0.097    -.8530669    10.11162
                        192  |   4.063019   4.087774     0.99   0.322    -4.023003    12.14904
                        193  |     4.2805   2.721774     1.57   0.118    -1.103438    9.664437
                        194  |    3.43001   2.603581     1.32   0.190     -1.72013     8.58015
                        195  |   9.433271   3.526246     2.68   0.008     2.458007    16.40853
                        196  |   16.24624   5.132718     3.17   0.002     6.093215    26.39926
                        197  |   8.455518   3.065737     2.76   0.007     2.391187    14.51985
                        198  |   9.882101   3.724985     2.65   0.009     2.513713    17.25049
                        199  |   12.69582   3.563849     3.56   0.001     5.646175    19.74547
                        200  |   1.867825    3.95118     0.47   0.637       -5.948     9.68365
                        201  |   8.240729   3.556003     2.32   0.022     1.206604    15.27485
                        202  |   .8703124   1.075442     0.81   0.420    -1.257017    2.997642
                        203  |   1.608618   1.221838     1.32   0.190    -.8082977    4.025533
                        204  |    1.30814   1.097128     1.19   0.235    -.8620869    3.478367
                        205  |   1.554051    1.13313     1.37   0.173     -.687393    3.795495
                        206  |   .7793318   1.057971     0.74   0.463     -1.31344    2.872104
                        207  |   1.147796   1.035874     1.11   0.270    -.9012662    3.196857
                        208  |   .6734256   1.331553     0.51   0.614    -1.960518    3.307369
                        209  |   .1621666    .868289     0.19   0.852    -1.555395    1.879728
                        210  |   1.378823   .9769813     1.41   0.161    -.5537424    3.311389
                        211  |   1.271435   .7697856     1.65   0.101    -.2512772    2.794147
                        212  |   2.198235   1.288721     1.71   0.090    -.3509823    4.747452
                        213  |   2.296725   1.226117     1.87   0.063    -.1286548    4.722105
                        214  |   1.794132   .7569187     2.37   0.019     .2968724    3.291392
                        215  |   2.532105   1.134156     2.23   0.027     .2886327    4.775578
                        216  |   2.486962   .9344627     2.66   0.009     .6385025    4.335422
                        217  |   2.107282   .8519065     2.47   0.015     .4221268    3.792437
                        218  |   2.604755   .9912652     2.63   0.010     .6439342    4.565575
                        219  |   2.258858   .6362073     3.55   0.001     1.000377    3.517339
                        220  |   1.659708   .8535926     1.94   0.054    -.0287828    3.348198
                        221  |   1.912985   1.169457     1.64   0.104    -.4003174    4.226288
                        222  |   1.595876   .6107158     2.61   0.010     .3878193    2.803932
                        223  |   2.688839   1.185928     2.27   0.025     .3429557    5.034723
                        224  |   .5796222   1.211211     0.48   0.633    -1.816273    2.975517
                        225  |   1.336593    .615738     2.17   0.032     .1186025    2.554584
                        226  |   .5550953   .4122213     1.35   0.180    -.2603192     1.37051
                        227  |          0  (omitted)
                             |
                    y088 |
                         L1. |   .5327861   .0476364    11.18   0.000     .4385566    .6270155
                             |
                       _cons |   19.05293   2.840656     6.71   0.000     13.43383    24.67203
-----------------------------+----------------------------------------------------------------
                     sigma_u |  10.175375
                     sigma_e |  8.5311931
                         rho |  .58721952   (fraction of variance due to u_i)
----------------------------------------------------------------------------------------------

. 
end of do-file

[Clyde Schechter]

Well, the first model is mis-specified. You included the post_cut_off#ib10.total_asset_cat interaction term, but neither of its component effects. So you really can't interpret that one at all (or at least not in any way that relates to treatment effects.) Was the omission of those terms deliberate or did you just mistype # when you meant ## in the -xtreg- command?

The second model corrects that problem and introduces an additional variable, lagged outcome. By the way, is y the same as y088? And I still don't see any representation of your control group in this model. So I'd be hesitant to offer any interpretation of it at all, other than to contrast the buckets among the treated.

With respect to contrasting the buckets among the treated, it is a bit difficult to see any consistent pattern. The effects on the buckets, judging from the interaction term coefficients, don't show any regularly increasing or decreasing pattern. And only in the 10 bucket is there what I would call a clear and unambiguous signal: 8.07 (95% CI 2.8 to 18.3). The others are so imprecisely estimated that even their signs are indeterminate. (Well, for bucket 9, the 95% CI is all in negative territory, but the upper limit of that is just barely negative, so I'm not really impressed with it.) And look at sigma_e: at 8.5 it's bigger than any of the regression coefficients (other than the constant term, which isn't really relevant), and, most of them by a long shot.

So my reading of this is that y088 is a very noisy measure that makes pinning down influences on it difficult, even with what looks like a reasonably large sample of data. Is there a theory that predicts that there would be a clear effect in the 10 bucket but little or nothing to see in any other group? If so, that theory would be the winner here. If not, it's hard to know what to make of an idiosyncratic finding like this.

By the way, while the effect of the transition to the post-treatment era for each bucket can be calculated by just adding the post_cutoff coefficient to each of the corresponding interaction coefficients (and to zero for the 10 bucket), it is easier to have Stata do it for you. And, actually, since the data are from an unbalanced panel, rather than just using the sum of coefficients it might be better to look at average marginal effects:

margins total_asset_cat, dydx(post_cut_off)

(You might have to add the -noestimcheck- option to get this to run, but I don't think you will.)

By the way there is something else that is peculiar about this output. Ordinarily in a fixed effects regression for a DID model, the treatment variable is colinear with the fixed effects, so you don't get any coefficients for the uninteracted treatment variable. But you got those. That tells me that some of your units are switching bundles over time. In my mind, that casts even greater doubt on the validity/usefulness of the bucket approach.

[Vats Prith]

Thanks for quick responses.

Is this what you mean by representation of the control group? Can this be interpreted?

Economics - The firms in the 9-10 asset size refused to grow in order to avoid the threshold, this should lead to some inefficiencies ( such as less loans if firms were banks). This should lead to some opertional inefficiency - the current y (or y088) It is noisy . My y variables are rudimentary right now.

I will check the correct y's to use - growth rates may be, I am just unsure If I have done the RHS correctly, if I am interested in how treatment affected all those above the threshold and those just below the threshold. If this does not make sense. I will use the continuous total variable with the post_cut_off##total_assets

. xtreg  y088 post_cut_off#i.total_asset_cat i.qdate l.y088,fe vce(cluster idrssd)
note: 1.post_cut_off#14.total_asset_cat omitted because of collinearity
 
Fixed-effects (within) regression               Number of obs     =     24,504
Group variable: idrssd                          Number of groups  =      1,055
 
R-sq:                                           Obs per group:
     within  = 0.5396                                         min =          1
     between = 0.9420                                         avg =       23.2
     overall = 0.7599                                         max =         47
 
                                                F(73,1054)        =      90.06
corr(u_i, Xb)  = 0.6095                         Prob > F          =     0.0000
 
                                             (Std. Err. adjusted for 1,055 clusters in idrssd)
----------------------------------------------------------------------------------------------
                             |               Robust
                    y088 |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
post_cut_off#total_asset_cat |
                       0  2  |  -.0643847   .4282769    -0.15   0.881     -.904757    .7759876
                       0  3  |  -.7642104   .6467229    -1.18   0.238    -2.033221    .5048004
                       0  4  |   .1880736   .9401858     0.20   0.841    -1.656775    2.032922
                       0  5  |  -1.708068   .7811179    -2.19   0.029    -3.240791   -.1753452
                       0  6  |  -1.535292   .9462828    -1.62   0.105    -3.392104    .3215206
                       0  7  |  -2.456292   1.057538    -2.32   0.020    -4.531411   -.3811738
                       0  8  |  -2.378391    1.19726    -1.99   0.047    -4.727676   -.0291064
                       0  9  |  -1.769125   1.188146    -1.49   0.137    -4.100525    .5622758
                       0 10  |  -4.333325   1.321751    -3.28   0.001    -6.926887   -1.739763
                       0 11  |  -2.402627   1.619664    -1.48   0.138     -5.58076    .7755067
                       0 12  |  -1.390223   1.591901    -0.87   0.383    -4.513879    1.733432
                       0 13  |  -1.370246   1.961933    -0.70   0.485    -5.219984    2.479492
                       0 14  |  -3.445865   1.463041    -2.36   0.019    -6.316668   -.5750615
                       1  1  |    3.92893   1.146194     3.43   0.001     1.679849    6.178011
                       1  2  |   2.680157   1.094537     2.45   0.015     .5324366    4.827877
                       1  3  |   2.123162   1.079066     1.97   0.049     .0058006    4.240524
                       1  4  |   2.339093   1.095797     2.13   0.033        .1889    4.489285
                       1  5  |   1.370578   1.032455     1.33   0.185    -.6553223    3.396479
                       1  6  |   .6414713    .968575     0.66   0.508    -1.259083    2.542026
                       1  7  |   .7625068    1.02982     0.74   0.459    -1.258224    2.783238
                       1  8  |   .7882203   .9082842     0.87   0.386    -.9940305    2.570471
                       1  9  |     .80353   .8433517     0.95   0.341    -.8513092    2.458369
                       1 10  |   .5246267   .7718767     0.68   0.497     -.989963    2.039216
                       1 11  |    .827249   .7123021     1.16   0.246    -.5704424    2.224941
                       1 12  |   .8116276   .9006639     0.90   0.368    -.9556706    2.578926
                       1 13  |   .1488912   .6487915     0.23   0.819    -1.124179    1.421961
                       1 14  |          0  (omitted)
                             |
                       qdate |
                        182  |  -.3514941   .2876252    -1.22   0.222    -.9158773     .212889
                        183  |   .1166799   .3018592     0.39   0.699    -.4756335    .7089932
                        184  |   .4596623   .4518253     1.02   0.309     -.426917    1.346242
                        185  |   -.209442   .2742618    -0.76   0.445    -.7476032    .3287192
                        186  |   -.278472   .2746018    -1.01   0.311    -.8173004    .2603564
                        187  |    .711557   .3510098     2.03   0.043     .0227994    1.400315
                        188  |   1.497191   .3964841     3.78   0.000     .7192027    2.275179
                        189  |   .6329619   .3623518     1.75   0.081    -.0780511    1.343975
                        190  |   .4541566   .3427429     1.33   0.185    -.2183794    1.126693
                        191  |   1.708043   .4065607     4.20   0.000     .9102831    2.505804
                        192  |   2.501199   .6167484     4.06   0.000     1.291005    3.711394
                        193  |   2.331838    .580423     4.02   0.000     1.192922    3.470754
                        194  |   2.523372   .4813821     5.24   0.000     1.578795    3.467948
                        195  |   7.137937   .8738144     8.17   0.000     5.423323     8.85255
                        196  |   5.389075   1.010295     5.33   0.000     3.406657    7.371492
                        197  |   6.909059   .7919506     8.72   0.000      5.35508    8.463038
                        198  |   4.960228   .6547781     7.58   0.000     3.675411    6.245045
                        199  |   6.708677   .7184335     9.34   0.000     5.298955      8.1184
                        200  |   .2339799   .7414696     0.32   0.752    -1.220945    1.688904
                        201  |   4.602149   .6002374     7.67   0.000     3.424352    5.779945
                        202  |   .5030697   1.219571     0.41   0.680    -1.889994    2.896133
                        203  |    1.74664   1.241277     1.41   0.160    -.6890148    4.182294
                        204  |   .1844356   1.279149     0.14   0.885    -2.325533    2.694404
                        205  |   .8656153   1.256444     0.69   0.491    -1.599801    3.331032
                        206  |   .0108217   1.207202     0.01   0.993     -2.35797    2.379614
                        207  |   1.351205   1.199625     1.13   0.260     -1.00272     3.70513
                        208  |  -.5830005   1.214177    -0.48   0.631     -2.96548    1.799479
                        209  |   .3541831   1.159884     0.31   0.760    -1.921761    2.630127
                        210  |   .7160715   1.156376     0.62   0.536    -1.552989    2.985132
                        211  |   1.392946   1.152114     1.21   0.227    -.8677522    3.653644
                        212  |   2.322638    1.24454     1.87   0.062    -.1194188    4.764695
                        213  |   .7054167   1.144027     0.62   0.538    -1.539412    2.950246
                        214  |   .8865723   1.151183     0.77   0.441    -1.372299    3.145443
                        215  |   1.729353   1.141374     1.52   0.130    -.5102711    3.968977
                        216  |   1.324136   1.160639     1.14   0.254    -.9532887    3.601561
                        217  |   .7307392   1.148188     0.64   0.525    -1.522255    2.983734
                        218  |   .8627631   1.125429     0.77   0.443    -1.345573    3.071099
                        219  |   1.293341   1.133208     1.14   0.254    -.9302591    3.516941
                        220  |   .4410718   1.151834     0.38   0.702    -1.819077     2.70122
                        221  |   .2605782   1.123201     0.23   0.817    -1.943385    2.464542
                        222  |   .6160928   1.115977     0.55   0.581    -1.573696    2.805881
                        223  |   .9532618   1.098185     0.87   0.386    -1.201617     3.10814
                        224  |    .481487   1.153998     0.42   0.677    -1.782907    2.745881
                        225  |    .302303   1.103285     0.27   0.784    -1.862583    2.467189
                        226  |   .3678794   1.111473     0.33   0.741    -1.813073    2.548832
                        227  |   .6880122   1.090633     0.63   0.528    -1.452047    2.828072
                             |
                    y088 |
                         L1. |   .7013456   .0146895    47.74   0.000     .6725217    .7301696
                             |
                       _cons |   16.46263   .9113322    18.06   0.000      14.6744    18.25087
-----------------------------+----------------------------------------------------------------
                     sigma_u |  5.5535803
                     sigma_e |  8.4028889
                         rho |  .30401195   (fraction of variance due to u_i)

[Vats Prith]

followed by :

margins total_asset_cat#post_cut_off, noestimcheck
marginsplot, xdimension(post_cut_off)

[Clyde Schechter]

No, you don't have the right hand side correct. It should be:

xtreg y088 post_cut_off##i.total_asset_cat i.qdate l.y088,fe vce(cluster idrssd)

The single # is not sufficient in this context. And it leads to a mis-specified model.

Remember you said:

For the treatment threshold, all the firms that had asset_size >=9.0 after 2010q3 are affected.

There is nobody in this regression who does not meet that criterion, is there? Where are the controls?

But when you say

The firms in the 9-10 asset size refused to grow in order to avoid the threshold, this should lead to some inefficiencies ( such as less loans if firms were banks). This should lead to some opertional inefficiency - the current y (or y088)

you seem to contradict that. Do you mean to say that those in the 9-9.99 range are the controls: they do not come under the registration because it's actually asset_size >= 10 that triggers the policy?

[Vats Prith]

I guess, I am terribly confused and tying myself up in knots here. appreciate your patience in advance.

If one had to run a diff in diff regression to see the effect of a policy change, I would define a byte variable post_cut_off that assumes value = 1 if the period is after 2010q3 and 0 otherwise ( the panel is from 2005q1 to 2016q4). I would have a byte variable treated that would assume a value = 1 if total_assets > 9.0 and I would run a regression and get the following output.

. xtreg  yvar post_cut_off##treated i.qdate,fe vce(cluster idrssd)
note: 227.qdate omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     25,798
Group variable: idrssd                          Number of groups  =      1,101

R-sq:                                           Obs per group:
     within  = 0.1122                                         min =          1
     between = 0.0423                                         avg =       23.4
     overall = 0.0608                                         max =         48

                                                F(49,1100)        =       9.59
corr(u_i, Xb)  = -0.0276                        Prob > F          =     0.0000

                                     (Std. Err. adjusted for 1,101 clusters in idrssd)
--------------------------------------------------------------------------------------
                     |               Robust
                yvar |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
---------------------+----------------------------------------------------------------
      1.post_cut_off |   9.417241   .8886583    10.60   0.000     7.673585     11.1609
           1.treated |  -4.272609   1.838207    -2.32   0.020    -7.879397   -.6658215
                     |
post_cut_off#treated |
                1 1  |   .3067492   1.675876     0.18   0.855    -2.981526    3.595024
                     |
               qdate |
                181  |   .4767523   .3472007     1.37   0.170    -.2044981    1.158003
                182  |   .0064599   .3717548     0.02   0.986    -.7229687    .7358886
                183  |   .2202802   .4276778     0.52   0.607    -.6188763    1.059437
                184  |   .4948353   .5462758     0.91   0.365    -.5770249    1.566696
                185  |   .1024716   .5623198     0.18   0.855    -1.000869    1.205812
                186  |  -.2864102   .5706373    -0.50   0.616    -1.406071    .8332503
                187  |   .4940948   .6063427     0.81   0.415     -.695624    1.683814
                188  |   1.859192   .6279276     2.96   0.003     .6271207    3.091263
                189  |   2.027316   .6573113     3.08   0.002     .7375904    3.317042
                190  |   2.019394   .6645901     3.04   0.002      .715387    3.323402
--more--

My diff in diff coefficient is 0.306 - and it is statistically insignificant. I do not have the control group here. My diff in diff coefficient is still 0.306, rt?

Instead of this, I wanted to see the effects across different asset size, above the threshold > i define a categorical variable as

 table total_asset_cat, contents(min total_assets max total_assets)

----------------------------------------
total_ass |
et_cat    | min(total_~s)  max(total_~s)
----------+-----------------------------
        1 |      1.000046       8.999551
        9 |      9.000186       9.998568
       10 |       10.0019       10.99883
       11 |      11.00476       11.99104
       12 |      12.00346       12.99395
       13 |      13.00074       13.98973
       14 |      14.00388       14.99417
----------------------------------------

. 
end of do-file

and I run

xtreg  yvar post_cut_off##total_asset_cat i.qdate,fe vce(cluster idrssd)

and I get output

 xtreg  yvar post_cut_off##total_asset_cat i.qdate,fe vce(cluster idrssd)
note: 227.qdate omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =      2,315
Group variable: idrssd                          Number of groups  =        168

R-sq:                                           Obs per group:
     within  = 0.1617                                         min =          1
     between = 0.0215                                         avg =       13.8
     overall = 0.0271                                         max =         48

                                                F(55,167)         =       4.98
corr(u_i, Xb)  = -0.1749                        Prob > F          =     0.0000

                                               (Std. Err. adjusted for 168 clusters in idrssd)
----------------------------------------------------------------------------------------------
                             |               Robust
                        yvar |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
              1.post_cut_off |   11.80535   4.725821     2.50   0.013     2.475301     21.1354
                             |
             total_asset_cat |
                          9  |  -1.290915   1.574733    -0.82   0.414    -4.399865    1.818034
                         10  |  -5.842063   2.492278    -2.34   0.020    -10.76249    -.921631
                         11  |  -2.570369    3.22315    -0.80   0.426    -8.933741    3.793002
                         12  |  -3.045107   2.696331    -1.13   0.260    -8.368394     2.27818
                             |
post_cut_off#total_asset_cat |
                       1  9  |   1.021009   1.964275     0.52   0.604    -2.857003     4.89902
                       1 10  |   5.198705   2.494071     2.08   0.039      .274732    10.12268
                       1 11  |    2.81625   3.796444     0.74   0.459    -4.678959    10.31146
                       1 12  |   3.837126    3.20806     1.20   0.233    -2.496453    10.17071
more

is this regression misspecified, are these not coefficients with respect to the control group - total_asset_cat = 1 ?

[Vats Prith]

 margins total_asset_cat##post_cut_off, noestimcheck

Predictive margins                              Number of obs     =      2,315
Model VCE    : Robust

Expression   : Linear prediction, predict()

----------------------------------------------------------------------------------------------
                             |            Delta-method
                             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-----------------------------+----------------------------------------------------------------
             total_asset_cat |
                          1  |   59.42765   .8003457    74.25   0.000       57.859     60.9963
                          9  |   58.70127   .7139673    82.22   0.000     57.30192    60.10062
                         10  |   56.46004   1.145474    49.29   0.000     54.21495    58.70512
                         11  |   58.41443   1.311278    44.55   0.000     55.84437    60.98449
                         12  |   58.50415   1.771596    33.02   0.000     55.03189    61.97642
                             |
                post_cut_off |
                          0  |   51.36416   2.493803    20.60   0.000     46.47639    56.25192
                          1  |   64.69117    2.03627    31.77   0.000     60.70015    68.68219
                             |
total_asset_cat#post_cut_off |
                        1 0  |   52.90029   2.184599    24.22   0.000     48.61855    57.18202
                        1 1  |   64.70564   2.698965    23.97   0.000     59.41576    69.99551
                        9 0  |   51.60937   2.596807    19.87   0.000     46.51972    56.69902
                        9 1  |   64.43573   2.110811    30.53   0.000     60.29862    68.57285
                       10 0  |   47.05822   3.564688    13.20   0.000     40.07156    54.04488
                       10 1  |   64.06228   1.968827    32.54   0.000     60.20345    67.92111
                       11 0  |   50.32992   4.044994    12.44   0.000     42.40187    58.25796
                       11 1  |   64.95152   1.838613    35.33   0.000      61.3479    68.55514

[Clyde Schechter]

Even reverting to your simple dichotomous treatment example, there is still something wrong here. You have a fixed effects regression, and yet your treatment variable has not been dropped due to colinearity. But it should be because it is supposed to be a time invariant property of each bank. (or whatever your units of analysis are). So that means that you have banks migrating in and out of the treatment group.

So first let me ask you a question about the real world: is this policy constructed in such a way that a bank might be subject to it one year and then exempt from it the next, and then subject to it again in some later year perhaps? If this kind of migration in status is actually possible, and, in particular if it is dependent on changes in the bank's assets, which, in turn may be affected by whether they were subject to the policy previously, you have a very complex web of endogeneity here and I do not think a DID model is up to the task of identifying treatment effects. And, to be honest, I'm not sure what kind of model is! Econometricians deal with complex things like this, and there are several econometricians in this forum. Perhaps one of them can speak to this question.

If the reality is that once you are subject to the policy, you remain subject to it, then your variable treat is somehow miscoded. The correct way to define it would be based on the assets in the year that the policy took effect (or, if the policy's implementation depended on some prior year's assets, then it would be based on the assets in that year instead.) So let's just say for the sake of argument that it's based on assets in 2010q3 for the sake of illustration. Then you would want:

by idrssd, sort: egen eligibility_assets = max(cond(qdate == tq(2010q3), assets, .))
gen treated = eligibility_assets > 9

This would result in identify those units whose assets exceed 9 at the 2010q3, and those would be considered the treated subset at all times. It is critical to a DID analysis that the treatment variable be a constant attribute of those units that are treated--even during the pre-treatment era.

Assuming you want to stick with your buckets, they, too, should be time-invariant and based on the assets as of the eligibility date; this should not change over time.

[Vats Prith]

I thought heterogeneity in treated is a good thing for treatment identification. The banks that have assets greater than 9 billion post 2010q3 will have to comply with the new policy. So if a bank has assets 9 billion or more after 2010q3 it will be treated. But it could happen that banks do not cross 9 bn at some point after 2010q3 even though previously they had crossed. So yes, I guess in that sense the cross section of those that are in the treated (and the control groups) at any given quarter after 2010q3 is changing across time which I thought was a good variation to isolate treatment effects.
perhaps I am wrong.
any other thoughts?

Is using post_cut_off ## c.total_assets and followed by margins a better way to go.?

[Clyde Schechter]

No other thoughts. You are jumping the gun asking about

Is using post_cut_off ## c.total_assets and followed by margins a better way to go.?

-margins- is a tool for interpreting estimation results. But if you interpret a mis-specified model you just get nonsense.

You need to get your model straightened out. There may be ways to model this situation where application of the policy to the same entity starts and stops based on the previous response of the outcome variable. That's a feedback mechanism. It's also called endogeneity. A DID model is simply too primitive to handle this. The model you are working with is not useful for this. You need a different approach. If none of the econometricians on the forum respond to this post, then I recommend you find and consult one for suggestions on how to model this data.

In terms of heterogeneity being a good thing for treatment identification, that is correct when the heterogeneity is due to exogenous influences.

[Vats Prith]

Asset size may change for various exogenous reasons.thanks for the e discussion so far. Any thoughts from an econometrician that would h3lp me corr3ctly sp3cify my model are most welcome