[Johanne Friedmann]

Dear Fernando,

Thank you very much for your responses; I really appreciate them!

I have tried around a lot, and unfortunately, my issue still occurs, and the results change every time I reload the data and run the code again.

In regard to your recommendations:

1) Stata does still tell me, "Program DRDID is outdated, or not installed," when running csdid.

When running "which drdid," stata tells me it's version 1.6. Additionally, when running "ssc install drdid, all replace," it tells me that the file already exists and is up to date. All web pages I checked suggest that 1.6 is the most recent version; however, I am not 100% certain if this is actually up-to date information. When using "which csdid" Stata tells me it's version 1.72. Is there anything else that I should check or have forgotten?

Moreover, even if we assume that my drdid is not up-to-date and the error "Program DRDID is outdated, or not installed. " is true, I do not understand how I get the same results every time I run the example code from your webpage and receive the results as you displayed there.

2) I assumed that I could use ivar() as I have panel data, which I now think was a bit naive as I could imagine that due to some missings, it's not understood as (at least balanced) panel data. However, when removing all entries that are missing and making sure the panel is balanced the message: "Panel is not balanced Will use observations with Pair balanced (observed at t0 and t1)..." is gone, but the issue of different results persists.

I really hope my post is understandable, and I want to thank you very much again!

Please let me add: I tested the code on a different computer with Stata 16. Unlike the previous attempt, I did not receive any warnings about the drdid command being outdated. The versions of drdid and csdid are 1.71 and 1.72, respectively. However, the results still differ. Your example code works correctly, just as it did before.

[Tiziana Giuliani]

Hello FernandoRios
thank you for the interesting discussion throughout this post. I am trying to apply the csdid methodology and commands to evaluate the impact on export of a Programme supporting MSMEs internationalization in Italy. I have an unbalanced set of panel data extending over 2014-2020 and treatment taking place in 2018 and 2019. To make it more coherent and homogeneous the trend before the treatment time, first I ran a psmatch2 command and got weights then I plugged in in the csdid command as weights:

csdid lvalesp ( log10dipendenti lfatturato log10vaggperaddetto lvalimp ) [weight=_weight], cluster(codimp) time(anno_2) gvar(anno_primo_trattamento2) method(dripw)

However, STATA returns me this message:

syntax error
In parsing newname,=nwhat follows the dash is invalid. The dash syntax is varname-varname, and the variable names cannot have
wildcard characters in them.
r(198);

Do you have any clue of why this happens?


. tab anno_2 anno_primo_trattamento2

| anno_primo_trattamento2
anno_2 | 0 1 2 | Total
-----------+---------------------------------+----------
-3 | 25,206 687 199 | 26,092
-2 | 26,021 746 213 | 26,980
-1 | 26,943 821 233 | 27,997
0 | 15,132 724 209 | 16,065
1 | 14,861 749 212 | 15,822
2 | 15,401 745 230 | 16,376
3 | 13,408 707 215 | 14,330
-----------+---------------------------------+----------
Total | 136,972 5,179 1,511 | 143,662

anno_2 is my time var, whereas my gvar is anno_primo_trattamento_2

nothing seems to be wrong here. I have also tried to replace the pre-treatment periods (-3, -2, -1) by dropping them from the data and leaving only t=0 but still this did not solve the problem

also weights seem ok


psmatch2: |
weight of |
matched |
controls | Freq. Percent Cum.
------------+-----------------------------------
.25 | 11,808 67.16 67.16
.5 | 1,585 9.01 76.17
.75 | 209 1.19 77.36
1 | 3,977 22.62 99.98
1.25 | 3 0.02 100.00
------------+-----------------------------------
Total | 17,582 100.00
Thank you very mucjh in advance for your kind support,

Tiziana

[Michael Bart]

Hi all,

This has been a very useful discussion already and I have learned a lot. I now have a very specific question and would appreciate your opinion:

I work with country level data where a policy is implemented in a staggered manner over the period between 2013 and 2024. My outcome variable is the usage of mobile phones in these countries. I am interested in estimating the effect of introducing the policy on mobile phone usage. The "problem" is that in many countries, mobile phone usage has been exploding during the time studied. Hence, my idea is to estimate (using csdid) a staggered diff-in-diff using the logarithm of the number of mobile phones as an outcome variable:

csdid log_phones, ivar(country_code) time(year) gvar(tax_intro) method (dripw)  notyet wboot

The event plot look qualitatively very nice and I see a significant reduction in the logarithm of the number of mobile phones. What confuses me, though, is the magnitude of the effects. For instance, the overall ATT is -0.65:

------------------------------------------------------------------------------
             | Coefficient  Std. err.      z    P>|z|     [95% conf. interval]
-------------+----------------------------------------------------------------
         ATT |  -.6544154   .2679143    -2.44   0.015    -1.179518   -.1293131
------------------------------------------------------------------------------

and if looking at the event plot, some of the post treatment coefficients go as low as -2. With my rusty grad school math, my interpretation of this is that the policy change induces a 65% reduction in the number of mobile phones and the peak event plots response is a 200% reduction (given the coefficient of -2).

My question is: is this the correct interpretation of the effect? Or is it rather a 6.5% or 0.65% reduction (and -20% or -2% for the peak event plot)? -65% is not very realistic and, even worse, I don't know how to interpret -200% so I am a bit stuck with this number.

Thanks a lot in advance for your help.

Michael

[FernandoRios]

Tiziana Giuliani I think the problem is because you are using "()" drop that and try again.
Michael Bart The interpretation is exactly as you say, a 65% (or exp(-.65)-1) reduction in Log_Phones. Not sure what its the "-2" coming from.
Now, the results do seem odd. Try other methods, perhaps dripw is too sensitive to this case.
F

[Tiziana Giuliani]

FernandoRios I tried taking out "()" but it gives me the same error message as before:

csdid lvalesp log10dipendenti lfatturato log10vaggperaddetto lvalimp [weight=_weight], cluster(codimp) time(anno_2) gvar(anno_primo _trattamento2) method(dripw)
(importance weights assumed)
(importance weights assumed)
.syntax error
In parsing newname,=nwhat follows the dash is invalid. The dash syntax is varname varname, and the variable names cannot have
wildcard characters in them.
r(198);

[FernandoRios]

Change names to something smaller
and make sure your time variable is measure in years

[Michael Bart]

FernandoRios Thanks a lot!
I think the very large ATTs come from the fact that the data is very dispersed, even in logs. The pre-treatment SD is around 2 and so an ATT of 2 is not even that large if measured in standard deviations. Probably it makes sense to display treatment effects in terms of standard deviations in that case?

[Mike Br]

Hello.
I have already received some good answers in this thread, so thank you for doing this important work.

I have one further query about the use of "csdid". I have ran by now quite many staggered event study analyses using this package, and I observe that generally the package does not normalize the "pre-treatment" period (0 or -1, whatever is the convention) effect to 0. The point estimates of effects in most of my analyses in period 0 are usually quite close to 0, but never actually 0. Does anyone know why this is the case?

thanks in advance again!

[FernandoRios]

FernandoRios Thanks a lot!
I think the very large ATTs come from the fact that the data is very dispersed, even in logs. The pre-treatment SD is around 2 and so an ATT of 2 is not even that large if measured in standard deviations. Probably it makes sense to display treatment effects in terms of standard deviations in that case?

Can say it for sure. To me it suggests to explore the data further, and analyze whythere is so much dispersion. Before even try to apply any other methodology

[FernandoRios]

Hello.
I have already received some good answers in this thread, so thank you for doing this important work.

I have one further query about the use of "csdid". I have ran by now quite many staggered event study analyses using this package, and I observe that generally the package does not normalize the "pre-treatment" period (0 or -1, whatever is the convention) effect to 0. The point estimates of effects in most of my analyses in period 0 are usually quite close to 0, but never actually 0. Does anyone know why this is the case?

thanks in advance again!

The normalization only happens at period T-1 when you request "long2" estimates.
Otherwise, the event effect will provide you short Gaps for pretreatment ATTS.

The fact that effects are close to zero at T=0 simply means that the treatment takes a bit longer to take effect, and there is no instant impact on the outcome

[Michael Bart]

Dear all,

I have another quick question regarding the CSDID command. I am studying the introduction of a certain tax across a number of countries. My data is a repeated cross section with multiple individuals in each country and for 4 different years (4 waves of a survey). I also know when the tax was introduced in each country. The crosstable of time (t) and gvar looks as follows:

tab t gvar

           |                    gvar
         t |         3          9         10         11 |     Total
-----------+--------------------------------------------+----------
         1 |     2,997        998      1,000      1,000 |     5,995 
         4 |     3,003        993      1,000        999 |     5,995 
         7 |     2,997        987        993        985 |     5,962 
        11 |     2,995        987      1,000        999 |     5,981 
-----------+--------------------------------------------+----------
     Total |    11,992      3,965      3,993      3,983 |    23,933

As you can see, most of the observations come from years in which the tax was not introduced. Whenever I run the following csdid command, everything is ommitted and I do not obtain any results:

csdid y $controls , time(t) gvar(gvar) cluster(country_code) method(reg) notyet

Intuitively, I feel like it should still be possible to estimate the treatment effects but stata tries to do comparisons such as t_1_7.666667, which is of course not possible because there is no time period 7.6666.

Do you know what the right approach is in this case?

Thanks a lot in advance.
Michael

[FernandoRios]

Hi Michael
two things. First, please update csdid to the latest iteration.
second, Im surprised that it even tried anything like that. For CSDID to work you need the treatment years and treatment periods to overlap. If they don't, then CSDID cannot correctly identified pre-post periods, nor which groups to analyze.
The only thing i can think of is recoded your data so that Gvar 3=4m and gvar 910 and 11 is equal to 11. But this will change how to interpret the results.
Might even make more sense to treat each case as separate, not using csdid, but relying in TWFE using 2 x 2 design.

[Michael Bart]

Hi Fernando,

Thanks for the quick reply! Will update csdid right away.

I see. Interesting that this is not possible using the csdid command. In my intuition there are a lot of ATT(g,t) that could be computed even if gvar and t do not perfectly overlap. For instance, the difference in the difference between t = 1 and t=4 could be computed using the group that was treated in 3 and the ones that were treated later. This should give us an estimate of ATT(g,t), or not?

Michael

[Mathieu Simoens]

Dear FernandoRios,

I have two questions regarding the use of clustering in csdid.
I am estimating a staggered DiD, investigating the impact of a staggered series of events on firm profitability, and I am running the following code:
csdid firm_profit, ivar(firm_id) time(year_id) gvar(year_treatment) long2

My two questions are the following:

1) I would like to cluster at state level (instead of the automatic clustering at firm level), because the intervention/treatment I am investigating happened at state level. If I understand the help file correctly, adding the option "cluster(state_id)" will result in double-clustering, right? However, because my firm_id variable is nested within my state_id variable (every firm is located in only one state), does this actually correspond with the simple clustering at state level that I require?

2) The help file describes the default "robust and asymptotic" standard errors as well as the option to compute "wild bootstrap" standard errors (and both are possible with clustering). Could you provide some guidance in why one might choose one option or the other?

Apologies if you already answered these questions at a certain point, but I could not find the answers on the forum.
Thank you in advance!

Mathieu

[FernandoRios]

Hi Fernando,

Thanks for the quick reply! Will update csdid right away.

I see. Interesting that this is not possible using the csdid command. In my intuition there are a lot of ATT(g,t) that could be computed even if gvar and t do not perfectly overlap. For instance, the difference in the difference between t = 1 and t=4 could be computed using the group that was treated in 3 and the ones that were treated later. This should give us an estimate of ATT(g,t), or not?

Michael

Correct. That is why i was suggesting doing each 2x2 manually. With CSDID, while facilitating many of the steps, it imposes the constrain of strict structure in your data. In absence of that structure CSDID cannot be used