SDID (Synthetic Difference-In-Difference) Diagnosis

[Dimitriy V. Masterov]

I am performing a synthetic diff-in-diff analysis using Clarke and Palanir's sdid command. I have daily data on 7 treated units, 35 untreated units, with 63 pre-treatment days and 25 days post-treatment. All units are treated on the same date.

The code is this:

sdid Y ID dailydate D, method(sdid) vce(placebo) reps(1000) covariates(X1 X2 X3 X4 X5, projected) graph

This produces the following graph:

I believe that divergence in mid-May is happening because a few of the treated units are expanding in size at that time. I have tried to control for that using the covariates, but I am not clear if sdid is averaging them over the entire pre-treatment period or just using daily values. Anticipation of treatment is not plausible in my setting.

The estimated effect is

Synthetic Difference-in-Differences Estimator

-----------------------------------------------------------------------------
games_booked |     ATT     Std. Err.     t      P>|t|    [95% Conf. Interval]
-------------+---------------------------------------------------------------
           D |   1.88892    0.60899     3.10    0.002     0.69533     3.08252
-----------------------------------------------------------------------------

I have tried trimming the pre-May 15th data, which shrinks the effect and leaves it insignificant.

Synthetic Difference-in-Differences Estimator

-----------------------------------------------------------------------------
games_booked |     ATT     Std. Err.     t      P>|t|    [95% Conf. Interval]
-------------+---------------------------------------------------------------
           D |   0.66752    0.57616     1.16    0.247    -0.46172     1.79677
-----------------------------------------------------------------------------

Am I correct to be worried here? Is there any way to handle this while keeping the April and May data?

[Jared Greathouse]

Hey what's up Dimitriy V. Masterov.

SDID as far as I understand is NOT predicated on quality of preintervention fit like normal SCM is (strictly speaking) . The reason is because unlike convex hull SCM, SDID naturally adds an intercept into the weights, so that there may indeed by systemic variation between your treated and untreated unit(s). Look at, for example, the Prop 99 example using SDID. So with this said.... SDID is likely averaging them under the hood.


But, even if true, such trends are a clear cause for concern! Why would there be this massive structural break in the pre period? I would check if the donor units also experience similar structural breaks. Otherwise, even if you were to get better fit, I'm afraid that any result you DO get would be invalid because of the linear factor model which is typically the theoretical motivation for SCM designs.

[Dimitriy V. Masterov]

The change is happening because some of the treated observations are "expanding" for exogenous reasons. For example, a store might lease the space next door and expand the inventory available for sale, a restaurant might add a parklet with additional tables, or East and West Germanies reunite.

I am trying to handle that by adding controls like floor space and the number of products as controls. But that does not seem to be enough.

[George Ford]

dump the pre-may data? looks pretty clean after that.

also might think about a monthly dummy to smooth out the cycle.

I'd think the TE would be negative looking the figure.