SCUL--command logic check/help with graphs

[Collin James]

Hi all,

I have the following code below. I am trying to see how the number of caseworkers who quit an office changes due to the implementation of a policy that was implemented in one of five offices in my county. Treated=1 if we are post the time period of implementation, and in that office. Count_* is a list of variables that counts the numbers of referrals, types of cases, etc that the offices see (basically, I want the other four offices to be combined into a control group that looks similar to the treatment office). Is this logic correct? SCUL produces two graphs it appears--the first one plots the real and synthetic office over time in number of quits, but this one goes away quickly as the next graph pops up. How can I save both graphs to look at? Thanks!

scul num_quits count_*, treated(treated) 
graph save "real_x"

[Andrew Musau]

scul is from SSC, as you are asked to explain (FAQ Advice #12). If you have a Windows system:

set autotabgraphs on, permanently

Thereafter, multiple graphs are created as tabs within one window. See

help autotabgraphs

[Collin James]

Thank you!

Could anyone additionally help me think about the interpretation of this event study? I have not used SCUL plotting commands before. It seems to me an increase in number of quits post event compared to control group?

[Andrew Musau]

You are graphing the effect size over time. In your case, the average effect size is .4316. The mean appears to be higher for the treated units in the post period, but you have to check the estimates to determine whether this is significant. The paper accompanying the command is at https://papers.ssrn.com/sol3/papers....act_id=4196189, but Jared Greathouse may have more to say.

[Collin James]

Thanks--where can I see the estimates at? I tried ereturn list but couldn't find them. Trying to figure this out. Ty!

[Andrew Musau]

The documentation states that the confidence bounds corresponding to the ATT estimate are among the stored results.

ereturn list

Scalars
e(ATT) Average Treatment Effect

e(MSE) Pre-Intervention Error

e(PMSE) Post-Intervention Error

e(LB) Lower Bound of ATT

e(UB) Upper Bound of ATT

e(ratio) Post error divided by pre-error

If not, Jared is the right person to talk to.

[Jared Greathouse]

I'll read more later, but my initial reaction is kinda that you have a short pretreatment period. This isn't wrong, the math still works, the issue is just that it's harder to have better out of sample predictions with 10 pretreatment periods... right?

Also, I'm kinda concerned about

basically, I want the other four offices to be combined into a control group that looks similar to the treatment office

Are you saying you have 4 donors? If so, then don't use scul!! In fact, I would almost even say don't use SCM at all, like why bother?

The benefit of the LASSO (in theory) is to find a sparse set of units to approximate the treatment unit. But this only makes sense when you have lots of controls. Like say we had 3 controls. At that point, what benefit is the lasso giving us even, finding the best.... two, donors? It would be one thing if you had 9 or even 10, but if you only have 4 I don't really get the benefit of how the Lasso would help with things.


Also, your outcome is the number of people quitting. Isn't this kinda a random event almost? Are we really convinced that there's a set of unit specific factors, affected by some common time shocks that generate the number of quits we see (the linear factor model justification for SCM usually)?

Anyways, to your interpretation, the ATT is .4. So, the policy increased quits by less than half a quit (in English, it didn't do anything.)