[Thomas Hegland]

OK, I think I see what you're asking. So, the right interpretation for the main ATT would be that it is an effect of 0.033 days per kid per time period - it is not cumulative across units or over time. Instead, the main ATT is the average of all unit-time period effects (i.e., i-t effects, so presumably kid-time period in your set up). If beta_i,t is the effect for kid i in time period t, then the main ATT is the mean of all the beta_i,t coefficients where (i,t) corresponds with a treated kid in a treated time period.

Assuming a balanced panel where all kids are observed for all time periods, the implication would be that the average kid experiences a cumulative effect of 0.033*6=0.198 days over the full study time period, and that the total effect over all 1000 kids summing across the full study time period is thus 198 days.

Hopefully that helps!

Thomas

[Andi Nygaard]

Hello Thomas, Yes, that helps a lot. Thank you so much. - Andi

[Haotian Wu]

Hi Thomas,

I was checking the document of wooldid, briefly looking over Wooldridge (2021), and have some questions.

1. Is the 'beta_it' in underlying regression a "within" treatment effect regarding "treated group only" (since non-treated are not in the cohort). I was a little confused by the notation of "sum_over_it_in_Z(beta_it * TreatedFlag_it * Tz_it)".

2. How is the reghdfe incorporated into the estimation (such as controls() and fe()). Are these included in the first step or should be measured later?

Appreciate your explanation!

Best,
Haotian Wu

[Thomas Hegland]

Hi Haotian!

Re: (1), I'm not entirely sure what you mean, so feel free to ask more questions if my answer doesn't cover what you have in mind. But the beta_it are coefficients for indicator variables that span all treated groups (i.e., when you run wooldid y i t ttre, all groups i where ttre specifies a within-sample year of treatment onset) and all treated time periods. So, if you look at a group i and a time period t, then there is an indicator variable included for that group in that time period if i is treated and if it t>=ttre (for that i). My Z notation was just meant to capture this - Z being the set of all (i,t) pairs where i is treated and where t>=ttre for that i. Note that when using an event study specification, Z gets expanded to include some pre-treatment time periods.

Reghdfe gets incorporated into the estimation principally in that the unit and time fixed effects are implemented via the reghdfe absorb() option. And anything you place in wooldid's fe() option gets placed in reghdfe's absorb(). If you put something in controls(), it just gets included as a regular control in exactly the fashion you specify. So, the controls enter into the first step regression, and do not really enter into the margins calculation step in an explicit fashion.

[Haotian Wu]

Hi Thomas,

Thank you for the reply. Currently, I have a couple of other questions.

1. I want to clarify step 2 of the 'wooldid' package (I mean by att method).

"The exact margins calculation performed depends on the estimands specified by the user. The overall average treatment effect would be obtained by computing fitted values from the underlying regression among all treated observations (those where TreatedFlag_it is equal to 1), subtracting the counterfactual fitted values for these observations that set TreatedFlag_it to 0, and then taking the average of these differences (applying any sample weights)."

Does this imply that you summarize the average treatment effect regarding all units having treatment and do a counterfactual analysis compared to the cases when these units are still "not yet treated"? For example, all units within a cohort started their treatment in 2007 (so they are supposed to get unified estimates of beta_it), I want to calculate a dynamic treatment effect for year three since treatment by att, so I need to take a look at the estimated beta_i,2010 and count the total unit number within the cohort, and incorporated their multiplication as a part for the att calculation?

2. What are the main advantages of the 'world' compared to some other event-study packages if I want to use linear regression for event-study analysis (and may incorporate some time-individual varying controls) and choose not to use time-invariant covariates proposed by Wooldridge (2021)? Is the cohort-specific estimations in the first step one advantage counted?

Appreciate again for your answer!

Best,
Haotian Wu

[Thomas Hegland]

Apologies about my delayed response.

Re: (1), the comparisons between the actual treated and control units only happen within the regression itself. What the margins calculation step is doing is more like a glorified method of computing observation count (or other specified weights) weighted averages of the treatment effect coefficients used in the underlying regression. I say 'glorified' since it can get a little more complicated than that simple case when we start to consider continuous treatments (or interactive controls), since the ATT then depends on the level of the continuous treatment variable within given subgroups as well. The exact way I implement this is by using stata's margins command, so implicitly what's happening is that I'm taking the real predicted values for each observation within a subgroup for which an ATT is getting generated, computing the predicted values after multiplying their treatment indicator to 0, taking the difference between these two things, and averaging. So for the dynamic treatment effect in the third year after treatment, the ATT I present is the average of the aforementioned difference in predicted values, but restricted to the set of observations corresponding with treated units in the third year after treatment.

Re: (2), in terms of advantages of wooldid versus other packages. Much of this I think comes down simply to taste. That said, there are a few things that I think currently are relatively unique to wooldid, though I could be wrong (I have not checked in some time). These include the option to specifically pin your choice of event study reference period to a particular flagged period relative to treatment, the included facilities for subgroup effect estimation, the options for automatically computing certain alternative estimands beyond the att (i.e., att_i, att_it, and att_itime), the option to produce Ibragimov and Muller style p-values (a method of which I am personally quite fond, but which strikes me as unfortunately underused), many of the facilities I have included for working with continuous treatment variables, and I think also some of the automatically included ancillary tests available when working with subgroups and event studies (specifically, the jointtest and adversarial time trend test options).

[Andrew Emmons]

Dr Hegland,
Thanks very much for the command and the note in the help file on DDD! I’m estimating the effect of unemployment insurance (UI) benefit amount changes on mental health. The data is repeated cross sections of individuals (Household Pulse Survey). The variation is different 1. Time, 2. States, & 3. Individual UI receipt
Questions:

• Graphs: Is it possible to graph an event study for a triple DDD? The graphs appear to be for the full estimation sample, subgroup=1, & subgroup=0. Although, there is not a graph for difference in subgroups?
• Event study estimates: Likewise, is there a way to estimate the coefficients at each event time period for the difference in subgroups?
• Odd Result: The following specification gives substantially different estimates than the standard event study code (e.g., - 1.3 rather than -10.0), which I’m unsure if it is the program delivering a weird result?

*Local for Outcomes:
local outcomes_regs  gad2_anxiety_3_up    phq2_depression_3_up
*Local for Control Variable List
local controls i.age i.household_size i.male i.hispanic i.black i.asian i.other_race i.less_high_school i.high_school i.married i.income_less_25k i.income_25_34k i.income_35_49k i.income_50_74k i.income_75_99k i.income_100_149k i.income_150_199k i.kids_in_household 
*WOOLDID
foreach Y of local outcomes_regs {
wooldid  `Y' state week treat_wool [pweight=scaled_pweight]       , att att_it cluster(state) robust controls(`controls') subgroups(ui_unemployed)  fe(i.ui_unemployed_week_fe  i.ui_unemployed_state_fe)     timetrends
est store WDID`Y'
}
* EVENT STUDY
foreach Y of local outcomes_regs {
areg `Y'   `ui_event_time'                  i.ui_unemployed_week_fe  i.ui_unemployed_state_fe       `controls'        [pweight=scaled_pweight],r cluster(state) absorb(state_week_fe)
est store Wevent`Y'

Thanks!

[Thomas Hegland]

Hi Andrew;


So, starting with the graph and event study estimates question. You are correct to note that currently, the program does not produce DDD event study results for the DDD coefficients. Sorry -- handling the DDD case just hasn't be a top priority. It's on my to do list to include eventually, but it could be a while. For now, the program can only do what you've identified - generate the overall effect estimate and the event study plots for the constituent diff-in-diffs. (Maybe there is a way to force wooldid to make an event study plot for the DDD over time coefficients by manually including a special set of fixed effects, but I have not really fully thought this through. Apologies.)

As a side note, I think your specification may not be correct. If I understand correctly, your subgroups are "People with UI vs People without UI". Meanwhile, your i and t variables are state and week. So, your i and t variables permit treatment effect heterogeneity at the state-year level, but the 'has UI vs does not have UI' subgroups ought to vary within state. Since your i and t variables do not permit variation at the "does this person have UI or not" level, the subgroup variation you find probably doesn't reflect what you want it to. Specifically, the 'has UI' effect will (more or less) be the average of state-year effects weighted by the number of people with UI in the given state-year (and the same for the 'without UI' effect).

To allow treatment effect variation at the 'UI vs no UI X state X year' level, you would probably want to replace your i variable state with something like the variable state_sg where state_sg is the result from a command like: egen state_sg = group(state ui_unemployed). If you do that, then you should get the type of variation you want. That said, you may or may not want that specification either. To get treatment effects robust to staggered diff-in-diff type issues, all you really need to do is allow treatment effect heterogeneity by the time period of treatment onset. So, if used g, where g is the output from egen g = groups(treat_wool), as your i variable in lieu of state (and used the result from egen g_sg = group(ui_unemployed treat_wool) in the case where you want treatment effect heterogeneity), your results would still be appropriately robust to staggered diff-in-diff problems. You would be allowing less treatment effect state out for a treatment onset groups i variable like the g I defined above, you would get faster estimation and maybe more stable results (given that the household pulse survey, if I recall correctly, has a fairly small sample and may not appreciate state-year-subgroup level estimates being produced). Note that to do this, you would also need to modify your fe to include state FE manually, if you want to keep them in the analysis.

Turning to the odd result. It is hard to say exactly what is going on without working with the data, but off the bat, I'd note that these are 2 very different specifications. They will yield similar looking results in a sense, yes, but underlyingly, the areg specification shown is not necessarily robust to staggered diff-in-diff type issues. So, with staggered onset, you could get very different results (all that treatment effect heterogeneity that wooldid estimates is estimated to address this problem). Also, it looks like the two commands include different fixed effects. Your wooldid command does not include i.state_week_fe -- instead it includes fixed effects for your i and t variables: i.state and i.week.

I hope that helps!

Thomas

[Andrew Emmons]

Thomas, Thanks very much for the incredibly helpful insights! I greatly appreciate you taking the time to rapidly & thoroughly address my questions.
Best,
Andrew