wooldid: Estimation of Diff-in-Diff Treatment Effects with Staggered Treatment Onset Using Heterogeneity-Robust TWFE Regressions

[Thomas Hegland]

Hello all;

I am posting to share that a new program, wooldid, is available on SSC.

This program offers a suite of tools in STATA for implementing difference-in-differences style analyses with staggered treatment onset using the two-way fixed effects approach proposed in Wooldridge (2021) and the high dimensional fixed effects estimators developed by Correia (2017). Features include:

• Estimation of multiple types of overall average treatment effects
• Estimation of event study style estimates of treatment effects in particular years relative to treatment using the user's choice of a pooled or fixed reference period
• Estimation of various pre-trend tests: various types of average pre-treatment effects, testing the hypothesis that all pre-treatment relative time period specific effects are zero, testing the hypothesis that all relative time period specific effects (pre-and post-treatment) are on a single line
• Estimation of subgroup/treatment-arm specific effects, including comparisons of effects across subgroups, tests for homogeneity of treatment effects across subgroups, and subgroup specific pre-trend tests
• Estimation of the average marginal treatment effect of a continuous treatment variable (experimental)
• Production of event study plots, histograms of treatment effects by cohorts, and plots of treatment effects across the distribution of a continuous treatment variable
• Inference using clustered standard errors, using unconditional standard errors, and using Ibragimov and Muller (2010) style cluster-robust inference
• Estimation of treatment effect elasticities and of treatment effects using poisson estimation

More information about the program is available at its github page here and its help file.

Feel free to reach out to me here or via email if you have any questions.

-Thomas

[Andi Nygaard]

Hello Thomas, many thanks for providing this STATA program. I've installed this today, but may have set up my i t tree variables incorrectly. My data contains 3500 observations (i) observed over 17 years (t), with staggered onset of ttreatment. I keep getting the error message that 'maxvar is too small'. maxvar is set at 32767 (STATA 18SE). Do i need to update my license to run this program (on this data). Or have I likely set up the data incorrectly? Many thanks for any assistance.
- Andi

[Thomas Hegland]

Hi Andi;

It depends on your setting. In wooldid, when you specify an i variable, wooldid will allow treatment effects to vary across the levels of i within a given time period (t). Your i variable does not have to be specified as finely as possible -- it can represent groupings of observations in your data. The important thing is just that you specify it so that it is fine enough to capture whatever degree of heterogeneity in treatment effects is important to do so in your setting.

For example, suppose you have a panel dataset consisting of specific individuals observed over time. Let's say your treatment variable, however, is based on the particular state in which each person lives, with different states entering into treatment at different times. Your options for the i-variable in this setting could be: i = individual identifiers (thus allowing each individual to have their own unique treatment effect), i = states (thus allowing each state to have its own treatment effect, but requiring all individuals in a given state to share an effect), and i = treatment onset groups (thus requiring all individuals in states treated at the same time to share a treatment effect, but allowing treatment effects to vary otherwise).

Which approach is best? The coarsest specification of the i-variable you can get away with is specifying it to consist of treatment onset groups (i.e., to specify something like: gen i = ttre; replace i = 9999 if i ==.). This will suffice for many purposes and will resolve the staggered diff-in-diff issue. It also doesn't really restrict you with respect to what fixed effects are included -- you can always use wooldid's fe() option to add additional fixed effects to your regression (e.g., in the case above, specifying i=state would only allow treatment effects to vary by state, but individual-level FE would still be included if you specified fe(i.person)). If you think it's important to allow treatment effects to vary at finer levels (at the state or individual level, in the example above) then you can specify i to do so. This can be especially useful if you are interested in the treatment effect heterogeneity itself, and wooldid gives you some options to explore that heterogeneity using the subgroup() option.

As for the maxvar issue. My guess is that your i-variable is currently setup at something like the individual-level in my example -- in your data, you'll need about 3500*16 variables to do an event study with that many levels of your i variable and that many periods. So if you want to keep your i-variable like that, you'll need to upgrade stata and also be willing to let your computer spend quite a long time chugging at estimation. That said, if you don't actually think allowing treatment effect heterogeneity at the i-level in your data is important and that allowing it at some higher level of aggregation instead would be fine, then specifying your setting's equivalent of i=state or i=treatment_onset_groups would probably work for you without having to upgrade stata.

Hope that helps!

Thomas

[Andi Nygaard]

Hello Thomas,
Many thanks for super quick reply. My data consists of people experiencing homelessness. There is a high degree of heterogeneity across this group, but dealing with that in the fe() option is a good suggestion. Many thanks again and wishing you a relaxed Christmas.

- Andi

[aneta zahei]

Hi,

I am also a novice when using wooldid and I am having trouble understanding the ttre variable. Specifically, in my case, I have individuals in 170 regions (I consider the regions to represent the i) and only 2 time points(t). All regions experience the treatment at the same time. How should my ttree variable look like?

Many thanks,
A.

[Ana Gouvea]

Hello Thomas, Thank you for providing this STATA program! I was wondering if I could use your command to estimate heterogeneous effects based on a covariate. For instance, if I have a treatment at the school level but obs at the student level with information on sex, define the cohorts by grouping school id and sex and calculating att subgroups( sex). Thank you!

[Thomas Hegland]

Hi,

I am also a novice when using wooldid and I am having trouble understanding the ttre variable. Specifically, in my case, I have individuals in 170 regions (I consider the regions to represent the i) and only 2 time points(t). All regions experience the treatment at the same time. How should my ttree variable look like?

Many thanks,
A.

Apologies for my delayed reply Aneta. In terms of setting up the ttre variable -- this variable should specify the time period of treatment onset for your observed units. With only 2 time points, presumably period 2 is treated. So, treated regions would have ttre = 2 and untreated regions woudl have ttre = missing (.).

[Thomas Hegland]

Hello Thomas, Thank you for providing this STATA program! I was wondering if I could use your command to estimate heterogeneous effects based on a covariate. For instance, if I have a treatment at the school level but obs at the student level with information on sex, define the cohorts by grouping school id and sex and calculating att subgroups( sex). Thank you!

Hi Ana;

You can definitely do exactly what you describe the way you describe it.

Thomas

[Meng Zhang]

Hi Thomas, would -wooldid- be appropriate for assessing the effects of the frequently changing minimum wages? I’m considering defining states that undergo significant increases in the minimum wage as the treatment group and using the year of increase as the treatment date.

[Thomas Hegland]

Hi Thomas, would -wooldid- be appropriate for assessing the effects of the frequently changing minimum wages? I’m considering defining states that undergo significant increases in the minimum wage as the treatment group and using the year of increase as the treatment date.

The minimum wage is tricky for the reason you mention - minimum wage laws change frequently. The approach you describe seems doable to me and could be implemented using wooldid. If you did do that, you might want to allow for some treatment effect heterogeneity by the size of the minimum wage increase and you might want to further report on that treatment effect heterogeneity. One way you could implement that would be making your i-variable (in wooldid's parlance) groups of states split by date of treatment onset and by magnitude of minimum wage hike (e.g., 0-25 cents, 25-50 cents, etc.), and then specifying a subgroup() variable that contains just those minimum wage hike magnitudes. Another way would be to set up a continuous treatment variable containing the change in the minimum wage.

That said, the frequency of the changes is tricky, and the question of how to deal with states that raise their minimum wage multiple times within your sample is a bit tricky. Conceptually, the new generation diff-in-diff estimators aim to be very agnostic about what kinds of treatment effect dynamics can result from a treatment. If a state raises the minimum wage in 2010 and again in 2015, the new estimators will generally have a hard time differentiating between those treatments: they won't know if a treatment effect in 2015 is a 5-year lagged effect of treatment in 2010 or an immediate effect of the new treatment in 2015. One easy but probably undesirable solution to this problem in your setting is to simply ignore later minimum wage increases. Another might be to set up your data along the lines of a stacked diff-in-diff and only consider very short run effects of minimum wage increases. Another still might be to try and conceptualize your treatment variable as continuous, perhaps reflecting a change in the minimum wage relative to some baseline period, and then proceed from there. There are probably other solutions still -- I think minimum wage laws are inherently a bit difficult to study, and it isn't clear to me that there is any single best default approach to research involving them.

[Solomiya Shpak]

Hi Thomas! Thank you for putting together this excellent program.

Could you please advise if it makes sense to use wooldid in my case? I am trying to estimate effect of loan receipt on firm performance but in addition to loan dummy, I would also like to estimate effect of loan amount. Firms get loans in different periods, so treatment is staggered.

Many thanks!

[Thomas Hegland]

Hi Thomas! Thank you for putting together this excellent program.

Could you please advise if it makes sense to use wooldid in my case? I am trying to estimate effect of loan receipt on firm performance but in addition to loan dummy, I would also like to estimate effect of loan amount. Firms get loans in different periods, so treatment is staggered.

Many thanks!

Wooldid should be able to accommodate this case, and estimate both average treatment effects of loan receipt (given the average loan amount) and average marginal effects of increasing loan amounts.

It is a bit tricky, though, as you will need to think hard about selection into loan amount levels. If you're dead confident there's no selection on loan amounts, then you can use the 'untreatedZero' option in wooldid to let it know that outside the treatment period, everyone has a loan amount of 0. Otherwise, though, you may wish to look into simulating how much each firm in your analysis would receive in loans were it to receive one. At minimum, you can always do this amount treated firms (the simulated amount for a loan receiving firm in the pre-treatment period is just the loan it receives when it receives the loan later) and do your analysis exclusively among them -- this will get you an AME, though not an accurate ATT if there is an effect of receiving a loan of size 0.

[Andi Nygaard]

Hello Thomas, still working with this great package after an extended break! When interpreting the main regression results (in my case Poisson) do the aggregation coefficients (e.g. main att, pre1, contemp, post1, etc) represent the the total treatment effect for the sample in each period, or the treatment effect on each individual in the sample for each period? My data is in 6-month windows - 4 pre, 6 post. For instance, in post2 the effect is -0.321 (after poisexp).
- Andi

[Thomas Hegland]

Hi Andi - I'm not 100% sure what you mean by the 'total' treatment effect - do you mean something like, a cumulative treatment effect (i.e., a case where the estimate reported in period X is the sum of treatment effects from the period of treatment onset to period X)? If so, then the results wooldid presents are not that - they are not cumulative. The "post1" effect would be the average effect observed just in the first period after treatment onset, and is not summed or otherwise directly connected with the effect observed in the prior period (the period contemporaneous with treatment onset). Similarly, the main ATT gives the average effect over all treated units over all treated time periods, and is not cumulative either.

[Andi Nygaard]

Hello Thomas, many thanks for quick reply. Apologies for lack of detail in previous message. Below an example of number of school days (attendance) per 6 months (kids are observed on a 6-monthly basis, 11 periods, data is count of school days in each 6-month period, staggered entry into treatment, 1000 kids in treated and control group alike). Results are based on pooled rather than fixed comparison period. Estimation uses poisson and poisexp)

Main ATT: 0.033 (p=0.586)

Pre4 -0.025 (p=0.676)
Pre3 -0.032 (p=0.624)
Pre2 -0.028 (p=0.695)
Pre1 0.182 (p=0.034)
Cont 0.193 (p=0.027)
Post1 0.047 (p=0.537)
Post2 -0.073 (p=0.354)
Post3 -0.178 (p=0.010)
Post4 -0.189 (p=0.042)
Post5 -0.256 (p=0.012)



Focusing only on the point estimated (i.e. setting aside the issue of significance for the time being), the main ATT would suggest for the kids who received treatment there was an average increase in school days of 0.033 days. My interpretation of this is that this is 0.033 days for all 1000 kids, over the 6 treatment period (so practically nothing, and also not significant), rather than 0.033 days per kid, over the 6 treatment periods (which would translate into 33 days across the 1000 kids, over the 6 treatment periods).

When looking at the time heterogeneity, the results suggests a likely initial positive effect (Pre1 and Cont are both positive) before a gradual decline in attendance from period Post3 onwards. The interpretation of Post3 would be that there is a 0.178 days reduction in school attendance across all treated kids in period Post3, and 0.256 days reduction in period Post5 (rather than the alternative 17.8 and 25.6 days if effect is per kid, the staggering would make these number somewhat smaller as not all kids are observed with 6 periods of treatment). In the first interpretation the results, while significant (Post3 onwards), is in practical terms neither here nor there, whereas in the alternative interpretation the practical result is more meaningful.

Thank you again for taking the time to reply to these messages.
- Andi