DiD with treatment status change over time (i.e., the status is switched on and off)?

[Juan Xi]

Hello,
I have a panel data, on which I want to evaluate the impact of adopting a program on the economic outcome.
The context is over 10,000 individual online sellers, the program is a special web-embedded tool, which allows the consumers to get a discount on that product, by sharing the product link on his/her social media like facebook.
The panel id is the 10,000+ online sellers, a period is a month, the dependent variable is monthly sales, and the treatment=1 if the seller enabled the web-embedded tool in that month.

I was trying to use DiD, then I realized I'm facing the following challenges:
1) the treatment, i.e.,whether or not to use the tool is endougeneous (sellers have the option to choose to adopt or not)
2) the treatment status can be switched on and off. That is, a seller could enable the tool in January, then in February he/she turned off the tool, then maybe in May, he/she turned the tool on again...

For 1), this is a classic self-selection bias problem, like in the job training program problem. I think there are some ways to address this concern.
For 2), however, I'm feeling troubled. If the treatment status flipped some, then how can I define a treatment and a control group?

Could you give me advice on the problem 2)? Is it possible to use DID for such a setting? If not DID, then what would be good method for the analysis? For example, I can think of things such as just run a seller-fixed effect regression:

areg sales tool_enabled  X i.month, absorb(seller_id)

Here we have the sales in each month sales as dependent variable, and the binary variable tool_enabled indicating the 'on or off' of the tool as the key independent variable and X is a set of covariate in each month.

Thank you!

[Clyde Schechter]

For 2), this is not a classic setup for a DID, but it falls within the framework of generalized DID. https://www.ipr.northwestern.edu/wor.../Day%204.2.pdf has a very readable introduction to this. From a coding point of view, it's very simple:

xtreg sales i.tool_enabled##i.month covariates, fe

(If your study design is more appropriately analyzed with -areg- just run the same regression that way.)

After the regression you will probably want to look at the predicted sales in each condition, on average, and perhaps in each month.

margins tool_enabled // AVERAGE PREDICTED SALES WITH & WITHOUT TOOL
margins tool_enabled#month // PREDICTED SALES WITH & WITHOUT TOOL EACH MONTH
margins, dydx(tool_enabled) // AVERAGE MARGINAL EFFECT OF TOOL

That said, the endogeneity you point out in 1) seems to me a far more difficult problem.

[Juan Xi]

For 2), this is not a classic setup for a DID, but it falls within the framework of generalized DID. https://www.ipr.northwestern.edu/wor.../Day%204.2.pdf has a very readable introduction to this. From a coding point of view, it's very simple:

xtreg sales i.tool_enabled##i.month covariates, fe

(If your study design is more appropriately analyzed with -areg- just run the same regression that way.)

After the regression you will probably want to look at the predicted sales in each condition, on average, and perhaps in each month.

margins tool_enabled // AVERAGE PREDICTED SALES WITH & WITHOUT TOOL
margins tool_enabled#month // PREDICTED SALES WITH & WITHOUT TOOL EACH MONTH
margins, dydx(tool_enabled) // AVERAGE MARGINAL EFFECT OF TOOL

That said, the endogeneity you point out in 1) seems to me a far more difficult problem.

Thank you Clyde!
For 2) The page of the link cannot be found. Could you maybe give me the title/author of the article, and I will google it?
For 1) I was not too bothered by this endogeneity problem because I was thinking that, if in a classic DID framework, we can do something like matching or weighting to balance the two groups, then do a set of robustness checks like ensure the parallel pre-treatment trend to validate the DID model. But maybe I was overlooked the complexity of this concern? What would you in such a situation to address the endogeneity issue, particularly we're dealing with a generalized DID framework?

Thank you very much~

[Clyde Schechter]

Try https://www.ipr.northwestern.edu/wor.../Day%204.2.pdf. I just went there and the link works. If not, Google generalized difference in differences and look for the hit that mentions northwestern.edu.

I don't have any good suggestions about the endogeneity issue. I would find it quite daunting to deal with. There are others on the Forum who deal with problems like this more often than I do, and I hope one of them will respond.

[Juan Xi]

Try https://www.ipr.northwestern.edu/wor.../Day%204.2.pdf. I just went there and the link works. If not, Google generalized difference in differences and look for the hit that mentions northwestern.edu.

I don't have any good suggestions about the endogeneity issue. I would find it quite daunting to deal with. There are others on the Forum who deal with problems like this more often than I do, and I hope one of them will respond.

Thank you Clyde. I tried the link, it worked. Yes as you suggested, in the document it says that a generalized DID is more flexible, with ' Switching (on/off) is allowed.' Thank you very much for the reference.

[Yingchen Xu]

Try https://www.ipr.northwestern.edu/wor.../Day%204.2.pdf. I just went there and the link works. If not, Google generalized difference in differences and look for the hit that mentions northwestern.edu.

I don't have any good suggestions about the endogeneity issue. I would find it quite daunting to deal with. There are others on the Forum who deal with problems like this more often than I do, and I hope one of them will respond.

Hello Clyde,

The link no longer works in 2024. Is there anywhere I can find this reference material? Thanks.

[Clyde Schechter]

That's unfortunate. I don't know where this can be found now, and I never downloaded a copy for myself. When I needed it I just relied on the link, and after doing enough of this kind of thing I no longer needed the reference for myself.

I'm attaching a PDF of a slide presentation on generalized DID. Generalized DID Model.pdf It's a bit more technical and notation-heavy than the reference(s) I have previously used, but it also covers some more advanced applications after doing a thorough presentation and analysis of classical DID. U nfortunately, the presentation does not identify the author(s), and I know longer remember him/her/them, so I cannot explicitly give appropriate credit.