how to generate placebo test for random false treatment ?

[Saleh sharmah]

Dear all

I would like to do a placebo test to ensure the result of my main staggered DID results. I have a country-level mandate that goes into effect in different years during my sample period, and in some countries, there was no mandate. I examine the effect of this mandate on firms' outcomes in an international sample from 2005 to 2020. Now I would like to do:

1. Placebo test by creating an indicator variable randomly by sorting firms randomly and assigning the first X # of firms to treatment based on how many actual X # firms are treated that year
2. after the placebo indicator is created, I would like to run it in regression using reghdfe?

Here is an example of my data: note that magnitude is my DV and treatedxmandate is my IV, and policy in effect indicates the year in which the mandate is passed. Thanks for your kind input.
input float(magnitude treatedxmandate) into Year float(country_id firm_id policy_in_effect)
0 0 2009 15 1101 0
0 0 2014 32 2431 0
0 0 2009 45 297 0
0 0 2008 45 297 0
0 0 2010 45 297 0
0 0 2016 50 2872 0
0 0 2015 50 2872 0
0 0 2017 50 2872 0
0 0 2019 50 2872 0
0 0 2016 50 2868 0
0 0 2016 2 9 0
0 0 2017 2 9 0
0 0 2012 37 2530 0
0 0 2013 37 2530 0
0 0 2016 45 295 0
0 0 2013 3 39 0
0 0 2015 3 39 0
0 0 2017 50 3275 0
0 0 2018 45 295 0
0 0 2011 42 1875 0
0 0 2006 49 919 0
0 0 2010 42 1875 0
0 0 2019 50 3239 0
0 0 2018 50 3080 0
0 0 2018 50 3249 0
0 0 2006 49 864 0
0 0 2020 31 2490 0
0 0 2007 49 864 0
0 0 2005 50 3324 0
0 0 2016 50 3239 0
0 0 2016 50 2834 0
0 0 2017 50 3147 0
0 0 2017 23 2204 0
0 0 2012 50 2921 0
0 0 2013 50 2921 0

[George Ford]

ritest will automate this, but make sure you set it up correctly (see Heb's website for how to do it with DID). (ssc install ritest)

if the treatment occurs in different years, you need to thing about staggered DID (csdid or xthdidregress or hdidregress) since the effect may vary by calendar year.

I think you'd want the false treatment to be before the true treatment so think about that.

[Saleh sharmah]

Thanks, George for your input. I tried to use a code to create an indicator variable by sorting firms randomly and assigning the first X # of firms to treatment based on how many actual X # firms are treated that year. However, the code did not work

gen random_treatment=runiform( firm_id ) if treated==1, by ( firm_id Year)

what would be the correct code here? thanks for your time and input. data example is provided above.

[George Ford]

Is the data balanced? What's the date range?

your data in #1 is unusable. look up dataex to provide a good sample with some treated units.

the trick here is that the false date should be prior to the treatment date. not a problem to create that, but keep that in mind.

[George Ford]

Do you have outcomes for countries when they are untreated?

[George Ford]

If you are treated in 2008 all the way through 2020, say, then just randomly creating a false treatment is problematic since the treatment effect, if any, is present in later periods.

You're going to need to use some type of staggered treatment design, which in most cases can automatically generate a pseudo-treatment.

eventdd, xthdidregress, csdid is where I'd start looking.

[Saleh sharmah]

Hi George

Thanks for your time and response. My data is not balanced and I am using staggered DID in this study since countries start mandating the policy at different points in time in the study sample.

I have reached out to an author who has conducted a similar model design and investigated the effect of the mandate and here is what he suggested:

1- create an indicator variable by sorting firms randomly and assigning the first X # of firms to treatment based on how many actual X # firms are treated that year.
so suppose the number of treated firms in 2005 = 50, i would like to randomly assign the first 50 firms in 2005 as treated, and the same goes for all the years. That is what the author suggested. how can I code this in Stata, please?

Here are descriptions of my variables below: magnitude is my DV, treatedxmadnate is my IV, policey_in _effect refers to a dummy = one once the country mandates a policy during a given year and remains one for the future years, and zero otherwise. treated x mandate refers to firms effected by the mandate. Policey1year refers to the first year a country mandates the policy.
input str44 Country int Year float(firm_id magnitude treatedxmandate policy_in_effect) int Policy1Year
"United States" 2019 3062 1.6 0 0 .
"Brazil" 2011 216 .74 0 0 .
"New Zealand" 2013 2420 1.11 0 0 .
"Hong Kong" 2019 137 .6 1 1 2014
"South Korea" 2013 1866 .63 1 1 2012
"Japan" 2019 1766 .590 1 1 2005
"United States" 2012 2965 .95 0 0 .
"Japan" 2007 1344 .83 1 1 2005
"United States" 2006 2832 .93 0 0 .
"Sweden" 2007 2631 .429 1 1 2005
"South Korea" 2017 1828 .32 1 1 2012
"Switzerland" 2016 319 .48 0 0 .
"Sweden" 2013 2572 .47 1 1 2005
"United States" 2008 3313 .63 0 0 .
"China" 2009 1084 1.09 1 1 2008
"United States" 2016 2994 .539 0 0 .
"United States" 2019 3025 .94 0 0 .
"United States" 2010 3254 .96 0 0 .
"Germany" 2016 584 1.04 1 1 2005
"Sri Lanka" 2013 2195 .97 0 0 .
"China" 2018 363 .309 1 1 2008
"South Korea" 2014 1829 .52 1 1 2012
"United Kingdom" 2008 777 1.02 1 1 2008
"Netherlands" 2015 2327 .35 1 1 2006
"United Kingdom" 2016 974 1.19 1 1 2008
"Germany" 2012 638 .55 1 1 2005
[/CODE]

Thank you in advance for your time and help, George.

[George Ford]

with staggered treatments, you'll get the pseudo treatment in the output if you ask for it. I'd go with that. I don't think the recommended approach is legit, depending on what you think about how the treatment plays out. That approach will catch treated units and untreated units, where the treated units do not satisfy the null, so you'll may get results even if there are none.

you could use shufflevar. it will simply shuffle the observed treatment dates so you always get the right number of treatment by year.