Parallel trends almost holds

[Oliver WB]

Hi,

I have created an event study plot for checking parallel trends, as shown here. However, about 3 out of the 40 pre-treatment periods have statistically significant coefficients. I am writing my bachelor thesis and I would appreciate some guidance on whether to treat this as parallel trends, and hence be able to interpret the DID estimates. If not, can I use any method/earlier study to argue that they are parallel "enough"? (x-axis = time to treatment). As seen in the graph there is a pretty visible effect of the treatment.

[Maxence Morlet]

You have a lot of periods pre-treatment, the only thing that is worrying is the significant coefficient at t-3 I think.

If you use the user written command eventdd, you can do a joint test of significance of the leads and lags, and that should give you more information.

But this is only a visual inspection, accompanied with some significance testing. You need to know yourself: was the reform adopted based on trends in the treatment group? Were there spillover effects from treatment to control? Was treatment anticipated?

Parallel trends must also be convincingly argued. Also, Roth (2022) criticises leads and lags tests on the basis of low statsitcal power.

[Luca Calianno]

I will suggest some tests that may support the parallel trends assumption.

• You can test that all pre treatment coeffients are jointly 0 (using test command in stata is quite easy)
• You can group pre-treatment coefficients in 5 or 10 years groups (group values between -39 and -35 into 1 coefficient; between -34 and -30; and so on...)
• Another approach is to show that using tools that allow for violations of the parallel trends assumption (Rambachan and Roth 2023) your estimates do not change much.

Probably there are other tests you can implement but starting with 1 and 2 can be enough to support your argument.

[Oliver WB]

Thanks! The variables used are the log_prices of housing prices for two Swedish regions, and the treatment is the establishing of a new factory in one of the regions (a large factory that has become the largest employer in that region). I am pretty sure that there are not any spillover effects since the treatment group is an aggregated control group consisting of 6 regions that are not relatively close to the treated region. I have used the date when the factory received its last environmental permits for starting to build as the treatment date (Q2 2018), and hence there might have been anticipation effects since the building plan was announced in middle 2017. The region was also in a public "competition" to compete for the building rights earlier than this, say beginning of 2017. What do you believe is my best way forward based on this?

[Oliver WB]

You have a lot of periods pre-treatment, the only thing that is worrying is the significant coefficient at t-3 I think.

If you use the user written command eventdd, you can do a joint test of significance of the leads and lags, and that should give you more information.

But this is only a visual inspection, accompanied with some significance testing. You need to know yourself: was the reform adopted based on trends in the treatment group? Were there spillover effects from treatment to control? Was treatment anticipated?

Parallel trends must also be convincingly argued. Also, Roth (2022) criticises leads and lags tests on the basis of low statsitcal power.

Thanks! The variables used are the log_prices of housing prices for two Swedish regions, and the treatment is the establishing of a new factory in one of the regions (a large factory that has become the largest employer in that region). I am pretty sure that there are not any spillover effects since the treatment group is an aggregated control group consisting of 6 regions that are not relatively close to the treated region. I have used the date when the factory received its last environmental permits for starting to build as the treatment date (Q2 2018), and hence there might have been anticipation effects since the building plan was announced in middle 2017. The region was also in a public "competition" to compete for the building rights earlier than this, say beginning of 2017. What do you believe is my best way forward based on this?

[Oliver WB]

I will suggest some tests that may support the parallel trends assumption.

• You can test that all pre treatment coeffients are jointly 0 (using test command in stata is quite easy)
• You can group pre-treatment coefficients in 5 or 10 years groups (group values between -39 and -35 into 1 coefficient; between -34 and -30; and so on...)
• Another approach is to show that using tools that allow for violations of the parallel trends assumption (Rambachan and Roth 2023) your estimates do not change much.

Probably there are other tests you can implement but starting with 1 and 2 can be enough to support your argument.

Thanks for the inputs! But won't the joint test also show that at least some of the coefficients pre-treatment are statistically significant, since at least 3 of the individual periods are? I.e. won't it show that the pre-treatment coefficients are jointly different from 0?

[Luca Calianno]

Yes, indeed. My reasoning may be wrong but my idea is to test whether those significant coefficents are significant because they are or because of type 2 error (you are in the 5% of the times the test does not rejects the null when it should).