Different results between a fixed effects model and an event study model

[Zhiya Zuo]

Dear Stata-ers, I am faced with a problem regarding the difference in results from a fixed effects model and an event study model. Briefly, I find an insignificant average effect when using the fixed effects model. However, the joint F-test with the post-period coefficients in the event study model show that there are indeed significant effects in one of the post periods. I wonder if this can be evidence that there's statistical significant dynamic effect of X on Y despite the absence of an average effect.

Thank you all for your help and I appreciate your help (Somehow the output/code gets messy when I copy paste and I had to manually format it as below... sorry for this)

Here's the setup:

• Study variable of interest: X. This is log-plus-one-transformed of a count variable.
• Control variables: C1 and C2
• Outcome variable Y1 and Y2
• Subscripts: I for user and t for week.

I start with a two-way fixed effect models with the following Stata code:

. qui eststo m1: xtreg Y1 X C1 i.t, fe r
. qui eststo m2: xtreg Y2 X C1 C2 i.t, fe r
. esttab m1 m2, p keep(X)
--------------------------------------------
                       Y1 Y2  
--------------------------------------------
X -0.378* -0.0456  
                 (0.028) (0.550)  
--------------------------------------------
N 1832 1832  
--------------------------------------------
p-values in parentheses
* p<0.05, ** p<0.01, *** p<0.001

Then I proceed with the event study setup, where I distinguish the pre- and post-periods based on the first time that the raw value (before the log1p transformation) was a positive value and created dummies. For all the post-period dummies, I multiplied them by X to indicate the treatment intensity. This also takes into account the fact that X may take a value of 0 after the first treatment. Since I only have 8 weeks of observations, I collapsed the relative times such that all relative time less than -3 (i.e., 3 weeks or more before the first treatment) or larger than 3 (ie.., 3 weeks ore more after the first treatment) are collapsed into the Post-3 period.

Below are my Stata commands and results:

. qui eststo m1_event: xtreg Y1 F2event F3event L*event C1 i.t, fe r
. qui eststo m2_event: xtreg Y2 F2event F3event L*event C1 C2 i.t, fe r
. esttab m1_event m2_event, p keep(*even*)
--------------------------------------------
                       Y1 Y2  
--------------------------------------------
F2event -0.00979 0.0556  
            (0.932) (0.419)  
F3event 0.115 0.00603  
                 (0.346) (0.943)  
L0event -0.186 -0.00963  
                 (0.163) (0.914)  
L1event -0.466 -0.583***
                  (0.617) (0.000)  
L2event -0.901*** 0.0462  
                 (0.001) (0.712)  
L3event -1.338*** -0.185  
                 (0.000) (0.227)  
--------------------------------------------
N 1832 1832  
--------------------------------------------
p-values in parentheses
* p<0.05, ** p<0.01, *** p<0.001

As I see that there are significant coefficients in the post-period (L2 and L3 for Y1 and L1 for Y2), I proceed with a joint F-test with the "test" commands, which reject the null hypothesis that the post-coefficients are jointly zero's. In this case, I wonder if I can say that there are dynamic effects (albeit the average effects are insignificant) of X on both Y2 (X on Y1 is consistently significantly negative.)

At the end, I also attach the event study plot.

. est restore m1_event
(results m1_event are active now)
. test (L0event=0) (L1event=0) (L2event=0) (L3event=0)
( 1) L0event = 0
( 2) L1event = 0
( 3) L2event = 0
( 4) L3event = 0
F( 4, 228) = 19.86
Prob > F = 0.0000

. test (F2event=0) (F3event=0)
( 1) F2event = 0
( 2) F3event = 0
F( 2, 228) = 0.60
Prob > F = 0.5497

. est restore m2_event
(results m2_event are active now)
. test (L0event=0) (L1event=0) (L2event=0) (L3event=0)
( 1) L0event = 0
( 2) L1event = 0
( 3) L2event = 0
( 4) L3event = 0
F( 4, 228) = 4.13
Prob > F = 0.0030

. test (F2event=0) (F3event=0)
( 1) F2event = 0
( 2) F3event = 0
F( 2, 228) = 0.41
Prob > F = 0.6632

event_plot m1_event m2_event, stub_lag(L#event) stub_lead(F#event) ///
    plottype(scatter) ciplottype(rcap) together noautolegend perturb(0 0.08 0.16) ///
    graph_opt(title("Event study estimators", size(medlarge)) ///
    xtitle("Periods since frist treatment X") ytitle("Average effect") xline(-1, lcolor(gs8) lpattern(dash)) yline(0, lcolor(gs8)) ///
    legend(order(2 "Y1" 4 "Y2") rows(3) ring(0) pos(1)) ///
    graphregion(color(white)) bgcolor(white) ylabel(, angle(horizontal))) ///
    lag_opt1(msymbol(O) color(purple)) lag_ci_opt1(color(purple)) ///
    lag_opt2(msymbol(Th) color(navy)) lag_ci_opt2(color(navy))