First effects vs first difference

[Nae Khar]

Hello all,

I am running two models with exactly the same variables (a fixed effects and a first difference).

If a variable carries a similar coefficient throughout both a fixed effects and first difference model, but loses significance when moving from fixed effects to first difference, does that indicate that this variable is not robust?

Thanks

[Joseph Coveney]

Do you meant the constant (intercept)?

version 15.1

clear *
set seed `=strreverse("1476127")'

quietly set obs 25
generate byte pid = _n
generate double pid_u = rnormal()

quietly expand 5
bysort pid: generate byte tim = _n
generate double rsp = pid_u + (tim - 2.5) + rnormal(0, 4)

*
* Begin here
*
quietly xtset pid tim
xtreg rsp i.tim, fe

bysort pid (tim): generate double fsd = rsp - rsp[1]
regress fsd i.tim ib(last).pid

exit

[Nae Khar]

Hi Joseph

Thank you for your reply. Yes I do. I have attached an image for clarification.


The first model is a fixed effects - xtreg lnstartup sound_money Tariffs_ iv3 i.year, cluster(country) fe -

The second model is first difference - regress d.lnstartup d.sound_money d.Tariffs_ d.iv3 i.year, cluster (country)



I was told that running a first difference is a good robustness check for my fixed effects model. They should produce the same results (in terms of the coefficients).


But what about significance? If one variable loses significance in the first difference but maintains similar coefficients (like sound money in this case above 0.006) - does that indicate that it is not robust?

[Joro Kolev]

Both fixed effects (= the within transformation) and the first differences (= first difference transformation) accomplish one task the same: they eliminate the fixed effect. So the coefficient magnitude should be the same under both. In other words, you cannot device a Hausman test on the basis of the difference between these two, because they are both consistent under the same assumption (= an unobserved fixed effect which might be correlated with the regressors).

The standard errors are expected to be different because, somewhat circularly, the transformations are different.

What I said so far is a fact. Now what I say next is most probably true, but might be marred by my fallible memory of what I have derived myself long ago, or read somewhere else:

The standard errors cannot be expected to be the same, because they are optimal under different dynamics of the idiosyncratic error term.
a) Fixed effects (= the within transformation) is optimal under independent identically distributed idiosyncratic error.
b) First differences is optimal under idiosyncratic error following random walk.

[Jorge Eduardo Perez Perez]

The fixed effects and first difference estimators are exactly the same for panels with T=2, but they will differ in finite samples for T>=3.

[Nae Khar]

Thank you for your answers. Highly appreciated.

I was advised to use the first difference model as a test of robustness for my fixed effects model by comparing their results.

Can you clarify specifically: what does this mean in terms of significance?

Are both models suppose to produce the same results (both in terms of coefficients and significance) if used for a panel of 8 years?

What does it mean when a variable does not carry significance over both models but maintains the same coefficient?

[Jorge Eduardo Perez Perez]

The coefficients will be the same for fixed effects and first difference estimation for T=2. The standard errors will be different.

For T>=3, both the coefficients and the standard errors will be different.

In any case where std. errors differ, significance may differ as well. My advice would be to account for the dependence between errors for the same individual, by using robust standard errors in both cases.

[Joro Kolev]

The coefficients will be the same for fixed effects and first difference estimation for T=2. The standard errors will be different.

This is not correct. For the special case of T=2 for each i, both the parameter estimates and their standard error are numerically equivalent:

. webuse grunfeld, clear

. xtset company year
       panel variable:  company (strongly balanced)
        time variable:  year, 1935 to 1954
                delta:  1 year

. keep if year<1937
(180 observations deleted)

. xtdes

 company:  1, 2, ..., 10                                     n =         10
    year:  1935, 1936, ..., 1936                             T =          2
           Delta(year) = 1 year
           Span(year)  = 2 periods
           (company*year uniquely identifies each observation)

Distribution of T_i:   min      5%     25%       50%       75%     95%     max
                         2       2       2         2         2       2       2

     Freq.  Percent    Cum. |  Pattern
 ---------------------------+---------
       10    100.00  100.00 |  11
 ---------------------------+---------
       10    100.00         |  XX

. xtreg invest mvalue kstock, fe

Fixed-effects (within) regression               Number of obs     =         20
Group variable: company                         Number of groups  =         10

R-sq:                                           Obs per group:
     within  = 0.4549                                         min =          2
     between = 0.6250                                         avg =        2.0
     overall = 0.6166                                         max =          2

                                                F(2,8)            =       3.34
corr(u_i, Xb)  = -0.1176                        Prob > F          =     0.0883

------------------------------------------------------------------------------
      invest |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      mvalue |   .0724024   .0323236     2.24   0.055    -.0021359    .1469408
      kstock |  -.6885403    1.00748    -0.68   0.514    -3.011793    1.634713
       _cons |   69.47996   52.82976     1.32   0.225    -52.34569    191.3056
-------------+----------------------------------------------------------------
     sigma_u |  76.530135
     sigma_e |  30.984468
         rho |  .85916804   (fraction of variance due to u_i)
------------------------------------------------------------------------------
F test that all u_i=0: F(9, 8) = 7.50                        Prob > F = 0.0046

. reg d.invest d.mvalue d.kstock, nocon

      Source |       SS           df       MS      Number of obs   =        10
-------------+----------------------------------   F(2, 8)         =      3.34
       Model |  12816.2962         2  6408.14808   Prob > F        =    0.0883
    Residual |  15360.5957         8  1920.07446   R-squared       =    0.4549
-------------+----------------------------------   Adj R-squared   =    0.3186
       Total |  28176.8919        10  2817.68919   Root MSE        =    43.819

------------------------------------------------------------------------------
    D.invest |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      mvalue |
         D1. |   .0724024   .0323236     2.24   0.055    -.0021359    .1469408
             |
      kstock |
         D1. |  -.6885403    1.00748    -0.68   0.514    -3.011793    1.634713
------------------------------------------------------------------------------

.

[Joro Kolev]

Thank you for your answers.

1) I was advised to use the first difference model as a test of robustness for my fixed effects model by comparing their results.

2) Can you clarify specifically: what does this mean in terms of significance?

3) Are both models suppose to produce the same results (both in terms of coefficients and significance) if used for a panel of 8 years?

4) What does it mean when a variable does not carry significance over both models but maintains the same coefficient?

1) Robustness with respect to what? There are countless notions of robustness in statistics and econometrics. For example the Ecker-White standard errors are robust with respect to heteroskedasticity. Robust regression (-rreg-) is robust with respect of presence of outliers. Etc. etc.

2) Parameter estimates are expected to be about the same for both fixed effect and first difference regression. They both eliminate the fixed effect and are consistent under the same conditions.

Their significance is not expected to be the same, it is expected to be different. It does not mean anything that their significance is different. I cannot think of any notion of robustness that can be demonstrated by finding similar significance of the two methods. It their significance is similar, this is just a coincidence.

3) In terms of coefficient estimates Yes. In terms of standard errors/significance, No.

4) It does not mean anything. This is the situation that is expected to occur. This is the normal situation.

[Sebastian Kripfganz]

There is a difference between the two methods that was not mentioned so far. Under the usual fixed-effects error-components assumption with homoskedasticity, no cross-sectional correlation and without serial correlation of the idiosyncratic error component, the fixed-effects estimator is the efficient estimator compared to the first-difference estimator. This can be seen by noting that the fixed-effects estimator is identical (for any T) to the first-difference GLS estimator that accounts for the first-order serial correlation in the first-differenced error term. The first-difference OLS estimator is obviously less efficient than the first-difference GLS estimator under this assumption.

Now suppose instead that the idiosyncratic error component in levels follows an AR(1) unit-root process. Then the first-differenced error term is serially uncorrelated and the first-difference OLS estimator is efficient. In that regard, the fixed-effects estimator tends to be better if there is little serial correlation in levels, the first-difference estimator if there is high serial correlation.

Another difference is that the intercept term cannot be compared. The intercept in the first-difference regression refers to the coefficient of a linear time trend in levels.

[Tiyo Ardiyono]

Dear Sebastian Kripfganz,
I'm interested in your explanation here. Do you have a reference or more detailed explanation for this?
Thank you.

[Sebastian Kripfganz]

For the equivalence of the fixed-effects within-groups estimator and the first-difference GLS estimator, see for example Bun and Kiviet (2006, Journal of Econometrics), Section 3.1.

For the comparison of the two estimators under different assumption about the serial correlation of the idiosyncratic error component, see Wooldridge's textbook on Cross Section and Panel Data, Chapter 10.7.1.