Announcement

Collapse
No announcement yet.
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Panel Data - Hausman Test

    Hello!

    I am using STATA/SE 15.1 and doing panel data analysis involving 100 countries and 12 years of time period. Panel is unbalanced and I have 600 obervations in total. I have read literature and few number of obervations and panel data analysis is the norm in this field. However, when I conduct the hausman test, it runs fine until I add Years and Country effects i.e. dummy variables of these i.year and i.countrynum.Now I have read previous posts about hausman test but I didn't get the answer. Maybe I overlooked something. This error (displayed below) comes when I run with year and country effects (Adding dummy variables to the regression equation).



    Code:
    Note: the rank of the differenced variance matrix (16) does not equal the number of coefficients being
            tested (17); be sure this is what you expect, or there may be problems computing the test.
            Examine the output of your estimators for anything unexpected and possibly consider scaling your
            variables so that the coefficients are on a similar scale.
    
                     ---- Coefficients ----
                 |      (b)          (B)            (b-B)     sqrt(diag(V_b-V_B))
                 |       fe           re         Difference          S.E.
    -------------+----------------------------------------------------------------
    totalterro~s |   -.0003332    -.0003332        6.66e-16               .
    ControlofC~n |    .3944104     .3944104        4.51e-12               .
    SavingsCur~g |    .5122627     .5122627       -9.89e-12               .
    Inflationwin |    .0189139     .0189139       -1.00e-13               .
    creditwins~g |     .006928      .006928       -1.02e-12               .
            Year |
           2007  |    .4605182     .4605182        1.98e-12               .
           2008  |   -.0723403    -.0723403        2.76e-12               .
           2009  |   -.0857283    -.0857283        8.63e-13               .
           2010  |    -.203764     -.203764        2.60e-12               .
           2011  |   -.1459416    -.1459416        4.59e-12               .
           2012  |    .1909434     .1909434        4.45e-12               .
           2013  |    .1695522     .1695522        4.52e-12               .
           2014  |    .2844599     .2844599        4.79e-12               .
           2015  |    .5150472     .5150472        3.58e-12               .
           2016  |    .7817107     .7817107        3.49e-12               .
           2017  |    .8225087     .8225087        4.54e-12               .
           2018  |    .9736898     .9736898        5.40e-12               .
    ------------------------------------------------------------------------------
                               b = consistent under Ho and Ha; obtained from xtreg
                B = inconsistent under Ha, efficient under Ho; obtained from xtreg
    
        Test:  Ho:  difference in coefficients not systematic
    
                     chi2(16) = (b-B)'[(V_b-V_B)^(-1)](b-B)
                              =    -0.00    chi2<0 ==> model fitted on these
                                            data fails to meet the asymptotic
                                            assumptions of the Hausman test;
                                            see suest for a generalized test


    Hoping to hear from the experts.





  • #2
    Abdullah:
    are you sure you actually have a panel-wise effect in your dataset?
    Could you please share with intresetd listers what you typed and what Stata gave you back before running -hausman-? Thanks.
    Kind regards,
    Carlo
    (StataNow 18.5)

    Comment


    • #3
      Carlo: Many thanks for your reply. Yes sure, I can show you that. Below is the result of panel data fe regression and then I have conducted RE too and hausman.

      Code:
      xtreg Entryratewin totalterrorattacks ControlofCorruption SavingsCurrentwinlog Inflationwin
      > creditwinsorlog,fe
      
      Fixed-effects (within) regression               Number of obs     =        668
      Group variable: countrynum                      Number of groups  =        103
      
      R-sq:                                           Obs per group:
           within  = 0.0588                                         min =          1
           between = 0.0761                                         avg =        6.5
           overall = 0.0655                                         max =         13
      
                                                      F(5,560)          =       7.00
      corr(u_i, Xb)  = -0.2453                        Prob > F          =     0.0000
      
      --------------------------------------------------------------------------------------
              Entryratewin |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
      ---------------------+----------------------------------------------------------------
        totalterrorattacks |  -.0001695   .0002155    -0.79   0.432    -.0005929    .0002538
       ControlofCorruption |   .4101753   .3226486     1.27   0.204     -.223574    1.043925
      SavingsCurrentwinlog |   .7537161   .1468609     5.13   0.000     .4652506    1.042182
              Inflationwin |   .0095524   .0107013     0.89   0.372    -.0114672     .030572
           creditwinsorlog |   .2060454   .2038465     1.01   0.313    -.1943517    .6064425
                     _cons |  -16.24582    3.63749    -4.47   0.000    -23.39061   -9.101025
      ---------------------+----------------------------------------------------------------
                   sigma_u |  4.1226609
                   sigma_e |  1.0484583
                       rho |  .93925236   (fraction of variance due to u_i)
      --------------------------------------------------------------------------------------
      F test that all u_i=0: F(102, 560) = 46.18                   Prob > F = 0.0000
      Hausman results show:

      Code:
      .  hausman fe re
      
                       ---- Coefficients ----
                   |      (b)          (B)            (b-B)     sqrt(diag(V_b-V_B))
                   |       fe           re         Difference          S.E.
      -------------+----------------------------------------------------------------
      totalterro~s |   -.0001695    -.0001463       -.0000232               .
      ControlofC~n |    .4101753     1.234152        -.823977        .1998601
      SavingsCur~g |    .7537161     .3031298        .4505864         .091397
      Inflationwin |    .0095524     .0041184         .005434               .
      creditwins~g |    .2060454     .3122487       -.1062033        .0590943
      ------------------------------------------------------------------------------
                                 b = consistent under Ho and Ha; obtained from xtreg
                  B = inconsistent under Ha, efficient under Ho; obtained from xtreg
      
          Test:  Ho:  difference in coefficients not systematic
      
                        chi2(5) = (b-B)'[(V_b-V_B)^(-1)](b-B)
                                =       47.80
                      Prob>chi2 =      0.0000
                      (V_b-V_B is not positive definite)


      but once I add year and country dummies in the regression equation i.e. i.Year and i.countrynum


      Code:
      Fixed-effects (within) regression               Number of obs     =        668
      Group variable: countrynum                      Number of groups  =        103
      
      R-sq:                                           Obs per group:
           within  = 0.1726                                         min =          1
           between = 0.0666                                         avg =        6.5
           overall = 0.0790                                         max =         13
      
                                                      F(17,548)         =       6.73
      corr(u_i, Xb)  = -0.0743                        Prob > F          =     0.0000
      
      --------------------------------------------------------------------------------------
              Entryratewin |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
      ---------------------+----------------------------------------------------------------
        totalterrorattacks |  -.0003332   .0002112    -1.58   0.115    -.0007482    .0000817
       ControlofCorruption |   .3944104   .3166452     1.25   0.213    -.2275765    1.016397
      SavingsCurrentwinlog |   .5122627    .159536     3.21   0.001     .1988857    .8256397
              Inflationwin |   .0189139   .0107877     1.75   0.080    -.0022765    .0401043
           creditwinsorlog |    .006928   .2057796     0.03   0.973    -.3972853    .4111414
                           |
                      Year |
                     2007  |   .4605182   .2408738     1.91   0.056    -.0126307    .9336672
                     2008  |  -.0723403   .2337967    -0.31   0.757    -.5315876    .3869071
                     2009  |  -.0857283   .2354963    -0.36   0.716    -.5483144    .3768577
                     2010  |   -.203764   .2409997    -0.85   0.398    -.6771603    .2696322
                     2011  |  -.1459416   .2482301    -0.59   0.557    -.6335406    .3416575
                     2012  |   .1909434   .2398698     0.80   0.426    -.2802335    .6621202
                     2013  |   .1695522   .2413717     0.70   0.483    -.3045748    .6436791
                     2014  |   .2844599   .2409312     1.18   0.238    -.1888018    .7577215
                     2015  |   .5150472   .2338447     2.20   0.028     .0557055    .9743889
                     2016  |   .7817107   .2324741     3.36   0.001     .3250613     1.23836
                     2017  |   .8225087   .2435492     3.38   0.001     .3441044    1.300913
                     2018  |   .9736898   .2549186     3.82   0.000     .4729526    1.474427
                           |
                countrynum |
                        2  |          0  (omitted)
                        3  |          0  (omitted)
                        6  |          0  (omitted)
                        7  |          0  (omitted)
                        8  |          0  (omitted)
                        9  |          0  (omitted)
                       11  |          0  (omitted)
                       12  |          0  (omitted)
                       13  |          0  (omitted)
                       14  |          0  (omitted)
                       17  |          0  (omitted)
                       18  |          0  (omitted)
                       19  |          0  (omitted)
                       20  |          0  (omitted)
                       21  |          0  (omitted)
                       22  |          0  (omitted)
                       24  |          0  (omitted)
                       29  |          0  (omitted)
                       31  |          0  (omitted)
                       34  |          0  (omitted)
                       36  |          0  (omitted)
                       37  |          0  (omitted)
                       38  |          0  (omitted)
                       39  |          0  (omitted)
                       41  |          0  (omitted)
                       42  |          0  (omitted)
                       47  |          0  (omitted)
                       50  |          0  (omitted)
                       51  |          0  (omitted)
                       54  |          0  (omitted)
                       55  |          0  (omitted)
                       57  |          0  (omitted)
                       58  |          0  (omitted)
                       59  |          0  (omitted)
                       62  |          0  (omitted)
                       64  |          0  (omitted)
                       65  |          0  (omitted)
                       66  |          0  (omitted)
                       67  |          0  (omitted)
                       68  |          0  (omitted)
                       70  |          0  (omitted)
                       71  |          0  (omitted)
                       72  |          0  (omitted)
                       73  |          0  (omitted)
                       74  |          0  (omitted)
                       75  |          0  (omitted)
                       76  |          0  (omitted)
                       77  |          0  (omitted)
                       78  |          0  (omitted)
                       79  |          0  (omitted)
                       81  |          0  (omitted)
                       83  |          0  (omitted)
                       85  |          0  (omitted)
                       88  |          0  (omitted)
                       89  |          0  (omitted)
                       90  |          0  (omitted)
                       92  |          0  (omitted)
                       94  |          0  (omitted)
                       95  |          0  (omitted)
                       97  |          0  (omitted)
                       98  |          0  (omitted)
                       99  |          0  (omitted)
                      100  |          0  (omitted)
                      102  |          0  (omitted)
                      104  |          0  (omitted)
                      105  |          0  (omitted)
                      106  |          0  (omitted)
                      108  |          0  (omitted)
                      109  |          0  (omitted)
                      110  |          0  (omitted)
                      111  |          0  (omitted)
                      112  |          0  (omitted)
                      114  |          0  (omitted)
                      115  |          0  (omitted)
                      116  |          0  (omitted)
                      117  |          0  (omitted)
                      118  |          0  (omitted)
                      119  |          0  (omitted)
                      121  |          0  (omitted)
                      122  |          0  (omitted)
                      123  |          0  (omitted)
                      124  |          0  (omitted)
                      125  |          0  (omitted)
                      126  |          0  (omitted)
                      128  |          0  (omitted)
                      129  |          0  (omitted)
                      133  |          0  (omitted)
                      134  |          0  (omitted)
                      135  |          0  (omitted)
                      136  |          0  (omitted)
                      137  |          0  (omitted)
                      141  |          0  (omitted)
                      142  |          0  (omitted)
                      145  |          0  (omitted)
                      146  |          0  (omitted)
                      147  |          0  (omitted)
                      150  |          0  (omitted)
                      151  |          0  (omitted)
                      153  |          0  (omitted)
                      154  |          0  (omitted)
                      156  |          0  (omitted)
                      158  |          0  (omitted)
                           |
                     _cons |  -9.981374   3.972446    -2.51   0.012    -17.78446   -2.178288
      ---------------------+----------------------------------------------------------------
                   sigma_u |  4.0810492
                   sigma_e |  .99374176
                       rho |  .94402587   (fraction of variance due to u_i)
      --------------------------------------------------------------------------------------
      F test that all u_i=0: F(102, 548) = 51.29                   Prob > F = 0.0000

      and hasuman result as :

      Code:
      hausman fe re
      
      Note: the rank of the differenced variance matrix (16) does not equal the number of
              coefficients being tested (17); be sure this is what you expect, or there may be
              problems computing the test.  Examine the output of your estimators for anything
              unexpected and possibly consider scaling your variables so that the coefficients are on
              a similar scale.
      
                       ---- Coefficients ----
                   |      (b)          (B)            (b-B)     sqrt(diag(V_b-V_B))
                   |       fe           re         Difference          S.E.
      -------------+----------------------------------------------------------------
      totalterro~s |   -.0003332    -.0003332        5.42e-16               .
      ControlofC~n |    .3944104     .3944104        4.29e-12               .
      SavingsCur~g |    .5122627     .5122627       -8.89e-12               .
      Inflationwin |    .0189139     .0189139       -9.38e-14               .
      creditwins~g |     .006928      .006928       -1.37e-12               .
              Year |
             2007  |    .4605182     .4605182        1.79e-12               .
             2008  |   -.0723403    -.0723403        2.55e-12               .
             2009  |   -.0857283    -.0857283        8.46e-13               .
             2010  |    -.203764     -.203764        2.42e-12               .
             2011  |   -.1459416    -.1459416        4.21e-12               .
             2012  |    .1909434     .1909434        4.08e-12               .
             2013  |    .1695522     .1695522        4.15e-12               .
             2014  |    .2844599     .2844599        4.41e-12               .
             2015  |    .5150472     .5150472        3.34e-12               .
             2016  |    .7817107     .7817107        3.24e-12               .
             2017  |    .8225087     .8225087        4.18e-12               .
             2018  |    .9736898     .9736898        4.95e-12               .
      ------------------------------------------------------------------------------
                                 b = consistent under Ho and Ha; obtained from xtreg
                  B = inconsistent under Ha, efficient under Ho; obtained from xtreg
      
          Test:  Ho:  difference in coefficients not systematic
      
                       chi2(16) = (b-B)'[(V_b-V_B)^(-1)](b-B)
                                =    -0.00    chi2<0 ==> model fitted on these
                                              data fails to meet the asymptotic
                                              assumptions of the Hausman test;
                                              see suest for a generalized test


      I didn't include RE results because otherwise it would have been too much tables.

      Also, I have noticed that if I add GDP into the equation I get similar haumsan results. I know the hausman test indicates that scaling of the variables. Is there any good read on rescaling of the variables, as in the literature, people have done various technqiues (most probably ) to get the right results. What I mean is that I have seen researchers changing the variable scale in different ways so there is no one way to rescale the variable. I will like to hear your feedback on this.

      Also, I know that r-square value is quite low .. I am aware of this. Is that responsible for any of the issues I am having?
      Last edited by Abdullah Ijaz; 11 Jun 2020, 06:39.

      Comment


      • #4
        Abdullah:
        the main issue here relates to the inclusion of the fixed effect as apredictor in your second panel data regression code: as it is obviously collinear with the fixed effect itself, all the coefficients of -countrynum- are omitted.
        Hence, stick with your first -xtreg- code (adding -i.year- in the set of predictors), but control whether the model is misspecified and/or you have little within panel variation in time-varying predictors.
        Besides, check for possible singletons, that can explain -hausman- results.
        Kind regards,
        Carlo
        (StataNow 18.5)

        Comment


        • #5
          Carlo: Many thanks for your valuable suggestion. It really means a lot.

          Is there any code for detecting singletons? I tried finding it online but I just wanted to be sure.

          Looking forward to your reply¬!

          Comment


          • #6
            The problem is that you included i.countrynum in both the FE and RE estimation, which simply does FE twice. Notice the 15+ RE and FE estimates are numerically identical to all reported decimal places. That's as close to a non-formal proof that the two estimators are identical than one can every hope for.

            FE estimation already takes care of the country fixed effects -- that's why they drop out. If you add country FEs to RE estimation, you're just doing FE.

            I wouldn't recommend the nonrobust Hausman test, anyway. Do a search on xtoverid after RE estimation.

            Code:
            xtreg y x1 ... xK i.year, re vce(cluster countrynum)
            xtoverid
            Or, you can use the Mundlak test that adds the time averages to the RE estimation.

            JW

            Comment


            • #7
              Dear Both,

              Thanks for your valuable comments. Much appreciated.

              Comment


              • #8
                Originally posted by Jeff Wooldridge View Post
                The problem is that you included i.countrynum in both the FE and RE estimation, which simply does FE twice. Notice the 15+ RE and FE estimates are numerically identical to all reported decimal places. That's as close to a non-formal proof that the two estimators are identical than one can every hope for.

                FE estimation already takes care of the country fixed effects -- that's why they drop out. If you add country FEs to RE estimation, you're just doing FE.

                I wouldn't recommend the nonrobust Hausman test, anyway. Do a search on xtoverid after RE estimation.

                Code:
                xtreg y x1 ... xK i.year, re vce(cluster countrynum)
                xtoverid
                Or, you can use the Mundlak test that adds the time averages to the RE estimation.

                JW
                Dear Jeff Wooldridge

                When I use this estimation, an error comes up

                Code:
                Code:
                xtoverid
                2006b:  operator invalid
                r(198);
                However, when I exclude i.Year , it works well. What am I doing wrong?

                Looking forward to your reply.

                Comment

                Working...
                X