Low R-Squared on Random Effects

[Arnola Putri]

Hi there, i would like to ask for opinions regarding low r-squared on a random effect model. I have a small panel data consist of 34 N and 8 T, with 1 dependent variable. and 7 independet variables. I use panel regression and found that the best model to use is the RE model. the result shows the R2 of my model is low, or 21,17%, but i have 4 independent variable that is significant.

my questions are:
- is this okay if i simply want to see the independent variables relation with the dependent variable?
- is there any published book about low r-squared model in panel regression that i can read?

below is my command:

 
 *run PLS FE RE reg lnCHL lnGOV TPT AMH lnPENG APK POV Q
xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, fe
xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, re

*restricted F-Test
reg lnCHL lnGOV TPT AMH lnPENG APK POV Q i.Provinsi
testparm i.Provinsi

*hausman
xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, fe
estimates store FEM
xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, re
estimates store REM
hausman FEM REM

*LM
xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, re
xttest0

xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, re

and below is my dataset

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(lnCHL lnGOV) double(TPT AMH) float lnPENG double(APK POV Q)
 9.007979  10.04755  9.06 96.99 15.762734  96.47 19.46                  .
        .  10.05314 10.12 97.04 15.834216  94.39  17.6  9.334591481469431
 9.100414 10.048844  9.02 97.42 15.914535  95.87 18.05  8.591249436175012
 8.860215  10.02235  9.93 97.63 16.015554  97.88 17.08  9.054137238385279
 8.977146 10.036062  7.57 97.74  16.08734  99.15 16.73  8.687864534336782
 9.796014 10.058905  6.57 97.94 16.198378  98.74 16.89  8.439582012020175
 9.687258 10.086384  6.34 98.03 16.273886   99.3 15.97  9.101293669556082
 9.006142 10.120303  6.17 98.21 16.293829  97.42 15.32  9.412389151401385
11.650127 10.241276  6.28 97.51 15.788024  88.59 10.67                  .
        .  10.28668  6.45 97.84 15.879025  86.84 10.06  9.667386055545814
11.314242  10.32473  6.23 98.57 15.942695  90.29  9.38 13.080245570407211
11.224936 10.362096  6.71 98.68 16.045769  95.41 10.53 16.204501202611386
11.760036 10.400775  5.84 98.88 16.142307  90.71 10.35   16.0126731941622
 11.68109   10.4395   5.6 98.89 16.205906  90.74 10.22 15.551748226696281
11.730268 10.479272  5.55 99.07 16.302126  89.98  9.22  15.27577219050321
11.685534 10.514709  5.39 99.15  16.36242  90.38  8.83 15.031404930420214
 9.672815 10.075086  6.65 97.23   15.9168  87.81  8.19  8.498373857828714
        .  10.12092  7.02 97.38 16.023094  85.46  8.14  8.281077713751753
 9.480978 10.165192   6.5 98.44 16.093369  88.05  7.41  8.545340426803755
 9.616472  10.20658  6.89 98.56 16.189154  90.94  7.31 10.787243120244614
 9.858751 10.245832  5.09 98.81  16.28533  91.05  7.09  10.38040509382427
10.115328 10.285758  5.58 98.85 16.352823   90.4  6.87   11.7573090345095
10.038717 10.324524  5.66 99.07 16.438988  90.92  6.65 11.450883312703949
10.078113 10.355432  5.38 99.17 16.453033  91.41  6.42  11.91703975816018
 9.729015  11.18991  4.37 98.45 16.121948   93.3  8.22                  .
        . 11.188539  5.48 98.48 16.172358  88.49  7.72                  .
 9.399555 11.189836  6.56 98.75 16.211702  90.79  8.12                  .
 9.714262 11.167188  7.83 98.87 16.305912  94.14  8.42                  .
 9.486531 11.164351  7.43 99.07  16.38178  93.84  7.98  9.808383910732461
  9.52208 11.166773  6.22 99.17 16.414583   92.4  7.78 13.305498294666616
 9.824336  11.16672  5.98  99.2 16.472496  94.18  7.39 12.711435978683566
 9.727645 11.191468  5.76 99.21 16.510067  93.95  7.08 12.578097639560132
 8.950922  10.38646   3.2  96.2 15.827815  88.11  8.42                  .
        . 10.434472  4.76 96.85  15.91829  85.17  8.07 11.493160813308688
 8.563122 10.487882  5.08 97.77 15.973303  87.83  7.92 10.759424769840736
 8.881558  10.51199  4.34 97.84 16.126892  91.78  8.86 10.141952239184139
 9.129456  10.53818     4 98.01  16.19981  90.75  8.41 10.219508057675997
 9.157256 10.567048  3.87 98.09 16.269157  86.37  8.19  12.14764486836088
 9.236982 10.597273  3.73 98.15 16.351511  87.55  7.92 11.875694666240818
 8.900276 10.640947  4.06  98.2 16.367128  87.31   7.6                  .
 9.958591 10.260388  5.66  97.5 15.786356  86.65 13.78                  .
        . 10.297445  4.84 97.62 15.859323  86.07 14.24                  .
 9.638285  10.32994  4.96 98.14 15.986528  88.43 13.91                  .
 9.317939 10.359306  6.07 98.22 15.987662  93.56 14.25 11.122488884127069
   9.7029 10.395115  4.31 98.46 16.125721  88.54 13.54 13.282928942807626
10.124108  10.43587  4.39 98.54  16.22229  89.09 13.19 13.454241211059678
 9.985344  10.48178  4.27 98.66 16.270039  86.51  12.8 14.011580055507704
 9.822331 10.522066  4.53 98.76 16.258364  86.97 12.71 13.260710616955707
 8.680841  9.806068  3.62 95.69 15.730476  95.93  17.7                  .
        .  9.847938  4.61 96.55  15.87646     85 18.34 12.064250614250614
 8.636043  9.884647  3.47 97.52 15.952038  88.23 17.48 18.118347895154884
 8.003029  9.918498  4.91 97.63 16.091024  88.79 17.88 25.382478879759287
 8.785692  9.954173   3.3 97.75 16.189255  90.38 17.32 11.484180035650624
 8.390722  9.987445  3.74  97.9 16.291285  90.52 16.45 12.301363366923534
 8.856519  10.02104  3.35 97.91 16.402773  92.08 15.43  12.52691116932603
 8.873188 10.064948  3.26 98.01 16.392403   89.5 15.23 11.808361204013378
10.561604  9.989429   5.2 95.13 15.642077  93.41 16.18 12.384853487412299
        .  10.03323  5.69 95.92 15.744654  85.47 14.86 11.716725627790987
 9.914279 10.071003  4.79 96.54 15.836014  86.76 14.28  10.91541477018601
  9.92818  10.10976  5.14 96.67  16.00094 100.83 14.35  10.51475614319571
  10.1367  10.14912  4.62 96.78 16.063505  93.58 14.29 12.530194564087331
10.042945 10.189226  4.33 96.89 16.137066  91.04 13.69 13.713367018833257
10.471128 10.230495  4.04 96.93 16.219898  94.18 13.14 13.104433962264151
10.227526 10.271406  4.03 97.11 16.226797  91.11 12.62 12.935907125797382
 8.497807  10.34729  3.43 95.88 16.100376  78.35  5.53 14.874682257244535
        .  10.37603  3.65 96.44  16.23825  73.38  5.21 15.585264254100494
  8.16109      10.4  5.14  97.6 16.347025  82.52  5.36  16.08360049321825
 8.158802 10.418715  6.29 97.63 16.412048   87.6   5.4 17.575134857436424
 8.406261 10.438016   2.6 97.66  16.49259  84.38  5.22  17.17645569620253
 8.563122 10.461202  3.78 97.79 16.608633  84.53   5.2 18.328905419766205
 8.515191 10.484642  3.61 97.76 16.650719  85.93  5.25 17.304216107043676
 8.078068 10.523342  3.58 98.09  16.71251  85.34  4.62 16.178430002329375
 8.074961  11.16945  5.08  97.8 16.298208  92.45  7.11 29.717004048582996
        . 11.208345  5.63 98.07 16.395967  90.21  6.46  16.74258600237248
 7.625107  11.24261  6.69 98.71 16.540663  91.06   6.7  15.31386046511628
 7.398786  11.27245   6.2 98.79 16.596598  92.38  6.24  14.96859860039476
 7.237059  11.29347  7.69 98.84 16.682356   90.4  5.98 12.403144864503178
 8.247482 11.286572  7.16 98.83 16.748224  90.45  6.06 17.014280614066404
 8.380457 11.304747  8.04 98.87 16.754286  92.44   6.2 16.627215937869323
 7.112328 11.303913   7.5    99  16.87599  92.82   5.9   15.3680649526387
 8.874308 11.727734  9.67 99.21   16.6391  94.58  3.69 13.980753696484388
        . 11.775754  8.63 99.22 16.724657  86.35  3.55 13.399066587786761
  8.78554 11.822704  8.47 99.54 16.835901  90.86  3.92 14.884636755104875
 8.848797 11.869996  7.23 99.59 16.873333  88.35  3.93                  .
 8.646642  11.91727  6.12 99.64 16.929905  90.89  3.75                  .
 9.028938 11.968048  7.14 99.67 16.992287  93.88  3.77 18.327396724729596
 8.113726  12.01835  6.65 99.72 17.012953  94.91  3.57                  .
 7.900637  12.07147  6.54 99.74 17.068724  91.02  3.47  17.84173198954834
10.849046 10.044813  9.08 96.39  15.87121  87.44 10.09                  .
        . 10.090727  9.16 96.87 15.981352  85.26  9.52                  .
10.675954 10.125304  8.45 97.96 16.069513   87.5  9.44                  .
 9.972501 10.159892  8.72 98.01   16.1916  90.07  9.53                  .
 10.16854 10.200755  8.89 98.22 16.284163  89.58  8.95  17.96664131662692
11.484166  10.23892  8.22 98.23 16.398756   88.8  8.71 18.995102105304735
11.291705 10.280555  8.23 98.48 16.497692  90.96  7.45 18.101378373427767
 10.85975 10.322638  8.04 98.53 16.536972  90.75  6.91 17.696342812182213
11.287704  9.949924  5.61 90.45   15.6117  91.51 15.34 14.378446778303925
        .  9.991721  6.01 91.71 15.720086  87.42 14.56 14.673779667854502
10.900842 10.035356  5.68 92.98 15.832085   89.4 14.46 14.246012005438928
10.429428 10.081092  4.99 93.12 15.937804   91.4 13.58 15.048319152678895
end

thank you

[Carlo Lazzaro]

Arnola:
please report the complete Stata outcome table that you got after typing -xtreg,re-. Thanks.

[Maxence Morlet]

There isn't such a thing as a threshold for an "appropriate" R-squared. It really depends on your field, your model, expectations, etc.

As an aside, the below is only relevant if you're interested in causal inference.

Also, I don't know your field, but all I know is that in economics, I've never seen results from a random-effects model reported in a paper even if the Hausman test fails to reject its null hypothesis, as economists are absolutely obsessed with causality (which to be fair, makes sense). The assumption made by random-effects that the unobserved heterogeneity is uncorrelated with the regressors is simply implausible generally, impeding causal inference.

Two-way fixed effects (with time and unit) is generally what economists tend to resort to by default (see de Chaisemartin et d'Haultfoeuille, 2020). The Hausman test also makes unrealistic assumptions, such as homoscedasticity, which rarely holds in reality.

[Carlo Lazzaro]

Arnola:
the interesting paper Maxence pointed out us to can be downloaded free of charge in its working-paper format from https://arxiv.org/pdf/1803.08807.pdf.

[Arnola Putri]

Carlo Lazzaro here is the result after -xtreg, re-

Random-effects GLS regression                   Number of obs     =        196
Group variable: Provinsi                        Number of groups  =         34

R-sq:                                           Obs per group:
     within  = 0.1983                                         min =          4
     between = 0.1979                                         avg =        5.8
     overall = 0.2117                                         max =          7

                                                Wald chi2(7)      =      41.59
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.0000

------------------------------------------------------------------------------
       lnCHL |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       lnGOV |   -.665847    .265709    -2.51   0.012    -1.186627    -.145067
         TPT |  -.0867136   .0302675    -2.86   0.004    -.1460368   -.0273904
         AMH |  -.1133696   .0252347    -4.49   0.000    -.1628287   -.0639105
      lnPENG |   .5164464   .2050701     2.52   0.012     .1145164    .9183764
         APK |   .0130344   .0095322     1.37   0.171    -.0056484    .0317171
         POV |  -.0238592   .0224302    -1.06   0.287    -.0678215    .0201032
           Q |  -.0099524   .0088758    -1.12   0.262    -.0273485    .0074438
       _cons |   18.54711   3.010631     6.16   0.000     12.64638    24.44784
-------------+----------------------------------------------------------------
     sigma_u |  .81074479
     sigma_e |  .28094152
         rho |  .89279495   (fraction of variance due to u_i)
------------------------------------------------------------------------------

[Carlo Lazzaro]

Arnola:
your within and between R_sq are really similar.

Have you already checked whether the functional form of your regressand is correctly specified?

[Arnola Putri]

thankyou for your response Maxence Morlet , i will look into it

[Arnola Putri]

Carlo Lazzaro how to check the functional form? sorry i'm not really good at econometrics

[Maxence Morlet]

If my memory serves, with a Ramsey RESET test explained here (partially): http://www.personal.rhul.ac.uk/uhte/...le%20Tests.pdf

[Carlo Lazzaro]

Arnola:
the procedure is identical to the one reported under -linktest- (that unfortunately cannot be invoked after -xtreg-):

 
 xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q, re predict fitted, xb g sq_fitted=fitted^2 xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q fitted fitted^2, re test sq_fitted

If -test- outcome reaches statistical significance, the functional form of the regressand is ill-specified.

[Maxence Morlet]

Arnola Putri definitely follow Carlo's advice here, it's been a while since checked for functional form misspecification so I'm a bit rusty in that field

[Arnola Putri]

Carlo Lazzaro thank you! here is the result from the command you gave:

. xtreg lnCHL lnGOV TPT AMH lnPENG APK POV Q fitted sq_fitted, re
note: fitted omitted because of collinearity

Random-effects GLS regression                   Number of obs     =        196
Group variable: Provinsi                        Number of groups  =         34

R-sq:                                           Obs per group:
     within  = 0.1989                                         min =          4
     between = 0.2005                                         avg =        5.8
     overall = 0.2144                                         max =          7

                                                Wald chi2(8)      =      41.49
corr(u_i, X)   = 0 (assumed)                    Prob > chi2       =     0.0000

------------------------------------------------------------------------------
       lnCHL |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       lnGOV |  -1.171963   1.960384    -0.60   0.550    -5.014246    2.670319
         TPT |  -.1522341   .2530583    -0.60   0.547    -.6482193    .3437512
         AMH |  -.2083263   .3642934    -0.57   0.567    -.9223282    .5056755
      lnPENG |   .9320917   1.597838     0.58   0.560    -2.199613    4.063796
         APK |   .0230522   .0397442     0.58   0.562     -.054845    .1009493
         POV |  -.0427268   .0772044    -0.55   0.580    -.1940447    .1085911
           Q |  -.0179994   .0321761    -0.56   0.576    -.0810633    .0450645
      fitted |          0  (omitted)
   sq_fitted |    -.04101   .1577999    -0.26   0.795    -.3502922    .2682721
       _cons |   29.60996   42.67639     0.69   0.488    -54.03423    113.2541
-------------+----------------------------------------------------------------
     sigma_u |   .8246315
     sigma_e |  .28184848
         rho |  .89540066   (fraction of variance due to u_i)
------------------------------------------------------------------------------

. test sq_fitted

 ( 1)  sq_fitted = 0

           chi2(  1) =    0.07
         Prob > chi2 =    0.7950

does this mean that the functional form of the regressand is ill-specified?

[Carlo Lazzaro]

Arnola:
no, the opposite holds.
Be happy with your model in #5.

[Arnola Putri]

Carlo Lazzaro thank you so much!