Missing F-statistic when using xtreg with fe, vce(cluster) and adding time-fixed effects

[Manni Hitzel]

Dear Statalist user,

I'm having an unbalanced panel-dataset and want to do a fixed-effects-estimation. I cluster SE at the country-level and also include quarterly time-fixed effects.
However, when adding time fixed effects, my F-statistic is not reported anymore. I checked the help-file, but do not really see why the F-statistic cannot be calculated.
My regression looks the following way:

xtreg ln_fdi_l elect_d L1.elect_d  F1.elect_d L2.elect_d  F2.elect_d ///
data497 data498 data658 data1120 i.q_date , fe vce(cluster  ccode)

The resulting output is as follows:

Fixed-effects (within) regression               Number of obs     =      9,646
Group variable: ccode                           Number of groups  =        119

R-sq:                                           Obs per group:
     within  = 0.2433                                         min =          9
     between = 0.4924                                         avg =       81.1
     overall = 0.2709                                         max =        156

                                                F(118,118)        =          .
corr(u_i, Xb)  = -0.7879                        Prob > F          =          .

                                (Std. Err. adjusted for 119 clusters in ccode)
------------------------------------------------------------------------------
             |               Robust
    ln_fdi_l |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     elect_d |
         --. |   .0114366   .0241773     0.47   0.637     -.036441    .0593142
         L1. |  -.0165092   .0208586    -0.79   0.430    -.0578149    .0247964
         F1. |   .0205511    .026126     0.79   0.433    -.0311854    .0722877
         L2. |  -.0352879   .0272838    -1.29   0.198    -.0893172    .0187415
         F2. |   .0347876   .0277045     1.26   0.212    -.0200748      .08965
             |
     data497 |   .0073437   .0022485     3.27   0.001     .0028912    .0117963
     data498 |   .0000386   .0000151     2.56   0.012     8.75e-06    .0000684
     data658 |  -.0001534   .0001051    -1.46   0.147    -.0003614    .0000547
    data1120 |   7.05e-09   1.94e-09     3.64   0.000     3.21e-09    1.09e-08
             |
      q_date |
         63  |  -.0242295   .0206485    -1.17   0.243    -.0651191    .0166601
         64  |  -.0786142   .0343174    -2.29   0.024     -.146572   -.0106565
         65  |  -.1319244   .0485814    -2.72   0.008    -.2281288     -.03572
         66  |   .0243135   .0730829     0.33   0.740    -.1204107    .1690376
         67  |   .0385069   .0716588     0.54   0.592     -.103397    .1804108
         68  |   .0375118   .0609079     0.62   0.539    -.0831024     .158126
         69  |   .0467123   .0629826     0.74   0.460    -.0780103     .171435
         70  |   .0898097   .0652482     1.38   0.171    -.0393994    .2190189
         71  |     .09821   .0706432     1.39   0.167    -.0416828    .2381027
         72  |    .069811   .0652464     1.07   0.287    -.0593946    .1990167
         73  |   .0741895   .0775977     0.96   0.341    -.0794751    .2278541
         74  |   .0921752   .0701882     1.31   0.192    -.0468166    .2311669
         75  |    .084116   .0775115     1.09   0.280    -.0693778    .2376098
         76  |   .0334981    .079253     0.42   0.673    -.1234445    .1904406
         77  |   .0818282   .0794639     1.03   0.305     -.075532    .2391885
         78  |   .0838553    .080813     1.04   0.302    -.0761764     .243887
         79  |   .0870894   .0789471     1.10   0.272    -.0692474    .2434261
         80  |   .0879042    .082177     1.07   0.287    -.0748287    .2506371
         81  |   .0897231   .0916741     0.98   0.330    -.0918165    .2712627
         82  |   .0826496   .1000497     0.83   0.410     -.115476    .2807751
         83  |   .0820208   .0789809     1.04   0.301    -.0743828    .2384244
         84  |   .0936449   .0913912     1.02   0.308    -.0873346    .2746245
         85  |   .0679835   .1159625     0.59   0.559    -.1616538    .2976208
         86  |   .0978981     .10179     0.96   0.338    -.1036739    .2994701
         87  |   .1420955   .1039156     1.37   0.174    -.0636858    .3478768
         88  |   .0114938   .1067359     0.11   0.914    -.1998723    .2228599
         89  |    .097311   .1029148     0.95   0.346    -.1064883    .3011103
         90  |   .0990987   .0932644     1.06   0.290    -.0855903    .2837876
         91  |   .0638884   .1024925     0.62   0.534    -.1390746    .2668514
         92  |   .0400605    .084723     0.47   0.637    -.1277141    .2078351
         93  |   .1125806   .0918243     1.23   0.223    -.0692564    .2944177
         94  |   .0569003   .0938625     0.61   0.546    -.1289731    .2427737
         95  |   .0700592   .0853321     0.82   0.413    -.0989215      .23904
         96  |   .0763071   .1020234     0.75   0.456     -.125727    .2783412
         97  |   .0032573   .1373814     0.02   0.981    -.2687954    .2753099
         98  |   .0697331   .1006435     0.69   0.490    -.1295685    .2690347
         99  |  -.0226479   .0846387    -0.27   0.789    -.1902556    .1449598
        100  |   .0272374    .093552     0.29   0.771    -.1580209    .2124958
        101  |   .0071744   .0913672     0.08   0.938    -.1737575    .1881062
        102  |   -.000967   .1038971    -0.01   0.993    -.2067115    .2047776
        103  |  -.0061613   .0814355    -0.08   0.940    -.1674257    .1551031
        104  |   .0524127   .0873148     0.60   0.549    -.1204943    .2253198
        105  |   .0838599   .1047117     0.80   0.425    -.1234977    .2912176
        106  |     .06797   .1072802     0.63   0.528     -.144474     .280414
        107  |   .0677287   .1341325     0.50   0.615    -.1978903    .3333476
        108  |    .133141   .1051835     1.27   0.208    -.0751511     .341433
        109  |   .0354388    .111138     0.32   0.750    -.1846447    .2555224
        110  |   .1360479   .1172748     1.16   0.248     -.096188    .3682839
        111  |   .1887732   .1130257     1.67   0.098    -.0350485    .4125949
        112  |   .1531871   .1133764     1.35   0.179    -.0713291    .3777032
        113  |   .1606089   .1233376     1.30   0.195     -.083633    .4048509
        114  |    .231678    .112153     2.07   0.041     .0095845    .4537715
        115  |   .2179022   .1367173     1.59   0.114    -.0528352    .4886396
        116  |   .1841801   .1404581     1.31   0.192    -.0939651    .4623253
        117  |   .2593443   .1269119     2.04   0.043     .0080241    .5106644
        118  |    .191552   .1295918     1.48   0.142    -.0650751    .4481791
        119  |   .3418829   .1222609     2.80   0.006     .0997729    .5839928
        120  |   .2706548   .1344509     2.01   0.046     .0044054    .5369042
        121  |   .2949105   .1300078     2.27   0.025     .0374597    .5523613
        122  |   .2759598   .1361354     2.03   0.045     .0063748    .5455449
        123  |    .365766   .1274778     2.87   0.005     .1133253    .6182067
        124  |   .1964606   .1342924     1.46   0.146    -.0694749    .4623962
        125  |   .1733727   .1385654     1.25   0.213    -.1010246    .4477699
        126  |   .2674171   .1190133     2.25   0.027     .0317383    .5030959
        127  |   .1504222   .1129293     1.33   0.185    -.0732085    .3740529
        128  |   .2013811   .1138394     1.77   0.079    -.0240519    .4268141
        129  |   .1925026   .1241354     1.55   0.124    -.0533192    .4383245
        130  |   .1992236   .1209873     1.65   0.102    -.0403643    .4388115
        131  |   .1586782   .1302114     1.22   0.225    -.0991758    .4165322
        132  |   .2130356    .124794     1.71   0.090    -.0340904    .4601616
        133  |   .2338484    .129068     1.81   0.073    -.0217414    .4894382
        134  |   .2428987   .1344151     1.81   0.073    -.0232798    .5090772
        135  |   .2116095   .1208867     1.75   0.083    -.0277791    .4509981
        136  |   .2664201   .1259652     2.12   0.037     .0169746    .5158656
        137  |   .2507599   .1370196     1.83   0.070    -.0205761    .5220959
        138  |     .26064   .1418966     1.84   0.069    -.0203539    .5416338
        139  |    .312543   .1319993     2.37   0.020     .0511484    .5739376
        140  |   .3261879   .1332058     2.45   0.016     .0624042    .5899716
        141  |   .2716943   .1424145     1.91   0.059    -.0103251    .5537138
        142  |   .3470382    .132471     2.62   0.010     .0847096    .6093668
        143  |   .4823822   .1379796     3.50   0.001     .2091451    .7556193
        144  |   .3124669   .1494053     2.09   0.039     .0166038    .6083301
        145  |   .3040236   .1411083     2.15   0.033     .0245908    .5834564
        146  |   .3725079   .1396533     2.67   0.009     .0959563    .6490596
        147  |   .4073707   .1494441     2.73   0.007     .1114307    .7033106
        148  |    .373385   .1445251     2.58   0.011     .0871858    .6595841
        149  |   .3993425    .146637     2.72   0.007     .1089613    .6897238
        150  |   .4483316   .1464479     3.06   0.003     .1583248    .7383383
        151  |   .4623383   .1594198     2.90   0.004     .1466437    .7780329
        152  |   .4193369   .1532172     2.74   0.007     .1159251    .7227488
        153  |   .5061071   .1607929     3.15   0.002     .1876935    .8245208
        154  |   .4078365    .160625     2.54   0.012     .0897552    .7259178
        155  |   .5285764   .1658856     3.19   0.002     .2000777    .8570751
        156  |    .459087   .1571823     2.92   0.004     .1478232    .7703508
        157  |    .602488   .1847386     3.26   0.001     .2366553    .9683206
        158  |   .4693611   .1569211     2.99   0.003     .1586145    .7801076
        159  |   .5876982   .1653972     3.55   0.001     .2601668    .9152296
        160  |   .5207756   .1693356     3.08   0.003      .185445    .8561062
        161  |   .5022064   .1776411     2.83   0.006     .1504286    .8539841
        162  |    .539381   .1737443     3.10   0.002       .19532    .8834419
        163  |   .5072167   .1790784     2.83   0.005     .1525927    .8618407
        164  |   .4608808   .1651695     2.79   0.006     .1338003    .7879613
        165  |   .4455947   .1755366     2.54   0.012     .0979844     .793205
        166  |   .3788913   .1563461     2.42   0.017     .0692835    .6884991
        167  |   .4239653   .1587057     2.67   0.009     .1096848    .7382458
        168  |   .3904816   .1644056     2.38   0.019     .0649137    .7160494
        169  |   .4128752   .1702503     2.43   0.017     .0757333    .7500171
        170  |   .3603665     .16198     2.22   0.028      .039602    .6811311
        171  |   .4887751   .1669516     2.93   0.004     .1581655    .8193847
        172  |   .4118723   .1663738     2.48   0.015     .0824069    .7413378
        173  |   .2929846   .1730262     1.69   0.093    -.0496544    .6356236
        174  |   .3336438    .161609     2.06   0.041     .0136139    .6536737
        175  |   .3632253   .1871201     1.94   0.055    -.0073235    .7337741
        176  |   .3305094   .1843405     1.79   0.076     -.034535    .6955538
        177  |   .3699709   .1881072     1.97   0.052    -.0025325    .7424744
        178  |     .40748   .1630304     2.50   0.014     .0846354    .7303247
        179  |   .5148226    .176015     2.92   0.004      .166265    .8633802
        180  |   .5284791   .1617879     3.27   0.001     .2080951     .848863
        181  |   .4767073   .1905825     2.50   0.014      .099302    .8541125
        182  |   .5958493   .1786448     3.34   0.001      .242084    .9496146
        183  |    .628413   .1931406     3.25   0.001      .245942    1.010884
        184  |   .6689007   .1829918     3.66   0.000     .3065271    1.031274
        185  |   .5422775    .189944     2.85   0.005     .1661367    .9184182
        186  |   .6869088   .1818347     3.78   0.000     .3268266    1.046991
        187  |   .6781345    .204431     3.32   0.001     .2733055    1.082963
        188  |    .815548   .1936808     4.21   0.000     .4320073    1.199089
        189  |   .7831085   .2019487     3.88   0.000     .3831952    1.183022
        190  |   .7627859   .1741594     4.38   0.000      .417903    1.107669
        191  |   .7991028   .2115317     3.78   0.000     .3802125    1.217993
        192  |   .7879531   .2080489     3.79   0.000     .3759596    1.199947
        193  |   .6997222   .1785461     3.92   0.000     .3461524    1.053292
        194  |   .7038953    .204256     3.45   0.001     .2994127    1.108378
        195  |    .575429   .2371584     2.43   0.017     .1057908    1.045067
        196  |   .5109115   .1784477     2.86   0.005     .1575365    .8642866
        197  |   .4207373   .2252714     1.87   0.064    -.0253613     .866836
        198  |   .5255269   .1799654     2.92   0.004     .1691464    .8819074
        199  |    .578441    .188611     3.07   0.003     .2049399    .9519421
        200  |    .629176   .1941593     3.24   0.002     .2446877    1.013664
        201  |   .4439077   .1844492     2.41   0.018     .0786482    .8091673
        202  |   .5919404   .2030554     2.92   0.004     .1898354    .9940455
        203  |   .5761113   .2327998     2.47   0.015     .1151042    1.037118
        204  |   .6507395   .2083066     3.12   0.002     .2382357    1.063243
        205  |   .6264729   .1910615     3.28   0.001     .2481191    1.004827
        206  |   .5633758   .1819777     3.10   0.002     .2030104    .9237411
        207  |   .6579037   .2084198     3.16   0.002     .2451757    1.070632
        208  |   .7332402    .207785     3.53   0.001     .3217693    1.144711
        209  |    .561871   .1778738     3.16   0.002     .2096325    .9141096
        210  |   .6223397   .1981317     3.14   0.002      .229985    1.014694
        211  |   .5404157   .2274915     2.38   0.019     .0899207    .9909108
        212  |   .4646142   .1821058     2.55   0.012     .1039951    .8252332
        213  |    .491699   .1902489     2.58   0.011     .1149543    .8684437
        214  |   .5222486   .2130461     2.45   0.016     .1003593    .9441378
        215  |     .37337   .2714913     1.38   0.172    -.1642566    .9109967
        216  |   .1944305   .4967526     0.39   0.696    -.7892749    1.178136
        217  |   .6758979   .2383564     2.84   0.005     .2038874    1.147908
             |
       _cons |  -.8194075   .1962445    -4.18   0.000    -1.208025   -.4307899
-------------+----------------------------------------------------------------
     sigma_u |  .93503296
     sigma_e |  .63360056
         rho |  .68531924   (fraction of variance due to u_i)
------------------------------------------------------------------------------

.
end of do-file

[Carlo Lazzaro]

Manni:
see, -help j_robustsingular-.
As an aside, you seem to have far too many predictors vs the number of your clusters.

[Manni Hitzel]

Thanks Carlo,
I already had a look at the help file, but I cannot see why one of the mentioned points is the case in my estimation.
As an explanation, the dataset is a panel on country-level ranging from 1975-2014. It contains quarterly data on country level. Since I want to control for time-fixed effects (as a kind of global shocks) I add dummies for each quarter. I thought this is a common approach and not of a bigger issue, but am grateful for any improvements or hints in this case.

Regards,
Manni

[Clyde Schechter]

As an aside, you seem to have far too many predictors vs the number of your clusters.

Carlo was being too modest. That's not an aside--it's the answer to your question. You have only 119 groups, but you have far more variables in your regression than that. That leaves no degrees of freedom for a model-wide F statistic.

[Carlo Lazzaro]

Thanks, Clyde: very flattering!

[Manni Hitzel]

Thanks for your quick answer. My question is now, whether the coefficients and standard errors for the other explanatory variables are still unbiased or if I must reduce the number of variables to get valid estimations.

[Clyde Schechter]

The notion that linear regression produces unbiased estimates of model parameters if conditional on the model. If confounding variables are not included, then the parameter estimates will suffer from omitted variable bias. Inclusion of superfluous variables in the model will not introduce any bias in its own right, though it may well result in inflated standard errors.

The coefficients of your time variables are actually larger, in general, than the coefficients for your other predictors. Without knowing the scale of the other variables, it is hard to compare all the coefficients. But if your other variables are also 0/1 variables, or continuous variables with a scale of order of magnitude 1, then the time variables actually dominate the regression model. So they could not, in that case, be considered superfluous. This suggests that you are looking for a rather weak signal in the presence of a lot of noise, but it does not argue for removal of any of these variables.

If, on the other hand, your non-time variables have a large scale (say typical values are on the order of 100,000 to take a fairly extreme example), then even though they get multiplied by very small coefficients, they make very large contributions to the regression, and the time variables become negligible contributors. That would argue that the time variables are superfluous.

So it may be worth your while to do a quick estimate of the scale of your variables and their relative contributions to the model.

Anyway, to return to your question in #6, having more predictors than groups does not affect the unbiasedness (or not) of the coefficients conditional on the model, but what variables are included in the model can affect the validity of the model, and can have diverse effects on the standard errors. But in any case, the number of variables exceeding the number of groups and the undefinedness of the omnibus F-statistic have no direct bearing on these issues.

Finally, I neglected to comment in my original post that the F test you are looking for is probably of no real use anyway. If it were defined, it would test the null hypothesis that all model coefficients are zero. Given that most of the model coefficients are time variables, included to control nuisance effects of idiosyncratic period disturbances, the hypothesis of all model coefficients being zero is probably pointless and of no interest to anybody. So you are no worse off for lack of that F statistic. The more interesting "omnibus" test would be the simultaneous test of elect_d, its lags and leads, and perhaps the four data* variables (if those are variables of interest in your analysis as opposed to also being thrown in just to adjust for nuisance effects). You have plenty of degrees of freedom available to run that F-test using the -test- command, and that one would be meaningful.

[Carlo Lazzaro]

Manni:
you can investigate via:

testparm i.qdate

whether it makes sense on not to include them among predictors.
The following example may sound clearer:

. use "http://www.stata-press.com/data/r14/nlswork.dta", clear
(National Longitudinal Survey.  Young Women 14-26 years of age in 1968)

. xtreg ln_wage age i.race i.year, fe
note: 2.race omitted because of collinearity
note: 3.race omitted because of collinearity

Fixed-effects (within) regression               Number of obs     =     28,510
Group variable: idcode                          Number of groups  =      4,710

R-sq:                                           Obs per group:
     within  = 0.1060                                         min =          1
     between = 0.0914                                         avg =        6.1
     overall = 0.0805                                         max =         15

                                                F(15,23785)       =     188.00
corr(u_i, Xb)  = 0.0467                         Prob > F          =     0.0000

------------------------------------------------------------------------------
     ln_wage |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
         age |   .0125992   .0102163     1.23   0.217    -.0074253    .0326238
             |
        race |
      black  |          0  (omitted)
      other  |          0  (omitted)
             |
        year |
         69  |   .0748621   .0159011     4.71   0.000      .043695    .1060292
         70  |   .0478697   .0235673     2.03   0.042     .0016763     .094063
         71  |   .0865577   .0327939     2.64   0.008     .0222795     .150836
         72  |   .0856757   .0424903     2.02   0.044     .0023919    .1689594
         73  |   .0880069    .052344     1.68   0.093    -.0145906    .1906044
         75  |   .0778607   .0720304     1.08   0.280    -.0633235    .2190449
         77  |    .108365   .0922272     1.17   0.240    -.0724063    .2891363
         78  |   .1309518   .1028143     1.27   0.203    -.0705707    .3324743
         80  |   .1142649    .122792     0.93   0.352    -.1264152     .354945
         82  |   .1090451   .1431112     0.76   0.446    -.1714619    .3895522
         83  |   .1211272   .1532018     0.79   0.429    -.1791581    .4214125
         85  |   .1465637   .1736146     0.84   0.399    -.1937321    .4868594
         87  |   .1382642   .1941163     0.71   0.476     -.242216    .5187445
         88  |   .1799741   .2079871     0.87   0.387    -.2276938     .587642
             |
       _cons |   1.203731   .1952306     6.17   0.000     .8210667    1.586396
-------------+----------------------------------------------------------------
     sigma_u |   .4058746
     sigma_e |  .30300411
         rho |  .64212421   (fraction of variance due to u_i)
------------------------------------------------------------------------------
F test that all u_i=0: F(4709, 23785) = 8.80                 Prob > F = 0.0000

. testparm i.year

 ( 1)  69.year = 0
 ( 2)  70.year = 0
 ( 3)  71.year = 0
 ( 4)  72.year = 0
 ( 5)  73.year = 0
 ( 6)  75.year = 0
 ( 7)  77.year = 0
 ( 8)  78.year = 0
 ( 9)  80.year = 0
 (10)  82.year = 0
 (11)  83.year = 0
 (12)  85.year = 0
 (13)  87.year = 0
 (14)  88.year = 0

       F( 14, 23785) =    6.50
            Prob > F =    0.0000

In the example -testparm- results support rejecting the null that, taken years together, the their coefficient equals 0.

[Clyde Schechter]

Yes, except that it seems to me that Manni has something like 150 time variables in his model. With only 119 groups, that F test will fail for the same reason that the omnibus F test failed: exhausted degrees of freedom.

[Carlo Lazzaro]

Clyde:
yes, I suspect the same; I am only curious about the result.

[Manni Hitzel]

Thanks to both of you for your detailled and very helpful answers!
The basic idea of the regression is to see wether elections (coded as dummy==1 in the respective quarters) do influence capital flows (the dependent variable). The four data* variables are simply controls like GDP p.c. or population to account for other factors determining capital flows --> Thanks Clyde for pointing to the differences in size of coefficients, but as the variables have large scale (e.g. population) this should not be a problem as you already mentioned.
The reason for using so many time controls is that I want to rule out that the effects of global shocks are mixed with election induced changes of capital flows within a country. Furthermore I can only control for changes within the receipient country of capital flows and not for the countries abroad. But if I understood you correctly, it is possible to include quarterly time-variables to the regression without issues for my explanatory election dummies.
Furthermore, I did the proposed testparm. It worked out with the following results:

F(118,   118) = 5416.65
Prob > F =    0.0000

But stated that several constraints have been dropped. I furthermore conducted testparm for splitted parts of the sample (by time periods) and always found the H_0 rejected.
Please correct me if I'm wrong, but I think I should keep the time-dummies.

Regards,
Manni

[Carlo Lazzaro]

Manni:
you're welcome, but the main issue remains the same: you have far too many predictors in your regression model.
From a practical standpoint (that is, set aside for a while any remark on the degrees of freedom), I would have hard times in disentagling the contribution of your predictors in explaining the variation of the -depvar- in a paper to be submitted to any decent peer-reviewed journal.

[Manni Hitzel]

My work is quite related to the recently published paper of Julio & Yook: "Policy uncertainty, irreversibility, and cross-border flows of capital" (Journal of International Economics, 2016). The authors have a panel dataset including the period 1994-2010 and 43 countries. They use similar controls like me and additionally use country-fixed effects as well as quarterly time-fixed-effects. Furthermore, they declare their SE to be clustered at country level. I think that this should be quite similar to my approach. Hence, to my understanding they should have the same issues concerning degrees of freedom mentioned by Clyde above - or do I miss something important what makes their regression different from mine?

Regards,
Manni

[Carlo Lazzaro]

Manni:
one approach would be to post the corresponding author directly and ask for clarifications.

[Clyde Schechter]

I'm going to disagree with Carlo's response in #12. You can certainly test the 5 elect* variables, individually and jointly, in your model, and those tests are valid, even though you can't do an omnibus test on the entire model. Too many variables, even when it doesn't exhaust the degrees of freedom, can cause overfitting problems, but in your case we are talking about ~300 explicit variables (+ 119 fixed effects) in ~9,600 observations (regardless of # of groups). While I wouldn't call that a generous observations to variables ratio, it's not so small as to be a serious problem. If everything except those 5 variables is there just to adjust for nuisance variation, I think this is quite acceptable.