Why I got this error "independent variables are collinear with the panel variable id" when running the xtivreg?

[Linghui Han]

When running the code "xtivreg y (x = z) i.year##i.id, fe vce(cluster id)", I got this error "independent variables are collinear with the panel variable id". How to address this issue?

Thanks!

[Andrew Musau]

i.year##i.id

implies

i.year i.id i.id#i.year

Assuming that you xtset your data using id as the panel identifier, the dummies

i.id

are collinear with the fixed effects. Perhaps you want

xtivreg y (x = z) i.year i.year#i.id, fe vce(cluster id)

[Linghui Han]

Thank you very much, Andrew!

I tried
xtivreg y (x = z) i.year i.year#i.id, fe vce(cluster id) and the same error returned.

[Andrew Musau]

Probably the interactions involving i.id are collinear with the fixed effects. Try

ssc install ivreghdfe

then

ivreghdfe y (x=z), absorb(i.id##i.year) vce(cluster id)

[Linghui Han]

Many thanks, Andrew!

I tried but got the following error: "struct ms_vcvorthog undefined
(817 lines skipped)
(error occurred while loading ivreghdfe.ado)
"

[Andrew Musau]

https://github.com/sergiocorreia/ivreghdfe/issues/14 suggests the following resolution:

ssc install ivreghdfe, replace
ssc install ranktest, replace

[Linghui Han]

Sorry, I tried "ivreghdfe pgratio (zipfprov = indshare6) fdipc nightlight popden, absorb(i.id##i.year)" as vce() option is not allowed. But an error "last estimates not found" returned.

[Andrew Musau]

I think it has -cluster()- option instead of -vce()-

ivreghdfe pgratio (zipfprov = indshare6) fdipc nightlight popden, absorb(i.id##i.year) cluster(id)

[Linghui Han]

Thank you very much, Andrew! I tried
ivreghdfe pgratio (zipfprov = indshare6) fdipc nightlight popden, absorb(i.id##i.year) cluster(id) An error "last estimates not found" returned.

[Andrew Musau]

That is unusual. If you can replicate the error with your first 100 observations, copy and paste the result of the following:

preserve
keep in 1/100
*CONFIRM THIS RUNS
ivreghdfe pgratio (zipfprov = indshare6) fdipc nightlight popden, absorb(i.id##i.year) cluster(id) 
*COPY AND PASTE THE RESULT OF THIS
dataex pgratio zipfprov  indshare6 fdipc nightlight popden id year 
restore

[Linghui Han]

ivreghdfe pgratio (zipfprov = indshare6) fdipc nightlight popden, absorb(i.id##i.year) cluster(id) I kept the first 100 observations and tried the above code. The same error "last estimates not found" returned.

[Linghui Han]

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(y x z x1 x2 x3) double(id year)
 4.327499         .           .         .        .  762.2936 11 1995
 6.173452         .           .  19802.11   192254  767.1683 11 1996
 7.391708         .           .  21294.61 183873.5  755.5908 11 1997
 8.644941         .  .006084234 22646.457   197034  759.2468 11 1998
 9.616824 1.0047407  .005588062 27855.924 193901.5  765.9496 11 1999
10.831456  .9761131  .005662846 24459.586 203082.5  826.8844 11 2000
14.964346  .9616128  .009293114 25213.613 227554.5  842.7274 11 2001
  5.30858  .9865574  .015815698   27559.3 239827.5  867.1013 11 2002
  7.93224 1.0532985  .014199456 26851.693   216472  887.4536 11 2003
 9.193118 1.0348686  .021070685 28154.943   243906  909.7557 11 2004
 10.97285  .9838305  .016893324  30510.92 236419.5  937.1763 11 2005
12.899198  .9999651  .017717516 34321.656 247893.5  963.3782 11 2006
19.924044         .           .         .        .  790.2684 12 1995
26.450306         .           . 16952.604   158589   795.302 12 1996
17.408012         .           . 19675.064   152950  799.4966 12 1997
17.102072         . .0010484996  21990.09 160609.5  802.8524 12 1998
14.845325  .9940882 .0024630355 27815.256   155067  804.5302 12 1999
 22.90859  .9854985 .0018687984 27574.584   157507  839.7651 12 2000
19.720493  .9866916 .0007756654  28559.68   175828  842.2819 12 2001
 26.60165  .9953374 .0004801207  32681.14   182034  844.7986 12 2002
 19.58373  1.036732 .0005046906 34634.258   168842  848.4061 12 2003
 26.71493 1.0137659  .004166286 35592.242 193203.5  859.0604 12 2004
20.625793  .9599513  .002372134  42153.54 190543.5       875 12 2005
21.588934  .9586476 .0028717364  49476.11 207289.5  901.8456 12 2006
 6.060403         .           .         .        .  342.9536 13 1995
 6.897939         .           .  1970.474   705081  345.4577 13 1996
 6.940691         .           .  2066.189   707188 347.64215 13 1997
 7.450578         .   .03093755 2057.0493   741582  349.9864 13 1998
 7.653945  .9999799   .02744761 2016.4906 707031.5  352.3839 13 1999
 7.999667 1.0138539   .02466945 1718.5964 726291.5  355.5807 13 2000
 10.17147 1.0195103  .025121726 1736.3822 802806.5  356.9126 13 2001
 13.17925 1.0009689    .0270602 1915.4476   818183  358.8307 13 2002
13.010253 1.0480394  .024097467 2012.0217   683901  360.6656 13 2003
16.913507 1.0326258  .025264023  2039.082   848877  362.7733 13 2004
 14.69758  .9845735  .026615873 2219.1855 827391.5   365.011 13 2005
15.840396  .9741256   .02650359  2656.591   879044   367.515 13 2006
  7.39903         .           .         .        .   196.865 14 1995
 6.435109         .           . 1081.3728   488111 198.91235 14 1996
 5.501362         .           . 1147.8309   490949  200.9597 14 1997
 5.000979         .   .23141885  966.7808   483157 202.94305 14 1998
 5.143294 1.0196248   .21766452 1307.6656   445525  204.9904 14 1999
 6.370831 1.0184317    .1950557 1277.7507   475848  207.8055 14 2000
 7.945537 1.0066813    .2159245 1233.4995   512472   209.341 14 2001
 8.899892  .9773062    .2292331 1395.0482   534334 210.74857 14 2002
 12.32518  1.039925   .22437054  1314.587 459888.5  212.0467 14 2003
16.761196 1.0158327   .24441063  1315.413   601920  213.3717 14 2004
 15.23741 1.0120355   .27374214  1449.547   573208  214.6513 14 2005
15.776987 1.0122598   .28836787 2307.0564   609110  215.9309 14 2006
20.952024         .           .         .        . 19.306847 15 1995
12.384887         .           .       833   220379  19.50127 15 1996
13.079862         .           .  813.0875   185347 19.661877 15 1997
17.483166         .    .0824706  773.0286   223459 19.822485 15 1998
17.917507   .983538   .07321272  807.9928 197303.5  19.96619 15 1999
24.205923 1.0107175     .075675     856.4 256035.5  20.05072 15 2000
 32.01301 1.0135767   .07750881  750.1818 254575.5 20.092983 15 2001
 42.90153 1.0091571    .0837925   848.103 267189.5  20.10989 15 2002
37.453632 1.0363934   .08462862 1301.9546   254857 20.115046 15 2003
 39.90929 1.0333651   .09300426 3456.1584 363978.5 20.152155 15 2004
 44.81824 1.0150399   .11284686  3843.016 368240.5 20.312763 15 2005
 51.88762 1.0171665   .11851845  4453.134 397844.5 20.262045 15 2006
 38.37186         .           .         .        . 277.42374 21 1995
 15.94024         .           .  6583.655   474679 279.05084 21 1996
 20.01357         .           .  7597.519   406585 280.54236 21 1997
20.188166         .  .016391968  8890.332   461336  281.8305 21 1998
 17.13689 1.0009549  .018104931  8752.343 394872.5 282.77966 21 1999
 17.93646 1.0101026   .01645696   11670.1   504262   283.661 21 2000
 19.52509 1.0026213  .016550554 11736.085 521676.5   284.339 21 2001
 13.80778  .9600466  .017200725 13561.827   471435 284.94916 21 2002
  13.8085 1.0303191  .015915826 14261.477   439973 285.42374 21 2003
17.448254  .9795876  .015704675 11998.565   544116 285.89832 21 2004
20.818985  .9646319  .016955303 13843.132 524763.5  286.1695 21 2005
24.856144  .9663287  .014540224 15789.224   543915 289.55933 21 2006
25.877035         .           .         .        . 137.09064 22 1995
 8.378799         .           . 1990.2635   255356 138.04265 22 1996
 8.814848         .           . 2138.9397   191105 138.99467 22 1997
 7.783555         .  .016872184 2311.3577 252893.5 139.84091 22 1998
12.378417  .9725556  .014731757 2330.4963 231551.5 140.58136 22 1999
14.943972  .9948452  .011940835 2244.7026   334194 141.85072 22 2000
14.675217 1.0127369   .01171727 2362.7493 285669.5 142.32674 22 2001
16.691565 1.0124046  .010216657   5554.39 259812.5 142.74985 22 2002
 17.58342 1.1180202   .01053317  5532.384   247924 142.99843 22 2003
 20.13629 1.0690529   .01281442  5486.529   316196 143.27875 22 2004
18.834095 1.0604678  .016883448  5619.026   328782 143.64899 22 2005
 19.67792 1.0839232   .01784413   8325.49   303416 144.01921 22 2006
 13.83915         .           .         .        .  81.37339 23 1995
 8.149414         .           . 2215.0024   428974  81.96703 23 1996
 8.681317         .           . 2140.9126   329039  82.47273 23 1997
 10.04869         .   .04572305 2178.9443 463579.5  82.95644 23 1998
 8.487381  .9384899   .04041757 1882.9446 413110.5  83.37419 23 1999
10.350446  .9656764  .030706927 1372.7903   557867  83.70399 23 2000
18.861177  .9451426   .03247723 1409.6432   505601  83.79195 23 2001
12.740572  .9100608  .035301827 1728.4348   577053  83.83591 23 2002
  15.0632  .9708339   .03431091  1610.293 444843.5  83.87989 23 2003
23.855875  .9106113  .036056183  1619.825 608792.5  83.92387 23 2004
16.228756  .9305188   .04257656 1757.4777 585856.5  83.98982 23 2005
 19.56946  .9366841   .04124046 2181.5225   576840  84.05579 23 2006
   46.663         .           .         .        . 2231.5093 31 1995
 15.60696         .           .  46660.91   168007 2237.8174 31 1996
17.460857         .           .  50030.48 179030.5 2297.7449 31 1997
18.997065         .           0  55878.76   179973 2308.7842 31 1998
end

[Andrew Musau]

I cannot replicate your error. Plus, you have panel data and not multiple observations per id-period, so you cannot have an indicator for each observation as you will violate the identification condition in IV2SLS. The best you can do is individual and time fixed effects separately.

*DOES NOT WORK WITH PANEL DATA
 ivreghdfe y (x=z), absorb(i.id##i.year) cluster(id)
* WORKS
ivreghdfe y (x=z), absorb(i.id i.year) cluster(id)

Res.:

. *DOES NOT WORK WITH PANEL DATA

.
.  ivreghdfe y (x=z), absorb(i.id##i.year) cluster(id)
(dropped 64 singleton observations)
insufficient observations
r(2001);

.
. * WORKS

.
. ivreghdfe y (x=z), absorb(i.id i.year) cluster(id)
(MWFE estimator converged in 2 iterations)

IV (2SLS) estimation
--------------------

Estimates efficient for homoskedasticity only
Statistics robust to heteroskedasticity and clustering on id

Number of clusters (id) =            8                Number of obs =       64
                                                      F(  1,     7) =     0.64
                                                      Prob > F      =   0.4499
Total (centered) SS     =  853.3052989                Centered R2   =  -5.3686
Total (uncentered) SS   =  853.3052989                Uncentered R2 =  -5.3686
Residual SS             =  5434.380023                Root MSE      =     9.94

------------------------------------------------------------------------------
             |               Robust
           y |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
           x |   449.6553   561.8957     0.80   0.450    -879.0169    1778.327
------------------------------------------------------------------------------
Underidentification test (Kleibergen-Paap rk LM statistic):              0.789
                                                   Chi-sq(1) P-val =    0.3743
------------------------------------------------------------------------------
Weak identification test (Cragg-Donald Wald F statistic):                0.954
                         (Kleibergen-Paap rk Wald F statistic):          0.546
Stock-Yogo weak ID test critical values: 10% maximal IV size             16.38
                                         15% maximal IV size              8.96
                                         20% maximal IV size              6.66
                                         25% maximal IV size              5.53
Source: Stock-Yogo (2005).  Reproduced by permission.
NB: Critical values are for Cragg-Donald F statistic and i.i.d. errors.
------------------------------------------------------------------------------
Hansen J statistic (overidentification test of all instruments):         0.000
                                                 (equation exactly identified)
------------------------------------------------------------------------------
Instrumented:         x
Excluded instruments: z
Partialled-out:       _cons
                      nb: total SS, model F and R2s are after partialling-out;
                          any small-sample adjustments include partialled-out
                          variables in regressor count K
------------------------------------------------------------------------------

Absorbed degrees of freedom:
-----------------------------------------------------+
 Absorbed FE | Categories  - Redundant  = Num. Coefs |
-------------+---------------------------------------|
          id |         8           8           0    *|
        year |         8           0           8     |
-----------------------------------------------------+
* = FE nested within cluster; treated as redundant for DoF computation

.

[Linghui Han]

It is weird. I tried
ivreghdfe y (x=z), absorb(i.id i.year) cluster(id) and I got the same error "last estimates not found".

And, with this code "xtreg y x i.year##i.id, fe vce(cluster, id)", the panel data works well.

[Andrew Musau]

It is weird. I tried
ivreghdfe y (x=z), absorb(i.id i.year) cluster(id) and I got the same error "last estimates not found".

You should re-install the command. As you can see, I do not get such an error.

And, with this code "xtreg y x i.year##i.id, fe vce(cluster, id)", the panel data works well.

Note that the fixed effects are at the individual level in your command, and Stata is free to drop indicators to estimate the model. A more accurate test would be

gen id2= c.id#c.year
xtset id2
xtreg y x i.year##i.id, fe vce(cluster id)

In any case, I have revised my description to make it more informative in #13. In IV2SLS regression, an identification condition is that the number of observations must be greater than the number of instruments. Creating the indicators i.id#i.year in a panel dataset results in an indicator for each observation. You will also have other instruments, so you will end up violating the identification condition by including such an interaction. To see this more clearly, put the interaction on the left-hand side.

ivreghdfe y (x=z) i.id#i.year, absorb(i.id i.year) cluster(id)

Res.:

. ivreghdfe y (x=z) i.id#i.year, absorb(i.id i.year) cluster(id)
Warning - endogenous variable(s) collinear with instruments
Vars now exogenous: x
Warning - collinearities detected
Vars dropped:       z
(MWFE estimator converged in 2 iterations)
Error: number of observations must be greater than number of instruments
       including constant.
insufficient observations
r(2001);