xtabond2: System GMM robust estimation - Do I use Hansen or Sargan test results??

[Tom Hardwick]

Hi,

I am currently investigating the relationship between financial development and GDP growth. My results are as below and I am trying to interpret my diagnostic test results for over identification of instruments.

Please can someone explain to me whether I should be looking at the Hansen or Sargan statistics. I have noticed that dropping robust SE usage will remove the Hansen statistic, does this mean I should be using Hansen under robust SE?

Also, if both tests have the same null hypotheses, how can each p value be so different? Should I be looking at both of the results to interpret?

. xtabond2 GDPG l.GDPG M3Y INVEST HCAP POPG, gmm (l.GDPG M3Y INVEST HCAP POPG, lag (1 2) eq(diff)) gmm(l.GDPG M3Y INVEST HCAP POPG,
> lag (1 1) eq(level)) iv(i.YEAR, eq(level)) robust
Favoring space over speed. To switch, type or click on mata: mata set matafavor speed, perm.
Warning: Number of instruments may be large relative to number of observations.
Warning: Two-step estimated covariance matrix of moments is singular.
Using a generalized inverse to calculate robust weighting matrix for Hansen test.
Difference-in-Sargan/Hansen statistics may be negative.

Dynamic panel-data estimation, one-step system GMM
------------------------------------------------------------------------------
Group variable: country Number of obs = 824
Time variable : YEAR Number of groups = 76
Number of instruments = 227 Obs per group: min = 1
Wald chi2(5) = 97.01 avg = 10.84
Prob > chi2 = 0.000 max = 15
------------------------------------------------------------------------------
| Robust
GDPG | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
GDPG |
L1. | .0989889 .0459696 2.15 0.031 .0088901 .1890878
|
M3Y | -.0179359 .0094577 -1.90 0.058 -.0364726 .0006008
INVEST | .2661551 .0492722 5.40 0.000 .1695835 .3627268
HCAP | -.0325017 .0177498 -1.83 0.067 -.0672906 .0022872
POPG | -.8862412 .207724 -4.27 0.000 -1.293373 -.4791096
_cons | .5630129 1.335452 0.42 0.673 -2.054424 3.18045
------------------------------------------------------------------------------
Instruments for first differences equation
GMM-type (missing=0, separate instruments for each period unless collapsed)
L(1/2).(L.GDPG M3Y INVEST HCAP POPG)
Instruments for levels equation
Standard
1994b.YEAR 1995.YEAR 1996.YEAR 1997.YEAR 1998.YEAR 1999.YEAR 2000.YEAR
2001.YEAR 2002.YEAR 2003.YEAR 2004.YEAR 2005.YEAR 2006.YEAR 2007.YEAR
2008.YEAR 2009.YEAR 2010.YEAR 2011.YEAR 2012.YEAR 2013.YEAR
_cons
GMM-type (missing=0, separate instruments for each period unless collapsed)
DL.(L.GDPG M3Y INVEST HCAP POPG)
------------------------------------------------------------------------------
Arellano-Bond test for AR(1) in first differences: z = -5.12 Pr > z = 0.000
Arellano-Bond test for AR(2) in first differences: z = -2.55 Pr > z = 0.011
------------------------------------------------------------------------------
Sargan test of overid. restrictions: chi2(221) = 546.45 Prob > chi2 = 0.000
(Not robust, but not weakened by many instruments.)
Hansen test of overid. restrictions: chi2(221) = 74.27 Prob > chi2 = 1.000
(Robust, but weakened by many instruments.)

Difference-in-Hansen tests of exogeneity of instrument subsets:
GMM instruments for levels
Hansen test excluding group: chi2(148) = 72.48 Prob > chi2 = 1.000
Difference (null H = exogenous): chi2(73) = 1.79 Prob > chi2 = 1.000
gmm(L.GDPG M3Y INVEST HCAP POPG, eq(diff) lag(1 2))
Hansen test excluding group: chi2(82) = 72.79 Prob > chi2 = 0.757
Difference (null H = exogenous): chi2(139) = 1.48 Prob > chi2 = 1.000
gmm(L.GDPG M3Y INVEST HCAP POPG, eq(level) lag(1 1))
Hansen test excluding group: chi2(148) = 72.48 Prob > chi2 = 1.000
Difference (null H = exogenous): chi2(73) = 1.79 Prob > chi2 = 1.000
iv(1994b.YEAR 1995.YEAR 1996.YEAR 1997.YEAR 1998.YEAR 1999.YEAR 2000.YEAR 2001.YEAR 2002.YEAR 2003.YEAR 2004.YEAR 2005.YEAR 2006.Y
> EAR 2007.YEAR 2008.YEAR 2009.YEAR 2010.YEAR 2011.YEAR 2012.YEAR 2013.YEAR, eq(level))
Hansen test excluding group: chi2(207) = 74.27 Prob > chi2 = 1.000
Difference (null H = exogenous): chi2(14) = 0.00 Prob > chi2 = 1.000

Thanks in Advance,

Tom Hardwick

[Sebastian Kripfganz]

Welcome to Statalist, Tom.

I have a couple of comments on your specification:

1. The Sargan test is only appropriate after a difference-GMM estimator and under the assumption of homoskedasticity and no serial correlation (in levels) of the idiosyncratic error term. In that case, it uses an optimal weighting matrix and thus has an asymptotic chi2 distribution. (You can still have an intercept and time dummies as instruments for the level equation.)
2. If you are instead using a system-GMM estimator (with additional instruments for the level equation besides the intercept and time dummies), as you have done in your example, or do not want to rely on the rather strong assumption of homoskedasticity and absence of serial correlation in the error term, then you should build your decision on the Hansen test that uses an optimal weighting matrix (while the Sargan test would no longer be based on an optimal weighting matrix).
3. Whenever the Hansen test should take precedence over the Sargan test, based on the above discussion, it is also highly recommended to use robust standard errors (and possibly the twostep estimator).
4. As indicated in the xtabond2 output, the Hansen test (and the GMM coefficient estimates as well) potentially suffers from a too-many-instruments problem. In your case, the number of instruments (227) is definitely too large. As a rule of thumb, the number of instruments should never exceed the number of groups (76) and ideally should stay far below that number. The easiest way to reduce the number of instruments is to use the collapse suboption of the gmm() option.
5. Adding time dummies as instruments is not sufficient to account for time effects. You need to specify them as regressors as well. But be aware that xtabond2 suffers from a severe bug when you specify time dummies with the factor variable notation; see my 2017 presentation at the UK Stata Users Group Meeting referenced below. Instead, you should manually create your time dummies in the first place and then only use the non-redundant dummies in the xtabond2 command syntax, i.e. always avoid that xtabond2 by itself omits dummies or other variables. Alternatively, you could also use my command xtseqreg; see below.

Further information:

• Roodman (2009). How to do xtabond2: An introduction to difference and system GMM in Stata. Stata Journal 9 (1), pp. 86-136.
• Kripfganz (2017). xtseqreg: Sequential (two-stage) estimation of linear panel data models; and some pitfalls in the estimation of dynamic panel models. 23rd UK Stata Users Group Meeting, London, September 7, 2017.
• Statalist topic: XTSEQREG: new Stata command for sequential / two-stage (GMM) estimation of linear panel models.
• Statalist topic: XTDPDGMM: new Stata command for efficient GMM estimation of linear (dynamic) panel models with nonlinear moment conditions.
• Statalist topic: System GMM - Time dummies.

[Chiraz KARAMTI]

Hi Sebastian
I just saw your new command for dynamic panel data and seems very intresting. What's the difference with the xtdpdsys command?

I have also a question please about the instruments validity test. I'm using xtdpdsys command, for example:

xtdpdsys y x1 x2 x3, lags(1) maxldep(3) maxlags(3) twostep artests(2)

How can I conduct a test of the relevance of the first-stage excluded instruments?

any advice will be very helpful to me.

Thanks.

[Sebastian Kripfganz]

xtdpdsys is less flexible than xtabond2, xtdpdgmm, or xtseqreg. All of the latter allow, for example, to collapse the instruments in order to avoid a too-many-instruments problem. They also compute Hansen overidentification tests while xtdpdsys only computes the Sargan overidentification test that would generally not be asymptotically valid for a system GMM estimator.

Unfortunately, none of these commands to date computes first-stage test statistics for the relevance of the instruments. You could try to replicate the results with the community-contributed command ivreg2. Please see the help file of ivreg2, in particular the section on "First-stage regressions, identification, and weak-id-robust inference".

[Chiraz KARAMTI]

Indeed the Sargan test gives a p-value=1.000!!!
. estat sargan
Sargan test of overidentifying restrictions
H0: overidentifying restrictions are valid

chi2(36) = 11.64032
Prob > chi2 = 1.0000

The reason for which I used xtdpdsys commande and not xtabon2 is that I have a short period (T=16 years and N=32 countries) so T<N.

I know the command ivreg2 but I wasn't sure if it works with dynamic panel. I'll take a look for some documentations and I'll try to replicate the first stage estimation.

Thank you very much for the clarifications.

[Hu Weiping]

Welcome to Statalist, Tom.

I have a couple of comments on your specification:

1. The Sargan test is only appropriate after a difference-GMM estimator and under the assumption of homoskedasticity and no serial correlation (in levels) of the idiosyncratic error term. In that case, it uses an optimal weighting matrix and thus has an asymptotic chi2 distribution. (You can still have an intercept and time dummies as instruments for the level equation.)
2. If you are instead using a system-GMM estimator (with additional instruments for the level equation besides the intercept and time dummies), as you have done in your example, or do not want to rely on the rather strong assumption of homoskedasticity and absence of serial correlation in the error term, then you should build your decision on the Hansen test that uses an optimal weighting matrix (while the Sargan test would no longer be based on an optimal weighting matrix).
3. Whenever the Hansen test should take precedence over the Sargan test, based on the above discussion, it is also highly recommended to use robust standard errors (and possibly the twostep estimator).
4. As indicated in the xtabond2 output, the Hansen test (and the GMM coefficient estimates as well) potentially suffers from a too-many-instruments problem. In your case, the number of instruments (227) is definitely too large. As a rule of thumb, the number of instruments should never exceed the number of groups (76) and ideally should stay far below that number. The easiest way to reduce the number of instruments is to use the collapse suboption of the gmm() option.
5. Adding time dummies as instruments is not sufficient to account for time effects. You need to specify them as regressors as well. But be aware that xtabond2 suffers from a severe bug when you specify time dummies with the factor variable notation; see my 2017 presentation at the UK Stata Users Group Meeting referenced below. Instead, you should manually create your time dummies in the first place and then only use the non-redundant dummies in the xtabond2 command syntax, i.e. always avoid that xtabond2 by itself omits dummies or other variables. Alternatively, you could also use my command xtseqreg; see below.

Further information:

• Roodman (2009). How to do xtabond2: An introduction to difference and system GMM in Stata. Stata Journal 9 (1), pp. 86-136.
• Kripfganz (2017). xtseqreg: Sequential (two-stage) estimation of linear panel data models; and some pitfalls in the estimation of dynamic panel models. 23rd UK Stata Users Group Meeting, London, September 7, 2017.
• Statalist topic: XTSEQREG: new Stata command for sequential / two-stage (GMM) estimation of linear panel models.
• Statalist topic: XTDPDGMM: new Stata command for efficient GMM estimation of linear (dynamic) panel models with nonlinear moment conditions.
• Statalist topic: System GMM - Time dummies.

Hi Sebastian, I am new here
i 've read lots of your posts about sysGMM。
i have a question here: if we use Hansen J test ,do we still need to consider Arellano-Bond test for AR(2)
you said “do not want to rely on the rather strong assumption of homoskedasticity and absence of serial correlation in the error term, then you should build your decision on the Hansen test”

i upload my regression results and this is my code

xi:xtabond2 INTinve l.INTinve l.INTinvesquare l.MB SalesGwth CashflowINT l.CashflowINT StkIssue ///
l.StkIssue DebtIssue l.DebtIssue DCash l.DCash i.fyear ,iv(i.fyear,eq(level)) ///
gmm(INTinve MB SalesGwth CashflowINT StkIssue Debt DCash,lag(2 2) eq(level)) ///
gmm(INTinve MB SalesGwth CashflowINT StkIssue Debt DCash,lag(3 4) eq(diff)) ///

Is there anything wrong with my results？what do we do if Hansen test and Sargen Tset are contradictory？
Any help is appreciated！
Thanks.

[Sebastian Kripfganz]

i have a question here: if we use Hansen J test ,do we still need to consider Arellano-Bond test for AR(2)
you said “do not want to rely on the rather strong assumption of homoskedasticity and absence of serial correlation in the error term, then you should build your decision on the Hansen test”

No test is perfect. I would recommend to generally look at both tests, in particular if some of the instruments by construction rely on the assumption that there is no second-order serial correlation in the first-differenced model. This is usually the case when we use the second lagged level of the dependent variable as an instrument. In your case, you are only using the 3rd and 4th lagged level of the dependent variable as instruments. This would allow for second-order serial correlation (but not third-order serial correlation) of the first-differenced errors. However, this is not commonly done in practice. In particular given that the AR(2) test in your example does not reject the null hypothesis of no second-order serial correlation, the second lagged level of the dependent variable would qualify as a valid instrument and it is typically a stronger instrument that just the 3rd or 4th lag.

If the AR(2) test rejects the null hypothesis, instead of choosing higher lags as instruments it is usually a better idea to directly amend the regression model by adding further lags of the regressors to capture that serial correlation.

Is there anything wrong with my results？what do we do if Hansen test and Sargen Tset are contradictory?

Aside from the above comment, your results appear to be fine. The number of instruments is quite large but this might be okay given that you also have a relatively large number of observations. Given that you are using a two-step system GMM estimator, you can safely ignore the Sargan test. It uses an inefficient weighting matrix and therefore is not asymptotically valid.

[Hu Weiping]

No test is perfect. I would recommend to generally look at both tests, in particular if some of the instruments by construction rely on the assumption that there is no second-order serial correlation in the first-differenced model. This is usually the case when we use the second lagged level of the dependent variable as an instrument. In your case, you are only using the 3rd and 4th lagged level of the dependent variable as instruments. This would allow for second-order serial correlation (but not third-order serial correlation) of the first-differenced errors. However, this is not commonly done in practice. In particular given that the AR(2) test in your example does not reject the null hypothesis of no second-order serial correlation, the second lagged level of the dependent variable would qualify as a valid instrument and it is typically a stronger instrument that just the 3rd or 4th lag.

If the AR(2) test rejects the null hypothesis, instead of choosing higher lags as instruments it is usually a better idea to directly amend the regression model by adding further lags of the regressors to capture that serial correlation.


Aside from the above comment, your results appear to be fine. The number of instruments is quite large but this might be okay given that you also have a relatively large number of observations. Given that you are using a two-step system GMM estimator, you can safely ignore the Sargan test. It uses an inefficient weighting matrix and therefore is not asymptotically valid.

Helped me a lot, Thanks!

[Hu Weiping]

No test is perfect. I would recommend to generally look at both tests, in particular if some of the instruments by construction rely on the assumption that there is no second-order serial correlation in the first-differenced model. This is usually the case when we use the second lagged level of the dependent variable as an instrument. In your case, you are only using the 3rd and 4th lagged level of the dependent variable as instruments. This would allow for second-order serial correlation (but not third-order serial correlation) of the first-differenced errors. However, this is not commonly done in practice. In particular given that the AR(2) test in your example does not reject the null hypothesis of no second-order serial correlation, the second lagged level of the dependent variable would qualify as a valid instrument and it is typically a stronger instrument that just the 3rd or 4th lag.

If the AR(2) test rejects the null hypothesis, instead of choosing higher lags as instruments it is usually a better idea to directly amend the regression model by adding further lags of the regressors to capture that serial correlation.


Aside from the above comment, your results appear to be fine. The number of instruments is quite large but this might be okay given that you also have a relatively large number of observations. Given that you are using a two-step system GMM estimator, you can safely ignore the Sargan test. It uses an inefficient weighting matrix and therefore is not asymptotically valid.

i guess this is the last question before completion of my undergraduate thesis.
sysGMM results tend to be very unstable，you said it‘s like a black box.
the results i poseted seem to be fine, but if remove Salesgwth（sales growth）and MB(Market to book ratio) from instrument variable,some important coefficients are no longer significant.
In this specification, the former papers does not use these two as instrumental variables（they just use these as control variable） . If I follow the previous papers, I cannot get good results before.

Is it valid to add these two instrument varibles in sysGMM when all test results are fine?

[Sebastian Kripfganz]

If you remove those instruments, the remaining instruments might be weak (at least with respect to the regressors sales growth and market-to-book ratio). This explaines the larger standard errors and thus the statistical insignificance when you no longer use those instruments. The original authors may have had good theoretical arguments for not using those instruments. If there are no such theoretical reasons, then not using them does not seem to be a good strategy, in particular if the specification tests are fine. (It is still "valid", as long as the remaining instruments are strong enough.)

[Beri Parfait]

Hi Sebastian, I am new here
i 've read lots of your posts about sysGMM。
i have a question here: if we use Hansen J test ,do we still need to consider Arellano-Bond test for AR(2)
you said “do not want to rely on the rather strong assumption of homoskedasticity and absence of serial correlation in the error term, then you should build your decision on the Hansen test”

i upload my regression results and this is my code

xi:xtabond2 INTinve l.INTinve l.INTinvesquare l.MB SalesGwth CashflowINT l.CashflowINT StkIssue ///
l.StkIssue DebtIssue l.DebtIssue DCash l.DCash i.fyear ,iv(i.fyear,eq(level)) ///
gmm(INTinve MB SalesGwth CashflowINT StkIssue Debt DCash,lag(2 2) eq(level)) ///
gmm(INTinve MB SalesGwth CashflowINT StkIssue Debt DCash,lag(3 4) eq(diff)) ///

Is there anything wrong with my results？what do we do if Hansen test and Sargen Tset are contradictory？
Any help is appreciated！
Thanks.

Dear sebastian, thank you for the insights. I have also been following and reading many comments from you.

You said these results were good but I read somewhere (can't recall) that the coefficient of the dynamic parameter (l.INTinve) should be less than 1(ideally closer to zero) for the model to be stable. I had similar results as @Hu but as a novice in GMM and panel data econometrics, I have been very sceptical about presenting them. Could you kindly shade more light on the coefficient of the dependent variable in a dynamic panel estimation?

Secondly, is it necessary to use the ROBUST option in system GMM estimations? From my previous readings, the sys GMM is already robust to heteroscedasticity.

Thank you!

[Sebastian Kripfganz]

The coefficient of the lagged dependent variable should normally be in the [0,1] interval. In finite samples, the estimate occasionally exceeds those bounds. Note that in the example you are referring to the confidence interval still includes 1.

If you treat the coefficient as equal to 1, you can essentially subtract the lagged dependent variable from both sides. This will give you a static model with the change of the dependent variable on the left-hand side. Essentially, your model cannot explain the level but only the change of the dependent variable.

Yes, the two-step estimator is already robust to heteroskedasticity. The robust option is usually still necessary because it implies the Windmeijer finite-sample correction to the standard errors. Without this correction, standard errors can be severely biased.

[Max Brauer]

Hi Sebastian,

I just saw this thread, and really appreciate these insights. For example, I was using i.Year and didn't realize it created a bug. I have some related questions about difference GMM posted here:

https://www.statalist.org/forums/for...llano-and-bond

I didn't get any replies, and I don't mean to trouble you, but if you read this, I would greatly appreciate any help you could provide.

Thanks!

[Fleur Van Crombrugge]

Hello,

I have been following all the comments on this discussion and it has helped me a lot! Thanks for all tips!
However, being a beginner in this type of model, some questions remain.

My analysis is the relationship between GHG (greenhouse gas emissions) and total Population, population density, GDP per capita and gva-shares between 2002 and 2018 for almost all European nuts 3 regions.

To analyse the relationship for my panel, I used the following command:
xtabond2 ln_edgar L.ln_edgar L(0/1).(ln_pop_density ln_pop_tot geo2 ln_gdp_pps_hab ln_gdp_pps_hab_sq gva_a_per gva_be_per gva_f_per gva_gj_per) i.year, gmmstyle(L.(ln_edgar ln_pop_density ln_pop_tot geo2 ln_gdp_pps_hab ln_gdp_pps_hab_sq gva_a_per gva_be_per gva_f_per gva_gj_per)) ivstyle(i.year, equation(level)) twostep small

However, I have many a lot of instruments (1258), almost the number of regions (i) but I have 19580 observations.. Is this correct? My Hansen test is 0.369 but when I add "collapse", Hansen's test drops to 0.0000

What could this mean?
Is there an error in my code?

The results are in the attached pdf

Thank you in advance for your attention and help.

Fleur

[Sebastian Kripfganz]

Please avoid double posting the same question.