RESET Test after running logistic regression

[Anabella Lamarche]

Hello,

I was informed by my thesis supervisor that I should consider conducting a RESET test. I have tried:

logit y x z

ovtest

Which then generated the error message "last estimates not found"
Then trying

logit y x z
estimates store m1
ovtest

Which then generated the error message "last estimates not found"
Also, I tried

logit y x z, robust ovtest
Did not work.

From what I have read the RESET test identifies omitted variable bias?
I have not been able to find an example of someone doing the same thing.
Any help would be appreciated.

[Jeff Wooldridge]

Whoever informed you that RESET is a test for omitted variables is wrong. It's a functional form test. That it might have some power against omitted variables in some cases is entirely incidental. I have a paper in the volume "A Companion to Theoretical Econometrics" that shows why thinking of RESET as a functional form test is a mistake.

Having said that, you might wish to test for functional form in a linear model or a logit model. But my understanding is that ovtest works only for linear regression.

It's easy to construct a test "by hand," and this is discussed in Papke and Wooldridge (1996, Journal of Applied Econometrics). Here is generic code.

logit y x1 x2 ... xk
predict xbhat, xb
gen xbhatsq = xbhat*xbhat
gen xbhatcu = xbhatsq*xbhat
logit y x1 x2 ... xk xbhatsq xbhatcu
test xbhatsq xbhatcu

If you get a significant test statistic, the functional form is rejected, and you should include some squares and interactions among the xj to get a better functional form. Of course, that might have little effect on the average partial effects. It isn't clear that included complicated nonlinear functions changes the bottom line much, but one can try.

JW

[Joao Santos Silva]

Dear Jeff Wooldridge,

Just to be clear, did you really mean to say

I have a paper in the volume "A Companion to Theoretical Econometrics" that shows why thinking of RESET as a functional form test is a mistake.

Best wishes,

Joao
PS: Welcome back to the forum, we missed you!

[Carlo Lazzaro]

Annabella:
a Stata built-in command that does a very similar job to the one accomplished by the test that Jeff suggested is -linktest-, as you can see from the following toy-example:

use http://www.stata-press.com/data/r15/auto.dta

. logit foreign i.rep78

note: 1.rep78 != 0 predicts failure perfectly
      1.rep78 dropped and 2 obs not used

note: 2.rep78 != 0 predicts failure perfectly
      2.rep78 dropped and 8 obs not used

note: 5.rep78 omitted because of collinearity
Iteration 0:   log likelihood = -38.411464
Iteration 1:   log likelihood = -27.676628
Iteration 2:   log likelihood = -27.446054
Iteration 3:   log likelihood = -27.444671
Iteration 4:   log likelihood = -27.444671

Logistic regression                             Number of obs     =         59
                                                LR chi2(2)        =      21.93
                                                Prob > chi2       =     0.0000
Log likelihood = -27.444671                     Pseudo R2         =     0.2855

------------------------------------------------------------------------------
     foreign |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |
          1  |          0  (empty)
          2  |          0  (empty)
          3  |  -3.701302   .9906975    -3.74   0.000    -5.643033   -1.759571
          4  |  -1.504077   .9128709    -1.65   0.099    -3.293271    .2851168
          5  |          0  (omitted)
             |
       _cons |   1.504077    .781736     1.92   0.054    -.0280969    3.036252
------------------------------------------------------------------------------

. linktest

Iteration 0:   log likelihood = -38.411464
Iteration 1:   log likelihood = -27.676628
Iteration 2:   log likelihood = -27.446054
Iteration 3:   log likelihood = -27.444671
Iteration 4:   log likelihood = -27.444671

Logistic regression                             Number of obs     =         59
                                                LR chi2(2)        =      21.93
                                                Prob > chi2       =     0.0000
Log likelihood = -27.444671                     Pseudo R2         =     0.2855

------------------------------------------------------------------------------
     foreign |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        _hat |          1   .3429873     2.92   0.004     .3277573    1.672243
      _hatsq |   1.30e-08    .213694     0.00   1.000    -.4188326    .4188326
       _cons |  -2.98e-16   .4714045    -0.00   1.000    -.9239359    .9239359
----------------------------------------------------------------------

As the prediction squared has no expalnatory power, the regression model shows no evidence of misspecification.

As an aside, actually -estat ovtest- works after -regress- only.

PS: I do share Joao's welcome back to Jeff.

[Anabella Lamarche]

I implemented both your suggestions.

Thank you very much. I appreciate the insight and advice.
Jeff Wooldridge Carlo Lazzaro

[Carlo Lazzaro]

Anabella:
happy with reading I was helpful.

[Jeff Wooldridge]

Dear Jeff Wooldridge,

Just to be clear, did you really mean to say


Best wishes,

Joao
PS: Welcome back to the forum, we missed you!

Ugh. Thanks for catching that, Joao. I meant "as an omitted variables test."

[Nitin Jain]

Annabella:
a Stata built-in command that does a very similar job to the one accomplished by the test that Jeff suggested is -linktest-, as you can see from the following toy-example:

use http://www.stata-press.com/data/r15/auto.dta

. logit foreign i.rep78

note: 1.rep78 != 0 predicts failure perfectly
1.rep78 dropped and 2 obs not used

note: 2.rep78 != 0 predicts failure perfectly
2.rep78 dropped and 8 obs not used

note: 5.rep78 omitted because of collinearity
Iteration 0: log likelihood = -38.411464
Iteration 1: log likelihood = -27.676628
Iteration 2: log likelihood = -27.446054
Iteration 3: log likelihood = -27.444671
Iteration 4: log likelihood = -27.444671

Logistic regression Number of obs = 59
LR chi2(2) = 21.93
Prob > chi2 = 0.0000
Log likelihood = -27.444671 Pseudo R2 = 0.2855

------------------------------------------------------------------------------
foreign | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
rep78 |
1 | 0 (empty)
2 | 0 (empty)
3 | -3.701302 .9906975 -3.74 0.000 -5.643033 -1.759571
4 | -1.504077 .9128709 -1.65 0.099 -3.293271 .2851168
5 | 0 (omitted)
|
_cons | 1.504077 .781736 1.92 0.054 -.0280969 3.036252
------------------------------------------------------------------------------

. linktest

Iteration 0: log likelihood = -38.411464
Iteration 1: log likelihood = -27.676628
Iteration 2: log likelihood = -27.446054
Iteration 3: log likelihood = -27.444671
Iteration 4: log likelihood = -27.444671

Logistic regression Number of obs = 59
LR chi2(2) = 21.93
Prob > chi2 = 0.0000
Log likelihood = -27.444671 Pseudo R2 = 0.2855

------------------------------------------------------------------------------
foreign | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------+----------------------------------------------------------------
_hat | 1 .3429873 2.92 0.004 .3277573 1.672243
_hatsq | 1.30e-08 .213694 0.00 1.000 -.4188326 .4188326
_cons | -2.98e-16 .4714045 -0.00 1.000 -.9239359 .9239359
----------------------------------------------------------------------

As the prediction squared has no expalnatory power, the regression model shows no evidence of misspecification.

As an aside, actually -estat ovtest- works after -regress- only.

PS: I do share Joao's welcome back to Jeff.

Hello Carlo,

I implemented the link test for a panel ordinal logit and also tried the approach that Jeff mentioned. The results from these two approaches are different. I am appending their outputs. Please let me know if there is some other approach for functional specification check for a panel ordinal logit model. Thanks, Nitin.

". test xbhatsq xbhatcu

( 1) [Conflict_Code_New]xbhatsq = 0
( 2) [Conflict_Code_New]xbhatcu = 0

chi2( 2) = 1.33
Prob > chi2 = 0.5144

. linktest

Iteration 0: log likelihood = -177.63452
Iteration 1: log likelihood = -177.63452

Mixed-effects GLM Number of obs = 172
Family: Gaussian
Link: identity

Wald chi2(2) = 24.45
Log likelihood = -177.63452 Prob > chi2 = 0.0000
------------------------------------------------------------------------------------------
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
_hat | .0085211 .0189385 0.45 0.653 -.0285977 .0456399
_hatsq | -.0042359 .0012209 -3.47 0.001 -.0066289 -.0018429
_cons | 3.564635 .2776186 12.84 0.000 3.020513 4.108758
-------------------------+----------------------------------------------------------------
var(e.Conflict_Code_New)| .4619219 .0498103 .3739221 .5706319
------------------------------------------------------------------------------------------

[Jeff Wooldridge]

I'm not entirely sure what linktest is doing in mixed estimation. And I can't tell from the truncated Stata output exactly what you did. You should look to see how Stata defines the index "xb" in the mixed case. Did you use regular logit in the first case and mixed logit in the second? Or mixed logit in both?

[Carlo Lazzaro]

Nitin:
I echo Jeff's wise advice of posting entirely what you typed and what Stata gave you back (as per FAQ).
There's no chance i heaven that interested listers can be of any help without taking a compehensive look at all the stuff. Thanks.

[Nitin Jain]

Whoever informed you that RESET is a test for omitted variables is wrong. It's a functional form test. That it might have some power against omitted variables in some cases is entirely incidental. I have a paper in the volume "A Companion to Theoretical Econometrics" that shows why thinking of RESET as a functional form test is a mistake.

Having said that, you might wish to test for functional form in a linear model or a logit model. But my understanding is that ovtest works only for linear regression.

It's easy to construct a test "by hand," and this is discussed in Papke and Wooldridge (1996, Journal of Applied Econometrics). Here is generic code.

logit y x1 x2 ... xk
predict xbhat, xb
gen xbhatsq = xbhat*xbhat
gen xbhatcu = xbhatsq*xbhat
logit y x1 x2 ... xk xbhatsq xbhatcu
test xbhatsq xbhatcu

If you get a significant test statistic, the functional form is rejected, and you should include some squares and interactions among the xj to get a better functional form. Of course, that might have little effect on the average partial effects. It isn't clear that included complicated nonlinear functions changes the bottom line much, but one can try.

JW

Dear Jeff and Carlo: Thank you for your replies. Sorry for not attaching the full code earlier. Please find below the full code and the output. For both the approaches (one suggested by Jeff and the link test) the model is panel ordered logit.

Are the results different because in the second approach (suggested by Jeff), I am using x's also along with the predicted values and their squares while link test considers only the predicted values and their squares?


1. This is the link test approach.

xtset Ticker_ID Year
xtologit $depvar $allvars4b $allvars5 $allvars7 i.Year i.Country_Code i.GICS_Ind_Code , vce(robust)
linktest

Iteration 0: log likelihood = -177.63452
Iteration 1: log likelihood = -177.63452

Mixed-effects GLM Number of obs = 172
Family: Gaussian
Link: identity

Wald chi2(2) = 24.45
Log likelihood = -177.63452 Prob > chi2 = 0.0000
------------------------------------------------------------------------------------------
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
_hat | .0085211 .0189385 0.45 0.653 -.0285977 .0456399
_hatsq | -.0042359 .0012209 -3.47 0.001 -.0066289 -.0018429
_cons | 3.564635 .2776186 12.84 0.000 3.020513 4.108758
-------------------------+----------------------------------------------------------------
var(e.Conflict_Code_New)| .4619219 .0498103 .3739221 .5706319
------------------------------------------------------------------------------------------


2. This is for the approach suggested by Jeff.

xtologit $depvar $allvars4b $allvars5 $allvars7 i.Year i.Country_Code i.GICS_Ind_Code , vce(robust)
predict xbhat, xb
gen xbhatsq = xbhat*xbhat
gen xbhatcu = xbhatsq*xbhat
xtologit $depvar xbhatsq xbhatcu $allvars4b $allvars5 $allvars7 i.Year i.Country_Code i.GICS_Ind_Code, vce(robust)
test xbhatsq xbhatcu


Fitting comparison model:

Iteration 0: log likelihood = -177.11926
Iteration 1: log likelihood = -158.01451
Iteration 2: log likelihood = -157.39311
Iteration 3: log likelihood = -157.13109
Iteration 4: log likelihood = -156.826
Iteration 5: log likelihood = -156.82287
Iteration 6: log likelihood = -156.82224
Iteration 7: log likelihood = -156.82213
Iteration 8: log likelihood = -156.82212

Refining starting values:

Grid node 0: log likelihood = -143.26068

Fitting full model:

Iteration 0: log pseudolikelihood = -143.26068
Iteration 1: log pseudolikelihood = -129.36829
Iteration 2: log pseudolikelihood = -125.1427
Iteration 3: log pseudolikelihood = -124.66083
Iteration 4: log pseudolikelihood = -124.63723
Iteration 5: log pseudolikelihood = -124.63723

Random-effects ordered logistic regression Number of obs = 172
Group variable: Ticker_ID Number of groups = 46

Random effects u_i ~ Gaussian Obs per group:
min = 2
avg = 3.7
max = 4

Integration method: mvaghermite Integration pts. = 12

Wald chi2(33) = 427.00
Log pseudolikelihood = -124.63723 Prob > chi2 = 0.0000

(Std. Err. adjusted for 46 clusters in Ticker_ID)
--------------------------------------------------------------------------------------------
| Robust
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
---------------------------+----------------------------------------------------------------
xbhatsq | -1.183515 1.094259 -1.08 0.279 -3.328222 .9611932
xbhatcu | -.0239547 .0235079 -1.02 0.308 -.0700293 .0221199
Lag_Debt_Equity_w5 | -.0742851 .0688139 -1.08 0.280 -.2091578 .0605876
Lag_DI_w5 | -276.9445 247.7946 -1.12 0.264 -762.6131 208.7241
Lag_Diversity_w5 | 67.66749 60.6285 1.12 0.264 -51.16219 186.4972
Lag_Effort_w5 | 28.20416 27.76884 1.02 0.310 -26.22177 82.63009
Lag_ESG_Disc_Score_w5 | -.2546338 .2422113 -1.05 0.293 -.7293592 .2200917
Lag_Horizon_Complex_w5 | .0478374 .043611 1.10 0.273 -.0376386 .1333135
Lag_Pagerank_Central_w5 | -69678.49 63804.28 -1.09 0.275 -194732.6 55375.6
Lag_Return_Assets_w5 | -.7989521 .7175565 -1.11 0.266 -2.205337 .6074328
Lag_Tobin_Q_w5 | 9.25654 8.563546 1.08 0.280 -7.527702 26.04078
Lag_Total_Assets_Growth_w5 | .6360576 .5803646 1.10 0.273 -.5014361 1.773551
Lag_Vertical_Complex_w5 | .7831137 .7229085 1.08 0.279 -.633761 2.199988
Ln_Lag_Total_Assets_w5 | -13.44019 12.26872 -1.10 0.273 -37.48645 10.60606
Ln_Lag_Spatial_Complex_w5 | 31.06967 29.07728 1.07 0.285 -25.92075 88.06009
Lag_Age_w5 | .0741569 .0738747 1.00 0.315 -.0706349 .2189486
Horz_Effw5 | -.0288132 .0252793 -1.14 0.254 -.0783598 .0207333
Vert_Effw5 | -.2362379 .2187544 -1.08 0.280 -.6649885 .1925128
Space_Effw5 | -1.80027 1.841396 -0.98 0.328 -5.40934 1.8088
Horz_Supremw5 | 33.38921 30.80824 1.08 0.278 -26.99383 93.77225
Vert_Supremw5 | 608.5288 565.9477 1.08 0.282 -500.7083 1717.766
Space_Supremw5 | 4791.526 4379.784 1.09 0.274 -3792.694 13375.75
|
Year |
2016 | -4.701005 4.33651 -1.08 0.278 -13.20041 3.798398
2017 | -14.91024 13.66101 -1.09 0.275 -41.68532 11.86484
2018 | -11.17551 10.14863 -1.10 0.271 -31.06646 8.715436
|
Country_Code |
2 | -4.605375 4.540278 -1.01 0.310 -13.50416 4.293407
3 | 67.29263 47.20546 1.43 0.154 -25.22836 159.8136
4 | 44.75912 41.95832 1.07 0.286 -37.47768 126.9959
5 | 145.5661 132.2435 1.10 0.271 -113.6264 404.7587
6 | 3.728719 3.91832 0.95 0.341 -3.951048 11.40849
7 | 42.79012 39.34275 1.09 0.277 -34.32025 119.9005
|
GICS_Ind_Code |
3 | 34.99635 32.05392 1.09 0.275 -27.82817 97.82087
4 | 23.32181 21.76984 1.07 0.284 -19.34629 65.9899
---------------------------+----------------------------------------------------------------
/cut1 | 75.69624 84.63313 -90.18164 241.5741
/cut2 | 80.41238 84.43477 -85.07673 245.9015
---------------------------+----------------------------------------------------------------
/sigma2_u | 11.93567 5.922211 4.513356 31.56413
--------------------------------------------------------------------------------------------

. test xbhatsq xbhatcu

( 1) [Conflict_Code_New]xbhatsq = 0
( 2) [Conflict_Code_New]xbhatcu = 0

chi2( 2) = 2.23
Prob > chi2 = 0.3287

Thanks,
Nitin

[Carlo Lazzaro]

Nitin:
some comments about your results:
1) you use -robust- in your baseline -xtlogit-, whereas -linktest- applies default standard errors;
2) as this might be the reason why the two approaches give back conflicting results as far as the specification of the functional form of the regressand is concerned, I would try to replicate -linktest- by hand simply adding -fitted and -sq_fitted- to your -xtologit- regression and see whether the results of the augmenteed regression are (or not) in line with Jeff's suggested procedure.

[Nitin Jain]

Nitin:
some comments about your results:
1) you use -robust- in your baseline -xtlogit-, whereas -linktest- applies default standard errors;
2) as this might be the reason why the two approaches give back conflicting results as far as the specification of the functional form of the regressand is concerned, I would try to replicate -linktest- by hand simply adding -fitted and -sq_fitted- to your -xtologit- regression and see whether the results of the augmenteed regression are (or not) in line with Jeff's suggested procedure.

Dear Carlo,

Thank you for the reply. I have removed vce(robust) now. But the results for the two approaches - one suggested by Jeff and the Link test - are still not matching. I have tried to replicate what you suggested in point 2). Please see approach 2 below.

Approach1 : Considering fitted, sq of fitted and cube of fitted values

xtologit $depvar $allvars4b $allvars5 $allvars7 i.Year i.Country_Code i.GICS_Ind_Code
predict xbhat1, xb
gen xbhat1sq = xbhat1*xbhat1
gen xbhat1cu = xbhat1sq*xbhat1
xtologit $depvar xbhat1sq xbhat1cu xbhat1
test xbhat1sq xbhat1cu
.
Wald chi2(3) = 14.28
Log likelihood = -128.07636 Prob > chi2 = 0.0025

-----------------------------------------------------------------------------------
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
------------------+----------------------------------------------------------------
xbhat1sq | -.0118234 .0330821 -0.36 0.721 -.0766631 .0530164
xbhat1cu | -.0012602 .0035191 -0.36 0.720 -.0081576 .0056372
xbhat1 | 1.039964 .9166578 1.13 0.257 -.7566519 2.836581
------------------+----------------------------------------------------------------
/cut1 | -11.28795 4.292325 -19.70075 -2.875146
/cut2 | -6.81599 4.136091 -14.92258 1.2906
------------------+----------------------------------------------------------------
/sigma2_u | 10.43725 4.047954 4.880432 22.321
-----------------------------------------------------------------------------------
LR test vs. ologit model: chibar2(01) = 74.40 Prob >= chibar2 = 0.0000

. test xbhat1sq xbhat1cu

( 1) [Conflict_Code_New]xbhat1sq = 0
( 2) [Conflict_Code_New]xbhat1cu = 0

chi2( 2) = 0.44
Prob > chi2 = 0.8011

------------------

Approach2 : Considering fitted and sq of fitted values

xtologit $depvar xbhat1sq xbhat1


Wald chi2(2) = 14.06
Log likelihood = -128.12376 Prob > chi2 = 0.0009

-----------------------------------------------------------------------------------
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
------------------+----------------------------------------------------------------
xbhat1sq | -.0021991 .087512 -0.03 0.980 -.1737195 .1693213
xbhat1 | .9684624 1.282045 0.76 0.450 -1.544299 3.481224
------------------+----------------------------------------------------------------
/cut1 | -10.77468 4.862318 -20.30465 -1.244715
/cut2 | -6.30888 4.729147 -15.57784 2.960078
------------------+----------------------------------------------------------------
/sigma2_u | 10.38203 4.02638 4.854758 22.20225
-----------------------------------------------------------------------------------
LR test vs. ologit model: chibar2(01) = 74.76 Prob >= chibar2 = 0.0000

. test xbhat1sq xbhat1 /* With vce(robust) p-value is 0.31, without vce(robust) p-value is 0.51*/

( 1) [Conflict_Code_New]xbhat1sq = 0
( 2) [Conflict_Code_New]xbhat1 = 0

chi2( 2) = 14.06
Prob > chi2 = 0.0009



------------------
Approach 3: Link Test

xtset Ticker_ID Year
xtologit $depvar $allvars4b $allvars5 $allvars7 i.Year i.Country_Code i.GICS_Ind_Code
linktest

Wald chi2(2) = 24.45
Log likelihood = -177.63452 Prob > chi2 = 0.0000
------------------------------------------------------------------------------------------
Conflict_Code_New | Coef. Std. Err. z P>|z| [95% Conf. Interval]
-------------------------+----------------------------------------------------------------
_hat | .0085211 .0189385 0.45 0.653 -.0285977 .0456399
_hatsq | -.0042359 .0012209 -3.47 0.001 -.0066289 -.0018429
_cons | 3.564635 .2776186 12.84 0.000 3.020513 4.108758
-------------------------+----------------------------------------------------------------
var(e.Conflict_Code_New)| .4619219 .0498103 .3739221 .5706319
------------------------------------------------------------------------------------------

Regards,
Nitin

[Carlo Lazzaro]

Nitin:
the issue may depend on:
1) the different approaches that Stata uses to estimate the two auxiliary regressions;
2) -linktest- stops at fitted^2.

Taht said, I would stick with Jeff's suggested test.

[Nitin Jain]

Nitin:
the issue may depend on:
1) the different approaches that Stata uses to estimate the two auxiliary regressions;
2) -linktest- stops at fitted^2.

Taht said, I would stick with Jeff's suggested test.

Dear Carlo,

Thank you for the reply. I just want to confirm my understanding.

Link test only considers fitted and fitted sq values. Whereas Jeff's suggestion is to also include cube of fitted value also in the model. So, you would recommend doing that.

Regards,
Nitin