Linear regression error "no observations" even though observation is there

[Farhan Hasnat]

I am using this code to run this following regression in the code. However, every time it's throwing error "no observations" as in the picture.

Before running the regression xtset my data

egen id = group (Ticker)

xtsest id Year, y

sort id Year

set more off

gen ABN_PROD = .                    

egen sicid = group(sic2 fyear)    
sum sicid
local max=r(max)

forvalues i = 1/`max' {

 di "Busy with regression " `i' " of " `max'
 qui reg PROD_SLAssset Inverse_LAssets Sales_SLAssets DSales_SLAssets Delta_SalesLag_SLAsset if sicid == `i' qui predict res if sicid == `i', res
 qui replace ABN_PROD = res if sicid == `i'
 qui drop res
 
}

* Example generated by -dataex-. For more info, type help dataex
clear
input str9 Ticker int Year str23 Industry byte SICCode float(PROD_SLAssset Inverse_LAssets Sales_SLAssets DSales_SLAssets Delta_SalesLag_SLAsset)
"TH:2S"  1994 "Industrials"             11         .             .         .            .            .
"TH:2S"  1995 "Industrials"             11         .             .         .            .            .
"TH:2S"  1996 "Industrials"             11         .             .         .            .            .
"TH:2S"  1997 "Industrials"             11         .             .         .            .            .
"TH:2S"  1998 "Industrials"             11         .             .         .            .            .
"TH:2S"  1999 "Industrials"             11         .             .         .            .            .
"TH:2S"  2000 "Industrials"             11         .             .         .            .            .
"TH:2S"  2001 "Industrials"             11         .             .         .            .            .
"TH:2S"  2002 "Industrials"             11         .             .         .            .            .
"TH:2S"  2003 "Industrials"             11         .             .         .            .            .
"TH:2S"  2004 "Industrials"             11         .             .         .            .            .
"TH:2S"  2005 "Industrials"             11         .             .         .            .            .
"TH:2S"  2006 "Industrials"             11         .             .         .            .            .
"TH:2S"  2007 "Industrials"             11 4.6497087 1.5962966e-06  4.737489     .9118366            .
"TH:2S"  2008 "Industrials"             11 3.2230334 1.3520456e-06  3.779143    -.2334577     .7723155
"TH:2S"  2009 "Industrials"             11  4.581761 1.7878214e-06  4.576771    -.4204223    -.3087031
"TH:2S"  2010 "Industrials"             11  2.906181  1.094789e-06  3.079102    .27647364   -.25744948
"TH:2S"  2011 "Industrials"             11 3.4135144  9.758983e-07 3.6054766     .8607559    .24644944
"TH:2S"  2012 "Industrials"             11   3.49226  9.435992e-07  3.829722    .34357485     .8322677
"TH:2S"  2013 "Industrials"             11  3.147663   7.98077e-07   3.26576   .026658963     .2905886
"TH:2S"  2014 "Industrials"             11  3.335886  8.197259e-07 3.4718614    .11751344   .027382124
"TH:7UP" 1994 "Resources"               33         .             .         .            .            .
"TH:7UP" 1995 "Resources"               33         .             .         .            .            .
"TH:7UP" 1996 "Resources"               33         .             .         .            .            .
"TH:7UP" 1997 "Resources"               33         .             .         .            .            .
"TH:7UP" 1998 "Resources"               33         .             .         .            .            .
"TH:7UP" 1999 "Resources"               33         .             .         .            .            .
"TH:7UP" 2000 "Resources"               33         .             .         .            .            .
"TH:7UP" 2001 "Resources"               33         .             .         .            .            .
"TH:7UP" 2002 "Resources"               33 4.0856404  8.065576e-07  4.338902      .416637            .
"TH:7UP" 2003 "Resources"               33  3.059031    6.9175e-07 3.2586355    -.4626548    .35733175
"TH:7UP" 2004 "Resources"               33 3.1488006   6.29042e-07   3.54184     .5786035    -.4207146
"TH:7UP" 2005 "Resources"               33  4.045958   3.90379e-07 4.0607376    1.8626966     .3590773
"TH:7UP" 2006 "Resources"               33 3.4078295 2.7327027e-07  3.619166     .7765983    1.3039112
"TH:7UP" 2007 "Resources"               33 2.3345294 2.4704937e-07 2.5703285    -.7015703     .7020819
"TH:7UP" 2008 "Resources"               33 2.0958078  2.648143e-07  2.221718   -.53343874    -.7520192
"TH:7UP" 2009 "Resources"               33  1.209359 2.3933214e-07 1.4020097      -.60592    -.4821078
"TH:7UP" 2010 "Resources"               33  1.425994 2.4231502e-07 1.5356272    .11614354    -.6134719
"TH:7UP" 2011 "Resources"               33 1.2415527  2.270738e-07 1.3578967   -.08114187    .10883828
"TH:7UP" 2012 "Resources"               33 1.0803432 2.4556655e-07 1.2565333    -.2119497      -.08775
"TH:7UP" 2013 "Resources"               33  3.894809  8.253958e-07  3.907181    -.3162661    -.7124032
"TH:7UP" 2014 "Resources"               33  1.725239  8.131668e-07 2.1056044   -1.7436882    -.3115803
"TH:A"   1994 "Property & Construction" 77         .             .         .            .            .
"TH:A"   1995 "Property & Construction" 77         .             .         .            .            .
"TH:A"   1996 "Property & Construction" 77         .             .         .            .            .
"TH:A"   1997 "Property & Construction" 77         .             .         .            .            .
"TH:A"   1998 "Property & Construction" 77         .             .         .            .            .
"TH:A"   1999 "Property & Construction" 77         .             .         .            .            .
"TH:A"   2000 "Property & Construction" 77         .             .         .            .            .
"TH:A"   2001 "Property & Construction" 77         .             .         .            .            .
"TH:A"   2002 "Property & Construction" 77         .             .         .            .            .
"TH:A"   2003 "Property & Construction" 77         .             .         .            .            .
"TH:A"   2004 "Property & Construction" 77 1.0460142 4.8528494e-07  .8610736     .3701438            .
"TH:A"   2005 "Property & Construction" 77 .22955093  2.489774e-07 .24012475    -.2016525    .18990377
"TH:A"   2006 "Property & Construction" 77 .13090853 2.3376775e-07  .2025974   -.02285851    -.1893339
"TH:A"   2007 "Property & Construction" 77  .3248509  2.328408e-07 .18617764    -.0156164   -.02276787
"TH:A"   2008 "Property & Construction" 77  .4043962 1.9669103e-07 .15804733   .000774766   -.01319187
"TH:A"   2009 "Property & Construction" 77  .3011982 1.7482624e-07  .6006953      .460217  .0006886406
"TH:A"   2010 "Property & Construction" 77  .4767359 1.8711587e-07  .6238623   -.01905962     .4925685
"TH:A"   2011 "Property & Construction" 77 .13769744  1.813995e-07 .28060448    -.3241988   -.01847735
"TH:A"   2012 "Property & Construction" 77  .3409032 1.7449793e-07  .3103165     .0403879    -.3118643
"TH:A"   2013 "Property & Construction" 77  .3273264 1.4564372e-07 .25672096 -.0022829655    .03370953
"TH:A"   2014 "Property & Construction" 77  .5052099 1.1142995e-07 .27166867     .0752551 -.0017466644
"TH:A5"  1994 "Services"                55         .             .         .            .            .
"TH:A5"  1995 "Services"                55         .             .         .            .            .
"TH:A5"  1996 "Services"                55         .             .         .            .            .
"TH:A5"  1997 "Services"                55         .             .         .            .            .
"TH:A5"  1998 "Services"                55         .             .         .            .            .
"TH:A5"  1999 "Services"                55         .             .         .            .            .
"TH:A5"  2000 "Services"                55         .             .         .            .            .
"TH:A5"  2001 "Services"                55         .             .         .            .            .
"TH:A5"  2002 "Services"                55         .             .         .            .            .
"TH:A5"  2003 "Services"                55 1.1525862   9.27463e-06   2.13323    1.6549838            .
"TH:A5"  2004 "Services"                55  .8782888 4.1007806e-06  1.278562     .3353536     .7317515
"TH:A5"  2005 "Services"                55  .9454873  3.986303e-06 1.0757637   -.16710582     .3259919
"TH:A5"  2006 "Services"                55  .8153177  4.708519e-06   .790372    -.4802925   -.19738112
"TH:A5"  2007 "Services"                55         .   7.53909e-06 1.9890834     .7235717    -.7690249
"TH:A5"  2008 "Services"                55         . 3.0673905e-06  .8288059   .019517805    .29439586
"TH:A5"  2009 "Services"                55 .50154984  2.404754e-06  .7490495    .09928747   .015301448
"TH:A5"  2010 "Services"                55 1.0619361  5.579828e-06  .8722108     -.865833    .23037992
"TH:A5"  2011 "Services"                55   2.92249  8.166732e-06  3.375139     2.098556    -1.267248
"TH:A5"  2012 "Services"                55 1.8582982  8.965635e-06  1.859912   -1.8453966    2.3038454
"TH:A5"  2013 "Services"                55   .777864 .000010775978  .9920258     -1.24344   -2.2180195
"TH:A5"  2014 "Services"                55  76.49998  .00001781642 1.1354939    -.5046679   -2.0558367
"TH:AA"  1994 "Industrials"             11 .15526025  5.469568e-07 .28518873     .1788937            .
"TH:AA"  1995 "Industrials"             11  .2614009  1.410538e-07 .12367302    .05012615    .04613461
"TH:AA"  1996 "Industrials"             11  .1655263  6.650534e-08 .18224686    .12393637    .02363394
"TH:AA"  1997 "Industrials"             11  .1580202  5.070889e-08  .3385713      .199612     .0944988
"TH:AA"  1998 "Industrials"             11  .1568674  2.425732e-08  .2310874   .069126986    .09548724
"TH:AA"  1999 "Industrials"             11 .21467777 2.3942054e-08  .3114601     .0833761    .06822856
"TH:AA"  2000 "Industrials"             11  .3019628  3.158392e-08  .4725594    .06168694    .10998823
"TH:AA"  2001 "Industrials"             11 .27098876 3.1228307e-08  .4884446   .021205833    .06099238
"TH:AA"  2002 "Industrials"             11  .3005794 3.3299195e-08  .4917017   -.02913373   .022612084
"TH:AA"  2003 "Industrials"             11  .4097156  3.456964e-08  .6128936    .10243217   -.03024526
"TH:AA"  2004 "Industrials"             11 .43288425  3.376023e-08  .7015701     .1030268    .10003382
"TH:AA"  2005 "Industrials"             11  .4946924  3.622872e-08  .7147999   -.03806784    .11055996
"TH:AA"  2006 "Industrials"             11 .50810826 3.3656402e-08  .7413872    .07733966  -.035364937
"TH:AA"  2007 "Industrials"             11  .5259682 3.3217717e-08  .7778341    .04611031     .0763316
"TH:AA"  2008 "Industrials"             11         . 3.5009858e-08         .            .    .04859802
"TH:AA"  2009 "Industrials"             11         .             .         .            .            .
end
format %ty Year

[Ken Chui]

Just a few clarifications:

• It's -xtset-, not -xtsest-.
• Two variables, sic2 and fyear, were not provided with the sample data, making replicating the error impossible.
• In the program, the regression and prediction commands were mixed into one line, separate them back into two lines.

Having no missing in sicid does not guarantee no missing data in the variables that the model uses. Since the regression got stuck at the first iteration, have you tried running the subset alone (using vanilla code, not the program and not made quiet), and see if there is any 0 case problem? For example, this:

reg PROD_SLAssset Inverse_LAssets Sales_SLAssets DSales_SLAssets Delta_SalesLag_SLAsset if sicid == 1

[Farhan Hasnat]

Ken Chui Thank you for your advice and my apologies for inputing the wrong commands in my question. I will check on what you have suggested now and see if the error still exits.

Just for reference the correct code is where in line 3 " egen sicid = group (SICCode Year)

set more off  
gen ABN_PROD = .                      
egen sicid = group(SICCode Year)    
sum sicid local max=r(max)

forvalues i = 1/`max' {  

di "Busy with regression " `i' " of " `max'  
qui reg PROD_SLAssset Inverse_LAssets Sales_SLAssets DSales_SLAssets Delta_SalesLag_SLAsset if sicid == `i'  
qui predict res if sicid == `i', res  
qui replace ABN_PROD = res if sicid == `i'  
qui drop res  
}

[Farhan Hasnat]

Ken Chui I tried what you have just suggested and it is indeed for 1 and 2 stops the iteration. However, just wanted to clear my understand when I input sicid =3 , it means that the subset is of sicid = 3 and for a particular year. If so, then is it normal for the program to calculate values of ABN_PROD for all sicid and Year ?

Thank you for your help once again

[Ken Chui]

Ken Chui I tried what you have just suggested and it is indeed for 1 and 2 stops the iteration. However, just wanted to clear my understand when I input sicid =3 , it means that the subset is of sicid = 3 and for a particular year. If so, then is it normal for the program to calculate values of ABN_PROD for all sicid and Year ?

Thank you for your help once again

Regarding the question"If so, then is it normal for the program to calculate values of ABN_PROD for all sicid and Year ?" Yes, it should work but the problem is one of the iterations got stuck. And once a loop is stuck, the whole process will stop.

Let me try to explain what happened, the current problem has two parts:

1) Regression needs 3 cases or more because one degree of freedom is used to estimate the intercept, one is used to estimated the regression coefficient, and one is to be divided by the residual sum of squares. If there are not enough cases, Stata will return error "no observation."

2) As the way you wrote your looping program, Stata will stop the loop when any one iteration was stopped.

To avoid this, it'd be better to first select the sicid-year combinations that has at lease three valid data points, and create a data subset with them before you applied that looping program.

[Farhan Hasnat]

Ken Chui Thank you so much for explaning the entire process to me.

Yes, what I want the program to do is calculate value for all sicid and year. So bascially I am running the regression by Industry by Year and sicid is just a proxy for the industry I am using for the sake of using non-string variables.

You are right going through my data, I did not notice there tons of missing vlaues in the earlier years i.e. 1994 ~ 2000. Proably that's what stopping the iteration.