r(2001) Insufficient observations while looping

[Yuxi Chen]

Dear All,

I have a unbalanced panel data for 691 firms from 2000 to 2015, and I am using the forvalue for looping and it keep showing error r(2001). I have read the previous threads and understand the command -capture- is the right way to skip the error. However, it seems I didn't code it right :/

* Example generated by -dataex-. To install: ssc install dataex
clear
input long stkcd float(year eps dps)
 2 2000    .5979245   .2182174
 2 2001    .7339562  .24246375
 2 2002    .7646937  .26011115
 2 2003    .4816828 .064508885
 2 2004    .4520849  .17563193
 2 2005    .4188644  .18476103
 2 2006    .6070246  .17239834
 2 2007    .7931072   .1024866
 2 2008   .14777902  .05135466
 2 2009    .5860879  .07015415
 2 2010    .8054049  .10017643
 2 2011   1.0567758  .13021822
 2 2012     .511426  .18030176
 2 2013    .7092986  .41088885
 2 2014   1.0787876   .5012027
 2 2015    .9016325   .7202358
 6 2000    .8649804   .3213946
 6 2001    .8389993  .20890647
 6 2002    .5162996  .12855783
 6 2003  -1.7667146          0
 6 2004    .3603315  .12052297
 6 2005    .6881822   .2892551
 6 2006   1.1381309  .46875715
 6 2007    1.154605  .11600561
 6 2008    .6425335  .08783888
 6 2009    .6794741  .12337156
 6 2010   .36603865  .06160907
 6 2011    .5198129  .04087448
 6 2012    .4975616  .10160732
 6 2013      .52158  .15636133
 6 2014    .3989851  .11897808
 6 2015     .156163  .12087108
12 2000    .5194232  .24604318
12 2001   .37492135   .2166489
12 2002   .43572605   .2499795
12 2003    .5346876   .2999754
12 2004    .9081318   .4166325
12 2005   .52024543   .2999754
12 2006    .6212344    .685862
12 2007    .3601305   .2134129
12 2008    .3223521    .123815
12 2009    .9006728   .4214269
12 2010    .3410782   .3567134
12 2011    .6519145  .18201007
12 2012   .37885505  .15066135
12 2013    .7849684    .301494
12 2014   .23706283  .50243556
12 2015    .2031785     .30126
16 2000    .8652775   .3135344
16 2001   -1.623514          0
16 2002   .10829102          0
16 2003    .2690636          0
16 2004    .3756118          0
16 2005   .04511677          0
16 2006   .22791146  .12461361
16 2007    .4429887          0
16 2008    .2356838  .05986432
16 2009   .14658122  .01197285
16 2010 -.005788817  .01197285
16 2011   -.1490676  .01197287
16 2012   .17880037  .01197279
16 2013   .05951122  .01197279
16 2014   -.1991864  .01197279
16 2015   -.3468329          0
21 2000    .7518119  .18270127
21 2001   -.1149535  .21924153
21 2002    .3019944  .36540255
21 2003   1.1272192  .36540255
21 2004   1.0918223   .5481038
21 2005   1.5669395   .5481038
21 2006   1.1155739  1.1287524
21 2007    1.720212   .9568117
21 2008    .6279721  .35787025
21 2009    .1907588   .2002099
21 2010   .27035543   .1501574
21 2011   .14765975  .10010495
21 2012  .006413673  .02002193
21 2013   .09637035  .05582117
21 2014   .22070815  .05004916
21 2015    .1229407  .05004916
22 2000    .7163461   .3133819
22 2001    .7902093  .24779037
22 2002   1.8383032   .6996434
22 2003    3.146192  1.2049414
22 2004      4.0788   1.548465
22 2005   2.1984901     1.8147
22 2006   3.4579666    1.35625
22 2007     3.26745  1.4708734
22 2008   1.4024465  1.0069561
22 2009    .4412922   .3251477
22 2010     1.28336    .463484
22 2011   1.0370003   .4004913
22 2012   .29470757   .3634145
22 2013    .9970883   .3902345
22 2014    .8221648   .3241847
22 2015   .41003665   .4102743
23 2000    .4191937       .152
23 2001  -1.3520644          0
23 2002     .183314          0
23 2003   .21227683          0
end

//Prepare panel data
xtset stkcd year, yearly
egen id = group(stkcd)

//Create target payout ratio as the median payout over a 6-year period
sort id year
by id: gen n=_n
gen tpr=.
 forvalues i = 1(1)11 {
 by id: egen l=median(dps/eps) if n>=`i'&n<=`i'+5
 replace tpr=l if n==`i'+5
 drop l
 }
 replace tpr=dps/eps if tpr==. 

//Create SOA and RelVol mesuarements
sort id year
by id:gen dps1=dps[_n-1]
gen ddps=dps-dps1
by id:gen dev=tpr*eps-dps1
gen soa=.
gen relvol=.
gen y=tpr*eps
 forvalues j=1(1)691{
 forvalues i= 1(1)11{
 qui reg ddps dev if id==`j'&n>=`i'&n<=`i'+5
 replace soa=_b[dev] if id==`j'&n==`i'+5
 gen t=1 if n==`i'
 replace t=2 if n==`i'+1
 replace t=3 if n==`i'+2
 replace t=4 if n==`i'+3
 replace t=5 if n==`i'+4
 replace t=6 if n==`i'+5
 gen t2=t^2
 reg dps t t2 if id==`j'&n>=`i'&n<=`i'+5
 gen m1=e(rmse)
 reg y t t2 if id==`j'&n>=`i'&n<=`i'+5
 gen m2=e(rmse)
 replace relvol=(m1/m2)/100 if id==`j'&n==`i'+5
 drop t t2 m1 m2
 }
}

should I do following:

 forvalues j=1(1)691{
 forvalues i= 1(1)11{
 qui reg ddps dev if id==`j'&n>=`i'&n<=`i'+5
 replace soa=_b[dev] if id==`j'&n==`i'+5
 gen t=1 if n==`i'
 replace t=2 if n==`i'+1
 replace t=3 if n==`i'+2
 replace t=4 if n==`i'+3
 replace t=5 if n==`i'+4
 replace t=6 if n==`i'+5
 gen t2=t^2
 cap reg dps t t2 if id==`j'&n>=`i'&n<=`i'+5
    if c(rc) == 0 {
       gen m1=e(rmse)
    }
    else if c(rc) == 2001 {
        display "Insufficient results for i == `i': moving on."
    }
    else {
        display "Unanticipated error in regression with i = `i'"
        exit `c(rc)'
    }
cap reg y t t2 if id==`j'&n>=`i'&n<=`i'+5
if c(rc) == 0 {
       gen m2=e(rmse)
    }
    else if c(rc) == 2001 {
        display "Insufficient results for i == `i': moving on."
    }
    else {
        display "Unanticipated error in regression with i = `i'"
        exit `c(rc)'
    }

 replace relvol=(m1/m2)/100 if id==`j'&n==`i'+5
 drop t t2 m1 m2
 }
}

[Clyde Schechter]

A couple of things. But "didn't code it right" isn't very informative: it doesn't say what specifically is going wrong.

So here are some possibilities, but perhaps none of them applies to your situation, which could only be fully assessed by seeing the output you are getting.

You are capturing the "insufficient observations" correctly, but sometimes there is a "no observations" condition which uses a different error code, 2000. So I would change all of the -if c(rc) == 2001- guards to -if inlist(c(rc), 2000, 2001)-.

But more to the point, your -reg ddps dev ...- command is not within the scope of a -capture- and that is the one that is breaking your code. So add the -capture- apparatus to that regression as well.

[Yuxi Chen]

Thanks for your reply, Clyde. I adjusted my codes as you suggested as

 forvalues j=1(1)691{
 forvalues i= 1(1)11{
  cap reg ddps dev if id==`j'&n>=`i'&n<=`i'+5
  if c(rc) == 0 {
 replace soa=_b[dev] if id==`j'&n==`i'+5
 gen t=1 if n==`i'
 replace t=2 if n==`i'+1
 replace t=3 if n==`i'+2
 replace t=4 if n==`i'+3
 replace t=5 if n==`i'+4
 replace t=6 if n==`i'+5
 gen t2=t^2
    }
 else if inlist(c(rc), 2000, 2001) {
        display "Insufficient results for i == `i': moving on."
    }
    else {
        display "Unanticipated error in regression with i = `i'"
        exit `c(rc)'
    }
 cap reg dps t t2 if id==`j'&n>=`i'&n<=`i'+5
    if c(rc) == 0 {
       gen m1=e(rmse)
    }
    else if inlist(c(rc), 2000, 2001) {
        display "Insufficient results for i == `i': moving on."
    }
    else {
        display "Unanticipated error in regression with i = `i'"
        exit `c(rc)'
    }
cap reg y t t2 if id==`j'&n>=`i'&n<=`i'+5
if c(rc) == 0 {
       gen m2=e(rmse)
    }
    else if inlist(c(rc), 2000, 2001) {
        display "Insufficient results for i == `i': moving on."
    }
    else {
        display "Unanticipated error in regression with i = `i'"
        exit `c(rc)'
    }

 replace relvol=(m1/m2)/100 if id==`j'&n==`i'+5
 drop t t2 m1 m2
 }
}

now it ends with another error(111), and I couldn't identify where went wrong.

[Clyde Schechter]

What is happening here is that when i = 11, your first regression, -reg ddps dev if id==`j'&n>=`i'&n<=`i'+5- fails due to insufficient observations. Because of that failure, the code where you create the variables t and t2 is skipped. So when you get to the next regression, -cap reg dps t t2 if id==`j'&n>=`i'&n<=`i'+5-, the variables t and t2 do not exist, and when Stata does not find them, it gives you error 111 and the code breaks at that point.

The way to fix the problem depends on what is appropriate to do when that -reg ddps dev...- regression fails. If the implication of that is that you should not attempt the other regressions, then change

else if inlist(c(rc), 2000, 2001) {
        display "Insufficient results for i == `i': moving on."
    }

to

else if inlist(c(rc), 2000, 2001) {
        display "Insufficient results for i == `i': skipping to next value of i"
        continue
    }

If, however it is reasonable to attempt the other regressions even when the regression of ddps has failed, then you need to take the code that creates variables t and t2 and move it up above that -cap reg ddps...- command so it gets executed regardless of how that regression works out.

By the way, the code for generating t can be simplified to a single line:

gen t = n - `i' + 1 if inlist(n, `i'+1, `i'+5)

For that matter, you don't need the t2 variable. All of your regressions involving t and t2 can be shortened to, for example,
-reg outcomevariable c.t##c.t etc.- using factor variable notation. See -help fvvarlist-.

[Yuxi Chen]

Beautifully solved, thanks very much!

Continue have to be used in order to carry on the regression in this case.

[Charilaos Mertzanis]

Hi. I am experiencing a similar problem (i.e., r(2001) Insufficient observations while looping) and I cannot solve it. I have tried the suggestions of Mr. Schechter, and still cannot solve it. Could you please provide some help?
Example generated by -dataex-. For more info, type help dataex
clear
input float Firm_Index int YEAR float(TACC Factor1 Factor2 Factor3 Jones_1995)
1 2016 . . . . .
1 2017 35389000 1.8763152e-10 -.005427617 1.5555236 .
1 2018 -904056000 1.0265888e-10 -.015780518 .8425337 .
1 2019 36087952 1.0190518e-10 .02275726 1.0588201 .
1 2020 90688992 8.74134e-11 -.04202696 .8113834 .
1 2021 121371008 9.394574e-11 .01849754 .7906287 .
2 2016 . . . . .
2 2017 -73562000 2.863679e-10 .01691289 .57818574 .
2 2018 -753073984 2.93357e-10 .007420465 .5794513 .
2 2019 19509000 3.000005e-10 -.02365084 .57579035 .
2 2020 444191008 2.8724276e-10 -.008648592 .5342997 .
2 2021 132315056 3.067924e-10 .42540845 2.2208157 .
3 2016 . . . . .
3 2017 -2504369920 7.246377e-11 .2126876 .3884058 .
3 2018 1587289984 7.212926e-11 .02545586 .3241489 .
3 2019 -1675010048 8.614007e-11 .00532001 .3798777 .
3 2020 716719936 8.520065e-11 -.0043179686 .29607224 .
3 2021 1954790016 1.126253e-10 .03099899 .41817775 .
4 2016 . . . . .
4 2017 539598976 1.6090804e-10 -.00729557 .08844905 .
4 2018 -796003008 1.5299308e-10 .02139287 .11186258 .
4 2019 -152830016 1.612093e-10 -.016163329 .1561201 .
4 2020 -112936976 1.602204e-10 .0018572748 .08888963 .
4 2021 52000016 1.7904593e-10 .02862604 .09591275 .
5 2016 . . . . .
5 2017 -156773824 1.5632606e-09 -.031504072 .3729086 .
5 2018 26926256 1.4591973e-09 -.06352371 .3318227 .
5 2019 -25089864 1.410321e-09 .01801143 .3012573 .
5 2020 -86857320 1.2555044e-09 .03828004 .2496621 .
5 2021 77716408 1.23075e-09 .022641104 .24591713 .
6 2016 . . . . .
6 2017 -2907741952 1.2734085e-10 -.036197655 1.0850543 .
6 2018 -1678034944 1.2441001e-10 .08924054 .9718296 .
6 2019 -1928727040 6.0587486e-11 -.05083932 .4690149 .
6 2020 -1289389952 5.942836e-11 -.073851384 .4791959 .
6 2021 -2862233088 6.3022906e-11 .02664722 .5714961 .
7 2016 . . . . .
7 2017 -2356053248 1.0407954e-11 .030005705 .11695832 .
7 2018 6905618432 8.849557e-12 .05704581 .0942268 .
7 2019 7105283072 9.00901e-12 -.013168252 .09820213 .
7 2020 178715040 8.547009e-12 -.0391499 .08785017 .
7 2021 -4579428352 8.62069e-12 .07832967 .09159664 .
8 2016 . . . . .
8 2017 -95587440 5.973989e-10 -.035686005 .7422693 .
8 2018 25908212 5.738925e-10 -.03433557 .7001962 .
8 2019 -20764192 5.584652e-10 .04386726 .664176 .
8 2020 -180363776 5.0586674e-10 -.006476262 .5878113 .
8 2021 -115618768 5.747978e-10 -.03315066 .6722986 .
9 2016 . . . . .
9 2017 -53825168 7.249162e-10 .009754134 .57727295 .
9 2018 -247624320 7.666501e-10 -.06614171 .7045328 .
9 2019 -65059472 8.025346e-10 -.03773109 .7027134 .
9 2020 -116292744 8.161055e-10 -.04143424 .54547274 .
9 2021 -113775680 1.0893342e-09 .03994713 .7180434 .
10 2016 . . . . .
10 2017 -142444000 4.9348137e-10 -.1536864 .4279041 .
10 2018 -1732008 5.2677435e-10 .030482326 .405238 .
10 2019 -22199000 6.812968e-10 .04613946 .5206143 .
10 2020 -15850000 7.201472e-10 -.01453833 .5361762 .
10 2021 -21859000 7.65125e-10 .01931405 .5567784 .
11 2016 . . . . .
11 2017 130051008 1.0428006e-09 .001208606 .8794501 .
11 2018 -121094000 9.292048e-10 .09576756 1.104949 .
11 2019 131973000 6.556105e-10 .01368718 .6415863 .
11 2020 -226358016 8.236079e-10 -.02047242 .53387994 .
11 2021 168432000 1.159435e-09 -.013252342 .7268313 .
12 2016 . . . . .
12 2017 -4.01e+07 2.879023e-10 -.04836759 .3330742 .
12 2018 -1.177e+09 2.834708e-10 -.018624032 .31930155 .
12 2019 -1.759e+08 3.457336e-10 -.07858525 .3508159 .
12 2020 5.302e+08 4.080966e-10 .08778159 .3678175 .
12 2021 -1.95e+08 4.2968246e-10 .22618484 .27727407 .
13 2016 . . . . .
13 2017 -163190992 2.279975e-10 -.021090906 .24387227 .
13 2018 23701016 2.3637028e-10 .12419864 .2487996 .
13 2019 -378560992 2.109086e-10 .28677619 .3046224 .
13 2020 -967404032 1.7340925e-10 .015061268 .4772361 .
13 2021 -482979808 1.3517508e-10 .43975565 .54310876 .
14 2016 . . . . .
14 2017 -24821000 1.737241e-10 -.007300755 .1974242 .
14 2018 -262844000 1.723856e-10 -.001428387 .2036393 .
14 2019 -335471008 1.819161e-10 -.03110183 .2099297 .
14 2020 -457974976 1.822565e-10 -.034083422 .2100105 .
14 2021 177950000 1.9056737e-10 .11017653 .20738228 .
15 2016 . . . . .
15 2017 -201482000 2.0051e-10 -.0587398 .11584885 .
15 2018 -520440000 1.938075e-07 -21.29557 135.60982 .
15 2019 -28784000 1.872282e-10 .015312832 .1544303 .
15 2020 109676992 1.7611404e-10 .00577566 .20651133 .
15 2021 -194920000 1.7253844e-10 .05423401 .2464936 .
16 2016 . . . . .
16 2017 -1001027 2.071964e-09 .008588193 .028868835 .
16 2018 -3642921 2.4078575e-09 -.026842665 .031886704 .
16 2019 -1352393 2.6001956e-09 -.01454815 .01118084 .
16 2020 -2192289 2.8359524e-09 -.015909353 .01452347 .
16 2021 -1792113 2.986435e-09 -.013456152 .003095103 .
17 2016 . . . . .
17 2017 -43257000 5.58142e-10 .016353559 .4879572 .
17 2018 -21768008 5.449045e-10 -.0476361 .53513706 .
17 2019 -115655000 4.920846e-10 -.02394287 .4931696 .
end
format %ty YEAR

My code:

gen Jones_1995 =.
forval t = 2017(1)2021 {
forval i = 1(1)20 {
display `i'
display `t'
reg TACC Factor1 Factor2 Factor3 if `i' == Firm_Index & `t' == YEAR, noconstant
predict r if `i' == Firm_Index & `t' == YEAR, resid
replace Jones_1995 = r if `i' == Firm_Index & `t' == YEAR
drop r
}
}

[Clyde Schechter]

Well, if your example data is representative of your full data set, you are engaged in a hopeless task. Your data is panel data at the Firm level. So there is only one observation per firm in each year. Consequently, when you try to -regress- in the observations corresponding to a particular value of Firm_Index and Year, there is only one observation, so no regression is possible. That is why the approach shown earlier in this thread does not rescue you: you are trying to do impossible regressions and the code correctly identifies that.

Whatever your overall research goal is, you need to rethink your approach. Either a different analytic strategy that does not rely on firm-year specific regressions, or a different data set that makes such regressions possible is needed.

[Charilaos Mertzanis]

I would like to transform this dataset (whose cells show years) into a long dataset where each variable that shows the year in which a country has signed an international treaty (cov1, cov2, etc.) is transformed into a dummy variable taking the value of 1 for each year the treaty is signed for this country. Variables cov1, cov2 etc., are different international treaties. I have never experience such a data format before, so despite my efforts I am unable to reach my target. Could anyone help out?

Example generated by -dataex-. For more info, type help dataex
clear
input str52 country int(conv1 conv2 conv3 conv4 conv5 conv6 conv7 conv8 conv9 conv10 conv11 conv12 conv13 conv14)
"Afghanistan" 2013 1985 2002 2015 2013 2004 2017 . 2013 2013 . 1995 2002 1979
"Albania" 1999 2003 1994 2001 2005 1999 2016 1996 2010 2004 2003 2000 1994 1989
"Algeria" 1998 1983 1995 2005 2005 1992 2016 1984 . 2006 1996 1996 1993 1974
"Andorra" 1999 . 2015 . . 2009 2017 2012 . . . 2002 2011 1997
"Angola" 2017 2013 1998 2006 2007 2000 . . . 2006 1990 1997 2000 1991
"Antigua and Barbuda" 1993 1997 1993 2007 1998 1992 2016 2005 2010 2003 1989 1997 1993 1983
"Argentina" 1991 1981 1995 1992 2005 1990 2016 1992 2004 2005 1995 1997 1994 1978
"Armenia" 1999 2008 1993 2011 2003 1999 2017 1993 2003 2003 2002 1997 1993 1993
"Australia" 1992 1976 1993 1991 2007 1989 2016 1975 2004 2004 1994 2000 1992 1974
"Austria" 1993 1982 1994 2005 2002 1989 2016 1983 2002 2002 1995 1997 1994 1992
"Azerbaijan" 2001 1998 2000 . 2000 1996 2017 2001 . 2004 2016 1998 1995 1993
"Bahamas" 1992 1979 1993 . 1999 1993 2016 1997 . 2005 1983 2000 1994 2014
"Bahrain" 1992 2012 1996 . 2006 1990 2016 1998 2012 2006 1985 1997 1994 1991
"Bangladesh" 1993 1981 1994 2005 2001 1990 2016 1992 . 2007 2001 1996 1994 1983
"Barbados" 1995 1992 1994 . 2000 1992 2016 2006 . 2004 1993 1997 1

[Nick Cox]

Does this help?

clear
input str52 country int(conv1 conv2 conv3 conv4 conv5 conv6 conv7 conv8 conv9 conv10 conv11 conv12 conv13 conv14)
"Afghanistan" 2013 1985 2002 2015 2013 2004 2017 . 2013 2013 . 1995 2002 1979
"Albania" 1999 2003 1994 2001 2005 1999 2016 1996 2010 2004 2003 2000 1994 1989
"Algeria" 1998 1983 1995 2005 2005 1992 2016 1984 . 2006 1996 1996 1993 1974
"Andorra" 1999 . 2015 . . 2009 2017 2012 . . . 2002 2011 1997
"Angola" 2017 2013 1998 2006 2007 2000 . . . 2006 1990 1997 2000 1991
"Antigua and Barbuda" 1993 1997 1993 2007 1998 1992 2016 2005 2010 2003 1989 1997 1993 1983
"Argentina" 1991 1981 1995 1992 2005 1990 2016 1992 2004 2005 1995 1997 1994 1978
"Armenia" 1999 2008 1993 2011 2003 1999 2017 1993 2003 2003 2002 1997 1993 1993
"Australia" 1992 1976 1993 1991 2007 1989 2016 1975 2004 2004 1994 2000 1992 1974
"Austria" 1993 1982 1994 2005 2002 1989 2016 1983 2002 2002 1995 1997 1994 1992
"Azerbaijan" 2001 1998 2000 . 2000 1996 2017 2001 . 2004 2016 1998 1995 1993
"Bahamas" 1992 1979 1993 . 1999 1993 2016 1997 . 2005 1983 2000 1994 2014
"Bahrain" 1992 2012 1996 . 2006 1990 2016 1998 2012 2006 1985 1997 1994 1991
"Bangladesh" 1993 1981 1994 2005 2001 1990 2016 1992 . 2007 2001 1996 1994 1983
end 

* dates in data example span 1974 to 2017 
expand 44 
bysort country : gen year = 1973 + _n 

forval j = 1/14 {
    gen signed`j' = year == conv`j' if conv`j' < . 
}

egen anysigned = rowtotal(signed*)

list country year signed*  in 1/44 if anysigned

[Clyde Schechter]

I don't understand what you want here. It sounds to me like you want to create a series of indicator ("dummy") variables, one variable for each of 14 international treaties and each year between 1974 and 2017 (inclusive), a total of 616 variables in all. While it is not hard to do that, it is hard for me to imagine what can usefully be done with that. So if it is what you want, you might want to post back with an explanation of how you plan to use all those variables. Knowing where you are going with this could perhaps help with coming up with a better approach.

If that's not what you want, I suggest that you post back showing a brief example of what the result you do want would look like.

Added: Crossed with #2.