Xtset error on preparing data to run regression by industry and year

[Farhan Hasnat]

My data is panel data and the ambition is to run regression by industry and year. I first grouped the data using egen id = group(Industry), but when I tried xtset the data using code "xtset id Year, y" it gives me error "repeated time values within panel".

Lastly, if please help me to evaluate if the below command is right to run the regression by industry and year

bysort Industry Year: regress y x1 x2 x3

* Example generated by -dataex-. For more info, type help dataex
clear
input str9 Ticker str7 Sector str23 Industry str3 Board int Year long(WC01651 WC04860) float(LagAssets ChangeInSales ChangeReceivables) long WC02301 float(TA TA_SLaggedAssets InvLaggedAssets DeltaSales_SLaggedAssets PPE_LaggedAssets id)
"TH:2S"  "STEEL" "Industrials"             "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2001        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2002        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2003        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2004        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2005        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2006    34020   -17390        .        .        .   179050    51410           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2007    55390     -260   626450   571220    28880   331370    55650    .0888339 1.5962966e-06     .9118366   .5289648 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2008   112840   361700   739620  -172670  -132590   361490  -248860   -.3364701 1.3520456e-06    -.2334577    .488751 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2009    91864  -202289   559340  -235159    69236   395128   294153   .52589303 1.7878214e-06    -.4204223   .7064183 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2010    71431   -26964   913418   252536    -3135   409757    98395   .10772177  1.094789e-06    .27647364   .4485975 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2011    64980    91636  1024697   882014    82286   444190   -26656 -.026013544  9.758983e-07     .8607559   .4334842 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2012    52643   -61615  1059772   364111     -470   510104   114258   .10781375  9.435992e-07    .34357485   .4813337 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2013    68031   148004  1253012    33404    12820   551650   -79973  -.06382461   7.98077e-07   .026658963   .4402592 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2014    41898   108184  1219920   143357   -17705   632275   -66286  -.05433635  8.197259e-07    .11751344   .5182922 3
"TH:7UP" "ENERG" "Resources"               "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2001    84000   137396        .        .        .   385317   -53396           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2002   301429  -368824  1239837   516562   -55806   422035   670253    .5405977  8.065576e-07      .416637   .3403955 6
"TH:7UP" "ENERG" "Resources"               "SET" 2003  -123386  -170268  1445609  -668818   -39021   444089    46882  .032430623    6.9175e-07    -.4626548  .30719855 6
"TH:7UP" "ENERG" "Resources"               "SET" 2004   178368   180634  1589719   919817   233355  1170788    -2266 -.001425409   6.29042e-07     .5786035   .7364748 6
"TH:7UP" "ENERG" "Resources"               "SET" 2005   179860  -477969  2561613  4771508    63040  1868867   657829   .25680265   3.90379e-07    1.8626966   .7295665 6
"TH:7UP" "ENERG" "Resources"               "SET" 2006    67330   354412  3659381  2841869    21737  2269883  -287082  -.07845097 2.7327027e-07     .7765983   .6202915 6
"TH:7UP" "ENERG" "Resources"               "SET" 2007   146175   551275  4047774 -2839798  -107338  2561405  -405100   -.1000797 2.4704937e-07    -.7015703   .6327935 6
"TH:7UP" "ENERG" "Resources"               "SET" 2008   125678   158321  3776231 -2014388   -24807  2961212   -32643 -.008644333  2.648143e-07   -.53343874   .7841713 6
"TH:7UP" "ENERG" "Resources"               "SET" 2009   126637   308781  4178294 -2531712   -87615  3360213  -182144  -.04359291 2.3933214e-07      -.60592   .8042069 6
"TH:7UP" "ENERG" "Resources"               "SET" 2010    66485   201337  4126859   479308  -119606  3584112  -134852 -.032676667 2.4231502e-07    .11614354   .8684843 6
"TH:7UP" "ENERG" "Resources"               "SET" 2011  -625090   166489  4403855  -357337   -37866  3542542  -791579  -.17974684  2.270738e-07   -.08114187   .8044184 6
"TH:7UP" "ENERG" "Resources"               "SET" 2012   962232  -128523  4072216  -863105   462476   410310  1090755   .26785293 2.4556655e-07    -.2119497   .1007584 6
"TH:7UP" "ENERG" "Resources"               "SET" 2013    58762    46188  1211540  -383169  -434653   412256    12574  .010378527  8.253958e-07    -.3162661   .3402744 6
"TH:7UP" "ENERG" "Resources"               "SET" 2014  -214396    97692  1229760 -2144318    88283   384919  -312088   -.2537796  8.131668e-07   -1.7436882   .3130034 6
"TH:A"   "PROP"  "Property & Construction" "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2001        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2002        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2003    51542  -594372        .        .        .   140447   645914           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2004   179015 -1578393  2060645   762735    14264  1061154  1757408    .8528436 4.8528494e-07     .3701438   .5149621 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2005     8721  -322179  4016429  -809923   -31812  1092831   330900   .08238662  2.489774e-07    -.2016525   .2720902 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2006     8191    70257  4277750   -97783   -17095   134391   -62066  -.01450903 2.3376775e-07   -.02285851 .031416282 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2007   -78636  -459811  4294780   -67069    -4824   146235   381175    .0887531  2.328408e-07    -.0156164 .034049474 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2008    32582  -192720  5084116     3939    11741   220299   225302   .04431488 1.9669103e-07   .000774766  .04333084 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2009   518008  1032357  5719965  2632425    21475   255151  -514349   -.0899217 1.7482624e-07      .460217  .04460709 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2010   427951   -73901  5344282  -101860   -24365   290103   501852   .09390447 1.8711587e-07   -.01905962  .05428288 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2011    76002  -204507  5512695 -1787209     2722   320140   280509   .05088419  1.813995e-07    -.3241988  .05807323 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2012    94353  -668051  5730727   231452     -115   394755   762404   .13303792 1.7449793e-07     .0403879  .06888393 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2013    13777 -1828784  6866070   -15675    25418   543052  1842561   .26835746 1.4564372e-07 -.0022829655  .07909212 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2014    41266 -2147707  8974248   675358    -9742   786954  2188973   .24391715 1.1142995e-07     .0752551  .08769024 5
end
format %ty Year

[Jared Greathouse]

I'm confused.

* Example generated by -dataex-. For more info, type help dataex
clear
input str9 Ticker str7 Sector str23 Industry str3 Board int Year long(WC01651 WC04860) float(LagAssets ChangeInSales ChangeReceivables) long WC02301 float(TA TA_SLaggedAssets InvLaggedAssets DeltaSales_SLaggedAssets PPE_LaggedAssets id)

"TH:2S"  "STEEL" "Industrials"             "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2001        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2002        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2003        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2004        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2005        .        .        .        .        .        .        .           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2006    34020   -17390        .        .        .   179050    51410           .             .            .          . 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2007    55390     -260   626450   571220    28880   331370    55650    .0888339 1.5962966e-06     .9118366   .5289648 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2008   112840   361700   739620  -172670  -132590   361490  -248860   -.3364701 1.3520456e-06    -.2334577    .488751 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2009    91864  -202289   559340  -235159    69236   395128   294153   .52589303 1.7878214e-06    -.4204223   .7064183 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2010    71431   -26964   913418   252536    -3135   409757    98395   .10772177  1.094789e-06    .27647364   .4485975 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2011    64980    91636  1024697   882014    82286   444190   -26656 -.026013544  9.758983e-07     .8607559   .4334842 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2012    52643   -61615  1059772   364111     -470   510104   114258   .10781375  9.435992e-07    .34357485   .4813337 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2013    68031   148004  1253012    33404    12820   551650   -79973  -.06382461   7.98077e-07   .026658963   .4402592 3
"TH:2S"  "STEEL" "Industrials"             "SET" 2014    41898   108184  1219920   143357   -17705   632275   -66286  -.05433635  8.197259e-07    .11751344   .5182922 3
"TH:7UP" "ENERG" "Resources"               "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2001    84000   137396        .        .        .   385317   -53396           .             .            .          . 6
"TH:7UP" "ENERG" "Resources"               "SET" 2002   301429  -368824  1239837   516562   -55806   422035   670253    .5405977  8.065576e-07      .416637   .3403955 6
"TH:7UP" "ENERG" "Resources"               "SET" 2003  -123386  -170268  1445609  -668818   -39021   444089    46882  .032430623    6.9175e-07    -.4626548  .30719855 6
"TH:7UP" "ENERG" "Resources"               "SET" 2004   178368   180634  1589719   919817   233355  1170788    -2266 -.001425409   6.29042e-07     .5786035   .7364748 6
"TH:7UP" "ENERG" "Resources"               "SET" 2005   179860  -477969  2561613  4771508    63040  1868867   657829   .25680265   3.90379e-07    1.8626966   .7295665 6
"TH:7UP" "ENERG" "Resources"               "SET" 2006    67330   354412  3659381  2841869    21737  2269883  -287082  -.07845097 2.7327027e-07     .7765983   .6202915 6
"TH:7UP" "ENERG" "Resources"               "SET" 2007   146175   551275  4047774 -2839798  -107338  2561405  -405100   -.1000797 2.4704937e-07    -.7015703   .6327935 6
"TH:7UP" "ENERG" "Resources"               "SET" 2008   125678   158321  3776231 -2014388   -24807  2961212   -32643 -.008644333  2.648143e-07   -.53343874   .7841713 6
"TH:7UP" "ENERG" "Resources"               "SET" 2009   126637   308781  4178294 -2531712   -87615  3360213  -182144  -.04359291 2.3933214e-07      -.60592   .8042069 6
"TH:7UP" "ENERG" "Resources"               "SET" 2010    66485   201337  4126859   479308  -119606  3584112  -134852 -.032676667 2.4231502e-07    .11614354   .8684843 6
"TH:7UP" "ENERG" "Resources"               "SET" 2011  -625090   166489  4403855  -357337   -37866  3542542  -791579  -.17974684  2.270738e-07   -.08114187   .8044184 6
"TH:7UP" "ENERG" "Resources"               "SET" 2012   962232  -128523  4072216  -863105   462476   410310  1090755   .26785293 2.4556655e-07    -.2119497   .1007584 6
"TH:7UP" "ENERG" "Resources"               "SET" 2013    58762    46188  1211540  -383169  -434653   412256    12574  .010378527  8.253958e-07    -.3162661   .3402744 6
"TH:7UP" "ENERG" "Resources"               "SET" 2014  -214396    97692  1229760 -2144318    88283   384919  -312088   -.2537796  8.131668e-07   -1.7436882   .3130034 6
"TH:A"   "PROP"  "Property & Construction" "SET" 1994        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1995        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1996        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1997        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1998        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 1999        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2000        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2001        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2002        .        .        .        .        .        .        .           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2003    51542  -594372        .        .        .   140447   645914           .             .            .          . 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2004   179015 -1578393  2060645   762735    14264  1061154  1757408    .8528436 4.8528494e-07     .3701438   .5149621 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2005     8721  -322179  4016429  -809923   -31812  1092831   330900   .08238662  2.489774e-07    -.2016525   .2720902 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2006     8191    70257  4277750   -97783   -17095   134391   -62066  -.01450903 2.3376775e-07   -.02285851 .031416282 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2007   -78636  -459811  4294780   -67069    -4824   146235   381175    .0887531  2.328408e-07    -.0156164 .034049474 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2008    32582  -192720  5084116     3939    11741   220299   225302   .04431488 1.9669103e-07   .000774766  .04333084 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2009   518008  1032357  5719965  2632425    21475   255151  -514349   -.0899217 1.7482624e-07      .460217  .04460709 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2010   427951   -73901  5344282  -101860   -24365   290103   501852   .09390447 1.8711587e-07   -.01905962  .05428288 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2011    76002  -204507  5512695 -1787209     2722   320140   280509   .05088419  1.813995e-07    -.3241988  .05807323 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2012    94353  -668051  5730727   231452     -115   394755   762404   .13303792 1.7449793e-07     .0403879  .06888393 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2013    13777 -1828784  6866070   -15675    25418   543052  1842561   .26835746 1.4564372e-07 -.0022829655  .07909212 5
"TH:A"   "PROP"  "Property & Construction" "SET" 2014    41266 -2147707  8974248   675358    -9742   786954  2188973   .24391715 1.1142995e-07     .0752551  .08769024 5
end
format %ty Year

cls

 xtset id Year, y

if !_rc {
    
    disp "Good job!!! Data are now panel set."
}

br

runs fine for me. The only sensible explanation is that in your real dataset, year is repeated more than once in a unique ID variable, where it should not be. To check this, do

tempvar dup
egen `dup' = tag(id Year)

cap as `dup' == 1

if _rc {

br if `dup' ~= 1
}

and this gives you the list of datapoints where id and Year are non-unique (that is, where id and Year are not unique IDs of one another).




EDIT: In fact, we've sort of been over this before. In this post I advised that you make ticker the unique ID variable because it seems like the case that multiple different (companies???) can be part of the same industry. Thus, if Carlsberg and Heineken are in the same dataset and the industry is "BEER", no wonder the ids will be the same because they're both beer making companies and thus share a common industrial name.

[Farhan Hasnat]

Jared Greathouse Thank you Jared for getting back on this. I actually followed the earlier advice you gave but sort of dangled on processing my data to run the by group industry regression.

But thank you for your detailed explanation, I was missing on how Ticker needs to be the unique identifier. So, I believe bysort industry year and regression would just work.

Could you please help me advice on getting the predicted value from one regression and using it in another regression ? I know that I might need to use

predict x1 x2 x3

[Carlo Lazzaro]

Farhan:
1) why -xtset-ting your panel dataset if you then go -regress-?;
2) why going -regress- as your first choice if you have a panel dataset?
3) the correct code for predicted values is:

predict <thenameyouwant>, xb

[Farhan Hasnat]

Carlo Lazzaro Sorry I am not sure if I understood your questions correctly, however below are my answers if these helps in providing your input.

1. So, I am quite new to Stata and research overall, I came across couple of videos and threads where they were - xtset/declaring their data as panel before running the below regression

TA = β0 x1+ β1 x2 + β2 x3+ ԑit

2. I need to run regression of the above equation by industry and year, which is why I was using regress. After which, I need to use the predicted values from the above equation to run my second regression

NAi,t = ^β1(1/Ai,t-1) + ^β2 (ΔSi,t -ΔARi,t)/Ai,t-1) + ^β3 (PPEit/Ai,t-1)

Thank you for providing me with the code for predicted value and your further input if you have any. This really helps me to understand what I am doing and overall sharpen my knowledge in research.

[Clyde Schechter]

Your stated request is to create a new variable containing the predictions of the first regression so that it can be used in the second regression. But there is nothing in the second equation that looks like it represents the predicted values from the first. Rather, it looks like you want to somehow make use of the coefficients derived in the first equation. I'm not sure what _^β2 means, but I'm guessing it has something to do with the β₂ coefficient that comes out of the first equation. But perhaps one of those other variables in the second regression is an opaque name for the predicted value of TA from the first regression. Anyway, the following code will give you both:

capture program drop one_regression
program define one_regression
    regress TA x1 x2 x3
    gen beta1 = _b[x1]
    gen beta2 = _b[x2]
    gen beta3 = _b[x3]
    predict predicted_TA, xb
    exit
end

runby one_regression, by(Industry Year) status

-runby- is written by Robert Picard and me, and is available from SSC.

Note: This code runs quickly. At the end you will get a report of how many groups (combinations of industry and year) were processed, and how many contained errors. The most likely errors would be groups for which there were insufficient complete-data observations to carry out the regression, a commonly encountered situation in this kind of setting. These errors do not terminate the program: it just moves on to the next group until all have been done (or attempted). The data set will contain new variables beta1, beta12, and beta3 that contain the coefficients from the regressions, and a new variable predicted_TA which gives the predicted values of TA from the regression.

But let me anticipate a problem you will encounter in the second regression: it is ill-defined, precisely because of the problem you encountered originally with -xtset id Year-. If the i subscript refers to industry, and t to Year, then it is impossible to identify what the subscript combination _it-1 refers to. Because there can be multiple observations with the same values of i and t-1, it is impossible to know which one of those is intended and Stata cannot know which one to use in the regression. It seems to me you will have to do this with the i index ranging over individual firms, not industries, to make this work.

[Jared Greathouse]

Indeed Clyde Schechter In fact, that's part of why I didn't bring up the regression in my first response. If we're working with panel data, it has to be xtset correctly; if not, it'll just cause problems and everything won't work. As Alberto Abadie once said, we can't whisper to the computer "this is the panel data variable", a comment I found humorous.


On a related point, I question if OP literally wants to run the regression by firm and year. "By" in Stata isn't the same thing as "year by firm fixed effects" in econometrics, so I just wanted to confirm that running one regression per firm per year is what you want, instead of xtreg. Farhan Hasnat ha

[Nick Cox]

Backing up to #2: Jared Greathouse gave code

tempvar dup
egen `dup' = tag(id Year)

cap as `dup' == 1

if _rc {

br if `dup' ~= 1
}

which is loosely equivalent to

duplicates tag id year, gen(tag)
browse if tag

[Farhan Hasnat]

Clyde Schechter, Jared Greathouse & Nick Cox, thank you for all your advice ! The points raised are something I need to absorb and think about for while then work on.

[Farhan Hasnat]

Clyde Schechter Hi Ciyde I have used the -runby- command that you have suggested and seems like it did exactly what I was looking for. Now, just to add, probably being new in research and data analytics field my explanation itself had created a lot of confusions earlier. I am using Modified Jones Model (1991) and the literature uses these equations in such manner earlier stated.

Besides the literature suggests to run the regression by industry by year not per firm per year (Jared Greathouse).

I have tried an alternative approach instead of the -runby- command as follows:

1. egen id = group (ticker)
2. xtset id Year, y
3. by Industry Year, sort : regress X Term1 Term2 Term3, noconstant
4. predict NA, xb
5. predict DA, residual

Just to give some context, using the first regression (no.3) I find total accrual earnings and the second equation "NA" (no.4) is used for predicting normal level of accruals where coefficient estimates from the first equation is used.

Lastly, "DA" is discretionary accruals (no.5) which is basically the residual of the regression, giving me the measure I need to. Mathematically, DA = "Variable X" - "NA"

I would be grateful if you could provide any comments on whether this seems I following the right methods which would eventually help me move forward.

[Jared Greathouse]

No idea, you'd need to ask someone who studies the field of economics you're interested in. Whether this is the appropriate method for what you're interested in can't be told by us without knowing your research question.

[Clyde Schechter]

There is one thing that catches me eye in #10 that suggests a problem.

You have the following code sequence, which makes no sense:
[code]
by Industry Year, sort : regress X Term1 Term2 Term3, noconstant
predict NA, xb
predict DA, residual
[/code[
What this code does is regress X on Term 1, Term2, and Term3 (with constant term constrained to zero) separately for each combination of industry and year in your data set. Then you create two new variables NA, and DA, which are predicted values and residuals from the regression on the last combination of industry and year only, applying them to the calculation of NA and DA in all industries and years. If you only want to use the results of the last combination of industry and year, then there is no reason to use -by Industry Year, sort:...-. Just do the one regression with an -if- clause restricting the industry and year accordingly.

If you actually intend to use each regression, perhaps calculating NA and DA for each combination of industry and year using the regression model from that industry and year, your code does not do that. To do that, you need to go back to -runby- or some equivalent method.

capture program drop one_regression
program define one_regression
    regress X Term1 Term2 Term3, noconstant
    predict NA, xb
    predict DA, xb
    exit
end

runby one_regression, by(Industry Year)

Note: Above code is untested. Beware of typos or other errors. But this is the gist of how you would use each Industry-Year combination's regression to calculate NA and DA for that industry and year.

By the way, as Jared Greathouse, I have no idea whether what you're doing here makes sense from an econometrics perspective or not. You need to consult someone in that discipline. I am not an economist, and I have never heard of the Jones model. Your reference is too incomplete for me to attempt to find it were I so inclined, but it probably would not be available to me anyway (paywalled?), and I don't have the time to invest in learning about it.

[Jared Greathouse]

Agreed, this is precisely what I meant here. OP appears (I hope, anyways) to be confusing estimating regressions by unit-time (in the standard panel sense) with literally estimating it by in Stata. As a matter of fact, I'll solve the issue for everyone that's commented so far: I just went ahead and Googled what this seemingly obscure model even was. Carlo Lazzaro Clyde Schechter Farhan Hasnat Perhaps it would be helpful.

The pdf, luckily available on the first search, does indeed say

The model for discretionary accruals is estimated by every year and industry [emphasis mine]. I require at least 15 observations for each industry-year grouping.

but when we look at the seemingly simple equation here (equation 2), the authors literally say here that this is just some sexed-up OLS and not anything like originally wanted. Okay they didn't literally say that, but this is just an OLS estimator we're discussing here. I'm not an economist (not of the kind OP appears to be anyways), but this seems like a straightforward issue.

So, this is a rather long way of saying we're overcomplicating things. Such programming gymnastics thankfully aren't needed here, this model you'd like to run Farhan Hasnat is easily accomplished with just few time series operators and data transformations.

[Carlo Lazzaro]

Jared Greathouse: thanks for sharing.