runby for rolling regressions/statistics

[River Huang]

Dear All, I found this question here (http://bbs.pinggu.org/thread-6392627-1-1.html). The purpose of this question is to obtain idiosyncratic volatility and skewness as follows.

1. For each `id' (but only one `id=1' here) and each month `t' (to obtain a single value only) , using daily observations from `t-2' to `t' to run the following regressions

   reg excessreturn hml smb rmrf

   and obtain the residuals `e'.
2. Calculate the standard deviation and skewness of `e'.

The sample data are:

* Example generated by -dataex-. To install: ssc install dataex
clear
input double(date ym excessreturn rmrf smb hml)
10503 345   -.01936289646965491 -.01127306  .00525148   .0006878
10504 345   .004428615495805104  .00170627 -.00085576 -.00139754
10505 345             -.0004251  .00996818 -.00446347  .00140606
10506 345             -.0004254   .0059196 -.00249881  .00250139
10507 345             -.0004254  .00319251  .00030991 -.00277563
10510 345             -.0004223 -.00024731  .00422974 -.00149079
10511 345             -.0004216 -.00274863  .00377518  .00549557
10512 345             -.0004223 -.01161097  .00538493  .00050242
10513 345             -.0004223  .00748774 -.00172834  .00512555
10514 345             -.0004223  .00450467 -.00003376  .00478568
10517 345   .023616446896690534  .01022073 -.00470417  .00443819
10518 345   .013620751277013725  -.0011541  .00067627 -.00090353
10519 345   .013474597771965517  .00363072  .00327984 -.00298672
10520 345             -.0004251  .00043836 -.00042181  .00095446
10521 345   .004160660287584015 -.00152961   .0025553   .0011501
10524 345   .004094270973417762 -.00565686  .00289626 -.00235665
10525 345   -.00491995430789627 -.00111718 -.00253788  .00240047
10526 345             -.0004223  .00032483  -.0014962  .00228125
10527 345   .017744790651005753  .00044753  .00076314  .00001995
10528 345   .008496248264910458  .00309713  .00082262  .00215505
10531 345             -.0004216 -.00123408  .00574452  .00172679
10532 346             -.0004251  .00287933 -.00061362     .00232
10533 346             -.0004233 -.00697585  .00488589  .00050676
10534 346             -.0004244 -.00285437  .00281202   .0029033
10535 346   .004018632959910915 -.00192175   .0009521  .00358065
10538 346             -.0004254 -.00776155   .0027741  .00311697
10539 346             -.0004264  .00952688 -.00659687  .00322087
10540 346             -.0004254 -.00755174  .00424434  .00203433
10541 346             -.0004244   .0000658 -.00040594  .00213749
10542 346             -.0004227 -.01219847  .00586826  .00206838
10545 346  -.013649725532307035 -.00454815 -.00256307  .00250378
10546 346             -.0004233  .00395039  .00248942  .00142837
10547 346             -.0004213  .00237563 -.00113014  .00247123
10548 346             -.0004223  .00695646 -.00743721 -.00091841
10549 346             -.0004244 -.00040684  .00169483  .00343632
10552 346   .004060976614823962 -.00631036  .00254483 -.00003972
10553 346  -.004886662993443917  .00421284 -.00169779  .00349698
10554 346              -.000424  .00715559 -.00209694  .00426515
10555 346  -.004908076614824184 -.00287966   .0015931  .00042118
10556 346  -.013841080905465748 -.02226113  .00126729 -.00038323
10559 346             -.0004548 -.00686149  .00134545  .00068337
10560 346 -.0050233243520851146  .00304422 -.00087426 -.00150591
10561 346             -.0004606  .00311911  .00006603  .00334213
10562 347  -.005045560287584237 -.00880611 -.00284232  .00227736
10563 347  -.005018397581225804 -.00939757 -.00306623 -.00021765
10566 347             -.0004585 -.00323499 -.00183136  .00373087
10567 347   -.00966786949185222  .00212448 -.00235887  .00124775
10568 347  -.005130805532577441  .00061133 -.00497766  -.0044441
10569 347  -.005102141236861527 -.00950524 -.00064092  .00103131
10570 347  -.005172627701375276 -.00722724 -.00339905  .00152378
10573 347  -.005192565455980971  .00015278 -.00530367 -.00236584
10574 347             -.0004548  .00223309  .00088146  .00437897
10575 347  -.005164832368107975 -.00032592 -.00276482  .00010165
10576 347             -.0004575  .00387507 -.00047641 -.00151807
10577 347    .00905806926717472  .00552013 -.00129807  .00101304
10580 347  -.005191265455980971 -.00128961 -7.196e-06  .00584483
10581 347             -.0004544  .00347369  -.0033825 -.00407795
10582 347             -.0004565 -.00266647 -.00067593 -.00204654
10583 347             -.0004551 -.00176227   .0036615 -.00504651
10584 347             -.0004517  .00185695 -.00313413  .00019805
10589 347             -.0004521  .00926918 -.00593432 -.00186907
10590 347             -.0004534  .00813243 -.00295587 -.00200299
10591 347             -.0004575 -.00457141  .00446324  .00128521
10595 348             -.0004603 -.00625428  .00348006  .00297693
10596 348  -.009930748234827456  .00525955  -.0027392 -.00272645
10597 348             -.0004603  .00329548 -.00122743 -.00069683
10598 348   .009101095144415997  .00577624 -.00208543 -.00008884
10601 348             -.0004593   .0099526 -.00551761 -.00026279
10602 348   .009059031733438961  .00179973  .00354946  .00461995
10603 348             -.0004603 -.00075754  .00440392  .00339012
10604 348             -.0004593  .00819852 -.00007103  .00246756
10605 348             -.0004593   .0060607   .0023039 -.00229719
10608 348   .004205411787819446  .00513847 -.00015631  .00334099
10609 348             -.0004613 -.00229445  .00057371 -.00277521
10610 348             -.0004613  .01093528 -.00588377 -.00158704
10611 348   .027600298631141437  .00921167 -.00284583 -.00459487
10612 348             -.0004613  .00313551 -.00084835 -.00063124
10615 348   .013138266298920526  .00374158  .00260237 -.00339525
10616 348             -.0004623  .00752605 -.00519193  .00170633
10617 348             -.0004617 -.00090837  .00317521  .00025433
10618 348             -.0004613  .00960258 -.00631583   .0045522
10619 348   .008546441027445393  .02269427 -.00559572 -.00016106
10622 348    .07504929897242767  .01651668 -.00309322 -.00189771
10623 348             -.0004613  .00320827  -.0016308  .00090841
10624 349             -.0004603  -.0049795  .00361908 -.00676923
10625 349  -.004567342497081826  .00225093  .00262478  .00050828
10626 349             -.0004599  .01362544  -.0009618  .00026414
10629 349             -.0004603 -.01070708  .00748479 -.00392056
10630 349             -.0004613  .01264815 -.00663474 -.00156815
10631 349   .003662679857614035   .0107934 -.00043577  .00168113
10632 349             -.0004613 -.00688215  .00523801  .00368968
10633 349  -.004567342497081826 -.01046152  .00620448  .00326649
10636 349  -.004626690171904928 -.01071695  .00314722 -.00381311
10637 349             -.0004593  .00668561 -.00352879  .00087794
10638 349             -.0004596 -.00106768  .00325114   .0002309
10639 349  .0037255299912816687  -.0052176  .00625345 -.00239602
10640 349  -.008750670029519073  .00303642 -.00170061 -.00133237
10643 349             -.0004579  .00946149 -.00533087  .00049292
10644 349             -.0004572 -.00169463  .00397498  .00187004
10645 349  -.004658732435981501 -.01182387  .00857389  .00020951
end
format %td date
format %tm ym

[Nick Cox]

You can run the regressions with rangestat (SSC) and then calculate the residuals. Then run rangestat again to get SD and skewness of residuals. Many of your questions result in advice to use rangestat, so it should be familiar to you.

Naturally there are other ways to do it with other commands but I think it does need two passes. You could of course wrap the two passes into one by writing a dedicated program to run with runby (SSC) but I would favour the approach above.

[River Huang]

Dear Nick, The one asked this question had tried a code like

rangestat (reg) excessreturn rmrf smb hml, interval(ym -2 0) 
gen Resid= excessreturn - b_cons - b_hml*hml - b_smb*smb - b_rmrf*rmrf
rangestat (sd) IV=Resid, int(ym -2 0)
rangestat (skewness) IS=Resid, int(ym -2 0)

but wondered if this is correct. In particular, he is worried about whether the calculation of residuals (`Resid') is adequate. I know that the third and fourth steps can be combined, though.

[Clyde Schechter]

Somebody else asked this same question, and gave the same example data, at https://www.statalist.org/forums/for...-of-this-month. My solution there uses -rangerun-, yet another approach.

[River Huang]

Dear Clyde, Thanks. I suppose that it is the same question raised by the same person. Ha! Ha!

[Nick Cox]

Note that the code in #3 can be simplified as SD and skewness can be calculated in one call.

 
 rangestat (sd) IV=Resid (skewness) IS=Resid, int(ym -2 0)

[River Huang]

Dear Nick, I see. But I found that the results from #3 are different from those provided by Clyde Schechter in #4.

[Robert Picard]

Clyde and Nick suggestions do not calculate the sd and skewness on the same set of residuals. When you first run the regressions using rangestat, the residuals are calculated using only the observations for that month. You cannot then get the same results that Clyde obtains when the residuals apply to all 3 months in the window. The results will match only for the first month as there are no prior months in the window. Here's Clyde's code tweaked to be more efficient since you only have to perform the regression once for each month. Load the data example in #1 first:

gen id = 1
save "dataex.dta", replace

capture program drop one_regression
program define one_regression
    regress excessreturn hml smb rmrf
    predict resid, resid
    summ resid, detail
    gen double sd = r(sd)
    gen double skewness = r(skewness)
    gen obs  = r(N)
    gen double mean = r(mean)
end

bysort id ym (date): gen high = cond(_n==1, ym, -99)
rangerun one_regression, by(id) interval(ym -2 high)
keep if !missing(sd)
keep id ym sd skewness obs mean
list

and the results:

. list

     +---------------------------------------------------------+
     |      ym   id          sd    skewness   obs         mean |
     |---------------------------------------------------------|
  1. | 1988m10    1   .00764541   .48224318    21    7.207e-11 |
  2. | 1988m11    1   .00630479   .86819698    43   -1.379e-11 |
  3. | 1988m12    1   .00561615   1.0268432    63   -3.811e-11 |
  4. |  1989m1    1   .00946129   4.2937221    63   -1.060e-11 |
  5. |  1989m2    1   .01003739   4.2176415    57   -4.027e-11 |
     +---------------------------------------------------------+

.

Now here's what you are trying to do in rangestat:

use "dataex.dta", clear
rangestat (reg) excessreturn hml smb rmrf, by(id) interval(ym -2 0)
gen Resid= excessreturn - (b_cons + b_hml*hml + b_smb*smb + b_rmrf*rmrf)
rangestat (count) N=Resid (sd) IV=Resid (skewness) IS=Resid (mean) Resid, by(id) int(ym -2 0)
bysort id ym (date): keep if _n == 1
list id ym IV IS N Resid_mean

and the results

. list id ym IV IS N Resid_mean

     +--------------------------------------------------------+
     | id        ym          IV          IS    N   Resid_mean |
     |--------------------------------------------------------|
  1. |  1   1988m10   .00764541   .48224318   21    7.207e-11 |
  2. |  1   1988m11   .00601843   .59611833   43    -.0005523 |
  3. |  1   1988m12   .00531221   .60336357   63   -.00064044 |
  4. |  1    1989m1   .00922525    4.707304   63   -.00026225 |
  5. |  1    1989m2   .00975446   4.5075617   57    -.0007265 |
     +--------------------------------------------------------+

.

[River Huang]

Dear Robert, Thank you so much. I will go through the code this afternoon.

[Micha]

Dear Ho-Chuan, were you able to confirm this approach? I tried the same approach (with a rolling window of 250 days) but the results dont pass some of the quality tests (e.g. IV is not correlated with CDS spread development) ...I am wondering if I implemented the approach in a wrong way or there are some issues with the data.

Best,
Micha