Updating Code

[Sartaj Hussain]

Below is the example data, We run the main code 1 that follows and it yields ew and vw returns on decile portfolios.

clear
input int stock_id str52 stock float(date mdate rt mcap idiovol)
1 "3M India Ltd."              19875 653           .   5078.8  .07464047
1 "3M India Ltd."              19905 654   .08684798  5519.88  .09564243
1 "3M India Ltd."              19936 655   .20035714  6625.83   .1153478
1 "3M India Ltd."              19967 656    .0796447  7153.54  .05842065
1 "3M India Ltd."              19997 657 -.015353846  7043.71  .05778759
1 "3M India Ltd."              20028 658  .004470022   7075.2  .05916266
1 "3M India Ltd."              20058 659  .030267583  7289.35  .07579362
1 "3M India Ltd."              20089 660     .049886  7652.98  .05058851
1 "3M India Ltd."              20120 661   .16551733  8919.68  .08370074
1 "3M India Ltd."              20148 662   .00643473  8977.08  .07558923
1 "3M India Ltd."              20179 663 -.002064286  8958.55  .11817545
1 "3M India Ltd."              20209 664  .005815781  9010.65  .05396339
1 "3M India Ltd."              20240 665   .03876228  9359.92  .12874845
2 "63 Moons Technologies Ltd." 19875 653           .   1162.1  .06185364
2 "63 Moons Technologies Ltd." 19905 654   .26605865  1471.29   .2051615
2 "63 Moons Technologies Ltd." 19936 655  -.20388347  1171.32   .1596595
2 "63 Moons Technologies Ltd." 19967 656  -.12077104  1029.86  .11630887
2 "63 Moons Technologies Ltd." 19997 657   -.2187919   804.53   .2429151
2 "63 Moons Technologies Ltd." 20028 658   .04925539   844.16  .08935963
2 "63 Moons Technologies Ltd." 20058 659   .10807862   935.39  .15593943
2 "63 Moons Technologies Ltd." 20089 660 -.007389163   928.48 .036555734
2 "63 Moons Technologies Ltd." 20120 661  -.11315138   823.42  .08614067
2 "63 Moons Technologies Ltd." 20148 662   .07302744   883.56    .237539
2 "63 Moons Technologies Ltd." 20179 663  -.10169491    793.7 .074082345
2 "63 Moons Technologies Ltd." 20209 664  -.05573298   749.47  .03879507
2 "63 Moons Technologies Ltd." 20240 665   -.0624039    702.7  .15120164
3 "A B B India Ltd."           19875 653           . 23777.18  .07483307
3 "A B B India Ltd."           19905 654    -.072323 22057.54  .07221169
3 "A B B India Ltd."           19936 655  -.05596119 20823.18  .03861913
3 "A B B India Ltd."           19967 656   .16755714 24312.25  .12428077
3 "A B B India Ltd."           19997 657  .016952803 24724.41  .06956107
3 "A B B India Ltd."           20028 658  -.04795368 23538.78  .06761845
3 "A B B India Ltd."           20058 659    .1579041 27255.66  .16287985
3 "A B B India Ltd."           20089 660 -.001088402 27225.99  .04814025
3 "A B B India Ltd."           20120 661    .0859277 29565.46  .10311798
3 "A B B India Ltd."           20148 662  -.09905387 26636.88  .05732526
3 "A B B India Ltd."           20179 663   .05962605 28225.14  .06130007
3 "A B B India Ltd."           20209 664  .006043807 28395.72  .06657626
3 "A B B India Ltd."           20240 665 -.012873135 28030.18  .05802079
4 "A B G Shipyard Ltd."        19875 653           .  1432.68  .09499458
4 "A B G Shipyard Ltd."        19905 654   -.0972099  1293.41  .05455963
4 "A B G Shipyard Ltd."        19936 655   -.0690945  1204.05  .11501205
4 "A B G Shipyard Ltd."        19967 656  -.04715582  1147.27  .12661067
4 "A B G Shipyard Ltd."        19997 657  .009542804  1188.16  .06451628
4 "A B G Shipyard Ltd."        20028 658 -.011870728  1174.06  .09569627
4 "A B G Shipyard Ltd."        20058 659   .02602895  1234.19    .063639
4 "A B G Shipyard Ltd."        20089 660  -.11513444  1092.09 .027290424
4 "A B G Shipyard Ltd."        20120 661   .15731433  1269.39   .2665481
4 "A B G Shipyard Ltd."        20148 662  -.03980519  1220.81  .21235485
4 "A B G Shipyard Ltd."        20179 663  -.09768464  1101.56  .07366299
4 "A B G Shipyard Ltd."        20209 664   -.3022972   769.47  .12029396
4 "A B G Shipyard Ltd."        20240 665    .2980736   998.83   .3044246
5 "A C C Ltd."                 19875 653           . 27561.96  .05712568
5 "A C C Ltd."                 19905 654   -.0501005 26181.09  .06022657
5 "A C C Ltd."                 19936 655  .069702365 28005.97  .05895481
5 "A C C Ltd."                 19967 656  -.05869138 26362.26  .04966506
5 "A C C Ltd."                 19997 657  .068902895  28178.7  .03969718
5 "A C C Ltd."                 20028 658 -.020354487 27605.14 .033498686
5 "A C C Ltd."                 20058 659  -.04825379 26273.09  .04008862
5 "A C C Ltd."                 20089 660   .11479922 29289.21  .05381659
5 "A C C Ltd."                 20120 661    .0789718 31602.24  .05813709
5 "A C C Ltd."                 20148 662  -.07286501 29299.54  .05722007
5 "A C C Ltd."                 20179 663    -.078111 27010.92  .08348867
5 "A C C Ltd."                 20209 664   .05272128 28434.97    .044722
5 "A C C Ltd."                 20240 665  -.04879339 27047.53  .05062318
6 "A I A Engineering Ltd."     19875 653           .  7324.45  .08130153
6 "A I A Engineering Ltd."     19905 654  .005601747  7365.48  .08059192
6 "A I A Engineering Ltd."     19936 655   .12549622  8289.82  .08216862
6 "A I A Engineering Ltd."     19967 656   .07094092   8877.9  .09176414
6 "A I A Engineering Ltd."     19997 657 -.001221806  8867.06  .06304555
6 "A I A Engineering Ltd."     20028 658    .2806085 11355.23  .10604648
6 "A I A Engineering Ltd."     20058 659  -.09942693 10226.21  .09121463
6 "A I A Engineering Ltd."     20089 660   .04819222 10719.04  .11355854
6 "A I A Engineering Ltd."     20120 661 -.004223615 10673.76  .07496843
6 "A I A Engineering Ltd."     20148 662   .10087043 11750.43  .09606886
6 "A I A Engineering Ltd."     20179 663  -.07617597 10855.33  .10489568
6 "A I A Engineering Ltd."     20209 664  -.11560522   9600.4  .09647164
6 "A I A Engineering Ltd."     20240 665  -.01709482  9436.28  .07773334
7 "Aarti Drugs Ltd."           19875 653           .   640.36  .12179978
7 "Aarti Drugs Ltd."           19905 654    .1787929   754.91  .08089033
7 "Aarti Drugs Ltd."           19936 655   .19337867   900.88  .16471484
7 "Aarti Drugs Ltd."           19967 656  .036881723   934.11  .08244735
7 "Aarti Drugs Ltd."           19997 657 -.063984215    874.3  .05832432
7 "Aarti Drugs Ltd."           20028 658   .10558385   966.69   .0699918
7 "Aarti Drugs Ltd."           20058 659  .067942664  1032.31  .08391946
7 "Aarti Drugs Ltd."           20089 660    .4081825  1453.75  .10373413
7 "Aarti Drugs Ltd."           20120 661    .0470447  1522.04  .14374593
7 "Aarti Drugs Ltd."           20148 662  .036975738  1578.35  .14562993
7 "Aarti Drugs Ltd."           20179 663  .011476587  1596.51  .20718463
7 "Aarti Drugs Ltd."           20209 664  .012559962  1616.49   .1337734
7 "Aarti Drugs Ltd."           20240 665  -.05447035  1528.46   .1502725
8 "Aarti Industries Ltd."      19875 653           .  1803.28  .11358362
8 "Aarti Industries Ltd."      19905 654   .05777171  1907.38   .0686042
8 "Aarti Industries Ltd."      19936 655    .2983466  2476.58  .10981288
8 "Aarti Industries Ltd."      19967 656  .002432368  2482.78  .08899546
8 "Aarti Industries Ltd."      19997 657   .08121614  2684.33  .09928838
8 "Aarti Industries Ltd."      20028 658  -.05333335  2541.25  .04957134
8 "Aarti Industries Ltd."      20058 659  -.11825407  2240.48  .08447535
8 "Aarti Industries Ltd."      20089 660    .1230429  2516.45  .06511408
8 "Aarti Industries Ltd."      20120 661  .018025612  2561.63   .1415287
end
format %td date
format %tm mdate

1. Main code

by mdate, sort: egen idiovol_decile = xtile(idiovol), nq(10)

capture program drop one_weighted_return
program define one_weighted_return
if !missing(idiovol_decile){
    egen numerator = total(mcap*rt)
    egen denominator = total(mcap)
    gen vw_mean_rt = numerator/denominator
    }
    exit
end

runby one_weighted_return, by(mdate idiovol_decile)

by mdate idiovol_decile, sort: egen ew_mean_rt = mean(rt)

keep mdate idiovol_decile *_mean_rt
drop if missing(idiovol_decile, vw_mean_rt, ew_mean_rt)

by mdate *_decile, sort: gen stock_count = _N
by mdate idiovol_decile, sort: keep if _n == 1

gen str32 group = "_idiovol_d" + string(idiovol_d) + "_"
drop *_decile
reshape wide @stock_count vw_mean_@rt ew_mean_@rt, i(mdate) j(group) string

On the output, i want to calculate:

a. mean and sd matrix
b. regression intercept, coefficients, t-stat, r-squared, adj r-squared.

We used following codes for this purpose previously but it was 5x5 bivariate sorts portfolios then and now it is univariate deciles. Request you to amend the following codes (code 2 and code 3) so that they are compatible with new requirement.

2. code for means and sd

frame create means_and_sds int (idiovol_group mcap_group) ///
    float(mean_vw sd_vw mean_unwtd sd_unwtd)
forvalues iv = 1/5 {
    forvalues mc = 1/5 {
        summ vw_mean_q`mc'_idiovol_q`iv'_rt
        local mean_vw = r(mean)
        local sd_vw = r(sd)
        summ unwtd_mean_q`mc'_idiovol_q`iv'_rt
        local mean_unwtd = r(mean)
        local sd_unwtd = r(sd)
        frame post means_and_sds (`iv') (`mc') (`mean_vw') (`sd_vw') ///
            (`mean_unwtd') (`sd_unwtd')
    }
}
frame change means_and_sds
rename (mean* sd*) =_idiovol_
reshape wide *_idiovol_, i(mcap_group) j(idiovol_group)

2. code for regression coefficients etc.

frame create regress_results int (idiovol_group mcap_group) ///
float(intercept tstat b_rmrf t_rmrf b_smb t_smb b_hml t_hml b_wml t_wml b_himli t_himli r2 adj_r2)
forvalues iv = 1/5 {
    forvalues mc = 1/5 {
        local group q`mc'_idiovol_q`iv'
        regress vw_mean_`group'_rt rmrf smb hml wml himli
        matrix M = r(table)
        local topost (`iv') (`mc') (M["b", "_cons"]) (M["t", "_cons"])
        foreach x in rmrf smb hml wml himli {
            local topost `topost' (M["b", "`x'"]) (M["t", "`x'"])
        }
        frame post regress_results `topost' (e(r2)) (e(r2_a))
    }
}
frame change regress_results
rename (intercept tstat b_* t_* r2 adj_r2) =_idiovol_
reshape wide *_idiovol_, i(mcap_group) j(idiovol_group)

I hope main code does sorts each month. Instead, if we wish to have:-

i. sorts only once in June month to form decile and weighting of vw returns for each month using june month mcap values observed at start of year.
ii. sorts each month on previous idiovol values to form deciles and weighting of vw returns by lagged mcap values of each rt observation.

What should the main code 1 be in case of i and ii.

[Clyde Schechter]

The codes for the means/standard deviations and regression results are:

frame create means_and_sds int (idiovol_group) ///
    float(mean_vw sd_vw mean_ew sd_ew)
forvalues iv = 1/10 {
    summ vw_mean__idiovol_d`iv'_rt
    local mean_vw = r(mean)
    local sd_vw = r(sd)
    summ ew_mean__idiovol_d`iv'_rt
    local mean_ew = r(mean)
    local sd_ew = r(sd)
    frame post means_and_sds (`iv')(`mean_vw') (`sd_vw') ///
        (`mean_ew') (`sd_ew')
}
frame change means_and_sds
rename (mean* sd*) =_idiovol

frame change default
frame create regress_results int (idiovol_group) ///
float(intercept tstat b_rmrf t_rmrf  b_smb t_smb b_himli t_himli b_wml t_wml b_himli t_himli r2 adj_r2)
forvalues iv = 1/ 10 {
    regress vw_mean__idiovol_d`iv'_rt rmrf smb hml wml himli

    matrix M = r(table)
    local topost (`iv') (M["b", "_cons"]) (M["t", "_cons"])
    foreach x in rmrf smb hml wml himli {
        local topost `topost' (M["b", "`x'"]) (M["t", "`x'"])

    frame post regress_results `topost' (e(r2)) (e(r2_a))
}
frame change regress_results
rename (intercept tstat b_* t_* r2 adj_r2) =_idiovol_

Notes:
1. The output with the example data has only 9 groups, not 10. I have written the code on the assumption that the full data will support 10 deciles.
2. As the data provided has no rmrf, smb, hml, wml, and himli variables, the regression code is untested.

I hope main code does sorts each month.

Yes, it does.

Instead, if we wish to have:-

i. sorts only once in June month to form decile and weighting of vw returns for each month using june month mcap values observed at start of year.
ii. sorts each month on previous idiovol values to form deciles and weighting of vw returns by lagged mcap values of each rt observation.

What should the main code 1 be in case of i and ii.

For i:

// CODE TO CALCULATE ONE PARTITION INTO DECILES PER FYEAR (JULY-JUNE)
// BASED ON PRECEDING JUNE MCAP
gen moy = month(dofm(mdate))
gen year = year(dofm(mdate))

//  CREATE A "FISCAL YEAR" RUNNING FROM JULY THROUGH SUBSEQUENT JUNE
gen fyear = cond(moy > 6, year, year-1)
frame put stock_id fyear mcap if moy == 6, into(mcap_idiovol_work)
frame change mcap_idiovol_work
collapse (count) n_mcap = mcap (firstnm) mcap, by(stock_id fyear)
assert n_mcap <= 1  // VERIFY UNIQUE VALUE OF MCAP AND idiovol
replace fyear = fyear + 1 // CHANGE THE FYEAR TO WHICH THEY WILL APPLY
rename mcap prior_june_=
frame change default

frlink m:1 stock_id fyear, frame(mcap_idiovol_work)
frget prior_june_*, from(mcap_idiovol_work)
frame drop mcap_idiovol_work
drop mcap_idiovol_work
by fyear, sort: egen mcap_group = xtile(prior_june_mcap), nq(10)

The variable mcap_group will contain the decile at the end of this code.

To get vw_mean_rt weighted by the June mcap immediately preceding the start of the fiscal year, substitute this code:

capture program drop one_weighted_return
program define one_weighted_return
if !missing(mcap_group){
    egen numerator = total(prior_june_mcap*rt)
    egen denominator = total(prior_june_mcap)
    gen vw_mean_rt = numerator/denominator
    }
    exit
end

runby one_weighted_return, by(mdate mcap_group)

End of for i.

For ii, I am uncertain whether the term "previous idiovol" refers to the one-month lag of idiovol or the idiovol from the June prior to the start of the fiscal year. Please clarify. The weighting of vw_mean_rt by lagged value of mcap looks like this:

xtset stock_id mdate
gen lagged_mcap = L1.mcap
capture program drop one_weighted_return
program define one_weighted_return
if !missing(mcap_group){
    egen numerator = total(lagged_mcap*rt)
    egen denominator = total(lagged_mcap)
    gen vw_mean_rt = numerator/denominator
    }
    exit
end

runby one_weighted_return, by(mdate mcap_group)

[Sartaj Hussain]

Thanks. I will try the codes. Well, can we add multicollinearity test to know VIF's to regressions part.

[Clyde Schechter]

You can, with the -estat vif- command. I'll let you figure out the details. But do note that testing for multicollinearity is a waste of time and effort. Do get a copy of Arthur Goldberger's A Textbook of Economics. There is a whole chapter devoted to why testing for multicollinearity makes no sense and should never be done.

[Sartaj Hussain]

Yes it says so. But does it stress on efficiency of SEs of coefficients instead, which is what i could understand can be resolved by having fairly large sample.

[Sartaj Hussain]

In case of #2 for i, it should be idiovol based sort and not mcap based. To keep things like previous code, it may be named as idiovol_decile and not with _group extension. However, weighting on mcap basis is ok. Therefore, code needs amendment again.Sorry i have made mistake in writing earlier.

 by fyear, sort: egen mcap_group = xtile(prior_june_mcap), nq(10)

[Clyde Schechter]

So the main changes take place in the early part of the code, replacing references to mcap with idiovol:

// CODE TO CALCULATE ONE PARTITION INTO DECILES PER FYEAR (JULY-JUNE)
// BASED ON PRECEDING JUNE IDIOVOL
gen moy = month(dofm(mdate))
gen year = year(dofm(mdate))

//  CREATE A "FISCAL YEAR" RUNNING FROM JULY THROUGH SUBSEQUENT JUNE
gen fyear = cond(moy > 6, year, year-1)
frame put stock_id fyear idiovol if moy == 6, into(mcap_idiovol_work)
frame change mcap_idiovol_work
collapse (count) n_idiovol = idiovol (firstnm) idiovol, by(stock_id fyear)
assert n_idiovol <= 1  // VERIFY UNIQUE VALUE OF MCAP AND idiovol
replace fyear = fyear + 1 // CHANGE THE FYEAR TO WHICH THEY WILL APPLY
rename idiovol prior_june_=
frame change default

frlink m:1 stock_id fyear, frame(mcap_idiovol_work)
frget prior_june_*, from(mcap_idiovol_work)
frame drop mcap_idiovol_work
drop mcap_idiovol_work
by fyear, sort: egen idiovol_decile = xtile(prior_june_idiovol), nq(10)

In addition, while the program to compute vw_mean_rt does not need to change, the -runby- command that invokes it needs to have idiovol_decile rather than mcap-group in its -by()- option.

[Sartaj Hussain]

Thanks. I had a query about it. Since we have idiovol values for june month only for each stock in data file. Same is case with mcap but mcap is not used for sorting. The rt values are however, given from July 2001 to June 2002 for each stock. So, does annual sort effectively take place with july month rt observations using june month idiovol.

[Clyde Schechter]

So, considering as an example the data from the fiscal year July 2001 to June 2002, those firms get partitioned into deciles based on their idiovol values from June 2001.

Looking back, I notice that in #2 (i) you want to calculate the value weighted mean rt's based also on the June 2001 mcap (though the mcap is not used for partitioning into deciles). To make that happen, requires a slight change from what I wrote in #7 (because the code in #7 doesn't capture the June 2001 mcap).

//  CREATE A "FISCAL YEAR" RUNNING FROM JULY THROUGH SUBSEQUENT JUNE
gen fyear = cond(moy > 6, year, year-1)
frame put stock_id fyear mcap idiovol if moy == 6, into(mcap_idiovol_work)
frame change mcap_idiovol_work
collapse (count) n_idiovol = idiovol n_mcap = mcap (firstnm) idiovol mcap, by(stock_id fyear)
assert n_idiovol <= 1 & n_mcap <= 1  // VERIFY UNIQUE VALUE OF MCAP AND idiovol
replace fyear = fyear + 1 // CHANGE THE FYEAR TO WHICH THEY WILL APPLY
rename (idiovol mcap) prior_june_=
frame change default

Note: The code in #7 that precedes and follows the above needs no changes.

[Sartaj Hussain]

When i run code #2 (1), i face following error. Kindly debug it.

1. unexpected end of file
r(612);

13 expressions expected and only 7 found
r(198);

[Clyde Schechter]

I had run that code to test it before posting it and did not encounter any error messages.

Moreover, the error messsage "13 expressions expected and only 7 found" is typically produced by -frame post- commands. But there is no -frame post- command in the code in #2 (1) which would expect 13 expressions. So I am forced to conclude that you have modified the code in some way you have not shown. As for the "unexpected end of file" message, if there were a problem in that code which caused that message, it would have occurred when I tested the code because it arises from either unmatched curly braces or a -program- without an -end-, regardless of what data it is run with. So, again, I am forced to conclude that you have modified the code.

In order for me to troubleshoot your problem you will need to post back with example data that exhibits these problems and with the exact code that you are using.

[Sartaj Hussain]

yes there is one error i modified that.

float(intercept tstat b_rmrf t_rmrf b_smb t_smb b_himli t_himli b_wml t_wml b_himli t_himli r2 adj_r2)

The bold part i changed to: b_hml t_hml

frame change default
frame create regress_results int (idiovol_group) ///
float(intercept tstat b_rmrf t_rmrf  b_smb t_smb b_hml t_hml b_wml t_wml b_himli t_himli r2 adj_r2)
forvalues iv = 1/ 10 {
    regress vw_mean__idiovol_d`iv'_rt rmrf smb hml wml himli

    matrix M = r(table)
    local topost (`iv') (M["b", "_cons"]) (M["t", "_cons"])
    foreach x in rmrf smb hml wml himli {
        local topost `topost' (M["b", "`x'"]) (M["t", "`x'"])

    frame post regress_results `topost' (e(r2)) (e(r2_a))
}
frame change regress_results
rename (intercept tstat b_* t_* r2 adj_r2) =_idiovol_

After running the above code, error faced:

15 expressions expected and only 7 found
program error: code follows on the same line as close brace
r(198);

Moreover, if i drop any regressors then what change is to be done?

[Clyde Schechter]

Ah, I see it:

frame change default
frame create regress_results int (idiovol_group) ///
float(intercept tstat b_rmrf t_rmrf  b_smb t_smb b_hml t_hml b_wml t_wml b_himli t_himli r2 adj_r2)
forvalues iv = 1/ 10 {
    regress vw_mean__idiovol_d`iv'_rt rmrf smb hml wml himli

    matrix M = r(table)
    local topost (`iv') (M["b", "_cons"]) (M["t", "_cons"])
    foreach x in rmrf smb hml wml himli {
        local topost `topost' (M["b", "`x'"]) (M["t", "`x'"])
    }
    frame post regress_results `topost' (e(r2)) (e(r2_a))
}
frame change regress_results
rename (intercept tstat b_* t_* r2 adj_r2) =_idiovol_

Added: Changing the list of regressors in this code is a bit hazardous. It is not enough to change the regression command. The variable list for frame regress_results has to change in parallel with it, as does the -foreach x...- command. That process is quite error prone. Had I anticipated you would be doing that, I would have written more robust code:

frame change default

local regressors rmrf smb hml wml himli // MAKE CHANGES TO LIST OF REGRESSORS HERE ONLY
local plist
foreach r of local regressors {
    local plist `plist' b_`r' t_`r'
}

frame create regress_results int (idiovol_group) ///
float(intercept tstat `plist' r2 adj_r2)
forvalues iv = 1/ 10 {
    regress vw_mean__idiovol_d`iv'_rt `regressors'
    matrix M = r(table)
    local topost (`iv') (M["b", "_cons"]) (M["t", "_cons"])
    foreach x in `regressors' {
        local topost `topost' (M["b", "`x'"]) (M["t", "`x'"])
    }
    frame post regress_results `topost' (e(r2)) (e(r2_a))
}
frame change regress_results
rename (intercept tstat b_* t_* r2 adj_r2) =_idiovol_

With this code, you can change the regressors just once in the -local regressors- command, and then the code will propagate those changes as needed to the rest of the code to keep it all parallel.

[Sartaj Hussain]

Got it. Thanks