ASREG: Rolling window regressions and by groups regressions (rolling betas)

[Attaullah Shah]

Hi everyone
Thanks to Kit Baum for uploading version 1 of asreg on SSC. The program can be downloaded by typing

ssc install asreg

. Short description and examples of asreg are presented below:


Description

---

To estimate rolling window regressions in Stata, the conventional method is to use the rolling command of Stata. However, that command is too slow, especially for larger data sets. As a speedy alternative, I have written asreg that fits a model of depvar on indepvars using linear regression in a user's defined rolling window or by a grouping variable. asreg is order of magnitude faster than estimating rolling window regressions through conventional methods such as Stata loops or using the Stata's official rolling command. asreg has the same speed efficiency as asrol (ssc des asrol). All calculations of the rolling windows, estimation of regression parameters, and writing the results to Stata variables are done in the Mata language.

Rolling window calculations require lots of looping over observations. The problem is compounded by different data structures such as unbalanced panel data, data with many duplicates, and data with many missing values. Yet, there might be data sets that have both time series gaps as well as many duplicate observations across groups. asreg does not use a static code for all types of data structures. Instead, asreg intelligently identifies data structures and matches one of its rolling window routines with the data characteristics. Therefore, the rolling window regressions are fast even in larger data sets.

asreg writes all regression outputs to the data in memory as separate variables. This eliminates the need for writing results to a separate file, and then merging those results back to the data for further processing. New variables from the regression results follow the following naming conventions:

observations variable containing number of observation is named as obs_N

regression slopes a prefix of _b_ is added to the name of each independent variables

constant variable containing constant of the regression is names as _b_cons

r-squared r-squared and adj. r-squared are named as _R2 and _AdjR2 , respectively

standard errors a prefix of _se_ is added to the name of each interdependent variables

residuals variable containing residuals is named as _residuals

fitted variable containing fitted values is named as _fitted.



Options

---

asreg has the following options.


1. window:

Option window(rangevar #) specifies length of the rolling window. The window option accepts up to two arguments. If we have already declared our data as panel or time series data, asreg will automatically pick the time variable. In such cases, option window can have one argument, that is the length of the window, e.g., window(5). If our data is not time series or panel, then we have to specify the time variable as a first argument of the option window. For example, if our time variable is year and we want a rolling window of 24, then option window will look like:

   window( year 24)

2. recursive:

The option recursive specifies that a recursive window be used. In time series analysis, a recursive window refers to a window where the starting period is held fixed, the ending period advances, and the window size grows. asreg allows a recursive window either by invoking the option recursive or setting the length of the window greater than or equal to the sample size per group. For example, if sample size of our data set is 1000 observation per group, we can use a recursive analysis by setting the window length equal to 1000 or greater than 1000


3. by:

asreg is byable. Hence, it can be run on groups as specified by option by(varlist) or the bysort varlist: prefix. An example of such regression might be Fama and MacBeth (1973) second stage regression, which is estimated cross-sectionally in each time period. Therefore, the grouping variable in this case would be the time variable. Assume that we have our dependent variable named as stock_returns, independent variable as stock_betas, and time variable as month_id, then to estimate the cross-sectional regression for each month, asreg command will look like:

  . bys month_id: asreg stock_return stock_betas

4. minimum:

asreg estimates regressions where number of observations are greater than number of regressors. However, there is a way to limit the regression estimates to a desired number of observations. The option minimum can be used for this purpose. If option min is used, asreg then finds the required number of observation for the regression estimated such that :

obs = max(number of regressors (including the intercept), minimum observation as specified by the option min)

For example, if we have 4 explanatory variables, then the number of regressors will be equal to 4 plus 1 i.e. 5. Therefore, if asreg receives the the value of 8 from the option min, the required number of observations will be : max(5,8) = 8. If a specific rolling window does not have that many observations, values of the new variable will be replaced with missing values.


Statistics_Options

---

fitted reports residuals and fitted values for the last observation in the rolling window. If option window is not specified, then
the residuals are calculated withing each group as specified by the option by(varlist) or the bysort varlist:

serror reports standard errors for each explanatory variable

other Most commonly used regression statistics such as number of observations, slope coefficients, r-squared, and adjusted r-squared
are written to new variables by default. Therefore, if these statistics are not needed, they can be dropped once asreg is
estimated.


Examples

---

The following examples can be easily run by clicking the clickable links inside the help file of asreg.
Example 1: Regression for each company in a rolling window of 10 years


.

 webuse grunfeld
 
    . bys company: asreg invest mvalue kstock, wind(year 10)

The grunfeld data set is a panel data set, so we can omit the word year from the option window. Therefore, the command can also be estimated
as shown below:

.

 bys company: asreg invest mvalue kstock, wind(10)

Example 2: Regression for each company in a recursive window

. webuse grunfeld
 
    . bys company: asreg invest mvalue kstock, wind(year 10) rec

OR


.

 bys company: asreg invest mvalue kstock, wind(year 1000)

Example 3: Using option minimum


.

 webuse grunfeld
 
    . bys company: asreg invest mvalue kstock, wind(10) min(5)

Example 4: Reporting standard errors


.

webuse grunfeld
 
    . bys company: asreg invest mvalue kstock, wind(10) se

Example 5: Reporting standard errors, fitted values and residuals


.

webuse grunfeld
 
    . bys company: asreg invest mvalue kstock, wind(10) se fit

Example 6: No window - by groups regressions


.

webuse grunfeld
 
    . bys company: asreg invest mvalue kstock

Example 7: Yearly cross-sectional regressions

 webuse grunfeld
 
    . bys year: asreg invest mvalue kstock

Please note: The beta version leaves behind a variable TimeDiff, which I shall work upon and upload a revised version soon to SSC .
Since this is a beta version, I would appreciate any bug reporting or suggestions on what other options can be added.

[Attaullah Shah]

As an update to the previous post, I am happy to share that an updated version of asreg is now available on SSC. New users can type

ssc install asreg

and existing users can update the ado files by

adoupdate asreg, update

The updated version fixes a minor bug in the earlier version that was leaving behind a temporary variable.

[Katharine Adame]

Hi Attaullah,

Thank you so much for this very helpful package. I was hoping you might be able to help me understand why I am receiving the following error:
asregw(): 3301 subscript invalid
<istmt>: - function returned error

I am running the following regression:
bys permno: asreg retlessrf mkt_prem, wind(10) se fit

I ran tsset on the data beforehand (unbalanced panel with gaps). Do you have any ideas? I'm really hoping to get this command to work as it seems much preferable to the rolling command.

Thank you!

[Nick Cox]

I can't try to debug asreg for you, or alternatively explain what's wrong with your code, not least because its Mata code is hidden and there is no data example here.

While Attaullah Shah is explaining what went wrong, an alternative is rangestat from SSC by Robert Picard and friends. Here is some token code: The regression here is presumably just silly.

webuse grunfeld , clear 
rangestat (reg) mvalue kstock, interval(year -9 0) by(company) 
gen fitted = b_cons + b_kstock * kstock

I don't understand the exact definition of windows in asreg from a quick glance at its help. If window(10) means (e.g.) the previous 10 values, not including the present observation, your rangestat equivalent would be interval(year -10 -1)

[Attaullah Shah]

Dear Katharine Adame , the updated version of asreg requires two arguments in the option window. Therefore, if you change the command to the following, it will solve the problem. Please note that I assume your time variable is named as year. If it is named differently, then replace year with actual name of the variable.

bys permno: asreg retlessrf mkt_prem, wind(year 10) se fit

[Stefanie Peiffer]

Dear Attaullah Shah ,

your programme seems very helpful for my intended analysis. However, I am unfortunately experiencing a problem when running the described command.

I have an unbalanced Panel Data set with:
xtset GlobalCompanyCode DataYearFiscal, yearly

After preparing my dataset (deleting observations with missing values, computing the variables and so on), I ran the following command

bys GlobalCompanyKey: asreg ActualLeverage_w FirmSize_w RDExp_w AdvertisingExp_w TangibleAssets_w , wind(DataYearFiscal 5)

the command runs smoothly without any error message and also generates the variables mentioned in your post, such as beta coefficients and R².
However, when I browse my data, those variables have only missing values.

Could you please tell me, what I am doing wrong?
Thank you so much in advance
Kind regards
Stefanie Peiffer

Student at Maastricht University, the Netherlands

[Nick Cox]

You are asking for an equivalent of

regress ActualLeverage_w FirmSize_w RDExp_w AdvertisingExp_w TangibleAssets_w

for windows of 5 observations. So with 4 predictors you are estimating 5 parameters from 5 values.

That can't be done usefully. It's on all fours with fitting a straight line to 2 distinct points. Here is a dopey example.

.  sysuse auto
(1978 Automobile Data)

. regress mpg price weight trunk length in 1/5

      Source |       SS           df       MS      Number of obs   =         5
-------------+----------------------------------   F(4, 0)         =         .
       Model |        38.8         4         9.7   Prob > F        =         .
    Residual |           0         0           .   R-squared       =    1.0000
-------------+----------------------------------   Adj R-squared   =         .
       Total |        38.8         4         9.7   Root MSE        =         0

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       price |   -.000583          .        .       .            .           .
      weight |  -.0072496          .        .       .            .           .
       trunk |  -.0949042          .        .       .            .           .
      length |   .1212445          .        .       .            .           .
       _cons |   24.12378          .        .       .            .           .
------------------------------------------------------------------------------

You need a longer window or a simpler model. No program can solve this otherwise. (If asreg doesn't give you R-square identically 1, that is a side puzzle.)

Alternatively, if you meant to pool your companies, don't supply the by: prefix.

[Stefanie Peiffer]

Thank you Nick Cox for your quick response.
I am using Stata for my Master Thesis for the first time, so I should probably tell more about what I want to do.

I am replicating an academic paper, which outlines the following model:

Actual Leverage = ß0 + ß1*FirmSize +ß2*RDExp + ß3*Advertising Exp + ß4*TangibleAssets + 20 Year Dummy Variables and 10 Industry Dummy Variables

I want to run this Regression for each Company in my Data set, based on a past number of years (the number of years, I can Chose freely, so I could also use window 10 if that helps).
So I thought I Need to use a rolling Regression, where asreg seemed a faster and better alternative as the new variables are stored in the original data set.

I will then use the all ß coefficients to calculate a new variable called TargetLeverage for each Observation in my data set.

So if I understand you correctly, I Need to Chose a window that is bigger than my number of Independent variables?

Thank you so much for your help

[Nick Cox]

10 years is far too few for the model you cite if you fit it the way you did. Put differently, it's not at all the same model, as it appears that it would be applied to data for several companies.

I never try to fit a model with 35 parameters, but if I did I would want hundreds of data points at least.I am not an economist and I fear that you need advice from someone who is.

[ivanpersie]

Hi guys, I have run the asreg and i keep getting the variable Oilret coefficent and Std.Err being omiited yet i would like to use them in th stage my model. What could be the reason?

xtset permno modate
panel variable: permno (unbalanced)
time variable: modate, 1990m2 to 2017m12, but with gaps
delta: 1 month

. asreg Ret_Rf mktrf Oilret, fmb

Fama-MacBeth (1973) Two-Step procedure Number of obs = 780240
Num. time periods = 335
F( 2, 334) = 1.46
Prob > F = 0.2328
avg. R-squared = 0.0000
------------------------------------------------------------------------------
| Fama-MacBeth
Ret_Rf | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
mktrf | -.0077381 .006395 -1.21 0.227 -.0203178 .0048415
Oilret | 0 (omitted)
_cons | -.0312543 .0054391 -5.75 0.000 -.0419536 -.0205551
------------------------------------------------------------------------------

[Nick Cox]

I suspect you need to tell us more about the gaps.

[Attaullah Shah]

asreg version 3.2 is now available for download from SSC (thanks to Kit Baum). This version fixes a minor bug in one of the Mata functions. Mata function ASREG4s0f1() would throw an error if option fit was used on a dataset that did not have enough observations for estimating a regression. To update,

ssc install asreg, replace

[Yousry Ahmed]

Hi,

Many thanks for your awesome command.

1. How I can create a table that presents a time series means of the coefficient estimates from the yearly regressions using asreg.

2. The table uses Newey-West adjustment with three lags to calculate p-values and reports them in parentheses

3. The significance levels are for a test of the hypothesis that the time series mean is equal to zero, using the time series standard error of the mean estimate for each coefficient.

[Attaullah Shah]

You can make a table of summary statistics using asdoc, see this video https://www.youtube.com/watch?v=zdI65G6AhdU
and this webpage for more details https://fintechprofessor.com/2018/01/31/asdoc/

[David Eschweiler]

Hi Attaullah Shah,

Many thanks for your command.
Is it possible to assign different weights to different observations at time t? In my case depending on the market capitalization of the respective company.

Thank you for your help