Adding variables in a loop

[Santosh Dash]

Hi,

I have several variables in model. I want to write a loop where it can add one variable in each turn.

Say I've variables, var1 to var20. First I want to use var1 iny regression, then var1and var2.

I wrote the following loop. But this only adds one variable in each regression.

foreach var of varlist var1 - var20{
reg Y X `var'
​​​​​}

Appreciate your help.
Thank you.

[Andrew Musau]

sysuse auto, clear
local list
foreach var of varlist rep78-weight{
    local list "`list' `var'"
    regress mpg `list'
}

Res.:

. foreach var of varlist rep78-weight{
  2. 
.     local list "`list' `var'"
  3. 
.     regress mpg `list'
  4. 
. }

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(1, 67)        =     12.94
       Model |  378.826865         1  378.826865   Prob > F        =    0.0006
    Residual |  1961.37603        67  29.2742692   R-squared       =    0.1619
-------------+----------------------------------   Adj R-squared   =    0.1494
       Total |   2340.2029        68  34.4147485   Root MSE        =    5.4106

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   2.384298   .6628009     3.60   0.001     1.061341    3.707254
       _cons |   13.16942    2.34946     5.61   0.000     8.479879    17.85896
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(2, 66)        =     12.84
       Model |  655.477096         2  327.738548   Prob > F        =    0.0000
    Residual |   1684.7258        66  25.5261485   R-squared       =    0.2801
-------------+----------------------------------   Adj R-squared   =    0.2583
       Total |   2340.2029        68  34.4147485   Root MSE        =    5.0523

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   2.079388   .6258091     3.32   0.001     .8299198    3.328856
    headroom |  -2.390408   .7261046    -3.29   0.002    -3.840123   -.9406935
       _cons |   21.37911   3.321449     6.44   0.000     14.74762    28.01059
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(3, 65)        =     16.71
       Model |  1019.11444         3  339.704812   Prob > F        =    0.0000
    Residual |  1321.08846        65  20.3244379   R-squared       =    0.4355
-------------+----------------------------------   Adj R-squared   =    0.4094
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.5083

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   1.889614   .5602162     3.37   0.001     .7707853    3.008443
    headroom |  -.0291626   .8552278    -0.03   0.973     -1.73717    1.678845
       trunk |  -.7116801   .1682518    -4.23   0.000    -1.047702   -.3756579
       _cons |   24.85365   3.075496     8.08   0.000     18.71146    30.99584
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(4, 64)        =     31.30
       Model |  1548.57333         4  387.143331   Prob > F        =    0.0000
    Residual |  791.629573        64  12.3692121   R-squared       =    0.6617
-------------+----------------------------------   Adj R-squared   =    0.6406
       Total |   2340.2029        68  34.4147485   Root MSE        =     3.517

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   .6425935   .4767913     1.35   0.182    -.3099066    1.595094
    headroom |   .1762834   .6679194     0.26   0.793    -1.158039    1.510606
       trunk |  -.1504935   .1567984    -0.96   0.341    -.4637343    .1627472
      weight |  -.0051783   .0007915    -6.54   0.000    -.0067594   -.0035971
       _cons |   36.36914   2.975633    12.22   0.000     30.42463    42.31365
------------------------------------------------------------------------------

.

[Nick Cox]

Andrew Musau gave exactly what was asked, but nestreg is a strong competitor here. (The example is dubious. but there you go.)

. sysuse auto, clear
(1978 Automobile Data)

. nestreg: regress mpg turn rep78 foreign

Block  1: turn

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(1, 67)        =     78.94
       Model |  1265.81367         1  1265.81367   Prob > F        =    0.0000
    Residual |  1074.38923        67  16.0356601   R-squared       =    0.5409
-------------+----------------------------------   Adj R-squared   =    0.5340
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.0045

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -.9715042   .1093461    -8.88   0.000     -1.18976   -.7532485
       _cons |   59.95291   4.378278    13.69   0.000     51.21383    68.69199
------------------------------------------------------------------------------

Block  2: rep78

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(2, 66)        =     39.17
       Model |  1270.16907         2  635.084534   Prob > F        =    0.0000
    Residual |  1070.03383        66  16.2126338   R-squared       =    0.5428
-------------+----------------------------------   Adj R-squared   =    0.5289
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.0265

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -.9389411   .1266318    -7.41   0.000     -1.19177   -.6861125
       rep78 |   .2944491   .5680979     0.52   0.606    -.8397949    1.428693
       _cons |   57.65415   6.249091     9.23   0.000     45.17744    70.13087
------------------------------------------------------------------------------

Block  3: foreign

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(3, 65)        =     26.46
       Model |  1286.71214         3  428.904048   Prob > F        =    0.0000
    Residual |  1053.49075        65  16.2075501   R-squared       =    0.5498
-------------+----------------------------------   Adj R-squared   =    0.5291
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.0259

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -1.022626      .1513    -6.76   0.000    -1.324793   -.7204589
       rep78 |   .5458106    .620109     0.88   0.382    -.6926325    1.784254
     foreign |  -1.578187   1.562101    -1.01   0.316    -4.697919    1.541544
       _cons |   60.60879   6.898675     8.79   0.000     46.83119     74.3864
------------------------------------------------------------------------------


  +-------------------------------------------------------------+
  |       |          Block  Residual                     Change |
  | Block |       F     df        df   Pr > F       R2    in R2 |
  |-------+-----------------------------------------------------|
  |     1 |   78.94      1        67   0.0000   0.5409          |
  |     2 |    0.27      1        66   0.6060   0.5428   0.0019 |
  |     3 |    1.02      1        65   0.3161   0.5498   0.0071 |
  +-------------------------------------------------------------+

.

[Chen Samulsion]

Hi, Andrew Musau could you please explain the principle of the local settings in

local list "`list' `var'"

Thank you very much.

[Andrew Musau]

You first define an empty local named "list"

local list

Then you initiate the loop:

foreach var of varlist rep78-weight{

For each iteration of the loop, you define the local list as the current local named list + `var'.

local list "`list' `var'"

So in the first iteration, list will just contain `var' because it is empty+`var'. In the second iteration, it contains one variable, namely rep78. So the new local list will be rep78 + `var'. For each iteration, the local keeps on expanding to include its previous contents + `var', where `var' is defined by the variable list in the -foreach- loop.

[Chen Samulsion]

Thank you so much Andrew Musau. I saw usage of it many times and in many occasions. Your detailed explanation really helps me a lot.

[Santosh Dash]

Andrew Musau Thank you very much for the code. and thank you a lot for the detailed explanation in #6.
This code sequentially adds one variable in each regression.
I was wondering if there is any way if can shuffle the variables in each step of regression. That is to say, it will randomly take the variables for each regression.
In the example, suppose I want to estimate the model the following way:

regress mpg turn
regress mpg turn rep78 foreign
regress mpg rep78 foreign
regress mpg trunk weight
regress mpg turn foreign

Is it possible to modify your code in #2?
Thank you very much.

[Santosh Dash]

Thank you very much, Nick Cox. I was not aware of the -nestreg- package before.

[Nick Cox]

#7 The code you cite is, so far as I can see, the simplest and clearest way to calculate a series of regressions if there is no discernible rule about change of predictors.

The only other way I can think of doing it is to put the predictors into some local macro -- with punctuation to separate clearly -- and then parse the local macro using that punctuation, which all seems futile unless you're playing some obfuscated code contest. Compare https://en.wikipedia.org/wiki/Intern...C_Code_Contest

[Andrew Musau]

If you want to regress the outcome with all combinations of the regressors, install tuples from SSC. Otherwise, as Nick notes, there is no rule that is apparent in your description.

*ssc install tuples
sysuse auto, clear
tuples turn rep78 foreign
forval i=1/`ntuples'{
    regress mpg `tuple`i''
}

Res.:

 forval i=1/`ntuples'{
  2. 
.     regress mpg `tuple`i''
  3. 
. }

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(1, 72)        =     13.18
       Model |  378.153515         1  378.153515   Prob > F        =    0.0005
    Residual |  2065.30594        72  28.6848048   R-squared       =    0.1548
-------------+----------------------------------   Adj R-squared   =    0.1430
       Total |  2443.45946        73  33.4720474   Root MSE        =    5.3558

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
     foreign |   4.945804   1.362162     3.63   0.001     2.230384    7.661225
       _cons |   19.82692   .7427186    26.70   0.000     18.34634    21.30751
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(1, 67)        =     12.94
       Model |  378.826865         1  378.826865   Prob > F        =    0.0006
    Residual |  1961.37603        67  29.2742692   R-squared       =    0.1619
-------------+----------------------------------   Adj R-squared   =    0.1494
       Total |   2340.2029        68  34.4147485   Root MSE        =    5.4106

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   2.384298   .6628009     3.60   0.001     1.061341    3.707254
       _cons |   13.16942    2.34946     5.61   0.000     8.479879    17.85896
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(1, 72)        =     77.14
       Model |  1263.82804         1  1263.82804   Prob > F        =    0.0000
    Residual |  1179.63142        72  16.3837697   R-squared       =    0.5172
-------------+----------------------------------   Adj R-squared   =    0.5105
       Total |  2443.45946        73  33.4720474   Root MSE        =    4.0477

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -.9457877   .1076853    -8.78   0.000    -1.160455   -.7311209
       _cons |    58.7965   4.295428    13.69   0.000     50.23372    67.35928
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(2, 66)        =     10.05
       Model |   546.30058         2   273.15029   Prob > F        =    0.0002
    Residual |  1793.90232        66  27.1803382   R-squared       =    0.2334
-------------+----------------------------------   Adj R-squared   =    0.2102
       Total |   2340.2029        68  34.4147485   Root MSE        =    5.2135

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       rep78 |   1.219096   .7926102     1.54   0.129    -.3634013    2.801593
     foreign |   4.202036   1.692832     2.48   0.016     .8221878    7.581885
       _cons |   15.85898   2.509808     6.32   0.000     10.84799    20.86997
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        74
-------------+----------------------------------   F(2, 71)        =     38.97
       Model |  1278.68109         2  639.340545   Prob > F        =    0.0000
    Residual |  1164.77837        71  16.4053292   R-squared       =    0.5233
-------------+----------------------------------   Adj R-squared   =    0.5099
       Total |  2443.45946        73  33.4720474   Root MSE        =    4.0503

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -1.029205    .138914    -7.41   0.000    -1.306192   -.7522186
     foreign |  -1.263614   1.328003    -0.95   0.345    -3.911577    1.384349
       _cons |   62.47956   5.784252    10.80   0.000     50.94609    74.01303
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(2, 66)        =     39.17
       Model |  1270.16907         2  635.084534   Prob > F        =    0.0000
    Residual |  1070.03383        66  16.2126338   R-squared       =    0.5428
-------------+----------------------------------   Adj R-squared   =    0.5289
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.0265

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -.9389411   .1266318    -7.41   0.000     -1.19177   -.6861125
       rep78 |   .2944491   .5680979     0.52   0.606    -.8397949    1.428693
       _cons |   57.65415   6.249091     9.23   0.000     45.17744    70.13087
------------------------------------------------------------------------------

      Source |       SS           df       MS      Number of obs   =        69
-------------+----------------------------------   F(3, 65)        =     26.46
       Model |  1286.71214         3  428.904048   Prob > F        =    0.0000
    Residual |  1053.49075        65  16.2075501   R-squared       =    0.5498
-------------+----------------------------------   Adj R-squared   =    0.5291
       Total |   2340.2029        68  34.4147485   Root MSE        =    4.0259

------------------------------------------------------------------------------
         mpg |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        turn |  -1.022626      .1513    -6.76   0.000    -1.324793   -.7204589
       rep78 |   .5458106    .620109     0.88   0.382    -.6926325    1.784254
     foreign |  -1.578187   1.562101    -1.01   0.316    -4.697919    1.541544
       _cons |   60.60879   6.898675     8.79   0.000     46.83119     74.3864
------------------------------------------------------------------------------

.

[Santosh Dash]

Thank you a lot, Andrew Musau. That is what I wanted.