Simplify the loop?

[River Huang]

Dear All, I'd like to simplify the code (with data, both can be downloaded) used in http://faculty.haas.berkeley.edu/ross_levine/papers.htm

"Big Bad Banks: The Winners and Losers from Bank Deregulation in the United States." (with Thorsten Beck and Alexey Levkov) Journal of Finance, October 2010, 1637-1667. Lead Article. Data Appendix. Data.

For example, the original code is

generate d_10 = 0; replace d_10 = 1 if _tintra == -10;
generate d_9  = 0; replace d_9  = 1 if _tintra == -9;
generate d_8  = 0; replace d_8  = 1 if _tintra == -8;
generate d_7  = 0; replace d_7  = 1 if _tintra == -7;
generate d_6  = 0; replace d_6  = 1 if _tintra == -6;
generate d_5  = 0; replace d_5  = 1 if _tintra == -5;
generate d_4  = 0; replace d_4  = 1 if _tintra == -4;
generate d_3  = 0; replace d_3  = 1 if _tintra == -3;
generate d_2  = 0; replace d_2  = 1 if _tintra == -2;
generate d_1  = 0; replace d_1  = 1 if _tintra == -1;

generate d1  = 0; replace d1  = 1 if _tintra == 1;
generate d2  = 0; replace d2  = 1 if _tintra == 2;
generate d3  = 0; replace d3  = 1 if _tintra == 3;
generate d4  = 0; replace d4  = 1 if _tintra == 4;
generate d5  = 0; replace d5  = 1 if _tintra == 5;
generate d6  = 0; replace d6  = 1 if _tintra == 6;
generate d7  = 0; replace d7  = 1 if _tintra == 7;
generate d8  = 0; replace d8  = 1 if _tintra == 8;
generate d9  = 0; replace d9  = 1 if _tintra == 9;
generate d10 = 0; replace d10 = 1 if _tintra == 10;
generate d11 = 0; replace d11 = 1 if _tintra == 11;
generate d12 = 0; replace d12 = 1 if _tintra == 12;
generate d13 = 0; replace d13 = 1 if _tintra == 13;
generate d14 = 0; replace d14 = 1 if _tintra == 14;
generate d15 = 0; replace d15 = 1 if _tintra == 15;

My modified code is

forvalues i = 10(-1)1 {
  gen d_`i' = 0
  replace d_`i' = 1 if _tintra == -`i'   
}
forvalues i = 1(1)15 {
  gen d`i' = 0
  replace d`i' = 1 if _tintra == `i'  
}

I wonder if these two loops can be combine together? Any suggestion is highly appreciated.

[Clyde Schechter]

They can be simplified:

forvalues i = 10(-1)1 {
    gen d_`i' = (_tintra == -`i')
}
forvalues i = 1(1)15 {
    gen d`i' = (_tintra == `i')
}

You could, in principle, combine them into a single loop going from -10 up to 15, but because the names of the variables being created differ depending on whether `i' is negative or positive (and nothing is done when _tintra == 0), the combined loop would have to have an -if- -else if- -else- branching structure that would, on balance, make the code more complicated and opaque than the two loops, in my opinion.

[Clyde Schechter]

On further thought, the following single loop isn't too terrible:

local dd d_
forvalues i = -10(1)15 {
    if `i' == 0 {
        local dd d
    }
    else {
        gen `dd'_`i' = (_tintra == `i')
    }
}

The idea is that we start with the variable prefix set to d_ and generate values from i = -10 through 1. When we hit i = 0, we change the variable prefix to d, but generate nothing. Then in the remaining iterations we generate again.

[River Huang]

Dear Clyde, Thanks, but there is an error message. I now post a simple data set

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(statefip _tintra)
1  -5
1  -4
1  -3
1  -2
1  -1
1   0
1   1
1   2
1   3
1   4
1   5
1   6
1   7
1   8
1   9
1  10
1  11
1  12
1  13
1  14
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5  -9
5  -8
5  -7
5  -6
5  -5
5  -4
5  -3
5  -2
5  -1
5   0
5   1
5   2
5   3
5   4
5   5
5   6
5   7
5   8
5   9
5  10
5  11
5  12
end

and wonder if you can have a look?

[Clyde Schechter]

Yes, you are right. Sorry. The error arose by trying to generate a variable named d_-10, and would persist throughout the negative values of i. The following is tested and works:

local dd d_
forvalues i = -10(1)15 {
    if `i' == 0 {
        local dd d
    }
    else {
        local ii = abs(`i')
        gen `dd'`ii' = (_tintra == `i')
    }
}

Changes shown in bold face.

[River Huang]

Dear Clyde, Many thanks. It works now.

[Romalpa Akzo]

Below code might be not a simplification, but just another way to go.

gen negative=-_tintra if _tintra<0
gen positive= _tintra if _tintra>0
tab negative, gen(d_) mi
tab positive, gen(d) mi
drop negative positive d_11 d16

[Clyde Schechter]

And here's another way. A bit opaque, but shorter:

forvalues i = -10(1)15 {
    gen `=strtoname("d`i'")' = (_tintra == `i')
}
drop d0

[River Huang]

Dear Romalpa, Thanks a lot for this alternative method.

[River Huang]

Dear Clyde, This is an interesting (and more concise) code. Thanks again.

[River Huang]

Dear Clyde, A similar question in the original code (after generating d_10-d15) is

quietly gen t5 = invttail(48,0.025)
quietly gen y = log(gini)
xi: xtreg y d_10-d15 i.wrkyr, fe robust

forvalues i = 1(1)10 {
  gen b_`i' = _b[d_`i'] 
  gen se_b_`i' = _se[d_`i']
  gen b_`i'LB = b_`i' - t5*se_b_`i'
  gen b_`i'UB = b_`i' + t5*se_b_`i' 
}
forvalues i = 1(1)15 {
  gen b`i' = _b[d`i'] 
  gen se_b`i' = _se[d`i']
  gen b`i'LB = b`i' - t5*se_b`i'
  gen b`i'UB = b`i' + t5*se_b`i' 
}

and later

gen b = .
gen LB = .
gen UB = .

forvalues i = 1(1)10 {
  replace b = b_`i' if _tintra == -`i'
  replace LB = b_`i'LB if _tintra == -`i'
  replace UB = b_`i'UB if _tintra == -`i'
}

forvalues i = 1(1)15 {
  replace b = b`i' if _tintra == `i'
  replace LB = b`i'LB if _tintra == `i'
  replace UB = b`i'UB if _tintra == `i'
}

Can they also be simplified as well?

[Clyde Schechter]

There are several ways this code can be made more compact. This code appears to have originally been written in an old version of Stata (or by somebody unfamiliar with recent improvements to Stata). Considerable economy can be obtained just be updating to modern coding practice. I am assuming that the "variables" b_1-b_10, b1-b10, se_1-se_10, se1-se10, LB_1-LB_10, LB1-LB10 and UB_1-UB_10 and UB1-UB10 are not actually wanted: they are actually constants, so there is no reason to store them as variables, and it appears that they are intended only as intermediate results including the final variables b, LB, and UB. So a considerable economy is gained by calculating B, LB and UB directly from the regression results without these unnecessary intermediate "variables." Similarly, the constant "variable" t5 is unnecessary because the code below will pull the upper and lower confidence limits directly from the matrix r(table) that -xtreg, fe- leaves behind in modern versions of Stata, so we don't need to calculate it..

I have also changed your example somewhat so that it can be run with your example data from earlier, data which does not contain all of the variables mentioned in your -xtreg- command. I also illustrate here with -regress- instead of -xtreg, fe- because the example data has no panel structure.

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(statefip _tintra)
1  -5
1  -4
1  -3
1  -2
1  -1
1   0
1   1
1   2
1   3
1   4
1   5
1   6
1   7
1   8
1   9
1  10
1  11
1  12
1  13
1  14
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
1  15
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5 -10
5  -9
5  -8
5  -7
5  -6
5  -5
5  -4
5  -3
5  -2
5  -1
5   0
5   1
5   2
5   3
5   4
5   5
5   6
5   7
5   8
5   9
5  10
5  11
5  12
end

forvalues i = -10(1)15 {
    gen `=strtoname("d`i'")' = (_tintra == `i')
}
drop d0

//    CREATE A Y- VARIABLE FOR THE REGRESSION
//    DON'T USE THESE TWO LINES IN DATA THAT ALREADY
//    HAS A Y VARIABLE
set seed 1234
gen y = rnormal()


//    REPLACE THIS LINE BY THE DESIRED -xtreg, fe-
//    COMMAND.  BE SURE TO KEEP d_10-d15 AS THE
//    FIRST PREDICTOR VARIABLES LISTED.
regress y d_10-d15, robust

gen b = .
gen LB = .
gen UB = .

//    PULL THE COEFFICIENTS AND CONFIDENCE
//    LIMITS FROM THE REGRESSION RESULTS
matrix M = r(table)

//    STORE THE RESULTS ACCORDING TO THE VALUE
//    OF _tintra
forvalues i = -10(1)15 {
    if `i' != 0 {
        local j = cond(`i' < 0, `i'+11, `i'+10)
        replace b = M[1, `j'] if _tintra == `i'
        replace LB = M[5, `j'] if _tintra == `i'
        replace UB = M[6, `j'] if _tintra == `i'
    }
}

In adapting this code back to your situation, replace the -regress- command with -xtreg- and specify the -fe- option. You can include variables other than d_10-d15, but the d_10-d15 variables must be the first variables listed after y. If you get them out of that order, the results will be incorrect. Also, do not use -xi:- with your -xtreg, fe- command. It is not necessary and it might mess things up here.

[River Huang]

Dear Clyde, Many thanks. Part of the data is

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(statefip _tintra y)
1  -5 -.8513666
1  -4 -.8671516
1  -3  -.808742
1  -2 -.7698559
1  -1 -.8335734
1   0 -.7889401
1   1   -.67467
1   2 -.7781308
1   3 -.7743035
1   4 -.7668957
1   5 -.7359415
1   6 -.8186591
1   7 -.8335625
1   8 -.8454594
1   9 -.7967452
1  10 -.8147821
1  11 -.7861173
1  12 -.8049591
1  13 -.7935148
1  14  -.773951
1  15 -.8454396
1  15 -.8334426
1  15 -.8274375
1  15 -.8957804
1  15 -.8107525
1  15 -.8092129
1  15 -.8147871
1  15  -.809782
1  15 -.8117705
1  15 -.7962877
1  15 -.8007851
5 -10 -.8769394
5 -10  -.869798
5 -10  -.855145
5 -10 -.8014046
5 -10 -.7937752
5 -10 -.8247868
5 -10 -.6868308
5 -10 -.7882528
5 -10 -.7636379
5  -9 -.8251348
5  -8 -.7939059
5  -7 -.7631558
5  -6  -.851346
5  -5 -.8377607
5  -4 -.8834409
5  -3 -.8863798
5  -2 -.8237171
5  -1 -.8201995
5   0 -.9153026
5   1 -.9224527
5   2 -.8700765
5   3   -.78951
5   4  -.854983
5   5  -.928569
5   6  -.826272
5   7 -.8744188
5   8  -.799176
5   9 -.8068259
5  10 -.8169147
5  11 -.8169862
5  12 -.8376288
end

The problem now is (I didn't mention before):

gen x = (b_10+b_9+b_8+b_7+b_6+b_5+b_4+b_3+b_2+b_1)/10
replace b = b-x
replace LB = LB-x
replace UB = UB-x

i.e., I need to generate a new variable x as above, but now the error message complained

b_10 not found
r(111);

I guess that x can be calculated from the matrix M but need some help!

[Clyde Schechter]

Well, b_10 wasn't found because I assumed that it wasn't actually needed for anything, so the code in #12 didn't create it. Even with this additional wrinkle, it's still not necessary to create those b* variables (other than b itself). Try this:

* Example generated by -dataex-. To install: ssc install dataex
clear
input float(statefip _tintra y)
1  -5 -.8513666
1  -4 -.8671516
1  -3  -.808742
1  -2 -.7698559
1  -1 -.8335734
1   0 -.7889401
1   1   -.67467
1   2 -.7781308
1   3 -.7743035
1   4 -.7668957
1   5 -.7359415
1   6 -.8186591
1   7 -.8335625
1   8 -.8454594
1   9 -.7967452
1  10 -.8147821
1  11 -.7861173
1  12 -.8049591
1  13 -.7935148
1  14  -.773951
1  15 -.8454396
1  15 -.8334426
1  15 -.8274375
1  15 -.8957804
1  15 -.8107525
1  15 -.8092129
1  15 -.8147871
1  15  -.809782
1  15 -.8117705
1  15 -.7962877
1  15 -.8007851
5 -10 -.8769394
5 -10  -.869798
5 -10  -.855145
5 -10 -.8014046
5 -10 -.7937752
5 -10 -.8247868
5 -10 -.6868308
5 -10 -.7882528
5 -10 -.7636379
5  -9 -.8251348
5  -8 -.7939059
5  -7 -.7631558
5  -6  -.851346
5  -5 -.8377607
5  -4 -.8834409
5  -3 -.8863798
5  -2 -.8237171
5  -1 -.8201995
5   0 -.9153026
5   1 -.9224527
5   2 -.8700765
5   3   -.78951
5   4  -.854983
5   5  -.928569
5   6  -.826272
5   7 -.8744188
5   8  -.799176
5   9 -.8068259
5  10 -.8169147
5  11 -.8169862
5  12 -.8376288
end
forvalues i = -10(1)15 {
    gen `=strtoname("d`i'")' = (_tintra == `i')
}
drop d0

regress y d_10-d15, robust

gen b = .
gen LB = .
gen UB = .
matrix M = r(table)
scalar x = 0
forvalues i = -10(1)15 {
    if `i' != 0 {
        local j = cond(`i' < 0, `i'+11, `i'+10)
        replace b = M[1, `j'] if _tintra == `i'
        replace LB = M[5, `j'] if _tintra == `i'
        replace UB = M[6, `j'] if _tintra == `i'
    }
    if `i' < 0 {
        scalar x = x + M[1, `j']
    }
}
scalar x = x/10

replace b = b-x
replace LB = LB-x
replace UB = UB-x

[River Huang]

Dear Clyde, Many thanks. It works perfectly.