Simple loop question

[sungwook kim]

Dear Listers,

I would like to ask you a very simple question regarding loop.

I want to make a variable for each of countries listed in my data. As there are many items, I would like to automate this process by using loop function.

global X 1 2 3 4 5 6 7 8 9 10 11 

foreach x of global X {

foreach y in 22.26    33.98    16.6    30.16    21.06    11.36    36.29    46.28    64.31    110.56    36.83 {



gen unit_physio_`x'= `y'

}
}

This is my code. In fact what I want is unit_physio_1 = 22.36, unit_physio_2=33.98, unit_physio_3=16.6

However, all of them are 22.26 after I run this code.

I would be happy if someone helps me with this problem.

Many thanks in advance.

Kind regards,

kim

[Nick Cox]

First, let's look at your outer loop:

global X 1 2 3 4 5 6 7 8 9 10 11  
foreach x of global X {  
}

There is an easier way to do that, namely

forval x = 1/11 {  
}

So, now we have -- with additions to make the code self-contained and reproducible --

clear  
set obs 1  
forval x = 1/11 {    
    foreach y in 22.26    33.98    16.6    30.16    21.06    11.36    36.29    46.28    64.31    110.56    36.83 {        
        gen unit_physio_`x'= `y'    
    }
}

which fails second time around the inner loop:

variable unit_physio_1 already defined
r(110);

Hence I can't believe your problem report.

Be that as it may, the major error is that you have just one loop, not two nested loops. Here is one way to implement what you want:

clear
set obs 1
local Y 22.26 33.98 16.6 30.16 21.06 11.36 36.29 46.28 64.31 110.56 36.83
forval x = 1/11 {
    gettoken y Y : Y
    gen unit_physio_`x'= `y'
}

[sungwook kim]

Dear Nick,

Thank you very much indeed for your detailed explanation. It is really helpful and much appreciated!

Kind regards,

Kim

[Chen Samulsion]

You can also use while to achieve your purpose.

set obs 1
local x=1 
while `x'<=11 {
 foreach y in 22.26 33.98 16.6 30.16 21.06 11.36 36.29 46.28 64.31 110.56 36.83 {
 gen unit_physio_`x'= `y'
 local ++x
}
}

[Alfonso Sánchez-Peñalver]

Nick Cox is right, there is only one loop because what you need to loop through are the values you want to enter in the variables. However, you need to have a counter to know what value it is, so that you can name the variable appropriately. The suggested forval x = 1/11 does the job of keeping that counter. Chen Samulsion's solution sets the counter and then adds 1 to it at the end of the inner loop, but includes nested loops where they are not needed. A simpler coded solution would be

clear all
set obs 1

local Y 22.26 33.98 16.6 30.16 21.06 11.36 36.29 46.28 64.31 110.56 36.83
local x = 1

foreach y of local Y {
    gen unit_physio_`x'= `y'
    local ++x
}

As you can see, all you need is to loop through the values in Y, and keep your counter increasing. Deciding between Nick Cox's solution and this one is a question of preferences, because as you can see they both have the same structure and lines of code.

[Nick Cox]

I agree with Alfonso. I see no advantage to while here.

while is great when you're watching for some condition to exit the loop. If you know how many times you are looping, forval or foreach is more direct.

Yet another way to do it:

clear
set obs 1
tokenize "22.26 33.98 16.6 30.16 21.06 11.36 36.29 46.28 64.31 110.56 36.83"
forval x = 1/11 {
    gen unit_physio_`x'= ``x''
}

[Chen Samulsion]

https://www.statalist.org/forums/for...orvalues-loops
This thread addressed a similar situation that can be used for reference. AND congratulations to Nick Cox on passing >17000 posts.

[Nick Cox]

Thanks!

[Chris Boulis]

Hi All.

I want to calculate the log of a number of variables (e.g. log(assets), ...) in my dataset, then calculate the average of these variables - one average for all waves, and separate averages by the wave in which they appear (2, 6, 10, 14, 18) to obtain, for example (e.g. lnassets_ave lnassets_ave2 lnassets_ave6 lnassets_ave10 lnassets_ave14 lnassets_ave18. Here's my attempt doing this using loops.

// taking the log of each variable

local varlist asset bank cash equity ins prop bus veh 
foreach i of local varlist {
    gen double ln`i' = log(`i') 
    su ln`i'
}

// taking the average of each variable

local varlist lnasset lnbank lncash lnequity lnins lnprop lnbus lnveh 
forvalues w = 2(4)18 {
    foreach v in local varlist  
    bys hgage1 ethnic: egen `v'"ave"`w' = mean(`v') // ethnic is a dichotomous variable (=1 if nth America; = 2 if sth America)
}

This second piece of code only goes so far as to find the 'total average of all waves for each variable, but not by wave for the five waves in which they appear.

If appropriate, I'd like to know how to run the second piece of code as a 'nested' loop in the first piece of code. Help appreciated.

* Example generated by -dataex-. To install: ssc install dataex
clear
input byte wave int hgage1 byte ethnic float asset
 6 19 2     905
 2 19 2  275020
14 20 1   38182
 2 20 1   30000
10 20 1   55862
14 20 1  970679
10 20 1   10150
14 20 2   13744
 6 20 2  228998
 6 20 2    4530
 2 20 2     716
 2 20 2    7745
14 20 2  197823
 2 22 2   53900
 2 22 2   45328
18 23 1 1058761
 2 23 1   41790
10 23 1  750297
10 23 1   33640
14 23 1   36500
14 23 1  489250
14 23 1  420760
 6 23 2   32569
10 23 2  501106
14 23 2 1456018
10 24 2  361700
10 24 2  663453
 6 24 2   20000
18 24 2   68256
 6 24 2  252088
14 24 2  469000
end

Stata v.15.1. I have panel data.

[Clyde Schechter]

I don't understand what you're trying to do here: the code you show does not correspond well to what you state is your goal. Your loop does not actually create a bunch of separate variables for each wave: it creates an overall average, and creates 5 copies of that variable.

You have panel data in long layout. There is no reason to loop over waves to calculate wave-specific averages, nor does it make sense to put those wave-specific averages in separate variables unless you have some special, strange purpose for doing that. You just need to include the wave variable in the -by- prefix of the -egen- command to get wave-specific averages.

So as best I can understand what you want, this is all you need:

local varlist asset bank cash equity ins prop bus veh
foreach i of local varlist {
    gen double ln`i' = log(`i')
    su ln`i'
    bysort hgage1 ethnic wave: egen ln`i'ave = mean(ln`i')
}

By the way, your example data does not include the variables bank, cash, equity, ins, prop, bus, and veh. It is best when you are showing code that you want troubleshooting for to make sure that the example data you show contains all the variables used in that code.

[Chris Boulis]

Thank you very much Clyde Schechter and congratulations for reaching over 20,000 posts - that's a lot of help you've provided. THANK YOU.

I don't understand what you're trying to do here

I want to graph each of these variables and compare the values between a few dichotomous variables (such as race, religion, etc). I do this with a 95% confidence band for each and overlaying that with the average of each of the same variables (by age) to see the change over a lifecycle (e.g below). My issue now is the scale - it no longer resembles actual $ values (which were in the 100,000s). I'm not sure how to interpret this now.

I appreciate you clarifying my error regarding creating different variables per wave when I can deal with this with an if statement -if wave==2- for example. As always, your help is appreciated. Thanks for your last point, I accidentally only included one asset type as an example. My mistake, thanks for highlighting this error on my part.

[Clyde Schechter]

My issue now is the scale - it no longer resembles actual $ values (which were in the 100,000s). I'm not sure how to interpret this now.

Well, that is what happens when you apply a log transformation. ln(100,000) = 11.5, and ln(1,000,000) = 13.8, and those numbers are roughly the range in which your data are now falling. I can also see, though, that with that large a range of values (especially with a few very low and very high outliers) you can't just plot the untransformed incomes on a regular graph, as the lower end of the range would be compressed to the point of invisibility. To gain the comprehensibility you seek and preserve the separation of the visual range, you might want to plot the untransformed incomes and use a logarithmic scale on the vertical axis (-yscale(log)- option). That way the graphs will look like they do now, but the axis will show labels in the original $ metric with familiar numbers.

[Nick Cox]

This problem has many solutions. For example if natural logarithms run between about 8 and 14 Stata is much better than I am in working back to the original scale.

. mata : exp((8, 10, 12, 14))
                 1             2             3             4
    +---------------------------------------------------------+
  1 |  2980.957987   22026.46579   162754.7914   1202604.284  |
    +---------------------------------------------------------+

I might then decide I want to show labels starting 1 and 3 (given that 3 is a little under the square root of 10, so labels like 1 3 10 30 100 are about equally spaced on logarithmic scale).

mylabels from SSC is a helper command. You say what you want to show and what different scale you ,are using and the command gives you the correspondence using a local macro.

. mylabels 3000 10000 30000 1e5  3e5 1e6, myscale(ln(@)) local(yla)
8.006367567650246 "3000" 9.210340371976184 "10000" 10.30895266064429 "30000"
11.51292546497023 "100000" 12.61153775363834 "300000" 13.81551055796427 "1000000"

See also https://www.stata-journal.com/articl...article=gr0072 and https://www.stata-journal.com/articl...article=gr0032

[Chris Boulis]

Thank you Clyde Schechter and Nick Cox for your thoughts and suggestions. Here's the graph before I transformed the values. As you can see I have a few 'extreme' values, and I thought transforming them to logs would address these.

tw (lpolyci totasset hgage1 if home == 1 & wave == 2, bwidth(3) lc("230 76 138") lw(medthick) ciplot(rarea) acolor("230 76 138%30") alw(5) level(95)) ///
    (lpolyci totasset hgage1 if home == 2 & wave == 2, bwidth(3) lc("25 154 222") lw(medthick) ciplot(rarea) acolor("25 154 222%30") alw(none) level(95)) ///
    (connect totassetave hgage1 if home == 1 & wave == 2, lc("230 76 138%70") lwidth(medthin) lpattern(shortdash) m(oh) mlw(vthin) mc("230 76 138%90"))  ///
    (connect totassetave hgage1 if home == 2 & wave == 2, lc("25 154 222%10") lwidth(medthin) lpattern(shortdash) m(oh) mlw(vthin) mc("25 154 222%90")), ///
    title("Wave 2", size(medsmall) position(11) justification(right)) /// 
    legend(region(lstyle(none)) order(2 "Household A" 4 "Household B") col(2) pos(0) ring(1) bplace(ne) rowgap(.1) colgap(1) size(small) color(none) region(fcolor(none))) ///
    ytitle("Total assets", size(small)) xtitle("Age", size(small)) ///   yscale(log) ///
    xla(20(10)100, format(%8.0fc) labsize(vsmall)) xtick(20(10)100) xmtick(15(10)95) ///
    yla(0(400000)2000000, format(%10.0fc) labsize(vsmall)) ytick(0(400000)2000000) ymtick(200000(400000)2000000), grid nogmin gex glc(gs12) glp(dot) glw(medthin)) ///
    ytick(0(.1).5) ///, add tpo(o) tl(1) tlw(thin) tlc(gs5) tlsty(grid)) ///
   plotr(margin(zero) lw(medthin)) scheme(burd) name("Fig4", replace) scale(1.2)

I included -yscale(log)- however, the scale (based on my code) was completely out (even after replacing the above -ylabel- to yla(3000 10000 30000 100000 300000 1000000). Although Nick's suggested code in #13 holds the solution to this, I did not know how to work this into my code. (Note I was provided this code, but have adapted it somewhat). By the way, Nick Cox I did install -mylabels- and I've been reading your attached papers - thank you. I feel like the answer is around p.277, but I do not yet understand how to apply it to my situation. I'll read more and try again tomorrow.

Could I kindly request an example of how I could work these suggestions into my code please?

[Nick Cox]

yscale(log) just takes the graph you would have got otherwise and changes the scale. It's not what to do when all calculations should be on the log scale. Also, in your case it seems that your smallest values are close to zero (or even negative) which could be problematic.


NB: This has morphed away from the thread title. Chris: Next different question, please start a new thread.