Issue combining foreach and runby

[Konstantina Boutsioukou]

Hi everyone,

I recently came across with the -runby- command which I find extremely helpful.

I have a cross-section of countries, regions and a number of variables reflecting different aspects of institutional quality as well as a population weight for each region. Example of how the data looks:

* Example generated by -dataex-. For more info, type help dataex
clear
input str10 iso3 str7 TL2 str8 TL3 float dem_pop_weight double eqi
"GRC" "EL30" "EL301" .3556652 -1.0455000400543213
"GRC" "EL30" "EL302" .3556652 -1.0455000400543213
"GRC" "EL30" "EL303" .3556652 -1.0455000400543213
"GRC" "EL30" "EL304" .3556652 -1.0455000400543213
"GRC" "EL30" "EL305" .3556652 -1.0455000400543213
"GRC" "EL30" "EL306" .3556652 -1.0455000400543213
"GRC" "EL30" "EL307" .3556652 -1.0455000400543213
"GRC" "EL41" "EL411" .018593332 -.9587500095367432
"GRC" "EL41" "EL412" .018593332 -.9587500095367432
"GRC" "EL41" "EL413" .018593332 -.9587500095367432
"GRC" "EL42" "EL421" .030548707 -.9900000095367432
"GRC" "EL42" "EL422" .030548707 -.9900000095367432
*


I want to create p90/p10 ratio for each institution for each country. I am using the following code which works fine:

capture program drop my_summarize
program define my_summarize
local statistics N mean p10 p90
summ eqi [aw=dem_pop_weight], detail
foreach s of local statistics {
gen `s' = r(`s') in 1
}
// keep in 1
// keep iso3 `statistics'
exit
end

runby my_summarize, by(iso3) verbose
exit
//////////

(Variables: eqi: institutional quality variable, dem_pop_weight is a population weight for each region and iso3: country)

I tried creating a loop to run the same code for all institutions. The code starts running and produces summary tables but as the loop runs I am getting the following error: invalid syntax. Also this is mentioned at the end
Number of by-groups = 42
by-groups with errors = 42


The code is the following:



capture program drop my_summarize
program define my_summarize

local statistics N mean p10 p90
foreach xvar of varlist $list_eqi {
global xvar `xvar'
foreach s of local statistics {
summarize ${xvar}[aw=dem_pop_weight], detail

gen `s'_${xvar} = r(`s')_${xvar}
}
}

exit
end

runby my_summarize, by(iso3) verbose

exit


Any help would be really appreciated.

Thank you in advance.

Best,
Konstantina

[Clyde Schechter]

There are two unrelated problems here, the fixing the first of which, reveals the second.

The first problem is that you do not define global macro list_eqi before you use it, so that -foreach xvar of varlist $list_eqi {- expandsd to -foreach xvar of varlist {- which, lacking an actual varlist, is a syntax error. Perhaps in your full real code you did define it and just didn't include that command in what you showed in your post.

Anyway, once that is taken care of, we then have a syntax error with -gen `s'_${xvar} = r(`s')_${xvar}-. The problem is that r(`s') is a number, like maybe -1.04. So you are asking Stata to -gen mean_eqi = -1.04_eqi-. But -1.04_eqi is not a legal variable name, nor can Stata interpret it as an algebraic expression because _ is not an operator.

Now, not a source of error messages, but an unecessary, and unwise, pracatice is overuse of global macros. You can't avoid using global macro $list_eqi to pass a list of variables from your top level code into program my_summarize. At least you can't do it without some effort. So we'll let that one go. But there is absolutely no reason to then create $xvar inside that program. Just use the local macro `xvar' that has already been created for you by -foreach-.

Putting this all together, my fix for this code would be

capture program drop my_summarize
program define my_summarize
set tracedepth 1
set trace on
    local statistics N mean p10 p90
    foreach xvar of varlist $list_eqi {
        foreach s of local statistics {
            summarize `xvar'[aw=dem_pop_weight], detail
            gen `s'_${xvar} = r(`s')
        }
    }

    exit
end

global list_eqi eqi // YOU MAY WANT TO ADD OTHER VARIABLES TO THIS
runby my_summarize, by(iso3) verbose

[Konstantina Boutsioukou]

Dear Clyde,

Thank you so much for the detailed answer, its extremely insightful. I run your code but i was getting an error that the variable N_xvar already exists so I tweaked the code a bit and now works fine.

capture program drop my_summarize
program define my_summarize
set tracedepth 1
set trace on
local statistics N mean p10 p90
foreach xvar of varlist $list_eqi {
global xvar `xvar'
foreach s of local statistics {
summarize `xvar'[aw=dem_pop_weight], detail
gen `s'_${xvar} = r(`s')
}
}

exit
end


runby my_summarize, by(iso3) verbose


Best,
Konstantina