Iteratively adding to a local

[Chris Larkin]

Hello Statalisters,

I am attempting to run some simulations, incrementing sample size and stratification factors to test for balance. The below code does most of what I need it to do.

1. It creates datasets of different sample sizes based on the values in -- local sample --
2. It creates covariate variables based on the categories in -- local cats --
3. It conducts stratified randomisation at different sizes of the dataset

The bit i'm struggling with is highlighted in red below. If you run the code you can see on the first loop that it creates what I need, iteratively adding "obs1" and then "obs1 obs2" and then "obs1 obs2 obs3", etc, until it is complete. But it then goes on because it is a nested loop. What I want to do is after each iteration stratify based just on obs1, and then obs1 and obs2, etc., until i'm stratifying my randomisation on all the observations generated in the first loop.

I've tried different things, such as moving both the bottom two snippets into the top bit of the code but to no avail. If anyone has any ideas, i'd welcome them!

**Clear memory
clear

local sample 100 200 400 800 1600 3200 6400 10000 //set up local for different sample sizes
local strata = ""
foreach size of local sample{
    preserve
    qui set obs `size' //set different sample sizes
    qui gen id = _n //generate a unique id
    local cats "`"2"' `"2"' `"2"' `"2"' `"2"' `"2"'" //set up local for category numbers of strat vars
    forval i = 1/6{
        local catind :word `i' of `cats'
        qui gen obs`i' = mod(_n,`catind') //generate 6 i number of strat vars based on values of local cats
    }
    
   qui ds obs*, skip(1)
        local stratlist "`r(varlist)'" //store all strat vars in a local
        *di `"`stratlist'"'{
        
        forval stratnum = 1/6{
            local strat: word `stratnum' of `stratlist'
            local strata `"`strata' `strat'"'
            di `"`strata'"'
            
        }
        qui egen strata=group(`strata') //gen a variable that makes unique groups based on strat vars
        set seed 31540 //setting a seed for replicability
        qui gen randomnum = runiform() //generating a random number
        qui bysort strata: egen order=rank(randomnum) //generating a rank order var based on the random number
        qui bysort strata: gen treat = (order <= _N/2) //assigning condition based on rank
            
        foreach var of local strata{
            tab `var' treat, r
        }
        des, s
        
    restore
}

[Clyde Schechter]

You just need to re-initialize local macro strata to the empty string before you enter the loop:

**Clear memory
clear

local sample 100 200 400 800 1600 3200 6400 10000 //set up local for different sample sizes
local strata = ""
foreach size of local sample{
    preserve
    qui set obs `size' //set different sample sizes
    qui gen id = _n //generate a unique id
    local cats "`"2"' `"2"' `"2"' `"2"' `"2"' `"2"'" //set up local for category numbers of strat vars
    forval i = 1/6{
        local catind :word `i' of `cats'
        qui gen obs`i' = mod(_n,`catind') //generate 6 i number of strat vars based on values of local cats
    }
    
   qui ds obs*, skip(1)
        local stratlist "`r(varlist)'" //store all strat vars in a local
        *di `"`stratlist'"'{
        
        local strata
        forval stratnum = 1/6{
            local strat: word `stratnum' of `stratlist'
            local strata `"`strata' `strat'"'
            di `"`strata'"'
            
        }

        qui egen strata=group(`strata') //gen a variable that makes unique groups based on strat vars
        set seed 31540 //setting a seed for replicability
        qui gen randomnum = runiform() //generating a random number
        qui bysort strata: egen order=rank(randomnum) //generating a rank order var based on the random number
        qui bysort strata: gen treat = (order <= _N/2) //assigning condition based on rank
            
        foreach var of local strata{
            tab `var' treat, r
        }
        des, s

        
    restore
}

[Chris Larkin]

Yes, that helps. I've re-initialised the local strata before I enter the second forval loop as suggested. But as the code is at the moment this means i only ever stratify on the final value of the local strata including all obs.

I've then had to move the randomisation snippet up into the same forval loop and -- drop strata randomnum rank treat -- before that loop repeats.

Cheers Clyde!

[Chris Larkin]

Hi Clyde,

I've attempted to add in some continuous variables to the simulation too, as well as some unobservables which are not stratified on but have assumed distributions. Below I show the output that i'm getting. I do not understand how the unobservable variables have found their way into the `strata' local. Can you see it?

obs1 obs2 obs3 obs4 obs5 obs6
the below randomisation is stratified on:  obs1
obs1 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
the below randomisation is stratified on:  obs1 obs2
obs1 obs2 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
the below randomisation is stratified on:  obs1 obs2 obs3
obs1 obs2 obs3 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
the below randomisation is stratified on:  obs1 obs2 obs3 obs4
obs1 obs2 obs3 obs4 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
the below randomisation is stratified on:  obs1 obs2 obs3 obs4 obs5
obs1 obs2 obs3 obs4 obs5 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
the below randomisation is stratified on:  obs1 obs2 obs3 obs4 obs5 obs6
obs1 obs2 obs3 obs4 obs5 obs6 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

Contains data
  obs:           100                          
 vars:            17                          
 size:         6,800                          
Sorted by: strata
     Note: Dataset has changed since last saved.
obs1 obs2 obs3 obs4 obs5 obs6
the below randomisation is stratified on:  obs1
obs1 unobs1 unobs2 unobs3 unobs4 unobs5 unobs6

**Setting up a log to veiw output
capture log close _all
log using "stratification_sims$S_DATE.txt", append text name("First Attempt")

local sample 100 200 400 800 1600 3200 6400 10000 //set up local for different sample sizes
local strata = ""

foreach size of local sample{
    preserve
        qui set obs `size' //set different sample sizes
        qui gen id = _n //generate a unique id
        local cats "`"2"' `"2"' `"2"' `"3"' `"4"' `"5"'" //set up local for category values of stratification vars
        quietly forval i = 1/6{
            local catind :word `i' of `cats'
            gen obs`i' = mod(_n,`catind') //generate i number of stratification vars based on values of local cats
            if `i'<=2                        local fcn rnormal() //generating a normally-distributed var
            else if `i' >= 3 & `i' <= 4   local fcn rbeta(1, 6) //generating a positively-skewed var 
            else if `i' >= 5 & `i' <= 6   local fcn rbeta(6, 1) //generating a negatively-skewed var
            qui gen unobs`i' = `fcn'
        }
        
            ds obs*, skip(1)
            local stratlist "`r(varlist)'" //store all stratification vars in a local
            local strata
            forval stratnum = 1/6{
                local strat: word `stratnum' of `stratlist'
                local strata `"`strata' `strat'"' //looping through stratification vars and adding a new one on each iteration
                di in red "the below randomisation is stratified on: " `"`strata'"'
                
                qui egen strata=group(`strata') //gen a variable that makes unique groups based on strat vars
                set seed 31540 //setting a seed for replicability
                qui gen randomnum = runiform() //generating a random number
                qui bysort strata: egen order=rank(randomnum) //generating a rank order var based on the random number
                qui bysort strata: gen treat = (order <= _N/2) //assigning condition based on rank
                
                qui ds unobs*, skip(1)
                local unobs "`r(varlist)'"
                local variables `strata' `unobs' //putting observables and unobservables in one local
                di `"`variables'"'
                foreach var of local variables{ //balance test
                    qui: reg treat `var', r //change this for an actual test of balance later
                }
                    des, s
                    qui drop strata randomnum order treat
            }
    restore
}

[Clyde Schechter]

I do not understand how the unobservable variables have found their way into the `strata' local.

They haven't.

The only place you -`display `"`strata'"'- what you get as output is exactly what you are looking for: first obs1, then obs1 obs2, etc. out to obs1 through obs6. None of the unobs variables is mentioned. However, the next command that displays on the screen is when yo get down to -di `"`variables'"'. The local macro variables is defined as containing unobs1 through unobs6, preceded by whatever is in local macro strata. So the first time you get obs1 unobs1...unobs6, the next time you get obs1 ob2 unobs1...unobs6, etc. through obs1...obs6 unobs1...unobs6. That's coming out of `variables', not `strata'.

Perhaps you are being confused by the color. When you -di in red "the below randomisation is stratified on: " `"`strata'"'-, the display color gets set to red. Nothing that happens before you get to -di `"`variables'"' changes that back to normal text. So you were probably expecting the obs-unobs hybrid line to show up as normal text, but it doesn't. If you change that line to -display as text `"`variables'"'- then it is easy to see what is coming from `strata' in red and what is coming from `variables'.

Added: Actually -display in red- is old syntax that is no longer documented, but does still work. The current equivalent syntax is -display as error-. Sometimes after you enter that mode of display, that mode remains turned on until it is overridden by another -display- statement that uses -as text- or as result or as input. The same is true of these other modes: they persist until overridden. (The override does not have to be from an explicit -display-: it may occur from a -display- command that is buried inside the execution of a different command. I say sometimes, because it is not consistently so, and I do not understand the circumstances that govern it. Anyway, if you are going to mix display modes in the output of a program, it is best to be explicit about the mode in every -display- command to avoid this kind of confusion.

[Chris Larkin]

Wow. I do feel a bit like an idiot... but thank you for your explanation. I did not know / understand that qualifying the - display - command to show up in red in the first instance would mean later outputs of the command would also show up red. I've changed it to - di as text - and it now does exactly what I was hoping.

Many thanks

[Clyde Schechter]

Don't feel like an idiot. This behavior of -display- is not consistent, and I'm not aware of it being documented anywhere. (And it probably wouldn't be in any case because -display in red- is no longer documented syntax in the first place.) This is just one of those things that you stumble over at some point using Stata and find out about.

[Chris Larkin]

Thanks. A closer read of my output and code would've brought me to the conclusion that I was in fact displaying `variables' and not `strata' though. It is, as you say though, one the things with coding; one can end up staring at a snippet for hours without seeing the block, but a fresh pair of eyes can often see the issue straight away

[Clyde Schechter]

It is, as you say though, one the things with coding; one can end up staring at a snippet for hours without seeing the block, but a fresh pair of eyes can often see the issue straight away

Quite so!