Include void in a loop

[Maximilian Koppenberg]

Hello,

I coded an optimization and I would like to repeat that optimization for subsamples randomly drawn from my sample.

*Here, I define the variables used in the optimization


mata: 
void GMMTL(todo,betas,crit,g,H)
{
PHI=st_data(.,("phi"))
LPHI = st_data(.,("lagphi"))
LED = st_data(.,("lagExportD"))
Z=st_data(.,("const","withinllogl","withinlogk1","withinllogl2","withinlogk2","withinlogkllogl","withinllogm","withinllogm2","withinllogmllogl","withinlogkllogm"))
X=st_data(.,("const","withinlogl1","withinlogk1","withinlogl2","withinlogk2","withinl1k1","withinlogm1","withinlogm2","withinl1m1","withink1m1"))
lagX = st_data(.,("const","withinllogl","withinllogk","withinllogl2","withinllogk2","withinllogkllogl","withinllogm","withinllogm2","withinllogmllogl","withinllogkllogm"))
Y=st_data(.,("withinlogy"))
C=st_data(.,("const"))
VAR = (C,PHI,LPHI,LED,Z,X,lagX,Y,C)

*Here, I draw a random subsample of my data and replace the original vectors. 

VAR[,1] = runiform(rows(VAR),1) :> .9
PHI = select(VAR[.,2], VAR[.,1]:~=0)
LPHI = select(VAR[.,3], VAR[.,1]:~=0)
LED = select(VAR[.,4], VAR[.,1]:~=0)
Z = select(VAR[.,4\5\6\7\8\9\10\11\12\13], VAR[.,1]:~=0)
X = select(VAR[.,14\15\16\17\18\19\20\21\22\23], VAR[.,1]:~=0)
lagX= select(VAR[.,24\25\26\27\28\29\30\31\32\33], VAR[.,1]:~=0)
Y=select(VAR[.,34], VAR[.,1]:~=0)
C=select(VAR[.,35], VAR[.,1]:~=0)

*Here, I define the system of equations to be solved by the optimizer.

OMEGA=PHI-X*betas'
lagOMEGA=LPHI-lagX*betas'
lagOMEGAlagED =diagonal((LPHI-lagX*betas')*LED')
lagOMEGAsq= diagonal((LPHI-lagX*betas')*(LPHI-lagX*betas')')
lagOMEGAsqlagED = diagonal(diagonal((LPHI-lagX*betas')*(LPHI-lagX*betas')')*LED')
lagOMEGAcub= diagonal(lagOMEGAsq*(LPHI-lagX*betas')')
lagOMEGAcublagED = diagonal(diagonal(lagOMEGAsq*(LPHI-lagX*betas')')*LED')
pollagOMEGA = (C,lagOMEGA,lagOMEGAsq,lagOMEGAcub,lagOMEGAlagED,lagOMEGAsqlagED,lagOMEGAcublagED,LED)
g_b = invsym(pollagOMEGA'pollagOMEGA)*pollagOMEGA'OMEGA
XI = OMEGA-pollagOMEGA*g_b
crit=(Z'XI)'(Z'XI)
}

*This is just fixing the optimization settings. 

S=optimize_init()
optimize_init_evaluator(S, &GMMTL())
optimize_init_evaluatortype(S,"d0")
optimize_init_technique(S,"nm")
optimize_init_nmsimplexdeltas(S, 0.1)
optimize_init_conv_maxiter(S, 1000)
optimize_init_which(S,"min")
optimize_init_params(S,(7,0.1911017,0.051978,0.0308493,0.0113114,0.0304485,0.5731468,0.0410968,-0.073725,-0.0350174))
p=optimize(S)

I would now like to loop the whole command 1,000 times, and save the betas to matrix. My first attempt was to put:

mata
bhat = J(1000, 10, .)
end

at the very beginning of the command followed by:

forval i=1/1000 {

and closing the loop by using

bhat[i,] = optimize_result_params(S)
}

to save the parameters to the matrix bhat. However, Stata keeps giving me the error message: <istmt>: 3499 i not found.

I have also tried to start the loop within mata using:

for (i=1; i<=1000; i++) {

However, Stata returns the error message expression invalid right after

void GMMTL(todo,betas,crit,g,H)

in that case.

I would appreciate any idea how to fix this.

[William Lisowski]

The i in your forvalues loop is a local macro so at a minimum you need

bhat[`i',] = optimize_result_params(S)

in the first version of your code, with the loop outside Mata.

[Maximilian Koppenberg]

Dear William,

thank you for pointing this out. However, now Stata performs the first iteration but gives the error message: expression invalid.

mata
bhat = J(1000,10,.)
end


forval i=1/1000 {
mata:
void GMMTLVA(todo,betas,crit,g,H)
{
PHI=st_data(.,("phi"))
LPHI = st_data(.,("lagphi"))
LED = st_data(.,("lagExportD"))
Z=st_data(.,("const","withinllogl","withinlogk1","withinllogl2","withinlogk2","withinlogkllogl","withinllogm","withinllogm2","withinllogmllogl","withinlogkllogm"))
X=st_data(.,("const","withinlogl1","withinlogk1","withinlogl2","withinlogk2","withinl1k1","withinlogm1","withinlogm2","withinl1m1","withink1m1"))
lagX = st_data(.,("const","withinllogl","withinllogk","withinllogl2","withinllogk2","withinllogkllogl","withinllogm","withinllogm2","withinllogmllogl","withinllogkllogm"))
Y=st_data(.,("withinlogy"))
C=st_data(.,("const"))
VAR = (C,PHI,LPHI,LED,Z,X,lagX,Y,C)
VAR[,1] = runiform(rows(VAR),1) :> .9
PHI = select(VAR[.,2], VAR[.,1]:~=0)
LPHI = select(VAR[.,3], VAR[.,1]:~=0)
LED = select(VAR[.,4], VAR[.,1]:~=0)
Z = select(VAR[.,4\5\6\7\8\9\10\11\12\13], VAR[.,1]:~=0)
X = select(VAR[.,14\15\16\17\18\19\20\21\22\23], VAR[.,1]:~=0)
lagX= select(VAR[.,24\25\26\27\28\29\30\31\32\33], VAR[.,1]:~=0)
Y=select(VAR[.,34], VAR[.,1]:~=0)
C=select(VAR[.,35], VAR[.,1]:~=0)
OMEGA=PHI-X*betas'
lagOMEGA=LPHI-lagX*betas'
lagOMEGAlagED =diagonal((LPHI-lagX*betas')*LED')
lagOMEGAsq= diagonal((LPHI-lagX*betas')*(LPHI-lagX*betas')')
lagOMEGAsqlagED = diagonal(diagonal((LPHI-lagX*betas')*(LPHI-lagX*betas')')*LED')
lagOMEGAcub= diagonal(lagOMEGAsq*(LPHI-lagX*betas')')
lagOMEGAcublagED = diagonal(diagonal(lagOMEGAsq*(LPHI-lagX*betas')')*LED')
pollagOMEGA = (C,lagOMEGA,lagOMEGAsq,lagOMEGAcub,lagOMEGAlagED,lagOMEGAsqlagED,lagOMEGAcublagED,LED)
g_b = invsym(pollagOMEGA'pollagOMEGA)*pollagOMEGA'OMEGA
XI = OMEGA-pollagOMEGA*g_b
crit=(Z'XI)'(Z'XI)
}
S=optimize_init()
optimize_init_evaluator(S, &GMMTLVA())
optimize_init_evaluatortype(S,"d0")
optimize_init_technique(S,"nm")
optimize_init_nmsimplexdeltas(S, 0.1)
optimize_init_conv_maxiter(S, 1000)
optimize_init_which(S,"min")
optimize_init_params(S,(7,0.1911017,0.051978,0.0308493,0.0113114,0.0304485,0.5731468,0.0410968,-0.073725,-0.0350174))
(void) _optimize(S)
        bhat[`i',] = optimize_result_params(S)
}

[William Lisowski]

Let's start with something basic.

void GMMTL(todo,betas,crit,g,H)

is not a "void" - it is the first line of a Mata function definition, the function name is GMMTL and it returns a void result. So your title for this topic would have been correctly "Include function definition in a loop". That title might have gotten more attention, it's a known trap, especially for those new to Mata. If you are new to Mata, you should spend some time with the Mata Reference Manual PDF included in your Stata installation and accessible from Stata's Help menu.

The root of your problem is that If you want

• a block of Mata code terminated by end

to run within

• a Stata block surrounded by braces {}, such as a forvalues loop

you need to

• define your Mata code as a Mata function outside the brace-enclosed block
• call the Mata function within the brace-enclosed block

You not only have your function definition to run, which in any event should not be within the forvalues loop, but also your code to run the optimization. So your code should look something like the following.

mata:
void GMMTLVA(todo,betas,crit,g,H)
{
...
}

void run_once(i,bhat)
{
S=optimize_init()
...
(void) _optimize(S)
bhat[i,] = optimize_result_params(S)
}

end

mata bhat = J(1000,10,.)
forval counter=1/1000 {
    mata: run_once(`counter',bhat)
}

Two things to notice. First, the bhat[i,1] is as it appeared in post #1 - at that point i is indeed a Mata variable, not a local macro. Second, when you're testing this, change the forval loop to end at something small, like 2 or 3, and then

mata bhat[(1..3),.]

and inspect the results carefully to be sure you're on the right path.