command "clear" effect on local macros is not clear

[Salah Mahmud]

Hello,
In the code below, I use mata functions to update and query 2 local macros (vtSuccessfulRuns and vtFailedRuns). By making these macros local to this file, I was hoping that these macros will be shielded from the effects of whatever code the caller of these functions might write between calls to them. If you run the tests at the bottom of this post, you will see that everything works fine even if I issued "clear" and "clear all" in the current file. But if you type "clear" in the command window, the code fails because both macros are gone (you will get missing values if not outright failures). The same thing happens if you run another do file that includes "clear".

Why clear behaves so differently?
What is the best way to have a set of variables hidden behind an interface like the one below? I tried to use structs in mata but the complier did not like struct vars declared outside the functions.
I guess what I need is the equivalent of a static variable in C or a module-level variable in Pascal.
Any suggestions?

Thanks,




mata: mata clear
set matastrict on //all vars must be pre-declared

local vtSuccessfulRuns = 0
local vtFailedRuns = 0


mata:
version 13.2

void vtRestTesting() {
st_local("vtSuccessfulRuns", "0")
st_local("vtFailedRuns", "0")
}

void vtIncSuccessCounter() {
real scalar n
n = vtSuccessfulRuns() + 1
st_local("vtSuccessfulRuns", strofreal(n))
st_global("r(SuccessfulRuns)", strofreal(n))
}

void vtIncFailCounter() {
real scalar n
n = vtFailedRuns() + 1
st_local("vtFailedRuns", strofreal(n))
st_global("r(FailedRuns)", strofreal(n))
}

real scalar vtSuccessfulRuns(){
real scalar n
n = strtoreal(st_local("vtSuccessfulRuns"))
st_global("r(SuccessfulRuns)", strofreal(n))
return(n)
}

real scalar vtFailedRuns(){
real scalar n
n = strtoreal(st_local("vtFailedRuns"))
st_global("r(FailedRuns)", strofreal(n))
return(n)
}


end //mata code



//testing routines

mata:
vtRestTesting()
if (vtSuccessfulRuns() != 0) errprintf("expected 0 got %g", vtSuccessfulRuns());;
vtIncSuccessCounter()
if (vtSuccessfulRuns() != 1) errprintf("expected 1 got %g", vtSuccessfulRuns());;
vtIncSuccessCounter()
if (vtSuccessfulRuns() != 2) errprintf("expected 2 got %g", vtSuccessfulRuns());;

if (vtFailedRuns() != 0) errprintf("expected 0 got %g", vtFailedRuns());;
vtIncFailCounter()
if (vtFailedRuns() != 1) errprintf("expected 1 got %g", vtFailedRuns());;
vtIncFailCounter()
if (vtFailedRuns() != 2) errprintf("expected 2 got %g", vtFailedRuns());;

end

mata: vtRestTesting()
mata: vtFailedRuns()
local nfail = r(FailedRuns)
mata: vtSuccessfulRuns()
local nsuccess = r(SuccessfulRuns)
noi di `nfail' `nsuccess'
mata: vtIncSuccessCounter()
local nsuccess = r(SuccessfulRuns)
mata: vtIncFailCounter()
local nfail = r(FailedRuns)
noi di `nfail' `nsuccess'
mata: vtIncSuccessCounter()
local nsuccess = r(SuccessfulRuns)
mata: vtIncFailCounter()
local nfail = r(FailedRuns)
noi di `nfail' `nsuccess'

mata: vtFailedRuns()
local nfail = r(FailedRuns)
mata: vtSuccessfulRuns()
local nsuccess = r(SuccessfulRuns)
noi di `nfail' `nsuccess'

clear

mata: vtFailedRuns()
local nfail = r(FailedRuns)
mata: vtSuccessfulRuns()
local nsuccess = r(SuccessfulRuns)
noi di `nfail' `nsuccess'


clear all

mata: vtFailedRuns()
local nfail = r(FailedRuns)
mata: vtSuccessfulRuns()
local nsuccess = r(SuccessfulRuns)
noi di `nfail' `nsuccess'

//Above code works fine, now type clear in the command window and run the following code. It will no longer works.

mata: vtFailedRuns()
local nfail = r(FailedRuns)
mata: vtSuccessfulRuns()
local nsuccess = r(SuccessfulRuns)
noi di `nfail' `nsuccess'

[William Lisowski]

I'm not entirely clear what you're doing, but I think it doesn't matter what the user types in the command window: once you exit the do-file within which the two local macros vtSuccessfulRuns and vtFailedRuns are defined, those local macros - which are "local" to the do-file within which they are deined - cease to exist, so when you attempt to use them the substitution result is null.

[Salah Mahmud]

Thanks,
What you said might be correct for Stata do files, but it is not the case for Mata functions. Using the Mata functions defined above, I am able to call e.g., mata: vtIncSuccessCounter() from the command window and get the underlying local macros updated correctly although the do file where the functions and macros defined is no longer in scope.
What is strange is that issuing "clear" in the command window seems to remove the macros. The documentation of clear does not mention that it affects local macros. But for some reason, it does in this case remove the macros defined in a different do file.

[William Lisowski]

Your example is unclear - I cannot tell which lines are executed from within do-files and which are executed from the command window, and it makes a meaningful difference.

So consider the following simple examples. We start with one where the local macro foo is undefined at the command window level before running my do-file.

. * in command window

. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)

. do "example.do"

. mata: mata clear

. 
. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)

. local foo gnxl

. macro list _foo
_foo:           gnxl

. 
. mata: 
------------------------------------------------- mata (type end to exit) -----------------------
: void setfoo() {
> st_local("foo","bar")
> }

: end //mata code
-------------------------------------------------------------------------------------------------

. 
. mata setfoo()

. macro list _foo
_foo:           bar

. 
end of do-file

. * in command window

. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)

.

What we see is that local macros created within a do-file cease to exist when the do-file ends.

In the next example the macro _foo is defined at the command window level before running my do-file.

. local foo whatever

. * in command window

. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)
_foo:           whatever

. do "/Users/lisowskiw/Research/Stata sandbox/170529/example.do"

. mata: mata clear

. 
. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)

. local foo gnxl

. macro list _foo
_foo:           gnxl

. 
. mata: 
------------------------------------------------- mata (type end to exit) -----------------------
: void setfoo() {
> st_local("foo","bar")
> }

: end //mata code
-------------------------------------------------------------------------------------------------

. 
. mata setfoo()

. macro list _foo
_foo:           bar

. 
end of do-file

. * in command window

. macro list _all
S_level:        95
S_ADO:          BASE;SITE;.;PERSONAL;PLUS;OLDPLACE
S_StataSE:      SE
S_FLAVOR:       Intercooled
S_OS:           MacOSX
S_OSDTL:        10.12.5
S_MACH:         Macintosh (Intel 64-bit)
_foo:           whatever

.

What we see is that local macro definitions within the do-file do not affect local macro definitions at the command window level.

[Robert Picard]

I think that William's advice about the scope of local macros is correct and I can't replicate what you claim. Here's a minimal example derived from your code:

clear all

local vtSuccessfulRuns = 100

version 13.2
mata:
mata clear
mata set matastrict on

real scalar vtSuccessfulRuns(){
    real scalar n
    n = strtoreal(st_local("vtSuccessfulRuns"))
    st_global("r(SuccessfulRuns)", strofreal(n))
    return(n)
}

void vtIncSuccessCounter() {
    real scalar n
    n = vtSuccessfulRuns() + 1
    st_local("vtSuccessfulRuns", strofreal(n))
    st_global("r(SuccessfulRuns)", strofreal(n))
}

end //mata code

clear
mata: vtIncSuccessCounter()
ret list
mata: vtIncSuccessCounter()
ret list

and the results from the last part when the code above is run from a do-file:

. clear

. mata: vtIncSuccessCounter()

. ret list

macros:
     r(SuccessfulRuns) : "101"

. mata: vtIncSuccessCounter()

. ret list

macros:
     r(SuccessfulRuns) : "102"

.

If I then proceed to the Command window and type:

mata: mata describe
mata: vtIncSuccessCounter()
ret list

I get:

. mata: mata describe

      # bytes   type                        name and extent
-------------------------------------------------------------------------------
          264   void                        vtIncSuccessCounter()
          228   real scalar                 vtSuccessfulRuns()
-------------------------------------------------------------------------------

. mata: vtIncSuccessCounter()

. ret list

macros:
     r(SuccessfulRuns) : "."

The mata functions are still defined because you are using mata in interactive more, even if you are running commands from a do-file. Your mata functions are therefore global and survive the end of the do-file. They also survive the clear statement in the do-file for the reasons explained in help clear.

If I return to the Command window and type:

local vtSuccessfulRuns = 200
mata: mata describe
mata: vtIncSuccessCounter()
ret list

I get:

. local vtSuccessfulRuns = 200

. mata: mata describe

      # bytes   type                        name and extent
-------------------------------------------------------------------------------
          264   void                        vtIncSuccessCounter()
          228   real scalar                 vtSuccessfulRuns()
-------------------------------------------------------------------------------

. mata: vtIncSuccessCounter()

. ret list

macros:
     r(SuccessfulRuns) : "201"

So the bottom line is that locals are just that, local to the do-file (or the current interactive session via the Command window). The clear command has no effect on macros. To clear all macros (locals and globals), you need macro drop _all.