[Leonardo Guizzetti]

#44 does not prevent other programs (or do-files) to reset the respective global macros, which as I understand it, is Clyde's primary concern.

Thanks Daniel. I did understand Clyde's meaning, and there's really no clean way to achieve such a concept right now. I was merely thinking of a hack-y way to "reset" globals that might have otherwise been altered. Not really the same thing, but can effect a vaguely similar idea.

[daniel klein]

Well, you could set up a small program that backs up (preserves) your globals and use it as a prefix for critical commands. Your globals would then remain unchanged when the command concludes.

Here is a quick demonstration:


Wrapper to preserve and restore global macros:

*! version 1.0.0  25may2023
program preserve_globals
    
    version 18
    
    local globals : all globals
    
    tempname fh
    tempfile restore_globals
    
    file open `fh' using "`restore_globals'" , write
    
    foreach gmname of local globals {
        
        local contents : copy global `gmname'
        
        file write `fh' `"global `gmname' `contents'"' _newline
        
    }
    
    file close `fh'
    
    nobreak {
        
        capture noisily break `0'
        local rc = _rc
        
        capture do "`restore_globals'"
        if ( _rc ) {
            display as err "could not restore global macros"
            exit 9
        }
        
    }
    
    exit `rc'
    
end

Proof of Concept

program redefine_global_foo
    // add all arguments to global foo
    global foo "`0' and foobar"
    display as res `"global macro foo is: $foo"'
    // define a new global macro
    global bar "foobar"
end

mac drop _all

global foo "bar"
display `"$foo"'
display `"`bar'"' // void

preserve_globals redefine_global_foo $foo

display `"$foo"'
display `"$bar"'

the relevant output

. global foo "bar"

. display `"$foo"'
bar

. display `"`bar'"' // void


.
. preserve_globals redefine_global_foo $foo
global macro foo is: bar and foobar

.
. display `"$foo"'
bar

. display `"$bar"'
foobar

Note that the commands you call can still define new global macros.


You could make the wrapper prefix more informative, e.g., list the global macros that would have been changed. This might help with debugging in other contexts.

[Daniel Fernandes]

Thanks Daniel. I did understand Clyde's meaning, and there's really no clean way to achieve such a concept right now. I was merely thinking of a hack-y way to "reset" globals that might have otherwise been altered. Not really the same thing, but can effect a vaguely similar idea.

I believe it would be better to give tools for programmers to avoid using globals as much as possible. I would suggest a couple of features that seem safer than redefining globals:
- A new macro that can be accessed within its scope and all its dependent scopes (but not in an outer scope).
- A new option in "program define" that allows the program to access (but not write) locals in the scope that it was called from.

- Possibility to create more "return lists". There's no reason to be limited to three return lists: r(macro), e(), and s(). Stata can expand return lists quite easily (for instance, d() or p() ). Each of these spaces may have a slighly different set of features. This is already a practice in existing spaces. For instance r() allows "return add". e() is tailored for estimation commands, so it supports things like "ereturn post" or "ereturn report".

For instance, we may have a new space that acts like a collection. Let's call it k(). k() could support commands such as kreturn add macro, kreturn remove macro, kreturn modify macro.
We may also have a p() class that works similarly to the c() class. However, instead of containing the values of system parameters and settings for the whole session, it could be done at the program (and subprogram) level.

[Clyde Schechter]

I love #47! It now lives in my ado/plus directory.

The suggestions in #48 are also excellent.

Thanks for a really rewarding discussion!

[Leonardo Guizzetti]

Thanks Daniel, #47 is great!

[daniel klein]

I love #47! It now lives in my ado/plus directory.

Thanks. That was fast. I have not really tested the program. There are a couple of issues with the code in #47. It will choke on global macros with nested quotes, e.g.

global foo `""bar" in quotes"'

Also, the code exits with error when a global macro cannot be restored -- other global macros might or might not have been restored by that time.


Here is an improved version that fixes some issues (and works with Stata 11.2 or newer):

*! version 1.1.0  25may2023
program preserve_globals
    
    version 11.2
    
    local globals : all globals
    
    local n_globals 0
    foreach gmname of local globals {
        local GLOBAL`++n_globals' `gmname' `"${`gmname'}"'
    }
    
    nobreak {
        
        capture noisily break `0'
        local rc = _rc
        
        forvalues i = 1/`n_globals' {
            
            capture global `GLOBAL`i''
            if ( _rc ) {
                gettoken gmname : GLOBAL`i'
                display as err "could not restore global macro `gmname'"
                local rc = 9 // but do not exit yet
            }
            
        }
        
    }
    
    exit `rc'
    
end

The code will still choke on left single quotes, e.g.

global foo "don`t know"

but it will restore all global macros that can be restored before exiting with error.

Note that using local macros instead of a temporary file to back up the global macros probably increases execution speed but makes the code vulnerable to commands that manipulate local macros in the caller, e.g.,

levelsof mpg , local(GLOBAL1)

Note also that in this example, levelsof will store the levels of mpg in the local macro GLOBAL1 inside preserve_globals' scope; the local macro will not be accessible in the program (or do-file) that calls preserve_globals.

You could address these remaining issues and improve further on the code but I feel this is not the place to do so. I will leave it at that.

[FernandoRios]

Not sure if I have mentioned this before, but it would be very convenient if in a future version of Stata, one could run Mata Code from the editor, even if one is already Mata
Currently, if you already activated Mata, running a dofile will give an error.

[John Mullahy]

Strongly endorse FernandoRios in #52.

[John Mullahy]

Might beta(.,.) and gamma(.) functions be implemented in Stata as they already are in Mata?

. mata gamma(3)
  2

. di gamma(3)
unknown function gamma()
r(133);

. mata beta(2,3)
  .0833333333

. di beta(2,3)
unknown function beta()
r(133);

Of course exp(lngamma(.)) is available but programming would often be streamlined if beta(.,.) and gamma(.) were directly available.

P.S. I have not yet updated to v18 so perhaps this has already been implemented.

[Clyde Schechter]

P.S. I have not yet updated to v18 so perhaps this has already been implemented.

No, it has not been implemented in version 18.

[Niels Henrik Bruun]

The stub in -reshape- should be able to handle * and ? such that one, e.g., can write -reshape long bs_*_tm?_, i(id) j(grp)- instead of -reshape long bs_hospA_tm1_ bs_hospA_tm2_ bs_hospB_tm1_ bs_hospB_tm2_, i(id) j(grp)-

[Niels Henrik Bruun]

The command -lookfor- should be integrated into data management commands like -codebook- and -summarize-. If the argument isn't a varlist then the varlist generated from using -lookfor- on the argument should be the argument for -codebook- and -summarize-.
There might be other commands that could benefit from this.

[Richard Williams]

I'll make my usual request that factor variable notation be expanded to include other transformations of variable, e.g. ln(X), cube root of X, or whatever. Fernando Rios-Avila's clever f_able routine does some of this but Ithink it could be made simpler if built in to Stata.

Other than that, I would like to see existing routines, especially sem, made faster. That probably isn't a sexy selling point for new sales, but a lot of new stuff I don't really care about, I mostly want existing stuff to run better. My long-time dream is for Stata to buy out mPlus (which is incredibly fast) and build it into Stata.

[Leonardo Guizzetti]

I strongly second both points raised in #58. As it happens, I’m using Stata to run simulations in Mplus because of the speed.

[Julian Reif]

I will repeat two older requests:

1. I'd like to see none be added as a vcetype option, at least for the regress command. This option would speed up estimation in situations where standard errors are not necessary. For example, when doing a non-parametric power analysis, a researcher simulates a large number of regressions to obtain a distribution of coefficients, and does not need their standard errors.

2. I'd like to see native support added for high dimensional fixed effects in regress and related commands, coded in C to maximize speed. These are necessary for standard analyses with big datasets.