Higher-Order Commands

[jhonny-econ]

Dear Stata-Users,

my goal is to set up a command to automatically track the number of observations that have been dropped during a cleaning process.

My idea is the following: instead of executing, say for example

drop if case==1

I would want to execute

trackobs drop if case==1

My own command, lets call it tackobs, would execute the drop-command and on top create a local with the number of observations that have been dropped or the number of remaining observations. I can collect those locals at the end of my dofile, organize that information in a table and export it.

The reason for doing that is that I want to "hide" the creation of locals inside another ado-file in order not to clutter my dofile and make it unreadable.

Is that a possible?

Thank you!

[Rafal Raciborski (StataCorp)]

As of Stata 15.1, drop stores the number of observations dropped in r(N_drop).

sysuse auto
drop if foreign
ret list

scalars:
             r(N_drop) =  22

[daniel klein]

Here is a quick draft of one possible way to do this:

*! version 1.0.0 11jul2018 daniel klein
program trackobs
    version 11.2
    
    gettoken subcmd zero : 0
    if (inlist(`"`subcmd'"', "set", "report", "clear")) {
        trackobs_`subcmd' `zero'
        exit
    }
    
    local i : char _dta[trackobs_counter]
    trackobs_assert_counter , counter(`i')
    
    nobreak {
        local Nbefore = c(N)
        capture noisily break `0'
        if (_rc) exit _rc
        local Nafter  = c(N)
        local ++i
        char _dta[trackobs_`i']     `Nbefore' `Nafter' `0'
        char _dta[trackobs_counter] `i'
    }
end

program trackobs_set
    version 11.2
    
    syntax // nothing allowed
    
    if (`"`: char _dta[trackobs_counter]'"' != "") {
        display as err "trackobs counter already set"
        exit 498
    }
    else char _dta[trackobs_counter] 0
end

program trackobs_clear
    version 11.2
    
    syntax // nothing allowed
    
    local I : char _dta[trackobs_counter]
    trackobs_assert_counter , counter(`I')
    
    nobreak {
        forvalues i = 1/`I' {
            char _dta[trackobs_`i'] // void
        }
        char _dta[trackobs_counter] // void
    }
end

program trackobs_report
    version 11.2
    
    syntax // nothing allowed
    
    local I : char _dta[trackobs_counter]
    trackobs_assert_counter , counter(`I')
    
    if (c(stata_version) >= 14) local u u
    display as txt _col(36) "Command"     _continue
    display as txt _col(50) "Obs. before" _continue
    display as txt _col(70) "Obs. after"
    display as txt "{hline 79}"
    forvalues i = 1/`I' {
        local trackobs_`i' : char _dta[trackobs_`i']
        gettoken Nbefore trackobs_`i' : trackobs_`i'
        gettoken Nafter  trackobs_`i' : trackobs_`i'
        display as res %42s `u'substr(`"`trackobs_`i''"', 1, 42) _continue
        display as res %18.0g `Nbefore'                          _continue
        display as txt char(32)                                  _continue
        display as res %18.0g `Nafter'
    }
    display as txt "{hline 79}"
end

program trackobs_assert_counter
    version 11.2
    capture syntax , COUNTER(numlist integer max=1 >=0)
    if (_rc) {
        display as err "trackobs counter not set"
        exit 499
    }
end

Here is an example

trackobs set
trackobs sysuse auto
trackobs drop if foreign
trackobs report
trackobs clear

and here is the output

. trackobs set

. trackobs sysuse auto
(1978 Automobile Data)

. trackobs drop if foreign
(22 observations deleted)

. trackobs report
                                   Command       Obs. before         Obs. after
-------------------------------------------------------------------------------
                               sysuse auto                 0                 74
                           drop if foreign                74                 52
-------------------------------------------------------------------------------

. trackobs clear

Best
Daniel

[daniel klein]

Anyone interested in the concept discussed here can download a modified version of the program in #3 form the SSC. Thanks to Kit Baum for the quick upload.

Best
Daniel

[jhonny-econ]

Daniel - thank you for your work and your template. Eventually, I rewrote it and added functionality to it to fit my use case.