Delaying the execution of a command till the end of do-file

[Cyrus Levy]

Is there a way to delay the complete execution of a command till the end of do-file? So wherever this command is typed in the do-file -even at the top of the do-file- it would wait till the end of do-file, and provide output just before the do-file ends. Something like "collect information and run this command now, but show the output only when the do-file ends." Note that this is a single line of command that would not require writing additional lines of code at the bottom of the do-file.

A simple example is below (but what I actually want to do is more complicated than this example).

use https://www.stata-press.com/data/r16/auto, clear

replace price=1000 if mpg>30
cap summ price

// Below is the -doend- command that would collect information at this
// position in the do-file but delay its execution till the end of do-file

doend: display r(mean)

drop if mpg>20

// So without any additional lines of command, when the do-file ends,
// it provides the output of "display r(mean)" but the output would reflect
// the status of data before "drop if mpg>20" command line.

[daniel klein]

I suspect this might turn into a XY problem. Better tell us about the more complicated situation and why you think you need this.

Best
Daniel

[Nick Cox]

The general mechanism here that you are seeking is just that of a program, whereby you can control when or even if results are displayed.
Incidentally,

cap summ price 

is not the best approach for what you want.

summarize price, meanonly

calculates the mean but does not display it.

In this and many other cases, where r- or e-class results are fragile, you can store results in scalars and then show the results when you want.

[Cyrus Levy]

Perhaps I could not explain it well in my original post. I want to program a generalized solution to what I outlined above in the simple example. The -doend- command collects information at the point where it is in the do file, but provides the output at the end of the do file. It always shows the output, but it delays showing it till the end of do file.

[Nick Cox]

Sorry, but I still don't have a clear idea of what you want. As said, you can always control when data and/or results are displayed: within the framework of a program. I would also recommend that you look at run as well as do.

[daniel klein]

Advice in #2 stands.

Either way, we would need more details. Where would such a command store the information? In a global? A file? Dataset characteristics? What kind of information is to be stored? One scalar? A matrix? Something else? What would the command do when the do-file does not conclude but exits with error? I could go on ...

Also, in general, what you ask for is not possible, at least not in the way you want it. There is no way for a command to know when the do-file in which it is called ends. An approach along the lines

doend do : mydofile.do

combined with commands like the one you suggest out in #1 inside mydofile.do might be possible. Personally, I would not be willing to think about the details of such an approach if I would not be sure that it was indeed the best (or at least a reasonable) way of getting what you ultimately want to do. Others might feel different about it.

Best
Daniel

[Cyrus Levy]

Sure, it is possible to control when data and/or results are analyzed and displayed within the framework of a program. So you can store information somewhere in the middle of a do-file and with additional lines of code at the bottom of do-file, you can analyze or display them at the end of the do-file. But the command that I want to program eliminates the need to type additional lines of code at the bottom of do file. So it is like timing what will happen at the end of the do file and simplifying the coding so that you wouldn't need to write additional lines of code at the end of your do file to display the information that has been collected at different locations inside the do file. My question is if something like this is already available, or it is even possible with Stata?

So if you look at the example again, there is not any code after the "drop if mpg>20" line. When this example is run in Stata, if the -doend- command is functional, the results window in Stata would show the output of "display r(mean)" right before "end of do-file"

[Cyrus Levy]

Also, in general, what you ask for is not possible, at least not in the way you want it. There is no way for a command to know when the do-file in which it is called ends.

Daniel, I think above is the ultimate question that I am asking, for a generalized solution.

So say you have a do-file that runs regressions, summary tables, tests etc at different points in the do file. When you are programming your program, you know you want to display the outputs of some of them not when they run in the order of the do-file BUT at the very end of your do-file like a report when it completes running. Now you can store all that in scalars, matrices, etc and every time you store something, you would need to go to the bottom of the do-file to enter additional lines of code so that they would get displayed eventually. But the command that I want to program eliminates that second step (additional lines of coding at the bottom of the do-file to display the output/report). So doend: tells the following command to run and store results now but display the output once the whole do-file ends running. It knows when the do-file completes running and then displays the output/report. So for a command to know when the do-file from which it is called ends is not possible? If so, what I want to do is probably not possible.

[daniel klein]

So for a command to know when the do-file from which it is called ends is not possible?

It might be possible but I am not aware of a straightforward solution. Why not just group the estimation commands at the end of the do-file?

Best
Daniel

[Cyrus Levy]

Why not just group the estimation commands at the end of the do-file?

^ Because data changes along the way in the do-file. Each doend: has to use the data at wherever they are coded in the do file, and these doend lines are at different locations within the do file.

Thinking about what you suggested, nesting do files within each other may make a general solution possible actually! Whenever the doend command is encountered by Stata, it would copy all the remaining part of the do file in a new temporary do file, and run that instead, so when it ends, the wrapping temporary do file would know when the do file ends, and then do whatever is asked to be displayed as a report. What do you think?

[daniel klein]

Because data changes along the way in the do-file.
[...]
Whenever the doend command is encountered by Stata, it would copy all the remaining part of the do file in a new temporary do file, and run that instead, [...]
What do you think?

To be honest this approach does not sound like a well-organized workflow and it is likely to get you in trouble. If you make so many radical changes to the dataset, you might be better off saving separate versions of the dataset to disk and having separate do-files for each of the datasets that you would call from a master do-file. I have not used it myself, but you might be interested in project (SSC). I do use so-called master do-files that call other do-files a lot, even for small projects. I also usually save multiple versions of the dataset to disk.

Also, your approach sounds like re-inventing wheels to me. Assuming that your ultimate goal is a (permanent) report outside of Stata, not just in the results window, have a look at Stata's tools for dynamic documents. There is also community-contributed software in direction of markdown. For something more basic, see

help log

in case you are not familiar with it.

Having said that, I believe that, technically, you will have a hard time when your doend command is called in do-files that call other do-files which, in turn, call doend themselves. You have stated that

[...] the command that I want to program eliminates that second step (additional lines of coding at the bottom of the do-file to display the output/report)

My guess is that you would have to write a lot of additional lines at the bottom of that do-file to justify the efforts that are probably needed to implement such a command.

Best
Daniel

[daniel klein]

Here is an example of how you could use a log file to re-display relevant output at the end of the do-file. Yes, you need one additional line of code. Also, the output is not suppressed when it is logged; that is just not possible. However, given that you will never need more than one line of additional code at the end of the do-file and given that you only want to look at the results in the very end, I would argue that both points are largely irrelevant to your ultimate goals.

program cleanlog
    version 11.2
    args logfile void
    if (`"`void'"' != "") error 198
    tempname fi fo
    tempfile tmp
    file open `fi' using `"`logfile'"' , r
    file open `fo' using "`tmp'" , w
    file read `fi' line
    if (`"`line'"' != "{smcl}") {
        file close `fi'
        file close `fo'
        display as err `"invalid log file `0'"'
        exit 698
    }
    file write `fo' `"`line'"' _newline
    while (!r(eof)) {
        file read  `fi' line
        if (  (`"`line'"' == "{com}{sf}{ul off}{txt}{.-}")        ///
            | (substr(`"`line'"', 1, 16) == "{com}. log close") ) ///
        {
            forvalues i = 1/5 {
                file read `fi' discard
            }
        }
        else file write `fo' `"`line'"' _newline
    }
    file close `fi'
    file close `fo'
    display
    type "`tmp'" , smcl
end

Your example (slight changes)

tempfile mylog // <- I use a temporary file

sysuse auto, clear // <- I use the shipped version

replace price=1000 if mpg>30
summ price , meanonly // <- I use Nick's code

local mean = r(mean) // <- the -log- command will wipe r()

log using "`mylog'" // <- start logging output
display `mean'
log close // <- stop logging

drop if mpg>20 // <- by the way, -drop- will also wipe r()

log using "`mylog'" , append // <- continue logging output
regress price weight mpg
log close // <- stop logging

cleanlog "`mylog'" // <- look at logged results

and the relevant output

. cleanlog "`mylog'" // <- look at logged results

. display
5936.1081

. regress price weight mpg

      Source |       SS       df       MS              Number of obs =      38
-------------+------------------------------           F(  2,    35) =   26.54
       Model |   237304791     2   118652395           Prob > F      =  0.0000
    Residual |   156493713    35  4471248.95           R-squared     =  0.6026
-------------+------------------------------           Adj R-squared =  0.5799
       Total |   393798504    37  10643202.8           Root MSE      =  2114.5

------------------------------------------------------------------------------
       price |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
      weight |  -1.403993   1.069681    -1.31   0.198    -3.575562    .7675752
         mpg |  -1355.763   239.3713    -5.66   0.000    -1841.713   -869.8137
       _cons |   34910.31   7382.303     4.73   0.000     19923.43    49897.18
------------------------------------------------------------------------------

Best
Daniel

[daniel klein]

I have reinvented the wheel in #12. There is already a cleanlog command on SSC that does essentially what I had in mind (although it does not seem to remove the headlines and log close statements). I am sorry for using that name; it was an honest mistake.

Best
Daniel

[Cyrus Levy]

Daniel, thank you so much for helping me with this. I haven't tried using it yet but I'll try it over the weekend and perhaps change a few details.