Send eveything to the log file even the parts in quietly

[Helder Costa]

Hello!

I´m writing a program that will be run by several other users. The majority of the code is encapsulated within `quietly {}' chunks, with only informative messages regarding the ongoing processes and progress displayed in the results window. However, if an unforeseen error occurs, for debugging purposes, it would be great if the user is able to send the log back, and in that case, it would be useful that everything was being logged. Is this possible?

Best,
Hélder

[Benno Schoenberger]

Hi Hélder,

you could define an additional option (e.g. verbose) in your program that then controls whether an output should be visible or not. The following mini-program shows an example of how you could do this...

program define logtest
    syntax, [verbose]
    
    if (`"`verbose'"'=="verbose") local q            // if option verbose is undefined, q is empty
    else local q quietly                            // contains "quietly" otherwise
    
    `q' {                
        display "{text}{bf:Output now depends on program option [verbose]}"
    }
    display "{text}{bf:Output you want to have irrespective of option [verbose] in any run}"
    exit 0
end

// whithout option
logtest

// with option included
logtest, verbose

Kind regards,
Benno

[Helder Costa]

Hi Benno,

Thanks a lot for your comment.

Thus, your suggestion is that there is an option that switches how much goes into the results window and logfile. This could be activated only if an error occurs and logfiles need to be shared. Am I interpreting this correctly?

Best,
Hélder

[Benno Schoenberger]

Hi Hélder,

you could just add the option "verbose" in case you want to have more output. I'm not sure what exactly you mean with "error" in your szenario, but if you refer to Stata terminating the execution of a program, then I think tracing the program would be a better choice for debugging. Maybe you want to have a look at https://www.stata.com/manuals13/ptrace.pdf

Kind regards,
Benno

[Helder Costa]

Hi Benno,

The problem is not turning on and off quietly nor it is to debug the code, when I´m the one running it.

As the program will be distributed by several users to be run on their servers, if an error/situation occurs that I do not detect while writing and testing, having the log file with the normal (by normal, I mean without quietly) Stata output would be helpful. However, under normal circumstances, I prefer that the only output the user sees are some messages that informs about which step is being run at the moment. The program has several thousands of lines, heavily looped. If I don't use quietly combined with noi display for occasional informative messages, the output that comes out will be almost unreadable to the user. If I do use quietly combined with noi display, then it is the logfile that non informative in the event that an error occurs. This is why I want to have separate things going to the results window and to the logfile.

As so far no one was able to offer a solution, what I thought you were suggesting was that the code would be run with quietly turned on. Then, if an error occurs, the user would be instructed to turn the "verbose" option off, and run the code again.

I hope this was a bit clearer than my initial post.

Thank you for your comments!

Best,
Hélder

[Benno Schoenberger]

Hi Hélder,
if I now got you right it would be nice for you to have the whole Stata output in a logfile that is generated in the background that would help you to adapt the program in case of unforeseen errors. I do not now whether such an option exists in Stata but would also be very interested in it.
My approach to your problem is exactly what you wrote at the end. In the normal scenario a user runs your program with the verbose option switched off and only turns it on in case of an error and reruns the program in order to provide the full output to you.

Best,
Benno

[daniel klein]

I still do not understand the problem. Here is what I consider the typical scenario:

1. You write your code quietly.
2. You distribute your code.
3. User runs the code; an error occurs.
4. User contacts you asking for help.
5. You tell User to (a) start a log file, (b) set trace on, and (c) send you the (lengthy) log-file.

You can then investigate, line by line, what went wrong.

What's the problem?

[Helder Costa]

Hi Benno,

Yes, that is exactly right. Thanks again for your comments on this, it was very helpful! I think I´ll implement something inspired by your verbose program.

Hi Daniel,

I think there are two problems. First, it requires the code to be run twice. The type of program we are discussion takes days or weeks to run. If the solution I´m looking for is possible, it would run once, and the log file would have immediately the required information. Second, setting trace on does nothing if quietly is on, at least from the a quick experiment I did (code bellow). Therefore, a solution similar to what Benno suggested would still be required. Furthermore, a solution such as this could be the perfect solution, as due to confidentiality reasons, that are steps (such as tabulations) that need to always be omitted, but this is different problem.

cd "folder_name"

capture log close
log using example_log.txt, text replace

set trace on

quietly {

clear
set obs 10

gen var1 = _n

sum
}

log close

[daniel klein]

I think there are two problems. First, it requires the code to be run twice. The type of program we are discussion takes days or weeks to run. If the solution I´m looking for is possible, it would run once, and the log file would have immediately the required information.

I think this approach is fundamentally flawed. You propose writing a (complete) log file every time the program is run, but only using that log file when an error occurs. Unless you expect frequent errors that occur very late during the program's running time (in which case you should really improve your program), writing a file to disk in the background is a complete waste of time. The time of running your program twice occasionally should pale in significance compared to the time wasted every time anyone runs your program.

Second, setting trace on does nothing if quietly is on, at least from the a quick experiment I did (code bellow).

That is because you have picked a built-in command. summarize is written in C; you cannot peak into what is doing. Try describe or another ado-based command.


Edit:

On the first point: A verbose option, which by default is set off, also implies running the code twice in case of errors.

On your example, set and generate are also built-in. Your log file shows the trace for clear, though.