Inexplicable 'Display not found' (error 111)

[Mark Franklin]

Inexplicable 'Display not found' (error 111). The error originally occurred when exiting an ado file that is the second in a hierarchy of three nested adofiles, but only after returning from the third ado file, thus: A.ado–>B.ado–>C.ado<>B.ado (where –> is a call and <> is a return). So the problem originally occurred at the end of B after returning from C. Curiously, when I comment-out all 'display' commands that will have been encountered in A, B or C prior to the error, the error then occurs at the end of C before returning to B. I have no variables or program named 'display'; the problem also occurs if I insert an 'exit' command (in which case it happens when that command is reached); also when I prefix the 'end' or 'exit' commands with 'capture'. Program flow is quite convoluted, with much use of preserve, restore, and tempfiles (I tried using frames but those were slower because of restrictions on the types of merge that frames permit). A less convoluted version worked fine but I have been trying to improve program efficiency (which has indeed improved by an order of magnitude). I already redesigned the program structure a couple of times so as to overcome earlier equally inexplicable problems. But I have not been able to find a way around this one, which happens after all the heavy lifting has been done. If I pause execution prior to the error and then BREAK back to Stata the results that have been saved by then are exactly what I want, so this last-minute glitch is very frustrating. Any suggestions gratefully received.
(I should have mentioned thatI am using Stata/MP 16.1 for Mac (Intel 64-bit) Revision 13 Jun 2023.)

[Nick Cox]

Repeats https://www.statalist.org/forums/for...ound-error-111

Whatever you do is wrong here as (1) a lengthy word description without code we can run is of limited use (2) we can't expect and wouldn't want you to post all of several linked ado files.

An ancient debugging technique is to sprinkle display [!} statements with flavour like

display "#1" 

display "#2"

within buggy code to isolate precisely where the error occurs. See also Joseph Coveney's replies to your earlier post.

[Daniel Feenberg]

log using foo,text replace
set trace on

but note that the error is genrally a score or more lines above the last statement shown in the trace and that Stata does not trace within a for loop. According to https://www.stata.com/manuals/perror.pdf#perror Stata thinks you are referring to a variable named "display", I am guessing that there is a missing newline or unintended continuation somewhere in the code. -trace- will help you find where.

[Mark Franklin]

Thank you for your replies. I have not been able to use trace and pause (my preferred debugging methods) to good effect because the bug only manifests itself when the program ends. Or did I miss something?. But your suggestions did lead me to try moving the `end' command progressively back towards the start of the first ado file (was wrong in thinking that 'exit' commands could substitute). But lobotomizing the program in this way produced errors of its own that had to be fixed before the program would run to end, so it took a while. Unfortunately I was not able to isolate small enough chunks of code in this way as to be able to identify the source of the problem. But I did isolate it to the first half of the A.ado program that I mentioned.
So now I propose to break that code down into smaller program chunks that can be invoked sequentially. That should get me closer to the problematic command line; but at the cost of a lot more time. If anyone can suggest a less labor-intensive method, I would be grateful to have that suggestion!

[daniel klein]

Nick has already pointed out that lengthy wordy descriptions aren't that useful. What is the error message, exactly? Is it really

Display not found
r(111);

with a capitalized Display and without additional description, like variable, label, subcommand, etc.?

[Nick Cox]

I should add that #2 was intended to be sympathetic too.

But the crunch remains that without seeing any code just about all we can do is make broad debugging suggestions. Also, although sometimes an experienced eye can just spot something wrong, being able to run the code is often essential.

We don't even have a sense of whether these programs are tens ... hundreds ... thousands of lines long.

[daniel klein]

To elaborate on #5, if the error message is exactly as shown, then Display might be the name of a matrix or the like. If so, then the program that created it should have used a temporary name instead; but that aside. I guess there is a clear, or clear mata, or matrix drop, or something along those lines either in one of the ado-files or in a community-contributed command that the ado-file calls. Those type of commands (except a simple clear when you preserve the data before). should generally not appear inside programs and/or ado-files, especially if they do not refer to temporary objects, because they might destroy objects the caller relies on, e.g., Display -- whatever it is.


Having said that, I cannot make sense off

But your suggestions did lead me to try moving the `end' command progressively back towards the start of the first ado file (was wrong in thinking that 'exit' commands could substitute).

Assuming no Mata (or Python or similar) code, there should be exactly as many end statements as programs in an ado-file. So what does moving the (single?) end statement mean here?

[Daniel Feenberg]

I don't understand what you mean by:

bug only manifests itself when the program ends

as the error 111 does by itself end execution, although as I said before, it doesn't always end immediately - there may be many lines of trace output after the error message appears in the trace. But the error message will (in my experience) appear just before trace prints the offending statement.

Daniel Klein has made a good point - the message would be "variable Display not found" (or similar) if Statacorp had written the message. This suggests the message was generated by user-written code, and cold be searched for in the .ado files. .

[daniel klein]

Daniel Klein has made a good point - the message would be "variable Display not found" (or similar) if Statacorp had written the message. This suggests the message was generated by user-written code, and cold be searched for in the .ado files. .

That was not exactly what I meant. StataCorp. would issue similar messages, e.g.,

. capture matrix drop Display

. matlist Display
Display not found
r(111);

I thought of community-contributed commands because StataCorp. would never destroy objects the user has created unless explicitly requested. Many community-contributed commands, especially but not exclusively those by inexperienced authors, would happily use permanent names for objects and destroy them, intentionally or by accident.

[Mark Franklin]

Some clarifications seem to be in order. I'm sorry my original post was so lacking in detail; I was trying to keep it short. I apologize for the fact that this post is at least as long, but this time I am trying for clarity rather than brevity.
1) I copied and pasted the error message exactly as it was displayed: capital D, no additional detail. I already mentioned that the first thing I did was to comment out all display commands in my own code. I should add that the word "display" (with or without capital initial letter) does not now appear anywhere in these ado files apart from the (now commented-out) display commands. And in none of these does the word start with an upper-case "D".
2) The program code that seems to contain the error is 860 lines long, but much of that is comments.
3) Only one `end' command can be executed in any program, but execution can be terminated early by moving that 'end' statement higher up the file. I hoped that by progressively positioning it earlier in the file I would eventually find the point before which the error did not occur.
4) It is probably clear from the above that the error message only appears when the program encounters an 'end' statement, as though it were the 'end' statement that was the offending one. That is what created the mystification that prompted me to post on statalist for the first time in twenty years of Stata programming. (Previously I have found what I needed by doing a Google search, which comes up with statalist responses as well as often useful items from elsewherel).
5) To the best of my knowledge I have not included any community-written commands in my code (other than the code itself, whose original version was posted on SSC in 2011 under the name '"ptvtools". That code did included other community-written code but one of my objectives in re-writing these commands is to replace such code with code of my own, even if more verbose (the package will probably also get a new name when I finally get done with it: perhaps "stackMe" or "multiLevelTools".

[daniel klein]

3) Only one `end' command can be executed in any program, but execution can be terminated early by moving that 'end' statement higher up the file.I hoped that by progressively positioning it earlier in the file I would eventually find the point before which the error did not occur.

A much better variation of this strategy was suggested in #2. Instead of gradually moving end repeatedly, add various

display as error "still here 1"
(code omitted)
display error "sill here 2"
(code omitted)
display error "still here 3"

all at once. Then execute the command once and look at the last "still here #". The problem occurs between that and "still here #-1". Having located the problem you can remove all "still here #" via search and replace using regular expressions.

4) It is probably clear from the above that the error message only appears when the program encounters an 'end' statement, as though it were the 'end' statement that was the offending one.

Sorry, but it's not at all clear to me. Why do you think the error has anything to do with an end statement? As I understand, you have not even tried to set trace on, another good suggestion, so how do you know where the error appears? You have not shown us what you have typed, exactly, yet. Anyway, unless your copy of Stata is corrupted, the error must be created by referring to an object named Display. By the way, make sure to run

update all

to make sure the problem is not with Stata. If there is no object named Display in your code and if your code does not refer to a hardwired named Display, it must come from somewhere else. A variable or value label name in the dataset (although that would probably result in a different error message), a scalar or matrix in memory, the contents of a global or local macro your code is referencing, or an entirely different command are likely candidates.

[Mark Franklin]

Trace has been on since long before the start of this conversation. Please tell me what benefit there would be from "still here" displays that would go beyond what I get from the 'pause' commands that also have been in place at the start of each logically distinct block of code (with 'pause on') since before the start of this conversation. There is NOTHING to see from either of these diagnostic tools until an 'end' command is reached, which is when I get the message that started this conversation. The real dinger is that there seems to be no way to bypass this message and the ensuing exit from Stata, which is REALLY frustrating since the program has by then correctly achieved everything it was supposed to achieve.
I tried 'update all'. The response was "Do nothing; all files are up to date."

[daniel klein]

Would you mind copying the last 25 or so lines of the trace that include the error message (with sufficient depth)? The trace typically does not include end statements. If yours does, there is something wrong. Without seeing any code, I am out of suggestions.

[Mark Franklin]

daniel klein asked "Would you mind copying the last 25 or so lines of the trace that include the error message (with sufficient depth)?"

Great idea. I hope it stimulates a helpful suggestion from someone! (I am really grateful for all the brainpower that has weighed in on this question).

Here you are . . .

25 LINES OF TRACE OUTPUT ENDING WITH FINAL STATA STATEMENT BEFORE ERROR EXIT (I added clarifying comments to this post) . . .

- local anything = "`prepipes'" // Initiates final phase of execution because there is no "||" introducing an additional user-supplied varlist
= local anything = ""
- local opts = ""
- }
- if "`anything'"=="" {
= if ""=="" {
- local more = 0 // local more is set =0 because there is no more user input (first line of trace output, above)
- }
- }
- while `more' {
= while 0 {
----------------------------------------------------------------------------------------------------
/* `while’ CODE (BY-PASSED DUE TO VALUE OF `more’) IS OMITTED */
----------------------------------------------------------------------------------------------------
}
- pause (9)
--------------------------------------------------------- begin pause ---
/* TRACING OF EXECUTION THROUGH PAUSE CODE IS OMITTED */
----------------------------------------------------------- end pause ---
- quietly use `origdta', clear // Tempfile containing the original data, as it was before a working subset was 'reshape'd
= quietly use /var/folders/0q/bssdjp717kx6bsyycqlyc26c0000gn/T//S_09136.000001, clear
- quietly merge 1:m SMunit using `temp', nogen // This is where reshaped working data is merged with un-reshaped original data
= quietly merge 1:m SMunit using /var/folders/0q/bssdjp717kx6bsyycqlyc26c0000gn/T//S_09136.000003, nogen
--------------------------------------------------------- begin merge ---
/* TRACING OF EXECUTION THROUGH MERGE CODE IS OMITTED */
----------------------------------------------------------- end merge ---
- capture erase `origdta' // Tempfile containing the original data, as it was before a working subset was 'reshape'd
= capture erase /var/folders/0q/bssdjp717kx6bsyycqlyc26c0000gn/T//S_09136.000001
- capture erase `temp'
= capture erase /var/folders/0q/bssdjp717kx6bsyycqlyc26c0000gn/T//S_09136.000003
- capture drop if stkmiss==. // This is where empty stacks (rows) of reshaped data are dropped from the dataset
- capture drop stkmiss // And the indicator variable flagging stacks that are all-missing is also dropped
display not found
r(111);
/* 'end' COMMAND (NOT DISPLAYED ON THE TRACE OUTPUT) IS NEXT */

It would be great if this trace output stimulated a brainwave in someone; but in any case you see here the basis for my original post. I am sorry I did not think to post it earlier. Thank you, Daniel, for the great suggestion.

[Mark Franklin]

How embarrassing! I just noticed that the error message "display not found" does NOT have an upper-case first character. On investigation I discover that, despite my memory of having pasted the message, I must actually have typed it into my apple notebook for ultimate transfer to statalist. The notebook automatically changes the first character of any sentence to upper case if the sentence is typed but not if it is pasted. Very sorry for misleading you. This error on my part does not affect the nature of the problem we address, though it might open the way to certain solutions that otherwise would have been ruled out.