program already defined

[paulvonhippel]

When I write:

program define foo
 set obs 100
end

it runs the first time, but the second time it returns an error "program already defined."

So instead I write:

program drop foo
program define foo
 set obs 100
end

but the first time I run that it returns an error "program foo not defined yet."

What can I do to get out of this? Many thanks.

[Clyde Schechter]

captureprogram drop foo
program define foo
    set obs 100
end

will work both the first time and subsequently.

If you are not familiar with the -capture- command in other contexts, do read about it now in the PDF documentation that comes with your Stata installation. It is very useful in situations like this where a certain type of error is expected to occur some, but not all of the time, and should be allowed to occur without interrupting execution.

[paulvonhippel]

Thanks! This works but seems a bit clumsy. I've never seen another programming language that doesn't allow you to define a program without first dropping it.

Is there another way -- maybe an option to program define that allows you to define a program even if it's already defined?

[Leonardo Guizzetti]

Thanks! This works but seems a bit clumsy. I've never seen another programming language that doesn't allow you to define a program without first dropping it.

Is there another way -- maybe an option to program define that allows you to define a program even if it's already defined?

Not unless you want to -clear all-.

[Clyde Schechter]

There is, in my view, a very good reason for Stata's behavior with regard to defining programs. When you are running your code in "production mode" it is likely that you will be calling the program more than once, perhaps even in a loop with many iterations. After all, if you were just going to run the program code once, why bother wrapping it in a program--you would just put it in line. Well, you don't want Stata to take the time to redefine the program each time you call it, do you? So Stata takes the position that if the program already exists, we use it the way we already have it--we don't redefine it at each call.

You pay the price for this while you are developing your code. In this situation you are probably only calling the code once or a few times for each run of the entire do-file, for testing purposes. And you are likely to be changing the program code each time you do that, to fix bugs or tweak the appearance of the output, or whatever. So you have to signal that fact to Stata so that it will redefine the program. That's what -capture program drop foo- does. It's a very small price to pay for the improved run-time efficiency that you get in "production" runs of the code.

By the way--very important: if you are writing your program in an ado-file to be saved for future use, once you have the program running the way you want it to, you should remove the -capture program drop foo- command. That's because whatever is in the ado file is taken to be part of the program code itself, and leaving -capture program drop...- in it would cause Stata to drop and redefine the program at each call, which is not what you want with a finished program in an ado file.

[daniel klein]

By the way--very important: if you are writing your program in an ado-file to be saved for future use, once you have the program running the way you want it to, you should remove the -capture program drop foo- command. That's because whatever is in the ado file is taken to be part of the program code itself, and leaving -capture program drop...- in it would cause Stata to drop and redefine the program at each call, which is not what you want with a finished program in an ado file.

I do not think so. Though I would recommend removing capture program drop from ado-files, the overhead is less than Clyde implies. I was thinking about writing the following up as a tip in the Stata Journal but decided that it was not relevant enough. Anyway, it might fit in here.

Write the following lines in a do-file and store it as foo.ado where Stata can find it:

capture noisily program drop foo // <- note the -noisily-

program foo
    
    version 17
    
    display as txt "foo"
    
end

Now, clear everything and call foo:

. clear all

. foo
program foo not found
foo

. foo
foo

Note that the repeated call to foo did not result in an error message. You might be tempted to conclude that this is because foo is now defined and, thus, program drop foo does not result in error. But this is not so. Let me demonstrate. I continue interactively where I left off above:

. program drop foo

. program foo
  1. version 17
  2. display "bar"
  3. end

. foo
bar

.

See how I drop foo, from memory to re-define foo. When I now call foo, it is the re-defined foo from memory that is executed. Stata never bothered running foo.ado, and the contained line capture noisily program drop foo. By the way, this works the same way for StataCorp. ado-files. Watch this:

. program regress
  1. version 17
  2. display "not the regress you are thinking of"
  3. end

. regress
not the regress you are thinking of

In general, whenever you call a command, Stata will first check whether the command is built-in. If it is, it is executed. This means, we cannot do the above with, say, summarize, because summarize is not implemented as an ado-file:

. program summarize
  1. version 17
  2. display "you will never see this line"
  3. end

. summarize

.

If the called command is not built-in, Stata will then check whether the command is defined in memory. If it is, it is executed. This is what happened with the re-defined foo and also with the phony regress above.

Only when the called command is not built-in and not defined in memory will Stata look for the respective ado-file and execute it. Thus, by the time the program drop statement in an ado-file is executed, it will always result in error. To reiterate: if the program you are trying to drop were in memory, Stata would not even execute the line capture program drop. So that is the only overhead. You do not need to drop the command from memory because you already know that the command is not defined yet.

I could go on and write about sub-programs in ado-files, how they are private, and how capture program drop subcommand would never drop the subcommand defined in the ado-file. Instead, capture program drop subcommand might drop subcommand, which is completely unrelated to the ado-file, from memory. That sounds more problematic than it really is. Next time you call subcommand, Stata simply redefines it from its own ado-file.

[paulvonhippel]

It takes very little time to redefine a program, right? After all, you're not loading or processing data, or calculating anything. If there's concern about inadvertently redefining a program (and I'm not sure why there would be), it seems that a replace option, something like this --

program define foo, replace

-- would be more elegant than what's currently required:

capture program drop foo
program define foo

I note that the replace option is used in other Stata commands, such as use.
Should I put this on the wishlist?

[daniel klein]

You typically write that line once at the beginning of development. Once the program is wrapped into an ado-file, the option would be useless anyway. I am not sure whether implementing (and certifying) a replace option is worth the effort. But you can naturally wish for whatever you want.

[Nick Cox]

You can ask for that, but I wouldn't support any such change myself, not that StataCorp is going to hold a vote.

I think it would be too tempting for some and could lead to more trouble than it solves. I think some user-programmers would use the option always. and that might be confusing not just to them, but to their users.

Making it a little harder to change important things is a key part of the Stata philosophy, as shown by the way that changed datasets have to be saved and by the need to spell out destructive commands or the need to use replace to change whatever was created by generate.

I don't think anyone mentioned discard. I use it a lot when I am developing a program and there is a multitude of minute revisions and corrections of silly errors. It makes Stata do more work and Stata has to load various programs repeatedly, but Stata doesn't complain and I don't think fast enough to notice the extra time.

[paulvonhippel]

My main objection to capture program drop is that it's not obvious and it took a few rounds of Statalist queries for me to find out that that's what I needed to do.

discard appeals more to me. Do you just put it once at the top of your do file?

[daniel klein]

My main objection to capture program drop is that it's not obvious and it took a few rounds of Statalist queries for me to find out that that's what I needed to do.

I won't argue with your experience. But I will point out that the help file for program lists drop as a subcommand and explains what it does. It also points to [U] 18 Programming Stata (pp. 4--5), where the problem and solution are discussed in detail.


Note that discard will not help with your (implied) workflow of developing the program in a do-file. discard only drops programs loaded from ado-files. Watch:

. sysuse auto
(1978 automobile data)

. regress mpg weight

(output omitted)

. program define regress
program regress already defined
r(110);

. program define foo
  1. display "foo"
  2. end

. discard

. program define foo
program foo already defined
r(110);

. program define regress
  1. display "not the regress you are thinking about"
  2. end

. regress
not the regress you are thinking about

So discard is useful if you already have wrapped your program(s) in ado-files, make changes therein, save the changes, and then run a test script from within a do-file (or interactively, in which case you will have to type discard repeatedly).

[Nick Cox]

discard is indeed what I use in developing and debugging an individual program and interactively and when I am doing that I make lots of small mistakes and think up small additions to the code, or indeed small subtractions.

It's a candidate for a test script that runs through various applications of a command, say foobar_test.do. I wouldn't use in a routine do-file at all. Alternatively what I sometimes do is put it in a tiny command that calls up my favourite text editor.

[paulvonhippel]

daniel klein is right that program drop is documented. What wasn't so obvious, to me, was the need to prefix program drop with capture, so it wouldn't return an error the first time.

I should add that I used program to modularize code, and not just to write ado files. This may not be common. If it's not, that might explain why the need for capture program drop bothers me more than others.

[daniel klein]

I should add that I used program to modularize code, and not just to write ado files. This may not be common. If it's not, that might explain why the need for capture program drop bothers me more than others.

You mean you define programs within one (longer) do-file? I do that, too, sometimes. I see how a replace option or something similar would be a little more convenient in this scenario. However, note that you could simply add

capture program drop _all

near the top of your do-file. It won't do any harm* and lets you define all the programs you want in the remainder of the do-file.

If you rely on programs across do-files, then you obviously need something else. An ado-file comes to mind. Or, if for whatever reason you do not want that, simply modularize your code in do-files instead of programs. Remember that you can pass arguments to do-files, too ([U] 16.4 Programming with do-files).


* Edit: except the minor delays due (re-)loading programs from ado-files that we have discussed earlier.