for var out-of-date command

[Christophe Misner]

Hello everyone,

I have a question regarding the "out-of-date" command "for var...".
I've been using it recently and I'm wondering if there's a risk of getting wrong results if I use it instead of the new version of the command.
It seems much more convenient than the "foreach var" command.

Example:
--------------------------------------------------------------------------------------------------------------------
"out-of-date"
for var sexe est_nais: bysort code_enfant: egen tp_X=nvals(X)

"new version"
foreach var of varlist sexe est_nais {
bysort code_enfant: egen tp_`var'=nvals(`var')
}
--------------------------------------------------------------------------------------------------------------------
I have the same result here. But can it in some cases distort my results?

Thanks,
Christophe

[Leonardo Guizzetti]

As the manual explains

for is an out-of-date command as of Stata 8.

Unless you really need to write code compatible with a version from before 2003 (~20 years ago!), you should use the new syntax. It is literally a difference of a few extra characters. -foreach- is more flexible but also doesn't limit you to repetitive one-liners. There is the benefit that others will be able to easily read your code, too.

[Nick Cox]

for was abandoned in Stata 7 for very good reasons. Some considerable time I will never get back was spent before that trying to explain what users couldn't follow at all (even when there was visible documentation) -- and indeed several of us got burned before we appreciated the pitfalls ourselves. There is a trace of this in historic FAQs.

So far as I can recall the code you cite is exactly equivalent, although (FAQ Advice #12) you should be explaining that nvals() is from the egenmore package on SSC.

In any case, sorry, but explaining precisely when for will give the wrong answer is too open a question, as there are so many ways to get wrong answers.

Stata has been written since the beginning by very smart people, but even then sometimes good ideas turned out to be not so good, really, and then the company came up with better ones. This is a case in point.

In this case, you are applying the same code to just two variables, so a competitive solution is just two separate commands, one being

bysort code_enfant: egen tp_sexe = nvals(sexe)

and the other being similar.

Please don't waste your time trying to understand what for can and cannot do. Otherwise, you're on your own! https://www.youtube.com/watch?v=__n5Bgxx-68

[Joro Kolev]

I still use the old -for- all the time from command line. The new construct with the opening and closing curly braces and the new lines is a pain if you are doing stuff from command line, like data exploratory analysis.

So my view is that Stata Corp did not think this through very well when they left the old -for- behind.

From a do file you should use the new looping constructs. They are faster, and they do not have the limitations the old -for- has -- the old -for- has some limitations as to how many elements you can have in the list and is slow as a turtle.

In short the old -for- will not give you wrong results if you know how to use it, but there is really no reason to use it from do files, and there are some good reasons not to use it, e.g., that it is slow.

[Jared Greathouse]

For went out of date in 2003 when I was like 6 years old. The only 'for' I use in my do files are when I'm writing loops for Python, and that's a separate animal.

I guess my real question for you, is why would you want to use for? If it helps you, okay cool, it isn't my do-file, not my circus, not my monkeys, but still, why would you want to use it, and what was the thought process behind deciding that you wanted to use it? I'm genuinely curious.

[Nick Cox]

I misspoke. for was abandoned as a documented command in Stata 8. The code is still bundled with Stata -- so as not to break ancient code -- but for a help file you would have to find one from an old copy of Stata.

For anything distributed publicly, the onus is on user-programmers who like and use the old for

* to be willing to explain to others what may seem cryptic code to anyone who does not remember the old for

* to support each other.

We can all savour one-liners -- when they are readable. If you grasp that X in #1 is behaving rather like a local macro, then you're ahead. If you know about local macros and start trying to use them with the old for, that usually won't work.

[Joro Kolev]

I just explained one application in the post above: the old -for- has simple syntax and is infinitely more convenient from command line, because you can do much stuff on one line.

I do not see anything wrong in using it in do files if you are doing something simple, like

. sysuse auto, clear
(1978 Automobile Data)

. for var headroom trunk weight: egen meanX = mean(X), by(rep) \ gen demeanX = X - meanX

->  egen meanheadroom = mean(headroom), by(rep)

->  gen demeanheadroom = headroom - meanheadroom

->  egen meantrunk = mean(trunk), by(rep)

->  gen demeantrunk = trunk - meantrunk

->  egen meanweight = mean(weight), by(rep)

->  gen demeanweight = weight - meanweight

I applied the fixed effect transformation on one line. If you want show me how many lines it is going to take you in any way how you know how to do it -- in Python, in Mata, or in standard Stata loop...

Another application of the old -for- which I had is that when I taught Stata to students, I could do a full blown course on Stata without ever speaking about macros (local and global). Some people might say that I am butchering Stata this way, but my views is that one can write American Economic Review papers without ever having to understand the aspects of Stata language -- just using Stata as regression software.

For went out of date in 2003 when I was like 6 years old. The only 'for' I use in my do files are when I'm writing loops for Python, and that's a separate animal.

I guess my real question for you, is why would you want to use for? If it helps you, okay cool, it isn't my do-file, not my circus, not my monkeys, but still, why would you want to use it, and what was the thought process behind deciding that you wanted to use it? I'm genuinely curious.

[Nick Cox]

Joro Kolev The issues are not at all that you like for and find it handy and know how to use it. The issues start with anyone else who asks "I like the look of that, but what are the rules so that I can use it for myself (and avoid whatever pitfalls might arise if I misunderstand it)?". Or with anyone who finds mentions of for in old code and struggles to know what is going on.

Local macros are a red herring here, as explaining for carries an obligation to explain what X (or whatever other character you use) is doing in the code. So, not having to explain something carries a burden of having to explain something else (idiosyncratic to one command).

[Joro Kolev]

Nick, you have the concept of a "place holder" in your -egenmore- commands. There are a few -egenmore- commands written by you, where you use the symbol @ to denote a place holder. I have used those -egenmore- commands by you, I have taught them to my students, I have taught my students the old -for- the same way, telling my students that X is a "place holder".

Many of my students outgrew me to publish in better journals than me, by the way, just doing stuff with place holders and not really understanding Stata as well as you do. Not ever understanding Stata as well as I do. There have never been a problem with "place holders" on my side.

Now back to the old -for-: the only thing I do not like about the old for, and I otherwise love it, is that it uses as a "place holder" the letter X. I wish it used as place holders special symbols such as @, #, % etc., as you do in your -egenmore- commands.

You seem to think that you are asking a hard question about what are the rules...

I taught it very simple:

You have the -for- command, which is followed by a declaration of the list type that will follow, e.g., - for var- or -for num-. Then after the colon ( you just use X as a place holder to refer to the elements of this list, which Stata substitutes sequentially.

And I am done. No macros (no local no global), and my students and myself know how to loop in Stata through the old -for-.

Joro Kolev The issues are not at all that you like for and find it handy and know how to use it. The issues start with anyone else who asks "I like the look of that, but what are the rules so that I can use it for myself (and avoid whatever pitfalls might arise if I misunderstand it)?". Or with anyone who finds mentions of for in old code and struggles to know what is going on.

Local macros are a red herring here, as explaining for carries an obligation to explain what X (or whatever other character you use) is doing in the code. So, not having to explain something carries a burden of having to explain something else (idiosyncratic to one command).

[Nick Cox]

Sure, I use placeholder ideas whenever local macros cannot be used, and I certainly didn’t invent them. The defence is that there’s always an accessible help file for my code that explains. Also, in the help file for egenmore there is explicit advice that there are now (since 2003) better ways to do it.

Bottom line is that you’re recommending code practices that are not documented, given the lack of a help file. My main point is directed to others so that they register this.

[Christophe Misner]

Thank you for your answers. I like to know what I'm doing to avoid unpleasant surprises and not having to double-check the commands I'm using.

So for code sharing and for large loops, the new form is certainly preferable. The question of the availability of the help file can also be crucial indeed, it's a pity that the help file has not been kept for the old "for". Otherwise, for personal code, it is clear that the old concise form for small loops is very convenient.

[Nick Cox]

I don't speak for StataCorp but it is all too easy to guess that StataCorp don't want to break existing code but also have no wish to encourage the use of for at all. It's clear enough that even two loop commands is confusing to some users, and -- for the sake of Tech Support alone -- almost nobody wants three, so far as I can tell.