Stata 14 Buggy??

[MB Ross]

So I have Stata 14 on two comps and everything is updated. I keep running into what appears to be a bug and I'm wondering if anyone else is getting it.

I'll run a section of code and I'll look back to find that Stata has tripped on what appears to be a typo. It'll say something like:

reg cne* yearsed tage sqtage
variable cne* not found
r(111);

where cne* is NOT one of my variables and appears to be a typo. I go back to the section of the .do file I've run and check it... no typo. I rerun the code without changing anything and it works fine.

I've had this happen like three times with different sets of code. I'm not a new Stata user so I dont think it is anything I am doing.

Anyone else having these weird issues?

[Richard Williams]

I'll have weird things like Stata will tell me -clear all- is not a legitimate command. Huh? Sometimes it will work when I rerun, sometimes I have to restart Stata. I can't consistently reproduce the error so I figure there is no point in reporting it. Maybe there is a problem with my machine, my installation of Stata, or some error on my part that I don't even realize, I have this suspicion that if I have been giving Stata a lot of heavy duty tasks it is a good idea to restart it every now and then. Commands I have copied and pasted sometimes give me grief -- maybe there are invisible characters causing problems. So I have a hunch there are some wildly esoteric bugs but I don't even know how I would report them.

[Sergiy Radyakin]

1. Insert describe cne* before running the regress.
2. It is not immediately obvious, but cne* is different from сne* ! So make sure cne* is spelled in the same language as intended.
3. Quote: "I go back to the section of the .do file I've run and check it... no typo." Is there any chance we can also peek into your .do file?

Best, Sergiy

PS: every program is buggy. some bugs are never found. to report a bug, provide a minimal illustrative example where the bug appears and explain why is this a bug.

[Joseph Coveney]

For do-files (and for ado-files, too, lately), I always put a star-bang (*!) comment as the first line. A couple three times, now, Stata 14 balked when I try to run a do-file with such a first-line comment, saying something like (paraphrasing) ! is not a valid command.

I look at what's there, thinking perhaps I might have accidentally put some kind of Unicode character in for the asterisk or exclamation point or something, but I always use the standard ASCII character set when I compose code, and can never see anything wrong in the first line.

In the few times that it's happened, to "correct" the problem, I've then re-typed the star-bang characters again separately in notepad, and then copied and pasted the new characters from there over the two offending characters in the do-file—to date, this always seems to have assuaged Stata 14.

[Richard Williams]

Here is an example. I copied and pasted the command from a help file. After I got the error, I tried to execute again and got the same error. Then, I tried the command, deleted the opening u and retyrped it, and it worked.

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear
file http://statisticalhorizons.com/wp-content/uploads/wages.dta not found
server says file temporarily redirected to /wp-content/uploads/wages.dta
r(601);

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear
file http://statisticalhorizons.com/wp-content/uploads/wages.dta not found
server says file temporarily redirected to /wp-content/uploads/wages.dta
r(601);

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear

.

I haven't had problems like this in earlier versions of Stata, as far as I can remember. I did switch from 13/SE to 14/MP2, in case that might matter. Platform is windows 7.

Anyway, something seems a ittle weird, although it is mostly at the minor nuisance level.

[Sergiy Radyakin]

Interestingly, after 2 fails it works flawlessly (Stata 13.0/Windows):

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear
file http://statisticalhorizons.com/wp-content/uploads/wages.dta not found
server says file temporarily redirected to /wp-content/uploads/wages.dta
r(601);

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear
file http://statisticalhorizons.com/wp-content/uploads/wages.dta not found
server says file temporarily redirected to /wp-content/uploads/wages.dta
r(601);

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear

. use http://statisticalhorizons.com/wp-content/uploads/wages.dta, clear

Opening another Stata session retains the "success" state. Looking into the server logs would be most informative.

Perhaps the site is doing something "smart" about the client or the requested resource, for example, redirecting to the same resource could be one of the strategies to gain time for processing a request (instead of risking a time out, we tell the client to repeat the request and provide the same request URI, while the client processes the server redirection response, the server extracts the data from some slow storage, so by the time the request comes again the data is ready to be consumed (and then it remains in a cache for a while).

Best, Sergiy

[Hua Peng (StataCorp)]

MB, I agree with Sergiy's advices, especially the third one, "Is there any chance we can also peek into your .do file?". On top of that, it will be helpful if you tell us the output of the following commands

about
query compilenumber

A screen shot of your desktop including Stata command and result windows when you encounter the issue will also be very helpful.

Joseph, if you run into the "*!" issue again, please send a copy of the do-file to Stata tech support at [email protected]. We would like to to track down the cause of the issue and your help is greatly appreciated.

[Joseph Coveney]

Huang, I think that I've figured it out.

When I begin a project by exploring received datasets and begin the initial coding, I invariably have a cmdlog running to provide a starting point for developing a do-file for the actual production run. cmdlog defaults to a .txt file extension. When I'm ready to start developing code from the command log file, I'll double click on the command log file and re-save it as a do-file with the .do file extension. In Windows, double clicking a command log file with its default extension will open the file in Notepad.

Suspecting that the problem has to do with Unicode, I went through a multitude of recent command log files and paid attention to what encoding Notepad would save them in. For nearly all of them, Notepad would save them in the ANSI encoding. But for a couple, it signaled its intention to save them "Encoding: UTF-8". I looked more closely at these command log files, and sure enough, I had made comments to myself, one with an em dash and one with "vis-à-vis" in the comment. (Using the alt+0151 and alt+224 keystrokes.)

It is not a problem for Stata to run Unicode do-files. However, Notepad inserts a byte-order marker (EF BB BF) at the beginning of its UTF-8 files, and Stata (and apparently a lot of other software, as well) does not expect to see BOMs in UTF-8 files. So, the problem appears to be an example of an interaction involving my use of Notepad to open the command log files to save them with the .do extension, and Stata's not recognizing the BOM.

I've attached a test of this (a running log in SMCL, the initial command log file "test.txt" which is in UTF-8 and which Stata runs without complaining, and the same command log file saved directly from Notepad as "test.do"). hexdump shows what's going on (see the running log).

[Sergiy Radyakin]

Joseph, you can manipulate the BOM with the tools from here:
http://www.radyakin.org/progs/bomtools/
This will be helpful for transferring both data and/or program files.

Best, Sergiy Radyakin

[Joseph Coveney]

First, Hua, please accept my apology for the errors in the post. In my haste, I failed to do a proper job of reviewing and editing it.

Sergiy, thanks for pointing out your BOM-removal software. Until this afternoon, I had never even heard of a BOM. I copied the hexadecimal codes from the hexdump output (the ones appearing in the output for "test.do" and not appearing in the output for "test.txt") off Stata's Results window and pasted them along with the word "hexadecimal" into Google, and found out what they were about. Now that I know—I think—what the problem is, I can open command log files directly in Stata's Do-file Editor instead of Notepad, but it's good to have such your BOM tools available whenever I (and others) need them.

[Hua Peng (StataCorp)]

Joseph and Sergiy, thanks for spending the time and track the issue down. Notepad automatically adds BOM when saving UTF-8 encoded file. And there does not seem to be a way to prevent Notepad from doing so. Opening the file directly in Stata's do-file editor or using a better editor than Notepad, for example, Notepad++, should avoid the problem. I will also look into add a tool in the do-file editor to remove the BOM.

[Sergiy Radyakin]

I think BOM is only relevant when the encoding is flexible (multiple encodings permitted). In the Stata documentation, there are repetitive assertions:

3. Representation of strings
Strings are encoded UTF-8 in Stata.

(http://www.stata.com/help.cgi?dta)

The other unicode encodings (Unicode Big Endian, etc) are never mentioned. So formally, Stata works correctly and the file should be saved as a utf-8 without a BOM to be read in Stata 14 correctly. The fact of life is though, that many users will use notepad.exe to make corrections to their do-files, and BOM will get saved there. Hence the bomtools.

Can anyone share the experience from other packages? How does R treat the BOM when present, e.g. during import of tab-delimited files?

Best regards, Sergiy

[Friedrich Huebler]

I've attached a test of this (a running log in SMCL, the initial command log file "test.txt" which is in UTF-8 and which Stata runs without complaining, and the same command log file saved directly from Notepad as "test.do"). hexdump shows what's going on (see the running log).

I experimented with your files.

Experiment 1: Execute test.do from the Stata Do-file Editor. Result: error message.

. *! (reserved for header comment)
is not a valid command name
r(199);

end of do-file

r(199);

.

Experiment 2: Open test.txt in Notepad++, save as test2.do (without changing the encoding) and execute test2.do from the Stata Do-file Editor. Result: no error message.

. *! (reserved for header comment)
. version 14.0

. * Try this here—em dash
. 
end of do-file

.

Experiment 3: Execute test.do from Notepad++ with rundolines, a program that sends commands from Notepad++ to Stata. Result: no error message.

. *! (reserved for header comment)
. version 14.0

. * Try this here—em dash
. 
. 
end of do-file

.

Experiment 4: Execute test.do from Notepad++ with rundo, a program related to rundolines that executes do-files opened in Notepad++ in Stata. Result: error message.

. *! (reserved for header comment)
is not a valid command name
r(199);

end of do-file

r(199);

.

The reason for the error in experiment 4 is that rundo treats do-files the same way as the Stata Do-file Editor. If a BOM is present there will be an error in Stata. rundolines doesn't have this problem because it sends the commands directly from Notepad++ to Stata, whereas rundo executes the saved do-file. If you are interested in using Notepad++ with Stata 14 and don't want to worry about file encoding I therefore recommend rundolines.

For a related discussion of Unicode and external text editors see http://www.statalist.org/forums/foru...t-for-stata-14.