"unexpected end of do-file" errors

[Scott Adams]

Hello,
I could not find an earlier post on Statalist regarding this, but please let me know if I overlooked it.
After upgrading to Stata 15, I am much more frequently getting the error, "unexpected end of do-file" and cannot figure out why. The code seems completely innocuous. What's more, it is difficult to reproduce.
Here is a snippet that stopped my do-file running overnight, a bit frustrating when a long file is interrupted. The code is not part of a loop, if, or any other structure that could be missing a "}". The error comes despite that fact that the next command is merely to store the estimates posted by margins (see second code set below). The do-file had been running for about 20 minutes before this error-- so there are several earlier estimations and some take a while to run, etc.

Error running entire do-file:

. est restore cms_hurd_ml
(results cms_hurd_ml are active now)

. margins, dydx(i.mhr) post expression( invlogit(predict(equation(first)))) vce(delta)

Average marginal effects                        Number of obs     =    324,020
Model VCE    : Robust

Expression   : invlogit(predict(equation(first)))
dy/dx w.r.t. : 1.mhr

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       1.mhr |  -.0124177   .0135112    -0.92   0.358    -.0388993    .0140639
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. unexpected end of do-file

end of do-file

No error, selecting snippet and running:

 do "C:\Users\VH8ADB~1\AppData\Local\Temp\26\STD1274_000000.tmp"

. est restore cms_hurd_ml
(results cms_hurd_ml are active now)

. margins, dydx(i.mhr) post expression( invlogit(predict(equation(first)))) vce(delta)

Average marginal effects                        Number of obs     =    324,020
Model VCE    : Robust

Expression   : invlogit(predict(equation(first)))
dy/dx w.r.t. : 1.mhr

------------------------------------------------------------------------------
             |            Delta-method
             |      dy/dx   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       1.mhr |  -.0124177   .0135112    -0.92   0.358    -.0388993    .0140639
------------------------------------------------------------------------------
Note: dy/dx for factor levels is the discrete change from the base level.

. est store cms_hurd_m1_ml

. 
. 
end of do-file

But, I want to emphasize, it is as far as I can tell NOT always related to -margins- or this series of steps-- it doesn't seem specific to -margins- or -est store-, or the particular estimation completed prior to the -margins- (in this case using -ml-). I can't rule any of these out as factors, but seeing the error over and over and looking for patterns I haven't noticed that. It happens with other commands. Moreover if I select and run the snippet, as shown above, it runs fine.
I thought for a while that it had to do with the CR and LF at the end of the lines, but these appear intact (when viewing in the editor). Sometimes I would add extra CR/LF and re-run and the do-file would work, but I think now this is a coincidence.

Mostly I am wondering if others are seeing this issue. As I cannot give exact conditions for generating the error, nor can I reproduce it reliably, it is difficult to debug. But I do see it with increased frequency recently, perhaps related to Stata 15.

That I see it somewhat "randomly" and usually when I am running a do-file that takes a relatively long time suggests to me perhaps it has something to do less with Stata execution, and more perhaps with the file system that is reading the do-file as execution proceeds...? Perhaps a temp file is disappearing over a long run? I am running Stata in a complex network environment, on a remote server, in a large bureaucracy so I have limited control over the implementation details, and all that is beyond my knowledge. But I could ask around/check stuff if I had specific ideas where to look.

Here is what the "About Stata" panel says:

Stata/MP 15.0 for Windows (64-bit x86-64)
Revision 20 Jul 2017
Available physical memory: 234785112 KB
Available Physical Memory: 187918540 KB
10-user 4-core Stata network perpetual license:...

Thanks for any clues,
Scott

[Jesse Wursten]

I think it's an issue of the temp file disappearing - I've had similar issues running local dofiles on a remote cluster where the network connection would occasionally disappear. Does it ever occur when you run entire (saved) dofiles?

One thing you could try is changing the temp folder, but it's a very crude solution (https://www.stata.com/support/faqs/d...ary-directory/) and you probably don't have the access rights to do that.

[Scott Adams]

Hi Jesse,
Thanks for your response.
I suspect you are right. It seems like maybe the network connections "blink" and that could be it. I'll look into it with the help of IT folks.

What do you mean by "Does it ever occur when you run entire (saved) dofiles?"
If this is the same as saving the do-file in the editor and pressing the Execute button on the editor (with nothing selected in the editor), this is when the error *does* happen. My do-files are stored on a network drive.
It *does not* seem to happen when I select a small section of code and Execute it.

Thanks again,
Scott

[William Lisowski]

I had the same question as Jesse - I think - and my interpretation was identical to the first of yours - saving the do-file in the editor and then hitting the button to run it (my button is labelled "Do"; I see no "Execute"). In that case, Stata is instructed to read the do-file from the saved location (as the generated do command in the Commands window confirms). When you select code and hit the button to run the selected code, Stata saves a temporary do-file locally and runs that. This is entirely consistent with Jesse's hypothesis about the network connection to the location of the do-file.

[Jesse Wursten]

Hi Jesse,
Thanks for your response.
I suspect you are right. It seems like maybe the network connections "blink" and that could be it. I'll look into it with the help of IT folks.

What do you mean by "Does it ever occur when you run entire (saved) dofiles?"
If this is the same as saving the do-file in the editor and pressing the Execute button on the editor (with nothing selected in the editor), this is when the error *does* happen. My do-files are stored on a network drive.
It *does not* seem to happen when I select a small section of code and Execute it.

Thanks again,
Scott

Ah, I misunderstood, I thought the issue occurred in the opposite situation (works when saved, not when running just a section). In that case, I think I can offer a solution - just add a space (or any other character) to the dofile before running it and do not save it. This will cause Stata to run a temporary copy of the dofile instead of the actual file. This temporary copy will be in the computer's temp folder and will thus remain accessible even if the network fails.

(No space added)
. do "C:\temp\sleep.do"
. sleep 5

(Space added, unsaved)
. do "C:\Users\Jesse\AppData\Local\Temp\STD00000000.tmp"
. sleep 5

[Chaza Abou Daher]

Hello everyone,
I am a new member to this forum and overall a Stata beginner. I am using stata 14.1 through Harvard's RCE and I am facing the following problem. After I run the first do file and save the dataset (called employee_national) to the directory (log named employee_national attached), I attempt to use it in another do file and it returns the following error:
.dta file corrupt the file unexpectedly ended before it should have

I thought i fixed it yesterday by adding the log close to the do file and it worked but today the problem came back. My data is very large in size.

Any help is much appreciated as I have noway of progressing my analysis if this problem persists.

[Nick Cox]

Chaza: Please start a new thread. Your problem isn't to do with do files.

Also, please do read the FAQ Advice carefully including https://www.statalist.org/forums/help#stata In general, active people here can't or won't open Word documents.

A problem with a corrupt .dta file is hard to advise on. Stata Technical Services may able to help.

[Chaza Abou Daher]

Thank you Mr. Cox. I will start a new thread. I already contacted Stata Technical Services, no answer so far. Many thanks

[Nick Cox]

As I write, it's 07:03 in College Station, TX, so you may need to wait a little.