Do file prematurely ending when ran in batch mode on Unix Server

[Kelsey OHollaren]

Hi everyone,

I'm new to using batch mode via a remote server to run Stata code. I have a do file that 1) loads in a .dta file, does some very basic data cleaning, and then runs a regression. Specific code below:

/*
Developer: Kelsey O'Hollaren
Initial Date: 
*/

** Universal settings 
clear all
set more off 

set maxvar 50000 // necessary given all the fixed effects! 

use "~/Data/Analytic_HMDA_Subset.dta"

sort ImportParcelID Year

drop if RepeatSale == 0

replace SellerBlack = blackHMDA[_n-1] 
replace SellerWhite = whiteHMDA[_n-1] 
replace SellerAPI = APIHMDA[_n-1]  
replace SellerLatino = latinoHMDA[_n-1] 


di "Testing to see if this thing works"


regress log_assesssale SellerBlack SellerLatino SellerAPI i.TaxJur i.Year , cluster(TaxJur)

I'm using the nohup stata-mp -b command to run the do-file, but the log output ends after the

di "Testing to see if this thing works"

command. I'm not sure if I'm using improper syntax or something OR that the log doesn't print anything until the regression output is available -- it's nearly 500,0000 observations with 17,000 fixed effects, in addition to covariates of interest, which obviously takes a while to run.



If anyone could provide some insight here, I'd appreciate it.

Thanks!
Kelsey

[Maarten Buis]

What happens when you add an empty line after the regress command?

[Bert Lloyd]

In addition to Maarten Buis 's good tip, I would try a test case with a small subset of your units so you don't have to wait around to see if the regress command completes successfully.

You could also look at some of the high-dimensional fixed effects estimators like reghdfe, those may handle your case better.

[Kelsey OHollaren]

Thanks for the speedy resposes:

Maarten Buis, do you mean an empty line between "regress" command and the variables in my model? ie

regress

log_assesssale SellerBlack SellerLatino SellerAPI i.TaxJur i.Year , cluster(TaxJur)

in that case, I get a "Error: last estimate not found" message.

Bert Lloyd, thanks for the suggestions! Similar issue when I use

 sample 10 
 log_assesssale SellerBlack SellerLatino SellerAPI i.TaxJur i.Year , cluster(TaxJur)

.I'll have to see if I can install packages on the remote version of Stata I'm using. I also tried running the regression through the GUI and it takes a long while, no surprise. It's possible I'm just being too impatient .

[Maarten Buis]

No, I mean an empty line at the very end of the file. In older versions Stata needed a end of line character to know where the command ends. If you did not add an empty line at the end, Stata would not know that the last command ended, as there was no end of line character there, so it would not execute the last command.

[Kelsey OHollaren]

Got it. I don't think that was an issue - I'm using Stata 18 anyways. I also got the model to run using a ten percent sample in combination with the reghdfe command (turns out I could install it after all). Currently running the job on the full data set. I'll be back in touch to report if it works or not.


Kelsey

[Kelsey OHollaren]

Happy to report everything works just fine now. Thanks for both your input, Maarten Buis and Bert Lloyd.

[Bert Lloyd]

Kelsey OHollaren please share what solved your problem in case others come across this in the future

[Kelsey OHollaren]

I *think* the problem was solved by using reghdfe -- much more efficient, it seems. running batch mode works as expected now. thanks all