STATA not recognizing commands

[Yosef Cohen]

Hi,
I've been running into the issue below where stata stops recognizing my commands and instead lists them as part of a numbered list.

Thanks for the help.

. use PSPS_10_SUPPRESS, clear

.
. forvalues x = 11/21{
2.
. append using PSPS_`x'_SUPPRESS, force
3. 4 5total patients if hcpcs_cd="93016"
6. . total submitted_service_cnt if hcpcs_cd="93016"
7. total submitted_service_cnt if hcpcs_cd="93016"
8. save "XXXX\PSPSall.dta"
9. exit, clear
10. exit, clear
11.

[Jared Greathouse]

I'm not understanding. Does it throw an error or what? This

forvalues x = 11/21{
append using PSPS_`x'_SUPPRESS, force

appears legal, the rest of the loop is not legal syntax. Why are you exiting and clearing? I'm so confused

[Clyde Schechter]

The problem is that you have begun a loop, but you have not ended it. The way Stata parses blocks of code that loop commands (-foreach-, a few others) create is that it reads them in, and echos them to the Results window as a numbered list until it reaches the closing brace. Then it begins execution of the entire loop. So once you close the loop with }, it will start running all those commands.

That said, the -exit, clear- command inside the loop will cause the loop to terminate after just one iteration, so you will only get results for x == 11. Also, what is the point of repeating the command -total submitted_service_cnt if hcpcs_cd="93016"- twice: nothing will have changed from the first to the second instance and you will just get the same results both times. And your -save- command will also subvert you. You need -save "XXXX\PSPSall.dta", replace-. Stata does not let you overwrite an existing data set unawares: so without the -replace- option, Stata will throw an error message and terminate execution there.

Finally, although it is legal and will not stop the code from running, using the -force- option on -append- is unwise. If the data sets are all compatible with each other, then this does no harm, but it also does no good because you don't need -force- to get -append- to put them all together. -force- only comes into play if the data sets are incompatible. By -force-ing Stata to put incompatible data sets together you will lose data. The resulting dataset will be incorrect and incomplete. It should only be done if you don't need any of the variables that are incompatible among the data sets: but in that case, a better way to go is to use -append-'s -keep()- option to tell Stata which variables you actually want. Because by using -force-, if it turns out that one of the incompatible variables is one that actually matters to you, you will lose information on that variable and you won't even know it until somewhere down the road you start getting ridiculous results. And if you are unlucky, you won't even notice that, but somebody else will while you are presenting your results! Moreover, then tracing back the problem will be time-consuming and difficult. Error messages are your friends, they live to save you from faulty data and your own mistakes. You suppress them at your own peril.

There is a handy little ado, -precombine-, by Mark Chatfield, available at http://www.stata-journal.com/software/sj15-3, which allows you to check your data sets for compatibility before combining them. I strongly recommend you use it. Then, if incompatibilities are found, fix the data sets before doing anything else with them. (Fixing might mean getting rid of the offending variables if you don't need them for anything.) And stop using -force- options except in those uncommon situations where you are 100% certain that they will not do you any harm. Doing otherwise is the coding-equivalent of writing a suicide note.

[Yosef Cohen]

Thank you everyone for your responses. To be clear, I ended my code by "force". When Stata stopped responding I kept trying different codes to see if something would work. Sorry for the confusion.
Clyde: I can't seem to download Mark Chatfield's code. The URL doesn't work and I cant seem to find it using net search in stata. Please advise.

Thanks

[Jared Greathouse]

Do

search precombine

[Yosef Cohen]

Thank you, that worked