looping problem

[Jake Magel]

Hello, I am using the loop below to import and append several csv files. The appended file contains Sheet 1 (oc_master) and Sheet 5 cases but none for sheets 2-3. I am hoping for some suggestions to modify the code to append sheets 2-5 to oc_master. Thank you.

import delimited oc_sheet1.csv, (stringcols(5) clear
gen sheet=1
gen row=_n
save oc_master1, replace

use oc_master1, clear
forval i=2/5 {
import delimited oc_sheet`i'.csv, stringcols(5) clear
gen sheet=`i'
gen row=_n
append using oc_master1, force
save oc_master1-5, replace
}

[Clyde Schechter]

Change -save oc_master1-5- to -save oc_master1-.

Or, better, since your intent is to have a final file that reflects inputs 1 through 5, change all of the references to oc_master1 to oc_master1-5. The problem you are encountering is because you are saving to one file but appending from a different one.

Added: As an aside, if you really need to use -force- to get your -append-s to run, then it means that your data sets are not really compatible for appending. By using -force- you are sweeping the incompatibility under the rug--you are not resolving it. Unless you are 100% certain that you will never need to make use of any of the variables that are incompatible, you are creating an incorrect data set that will, ultimately, give you incorrect analytic results. It is just a question of when, not if, this will bite you. If you are 100% certain that you will never need any of the incompatible variables, then the better solution is to -drop- them inside the loop before using -append-. That way you won't have a data set that looks like it contains valid data but doesn't.

I recommend you use Mark Chatfield's -precombine- program, available from Stata Journal, before you -append- these files to get a complete picture of any incompatibilities. If they are irrelevant to your work, then -drop- those variables before -append-ing. If relevant, then fix them.