Looping csv file - save as dta with file name

[Marco Errico]

Dear all,

I'm trying to work on two loops over 200 files:
1- First loop open the csv and save as dta keeping as file name the same one which is already save as csv
2- Appending all dta files

1-First loop I've tried to do the following:

local filenames: dir . files "Path folder where I stored all csv files"

foreach file of local filenames {
import delimited "`file'"
//Here I should write the line of codes to save them as dta with same csv filename in  a new folder path
}

2-Second loop:

Here I should append the dta files contained in the new folder


Would you be so kind to give me any tips?
Many thanks in advance

[Daniel Feenberg]

It would be nice if you could say

append *

but Stata provides a more complicated mechanism.

 
 append using `: dir . files "*.dta"'M

More information is documented in "extended macros".

[Daniel Feenberg]

That trailing M is a typo. The code should be:

 
 append using `: dir . files "*.dta"'

[William Lisowski]

Posts #2/#3 solve the second step. This addresses the first step.

local csvpath `"Path to folder where I stored all csv files"'
local dtapath `"Path to folder in which to store all dta files"'

local csvnames: dir `"`csvpath'"' files "*.csv"
local files : subinstr local filenames ".csv" "", all

foreach file of local files {
    import delimited `"`csvpath'/`file'.csv"'
    save             `"`dtapath'/`file'.dta"', replace
}

[Marco Errico]

Hi William Lisowski

Thanks a lot for your kind advice for the first step.

I implemented the code that you suggest but seems that I have some problem.
Specifically, I don't receive any error message from Stata, the work station shows that the loop is done but there is no dta files in the folder.

I copy and paste the line of codes that I'm running after you suggestion

cd "C:\Users\M.ERRICO\OneDrive - Universitat Ramón Llull\Projects\Tick Size Project\ESMA data\CSV_files"

local csvpath `"C:\Users\M.ERRICO\OneDrive - Universitat Ramón Llull\Projects\Tick Size Project\ESMA data\CSV_files"'
local dtapath `"C:\Users\M.ERRICO\OneDrive - Universitat Ramón Llull\Projects\Tick Size Project\ESMA data\DTA_files"'

local csvnames: dir `"`csvpath'"' files "*.csv"
local files : subinstr local filenames ".csv" "", all

foreach file of local files {
    import delimited `"`csvpath'/`file'.csv"'
    save             `"`dtapath'/`file'.dta"', replace
}

Thanks a lot for your time

[William Lisowski]

My apologies for an error on my part: try the following correction.

local files : subinstr local csvnames ".csv" "", all

[Marco Errico]

Thank you William Lisowski .

Now the loop starts but I got immediately an error message below

"file C:\Users\M.ERRICO\OneDrive - Universitat Ramón Llull\Projects\Tick Size Project\ESMA data\DTA_files/fulecr_20180106_e_1of1
> .dta saved
no; dataset in memory has changed since last saved
r(4);"

I've explored what might be the issue and as far as I got, by adding

foreach file of local files {
    import delimited `"`csvpath'/`file'.csv"'
    save             `"`dtapath'/`file'.dta"', replace
    clear
}

at the end of the loop, it solves the issue.

Thanks again for your time and suggestions. They have been really helpful

[William Lisowski]

The output of the help import delimited command tells us that the following change would also have solved the problem.

    import delimited `"`csvpath'/`file'.csv"', clear

[Marco Errico]

Thanks William Lisowski

I'm trying to implement the second loop (i.e. append all dataset and store the finale one in a different folder path)

I tried to implement [email protected] solution with this line of codes

     append using `: dir . files "*.dta"'

Nevertheless, I got this message
"invalid file specification"

Any suggestion?
Thanks again for your help


Edit: Solved it.
My silly typos in not setting

cd "dta folder"

Before the appending command

[Sanika Rr]

Hello all,
I am using Stata 16. I am trying to import multiple csv files and then append them into a single dta file in the same folder. I used the following

cd "G:\climate\river"
local files : dir "G:\climate\river" files "*.csv"
foreach file in 'files' {
import delimited 'file'
save 'file'.dta, replace
append using 'files'
}

I get the following error message

invalid 'file'
r(198);

Please tell me what I may be doing wrong, would be grateful for any help. Thank you.

[Clyde Schechter]

Your punctuation is wrong. 'file' is not valid Stata syntax. When you refer to local macros, the character that precedes the name is `, not '. On a US keyboard, ` is the upper case character immediately to the left of the !1 key. The character that follows the name, however, is, indeed, ', just as you have it.

[Sanika Rr]

Thank you Sir , I got it now!