Working With Data / Do Files On An External Hard Drive

[Sepehr Hashemi]

Hello,

I am using large datasets (~15Gb), which I cannot use in my local computer due to space limitations. Therefore I have the data and their respective load files (do files) in an external hard drive. However, I am not sure why I am not able to use or save any files in any directory that is not the immediate directory I "cd" 'd to .

For example, I set directory to my external hard drive's "folder 1"

.cd "/Volumes/external hard drive/folder 1"
/Volumes/external hard drive/folder 1

. pwd
/Volumes/external hard drive/folder 1

However, I cannot work with any files in any other folder in the hard drive any files in the external hard drive:

. run "/Volumes/external hard drive/folder1/sub-folder 1/ specific-do-file.do"
file /Volumes/external hard drive/folder1/sub-folder 1/ specific-do-file.do not found

I get a similar error when attempting to save files:

. save "/Volumes/external hard drive/folder1/sub-folder 1/testsavedata.dta", replace
(file /Volumes/external hard drive/folder1/sub-folder 1/testsavedata.dta not found)
file /Volumes/external hard drive/folder1/sub-folder 1/testsavedata.dta could not be opened

However, running do files and saving the results works when attempting to do them in the original directory. Can you guide as to why I am not able to use sub-directories in the main chosen directory in my external hard drive? I am using a Mac OS for this project.

Thank you for the support!
Sepehr

[Dirk Enzmann]

If you did paste the commands as you actually used them (I don't believe): in

run "/Volumes/external hard drive/folder1/sub-folder 1/ specific-do-file.do"

the name of the .do-file starts with a space which will not work.

You need not to specify the complete path once you changed your current directory to the working directory. Alternatively to moving to the working directory, you could use a global macro to specify the path and subsequently use the global macro for the path information (don't forget the slash at the end of the path):

global mypath = "/Volumes/external hard drive/folder 1/"
run "${mypath}specific-do-file.do"

By the way: You should avoid space characters in names of paths or files.

[William Lisowski]

I agree with Dirk that what you typed has a number of inconsistencies.

. pwd
/Volumes/external hard drive/folder 1

is inconsistent with

. save "/Volumes/external hard drive/folder1/sub-folder 1/testsavedata.dta", replace

and could explain your problem, since if folder1 does not exist, the file does not exist, but like most applications, Stata will not create the folder if it does not exist.

[Sepehr Hashemi]

Dear Dirk and William,

Thank you for your prompt and helpful response and tips! My apologies, I had simplified the code to make it more interpretable, but I see I was better off to include the original code. Here is my current actual code:

clear
    cd "/Volumes/externalhhd/HCUP NIS Data" // setting to directory of the HCUP data in external harddrive
    
    global externalhhd_loads "/HCUP NIS Data/All Load Programs" // using these globals so it's easier to change the paths/folder names later if needed
    global externalhhd_data "/HCUP NIS Data/All Loaded Data"
        
        * 2016
            clear
                run "$externalhhd_loads/StataLoad_NIS_2016_Core.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2016_Core.dta",replace
            
            clear    
                run "$externalhhd_loads/StataLoad_NIS_2016_Hospital.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2016_Hospital.dta", replace
            
            
        * 2017
            clear
                run "$externalhhd_loads/StataLoad_NIS_2017_Core.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2017_Core.dta", replace
                
            clear
                run "$externalhhd_loads/StataLoad_NIS_2017_Hospital.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2017_Hospital.dta", replace
                
        
        * 2018
            clear
                run "$externalhhd_loads/StataLoad_NIS_2018_Core.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2018/NIS_2018_Core.dta", replace
            clear
                run "$externalhhd_loads/StataLoad_NIS_2018_Hospital.Do"
                rename *, lower
                save "$externalhhd_data/NIS_2018_Hospital.dta", replace

"externalhhd" is the actual name of the external harddrive. I confirmed that all the do files exist in the correct folder, and are correctly named. When running this code, I get the following error at the very first command that navigates to a different path than current directory, hence the original obstacle I opened this thread for:

. do "/var/folders/yh/m7tpxqsn0714dbxf0k5rssnh0000gn/T//SD01433.000000"

.         clear

.         cd "/Volumes/data/HCUP NIS Data" // setting to directory of the HCUP data in externa
> l harddrive
/Volumes/data/HCUP NIS Data

.        
.         global externalhhd_loads "/HCUP NIS Data/All Load Programs" // using these globals s
> o it's easier to change the paths/folder names later if needed

.         global externalhhd_data "/HCUP NIS Data/All Loaded Data"

.                
.                 * 2016
.                         clear

.                                 run "$externalhhd_loads/StataLoad_NIS_2016_Core.Do"
file /HCUP NIS Data/All Load Programs/StataLoad_NIS_2016_Core.Do not found
r(601);

end of do-file

r(601);

However, I am able to run the do files and save the loaded data if strictly working within the same directory... but there is no organization and it will be a very cluttered folder. I do not have a similar issue when using local directories. Do you know why I am having issues navigating paths on the external drive?

[William Lisowski]

Change

    global externalhhd_loads "/HCUP NIS Data/All Load Programs" // using these globals so it's easier to change the paths/folder names later if needed
    global externalhhd_data "/HCUP NIS Data/All Loaded Data"

to

    global externalhhd_loads "/Volumes/externalhhd/HCUP NIS Data/All Load Programs" // using these globals so it's easier to change the paths/folder names later if needed
    global externalhhd_data "/Volumes/externalhhd/HCUP NIS Data/All Loaded Data"

and try again.

[Sepehr Hashemi]

Hi William, thank you for the prompt support, this was it!