Download files from FTP server with login and password

[Francois Durant]

Hi,

I would like to know if it is possible to download files from an FTP server with login and password from Stata. In particular, I would like to download data from the Wharton database (WRDS). Please let me know if it is possible. Thanks.

[Paul Dickman]

It works for me with Stata 16.1 Windows.

. use ftp://username:[email protected]/public_html/colon.dta
(Colon carcinoma, diagnosed 1975-94, follow-up to 1995)

You can also use:

. copy ftp://username:[email protected]/public_html/colon.dta foo.dta
. use foo

You could (depending on your OS) also map a directory on the ftp site as a network drive on your OS. That is definitely possible on Windows.

[Francois Durant]

I could not connect using your method. Trying to find more information on how to connect Stata to the Wharton Database WRDS, I found the following page and tutorial:
https://blog.stata.com/2017/09/19/im...ta-into-stata/

Tutorial that I followed:
https://www.stata.com/blog/wrds/Stata_WRDS.pdf

I followed every single step, downloaded PostgreSQL ODBC driver and installed it. I am stuck at the bottom of page 10:
"Provided you entered everything correctly, you can now click the "Test" button, and the driver will attempt to connect to the WRDS PostgreSQL server. If it does not return "Connection successful", please double-check all your settings; if it continues to fail, please contact us and provide the exact error message you get."

I checked every single entry several times, but I'm getting an error message to connect. I enclose a screenshot. Please can you help?

[Bjarte Aagnes]

If you followed the instructions and it failed; contact Stata tech support

[Francois Durant]

I have sent an email to the following:
[email protected]

Anything else I can / should do to get an answer? Thanks.

[William Lisowski]

The documentation tells you

Server: The hostname of the server to connect to. This must be "wrds-
pgdata.wharton.upenn.edu".

That hyphen is part of the server name.

wrds-pgdata.wharton.upenn.edu

% nslookup wrds-pgdata.wharton.upenn.edu

Non-authoritative answer:
Name:    wrds-pgdata.wharton.upenn.edu
Address: 165.123.60.118

[Francois Durant]

Great catch William, I thought I was safe with a copy/paste from the tutorial PDF but it did not seem to take into account "-". Thanks you so much William!

I am now able to load the data from WRDS to Stata, for instance:

odbc load, table("comp_na_daily_all.funda") noquote

Is there any way I could download the data without having to load it in Stata? Something similar to Paul's initial suggestion:

 
 copy ftp://.../colon.dta foo.dta, replace

[William Lisowski]

Well, in post #3 the error message lacks the hyphen in the server name, which it presumably obtained from your dialog box, and the error message is consistent with a failure of DNS to translate the domain name into an IP address, which what happens when I omit the hyphen (see below).

If you position your cursor in the Server box of the Setup window and use the left-arrow key repeatedly, you should be able to get the server name to scroll right so you can see the beginning and confirm that you did indeed include the hyphen.

Or do you now have a different error message?

% nslookup wrdspgdata.wharton.upenn.edu

** server can't find wrdspgdata.wharton.upenn.edu: NXDOMAIN

[Francois Durant]

You were right William, I edited my previous message but you may have read the original version. Any idea on how to download and save the dataset without loading into Stata first?

[William Lisowski]

The process of loading it into Stata is what creates a Stata dataset from output returned by a database as a stream of ODBC-style data. So without Stata in the middle, you won't get a Stata dataset.

I assume there are other ways to access WRDS data but I know nothing about WRDS. Your problem was a DNS problem caused by an incorrect server name: that I do have experience with.

[Francois Durant]

Thanks for the clarification William.

Does Paul Dickman knows what would be the syntax to download files from WRDS? I currently do it manually with an SFTP (different encryption method than FTP) software but would like to integrate that in my Stata code. Thanks a lot in advance.

[Paul Dickman]

Does Paul Dickman knows what would be the syntax to download files from WRDS? I currently do it manually with an SFTP (different encryption method than FTP) software but would like to integrate that in my Stata code. Thanks a lot in advance.

No. Sorry. Stata technical support might know, but my guess is that it you won't be able to read a file with sftp in the same manner as you can with ftp (as in #2).

Mapping a network drive should work. My understanding is linux and MacOS offer native support for mapping an SFTP directory as a network drive, whereas additional software is required to do this in Windows.

[Francois Durant]

I just realized that I can import data from WRDS via the graphic interface as shown on the screenshot but the code generated by the intefarce produces in error when ran without the graphic interface

. odbc load, table("comp_na_daily_all.adsprate") noquote
dsn() or connectionstring() required

This is a problem because I want to access the data via the Stata code, without looking at each file via the graphic interface every time I want the fresh feed. Can someone help? Thanks.