Load dataset from OneDrive web folder

[Luigi Rombi]

Dear fellows,
I am trying to load a dataset "A.dta" from my personal OneDrive folder, but I have some issues.

Essentially, I followed the instructions on the reference manual, and I set my URL and then checked that I effectively modified it through the commands.

webuse set https://XXXX the URL of my OneDrive folder XXXX
webuse query
prefix now "https://XXXX the URL of my OneDrive folder XXXX"

However, when I try to use my dataset, this is what Stata reports:

webuse A
web error 403
r(679);

Unfortunately, as I click on r(679), this is what I find:

And r(679) is not even in this list of common error messages.


I am sure that I must modify something in my Preferences>Internet, but I don't know how to do it. I tried to tick "Use authentication for proxy server" and insert my OneDrive user and PW, but it did not work.

Thank you for your time.

[William Lisowski]

I don't know what "instruction manual" you followed, and I'm not a user of OneDrive.

However I do know that the "web error 403" means that Stata was told it does not have permission to access the folder or perhaps the file within the folder.

Did you obtain the URL you used on the webuse set command following the procedures for obtaining a OneDrive link to share as described for example in

https://www.dummies.com/computers/op...onedrive-link/

I know you are the OneDrive user associated with the OneDrive directory, but I expect that accessing OneDrive over the web requires some sort of authentication (username/password), which Stata is not in a position to give.

[Luigi Rombi]

Hi William,
thank you for clarifying the meaning of "web error 403".
About the manual, I just followed the instructions after typing "help webuse" in Stata.
About the link, I tried the "dummy" alternatives #1 (copy-and-paste from the URL bar) and #2 (essentially what is described in the link that you posted) but the results are the same.

I expect that accessing OneDrive over the web requires some sort of authentication (username/password), which Stata is not in a position to give.

You are right, accessing OneDrive requires authentication. That's why I tried to "improvise" something on my internet preferences hoping that it would work but, based on your last comment, I don't think that there is a way to do it other than downloading the files that I need. Do you have some valid alternatives to do it?

[William Lisowski]

My hope was that accessing OneDrive using the URL for a shared folder would not require authentication.

In your web browser, are you able to access the folder and see the file by entering

https://XXXX the URL of my OneDrive folder XXXX/

You could create a reproducible example and others can try to help you. Create a new folder on OneDrive, place within it a Stata dataset, get the link for sharing the folder, and in a new post here paste the link to the new folder and tell us the name of the dataset within it.

[Luigi Rombi]

In your web browser, are you able to access the folder and see the file by entering
Code:
https://XXXX the URL of my OneDrive folder XXXX/

No, I cannot see the file because it is in a "restricted area". If I paste the link, created by following the procedure you suggested, in a private browsing window, I am redirected to the login page of OneDrive.

[William Lisowski]

I realized I have access to OneDrive through my Office subscription. I created a StatalistTesting folder and have uploaded a png into it, and I copied the link that was created for sharing the folder

https://1drv.ms/u/s!AhBsWZ98ANCOiGJV...0enCg?e=DBJPe5

which the Statalist Forum software may have abbreviated above, so let me see if pasting it into a code block presents the full link.

https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5

Did the sharing link you created look like that?

I tested this by using a different browser that has no prior experience with my Microsoft account (so, no remembered passwords) and found that opening the shared link takes me to the StatalistTesting folder, showing, and allowing me to view, the png file. I note that the link above is an indirect link to a onedrive.live.com URL that is shown in the web browser when it displays the StatalistTesting folder.

See if you can use this link in your web browser. If so, I'll move forward to putting a dataset there to open with webuse.

[Luigi Rombi]

Great! I see the ORCID.png file in your folder.

Did the sharing link you created look like that?

Absolutely not. Mine looks totally different and presents my name and surname and other "sensible" data so that I am unsure about copying and pasting it here.

Can you try to substitute the png file with a data file in your folder so that we can see whether it works at least in your folder?

Thank you so much for your time.

[Luigi Rombi]

Hi William,
I changed the privacy option so that anyone that possesses the link can access the folder (by trying on private browsing, I can now see the folder and the StataWebuse_FILE.dta file), but when I copy and paste it on Stata, this is what I get:

webuse StataWebuse_FILE
server reported server error
r(675);

Thank you again for your time.

[William Lisowski]

I created a version of Stata's auto.dta example dataset with the make variable changed to uppercase, and uploaded that to OneDrive. both attempts to read my example dataset directly from OneDrive fail, telling me the dataset is "not Stata format". Using my browser to download the dataset from OneDrive to my computer resulted in a dataset that read successfully. I attempted to have OneDrive give me a sharing link for the dataset itself, but my browser just spun its wheels, no link.

. // demo original uploaded to OneDrive
. // make changed to uppercase to distinguish it from sysuse version
.
. use ~/Desktop/auto, clear
(1978 Automobile Data)

. if _rc==0 list make in 1/5

     +---------------+
     | make          |
     |---------------|
  1. | AMC CONCORD   |
  2. | AMC PACER     |
  3. | AMC SPIRIT    |
  4. | BUICK CENTURY |
  5. | BUICK ELECTRA |
     +---------------+

.
. // try webuse from OneDrive
.
. webuse set "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5"
(prefix now "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5")

. capture noisily webdescribe auto
file https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta not Stata format

. capture noisily webuse auto, clear
file https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta not Stata format

. if _rc==0 list make in 1/5

.
. // try accessing directly from OneDrive
.
. capture noisily describe using "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta"
file https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta not Stata format

. capture noisily use "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta", clear
file https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta not Stata format

. if _rc==0 list make in 1/5

.
. // try version downloaded from OneDrive using web browser
.
. capture noisily use ~/Downloads/auto.dta, clear
(1978 Automobile Data)

. if _rc==0 list make in 1/5

     +---------------+
     | make          |
     |---------------|
  1. | AMC CONCORD   |
  2. | AMC PACER     |
  3. | AMC SPIRIT    |
  4. | BUICK CENTURY |
  5. | BUICK ELECTRA |
     +---------------+

.

Added in edit: An attempt to have Stata download the dataset with the copy command similarly produced a "not Stata format" dataset.

. copy "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta" ~/Desktop/auto_copy.dta

. describe using ~/Desktop/auto_copy
file ~/Desktop/auto_copy.dta not Stata format

Added in second edit: I managed to have OneDrive give me a sharing link for the file, but that too did not work. When I open the link in my browser, I see that it produces the preview page for the file, it does not download the file.

Summary: It does not appear that programmatic access to files stored in OneDrive is possible, given my limited knowledge. All the sharing links just take you to OneDrive web pages.

[Luigi Rombi]

Hi William,
I tried the easiest way possible.

webuse set "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5"
(prefix now "https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5")

webuse auto
file https://1drv.ms/u/s!AhBsWZ98ANCOiGJVrT_yugK0enCg?e=DBJPe5/auto.dta not Stata format
r(610);

As you can see, the problem persists.

Apparently, there is no easy solution to this issue, at least for my (very) limited knowledge.
Let's wait if someone sees this and try to solve it. Otherwise, I think that future releases of Stata should take into consideration to have:
#1 a dedicated cloud area for each Stata user (i.e., what OneDrive does for Office package softwares);
#2 the possibility to access "protected" cloud folders such as OneDrive, Dropbox, etc.

Let's hope that someone will see this post and future releases of Stata will solve this issue.

I appreciate a lot that you tried to help me anyway. Thank you so much for your time.