scrape data from linkedin

[Caroline Calogero]

hi guys, I want to scrape data from "https://www.linkedin.com/jobs/search/?currentJobId=3837155538&keywords=economics&origin =JOBS_HOME_SEARCH_BUTTON&refresh=true". does anyone have an example of code I can use to get observations of job titles and variables including job description, salary, experience level, etc?

[Sebastian Schirner]

Hey Caroline,

I have been webscraping in the past. My impression is that for such tasks Stata is not ideal. Instead, I recommend Python and the very useful package "Beautiful soup". With a few lines of code you can set up a scraper even without a lot of experience in Python.

That being said, I advise a great deal of caution when scraping data. There may be legal issues regarding data ownership. Especially if you want to scrape from commercial firms like LinkedIn. My guess would be that they take technological precautions to block scraping.

See https://www.linkedin.com/pulse/how-s...ankar-mallick-

Best,
Sebastian

[Andrew Musau]

Using official Stata commands and functions, see https://www.statalist.org/forums/for...using-fileread. A written tip on this is forthcoming in the next issue of the Stata Journal.