While visiting WU Vienna for a talk, I have given a workshop on collecting web data using web scraping and APIs. Download the slide deck here.
Workshop Invitation
Learning Goals
- How to select web data sources and extraction methods for academic research?
- How do a researcher’s design decisions affect research validity, technical feasibility, and ethical/legal risks of collecting web data?
- Receive feedback on own the design of one’s web data collection
Preparation
Required reading: Fields of Gold: Web Scraping and APIs for Impactful Marketing Insights
Familiarize yourself with web scraping and APIs. If unfamiliar with web scraping or APIs, please follow the interactive tutorial in Google Colab
Submit an activity for feedback
- Present an idea for an interesting data context to study
- Provide a first (conceptual) design of your data collection
- Submit an initial prototype (no matter whether in Python, R, etc.).
Submit any question you may have concerning the use and/or collection of web data in academic research.
Workshop Agenda
| Time | |
|---|---|
| 10.00-10:15 | Introduction & why to scrape/use APIs |
| 10:15-10:45 | Data source selection + feedback on submissions |
| 10:45-11:00 | Break |
| 11:00-11:40 | Extraction design + feedback on submissions |
| 11:40-12:00 | Future Research Opportunities |
Any PhD student and faculty member, including research master students, interested in collecting data from websites or APIs for academic research. No technical skills for scraping/APIs required.