Stop Scratching and Start Scraping!

Web Scraping in Python

Thomas Laetsch

Data Scientist, NYU

Feeding the Machine

Web Scraping in Python

Scraping Skills

  • Objective: Scrape a website computationally
  • How? We decide to use scrapy
  • How? We need to work with:
    • Selector and Response objects
    • Maybe even create a Spider
  • How? We need to learn XPath or CSS Locator notation
  • How? Understand the structure of HTML
Web Scraping in Python

What'd'ya Know?

  • Structure of HTML
  • XPath and CSS Locator notation
  • How to use Selector and Response objects in scrapy
  • How to set up a spider
  • How to scrape the web
Web Scraping in Python

EOT

Web Scraping in Python

Preparing Video For Download...