Web Scraping With Python

Web Scraping in Python

Thomas Laetsch

Data Scientist, NYU

Business Savvy

What are businesses looking for?

  • Comparing prices
  • Satisfaction of customers
  • Generating potential leads
  • ...and much more!
Web Scraping in Python

It's Personal

What could you do?

  • Search for your favorite memes on your favorite sites.
  • Automatically look through classified ads for your favorite gadgets.
  • Scrape social site content looking for hot topics.
  • Scrape cooking blogs looking for particular recipes, or recipe reviews.
  • ...and much more!
Web Scraping in Python

About My Work

AVorg.png

Web Scraping in Python

Pipe Dream

pipeline_setup_acq_proc.png

Web Scraping in Python

Pipe Dream: Setup

pipeline_setup.png

Setup

  • Understand what we want to do.
  • Find sources to help us do it.
Web Scraping in Python

Pipe Dream: Acquisition

pipeline_setup_acq.png

Acquisition

  • Read in the raw data from online.
  • Format these data to be usable.
Web Scraping in Python

Pipe Dream: Processing

pipeline_setup_acq_proc.png

Processing

  • Many options!
Web Scraping in Python

How do you do?

Our Focus

  • Acquisition!
  • (Using scrapy via python)
Web Scraping in Python

Are you in?

Web Scraping in Python

Preparing Video For Download...