The curious case of data growth

Introduction to Data

Maarten Van den Broeck

Senior Content Developer at DataCamp

The volume of data has grown exponentially

Data Growth

1 zettabyte = a one followed by 21 zero's in bytes = 1 billion terrabyte

1 Source: Statista
Introduction to Data

Data storage is changing

datastorage.gif

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings

Cave painting

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment

Scroll

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment
19th and 20th century
  • Punch cards

Punchcard

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment
19th and 20th century
  • Punch cards
  • Magnetic tape, floppy disks

Floppy disk

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment
19th and 20th century
  • Punch cards
  • Magnetic tape, floppy disks
20th and 21st century
  • More data on smaller media

CD

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment
19th and 20th century
  • Punch cards
  • Magnetic tape, floppy disks
20th and 21st century
  • More data on smaller media
  • CDs and hard/solid state drives (local)

Hard drive

Introduction to Data

Data storage is changing

Historical data storage
  • Genetic information in DNA
  • Cave and wall paintings
  • Scrolls and books of papyrus/parchment
19th and 20th century
  • Punch cards
  • Magnetic tape, floppy disks
20th and 21st century
  • More data on smaller media
  • CDs and hard/solid state drives (local)
  • Data centers (cloud)

Cloud storage

Introduction to Data

Data - where does it come from?

Ice cream shop #1 in New York

Competitor Ice cream shop in NY, USA.jpg

  • Sells vanilla, chocolate, and strawberry
  • Has a rough idea of sale transactions

Ice cream shop #2 in New York

New Market.jpg

  • Sells 20+ ice cream flavors
  • Also sells coffees and milkshakes
  • Tracks all sales
Introduction to Data

Capturing data

Data captured

  • Sales per product type and ice cream flavor
  • Stock per product type and flavor
  • Weather data

Optimizations

  • Avoid popular flavors being out of stock
  • Replace poor selling flavors with new ones
  • Predict sale spikes due to high temperature
  • Optimize prices

Ice cream shop #2 in NY, USA New Market.jpg

Introduction to Data

Which ice cream shop would fare better?

Ice cream shop #1 in New York Competitor Ice cream shop in NY, USA.jpg

  • Uses gut feeling to make decisions
  • Randomly switches ice cream flavors

Ice cream shop #2 in New York New Market.jpg

  • Uses data to make decisions
  • Searching for the best flavors
Introduction to Data

Companies are more complex than ice cream shops

3D Manufacturing companies

  • Beam heat
  • Layer thickness
  • Structural stability

Financial institutions

  • Mortgage applications
  • Fraud detection

Factory Data.jpg

iStock-1363104923.jpg

Introduction to Data

Let's practice!

Introduction to Data

Preparing Video For Download...