Cloud computing

Understanding Data Engineering

Hadrien Lacroix

Content Developer

Cloud computing for data processing

Servers on premises

  • Bought
  • Need space
  • Electrical and maintenance cost
  • Enough power for peak moments
  • Processing power unused at quieter times

Servers on the cloud

  • Rented
  • Don't need space
  • Use just the resources we need
  • When we need them
  • The closer to the user the better
Understanding Data Engineering

Cloud computing for data storage

  • Database reliability: data replication
  • Risk with sensitive data
Understanding Data Engineering

AWS

Understanding Data Engineering

Microsoft Azure

Understanding Data Engineering

Google Cloud

Understanding Data Engineering

file storage

Understanding Data Engineering

S3

Understanding Data Engineering

Azure Blob Storage

Understanding Data Engineering

Google Cloud Storage

Understanding Data Engineering

computation

Understanding Data Engineering

EC2

Understanding Data Engineering

Azure Virtual Machines

Understanding Data Engineering

Google Compute Engine

Understanding Data Engineering

databases

Understanding Data Engineering

AWS RDS

Understanding Data Engineering

Azure SQL Database

Understanding Data Engineering

Gogle Cloud SQL

Understanding Data Engineering

data pipeline

Understanding Data Engineering

using AWS S3 to store cover albums

Understanding Data Engineering

using EC2 to convert songs

Understanding Data Engineering

using RDS to store employees

Understanding Data Engineering

Multicloud

Pros

  • Reducing reliance on a single vendor
  • Cost-efficiencies
  • Local laws requiring certain data to be physically present within the country
  • Mitigating against disasters

Cons

  • Cloud providers try to lock in consumers
  • Incompatibility
  • Security and governance
Understanding Data Engineering

Summary

  • Benefits and risks of cloud computing
  • How it is implemented at Spotflix
  • Can cite the main cloud providers and their services
Understanding Data Engineering

Let's practice!

Understanding Data Engineering

Preparing Video For Download...