What's the difference between data warehouses and data lakes?

Data Warehousing Concepts

Aaren Stubberfield

Data Scientist

Database

• Structured data in rows and columns
• Transactional databases store transactions

Data warehouse

• Gather data, integrate, and make available for analysis
• Many input data sources
• Stores structured data
• Complex to change
  • Upstream and downstream effects must be considered
• Typically >100 GB in size

Why the data warehouse?

• How quickly the query will run on a large amount of data
• Avoid slowing down transactional database

Data marts

• A relational database for analysis
• Data is focused on one subject area
• Few input data sources
• Typically <100 GB in size

Data lake

• Entire organization store of data
  • Contains data from many departments
  • Many data input sources
  • Typically >100 GB in size
• Stores structured and unstructured data
  • Examples: video, audio, and documents

Data lake

• Less complex to make changes
  • Fewer upstream and downstream effects to consider
• Purpose to store data may not be known
  • Less organized

Summary

Feature	Data Warehouse	Data Mart	Data Lake
Data structure	Structured	Structured	Structured & Unstructured
Complexity to change	Complex	Complex	Less complex
Purpose of data	Known	Known	May not be known
Coverage of departments	Covers many	Covers only one	Covers many
Data sources	Many source systems	Few sources	Many source systems
Typical size	>100 GB	<100 GB	>100 GB

Let's practice!

Data Warehousing Concepts