What is Modern Data Architecture?

Understanding Modern Data Architecture

Miller Trujillo

Senior Software Engineer

What will we learn?

  • Modern data architectures
  • Different types of data architectures
  • Components
    • Ingestion to serving
    • Data governance
    • Orchestration
  • Design a modern data solution
Understanding Modern Data Architecture

Modern data architectures

Characteristic Modern data architecture Traditional data architecture
Volume and variety Massive volumes and multiple formats Smaller volumes of structured data
Storage Data lakes, NO-SQL, data warehouse Relational databases, data warehouse
Processing approach Real-time or near real-time processing Batch only
Scalability and flexibility Cloud-based solutions Hardware investments
Democratization Self-service analytics tools Heavy dependence on IT departments
Understanding Modern Data Architecture

Modern data architectures requirements

  • Flexibility and extensibility
  • Uses cloud platforms to scale
  • Handles rising data volumes
  • Critical business paths
    • Billing
  • Integrate distributed domains
  • Data governance and security
1 https://www.ibm.com/resources/the-data-differentiator/data-architecture
Understanding Modern Data Architecture

Components

  • Data sources
  • Data ingestion
  • Data storage
  • Data processing
  • Data orchestration
  • Data governance and security

Generic Big data architecture including data sources, storage, batch and stream processing, orchestration, and analytics storage and reporting

1 https://learn.microsoft.com/en-us/azure/architecture/guide/architecture-styles/big-data
Understanding Modern Data Architecture

Let's practice!

Understanding Modern Data Architecture

Preparing Video For Download...