Observability

Understanding Modern Data Architecture

Miller Trujillo

Senior Software Engineer

What is observability?

  • Understanding the insides from the outside.
  • Data observability
    • Understand data as it moves
  • Complex distributed systems
  • Problem resolution
  • Reliability
Understanding Modern Data Architecture

Key aspects of observability: Monitoring & metrics

  • Monitoring, logging, and tracing.
  • Monitoring: Continuously check of system's status
  • Metrics: Numerical values emitted by the system's

Sample monitoring dashboard from Grafana playground

Understanding Modern Data Architecture

Key aspects of observability: Logging & tracing

logging

  • Records of events
    • Examples:
      • Information or debug messages
      • Exception stack trace
  • Audit logs
    • Who did what and when

Tracing

  • Track an specific request or element through different stages

Toy with map to exemplify tracing

Understanding Modern Data Architecture

Observability platforms

Observability platforms logos from GCP and AWS

Various observability platforms logos

Understanding Modern Data Architecture

Let's practice!

Understanding Modern Data Architecture

Preparing Video For Download...