Intro to real-time streaming

Streaming Concepts

Mike Metzger

Data Engineer

What is 'real-time'?

  • Definition varies depending on context
  • Typically defines a response timeframe
  • The response timeframe is defined as a sort of guarantee
  • Could be:
    • 1 day
    • 1 hour
    • 1 minute
Streaming Concepts

Real world example

Post office

  • Different classes of service
  • Delivery timeframe varies based on service class
  • Only so much capacity for faster service
  • Costs are proportional to service speed
  • Service selection is up to the sender based on options
Streaming Concepts

Relationship to streaming?

How does real-time relate to streaming data?

  • Streaming processes are limited by available resources
    • How quickly can data be transported?
    • ... processed?
    • ... delivered?
    • How much does it cost?
Streaming Concepts

Resources define implementation

  • Helps define our requirements for streaming data processes
  • Speed of transport
  • Processing latency
  • Delivery
  • Data storage
  • Cost!
Streaming Concepts

Let's practice!

Streaming Concepts

Preparing Video For Download...