Visualizing streaming data

Streaming Data with AWS Kinesis and Lambda

Maksim Pecherskiy

Data Engineer

Our pipeline so far

2020-08-02_15-29.png

Streaming Data with AWS Kinesis and Lambda

Kibana Discover view

2020-08-02_16-03.png

Streaming Data with AWS Kinesis and Lambda

Kibana visualizations

2020-08-11_07-16.png

Streaming Data with AWS Kinesis and Lambda

Dashboards and alerts

2020-08-11_07-32.png

Streaming Data with AWS Kinesis and Lambda

Elasticsearch vs CloudWatch

CloudWatch

  • AWS centric
  • Can accept custom data, but not primary use case
  • Great for working with logs

Elasticsearch

  • General, open source tool
  • Accepts custom data, including logs
  • Better viz than CloudWatch
  • Robust plugin ecosystem
Streaming Data with AWS Kinesis and Lambda

Let's practice!

Streaming Data with AWS Kinesis and Lambda

Preparing Video For Download...