Getting ready for the first stream

Streaming Data with AWS Kinesis and Lambda

Maksim Pecherskiy

Data Engineer

End goal

chapter1_take_ms_liQuR.png

Streaming Data with AWS Kinesis and Lambda

End goal

gGuLC.png

Streaming Data with AWS Kinesis and Lambda

End goal

IIFUE.png

Streaming Data with AWS Kinesis and Lambda

End goal

XpcGO.png

Streaming Data with AWS Kinesis and Lambda

p5yig.png

Streaming Data with AWS Kinesis and Lambda

dcUser

Drh4F.png

Streaming Data with AWS Kinesis and Lambda

A destination S3 bucket

s3 = boto3.client('s3', 
                  aws_access_key_id=AWS_KEY_ID, 
                  aws_secret_access_key=AWS_SECRET, 
                  region_name='us-east-1')

s3.create_bucket(Bucket='sd-vehicle-data')
Streaming Data with AWS Kinesis and Lambda

LDGmc.png

Streaming Data with AWS Kinesis and Lambda

bFQoE.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

SSdiv.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

Yqb6Y.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

M4HOZ.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

NBLCR.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

ncOZ7.png

Streaming Data with AWS Kinesis and Lambda

Add permissions to user

1xvfb.png

Streaming Data with AWS Kinesis and Lambda

New powers

bEbSB.png

Streaming Data with AWS Kinesis and Lambda

A note on security

MBXWT.png

Streaming Data with AWS Kinesis and Lambda

A note on security

OsJnK.png

Streaming Data with AWS Kinesis and Lambda

vroeW.png

Streaming Data with AWS Kinesis and Lambda

New powers

bEbSB.png

Streaming Data with AWS Kinesis and Lambda

Streaming Data with AWS Kinesis and Lambda

Firehose stream permissions

m1sn3.png

Streaming Data with AWS Kinesis and Lambda

BLo0S.png

Streaming Data with AWS Kinesis and Lambda

RAFOr.png

Streaming Data with AWS Kinesis and Lambda

vx0iX.png

Streaming Data with AWS Kinesis and Lambda

xVkJU.png

Streaming Data with AWS Kinesis and Lambda

Roles vs users

rGbOt.png

  • Live in IAM
  • Have permissions policies
  • Only attach to other services
  • Do not have keys nor login

qQ1pa.png

  • Live in IAM
  • Have permissions policies
  • Can act on their own
  • Can have keys or login
Streaming Data with AWS Kinesis and Lambda

Review - end goal

WlEwO.png

Streaming Data with AWS Kinesis and Lambda

Review

m0gxE.png

Streaming Data with AWS Kinesis and Lambda

Review

Drh4F.png

Streaming Data with AWS Kinesis and Lambda

Review

BLo0S.png

Streaming Data with AWS Kinesis and Lambda

Let's do it!

Streaming Data with AWS Kinesis and Lambda

Preparing Video For Download...