Case Study: Scooting Around!

Introduction to AWS Boto in Python

Maksim Pecherskiy

Data Engineer

The Quandary

Scooters

Introduction to AWS Boto in Python

The Quandary

Scooters

Introduction to AWS Boto in Python

The Data

service_request_id image lat long public_description
93494 report_113439.jpg 32.723138 -117.128237 Hay un scooter electrico en sidewalk
101502 report_134938839.jpg 32.7077658 -117.1281408 This scooter helped me move a mattress!
101520 report_272819.jpg 32.77605567 -117.100004 There is a scooter blocking the sidewalk
101576 report_3722938.jpg 32.68899358 -117.0584723 I tripped on a stupid scooter
Introduction to AWS Boto in Python

Final Product

Final product

Introduction to AWS Boto in Python

Final Product

Final product

Introduction to AWS Boto in Python

Final Product

Final product

Introduction to AWS Boto in Python

Final Product

Final product

Introduction to AWS Boto in Python

Final Product

Final product

Introduction to AWS Boto in Python

Initialize boto3 service clients.

Initialize rekognition client 

rekog = boto3.client('rekognition',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Initialize comprehend client 

comprehend = boto3.client('comprehend',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)
Introduction to AWS Boto in Python

Initialize boto3 service clients

Initialize translate client 

translate = boto3.client('translate', 
                         region_name='us-east-1', 
                         aws_access_key_id=AWS_KEY_ID, 
                         aws_secret_access_key=AWS_SECRET)
Introduction to AWS Boto in Python

Translate all descriptions to English

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = translate_fake.translate_text(
          Text=desc, 
          SourceLanguageCode='auto', 
          TargetLanguageCode='en')
        df.loc[index, 'public_description'] = resp['TranslatedText']
Introduction to AWS Boto in Python

Translate all descriptions to English

service_request_id image lat long public_description
93494 report_113439.jpg 32.723138 -117.128237 Electric scooter on sidewalk
101502 report_134938839.jpg 32.7077658 -117.1281408 This scooter helped me move a mattress!
101520 report_272819.jpg 32.77605567 -117.100004 There is a scooter blocking the sidewalk
101576 report_3722938.jpg 32.68899358 -117.0584723 I tripped on a stupid scooter
Introduction to AWS Boto in Python

Detect text sentiment

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = comprehend.detect_sentiment(
          Text=desc, 
          LanguageCode='en')
        df.loc[index, 'sentiment'] = resp['Sentiment']
Introduction to AWS Boto in Python

Detect text sentiment

service_request_id image lat long sentiment public_description
93494 report_113439.jpg 32.723138 -117.128237 NEGATIVE Electric scooter on sidewalk
101502 report_134938839.jpg 32.7077658 -117.1281408 POSITIVE This scooter helped me move a mattress!
101520 report_272819.jpg 32.77605567 -117.100004 NEGATIVE There is a scooter blocking the sidewalk
101576 report_3722938.jpg 32.68899358 -117.0584723 NEGATIVE I tripped on a stupid scooter
Introduction to AWS Boto in Python

Detect scooter in image

df['img_scooter'] = 0
for index, row in df.iterrows():
    image = df.loc[index, 'image']
    response = rekog.detect_labels(
      # Specify the image as an S3Object
      Image={'S3Object': {'Bucket': 'gid-images', 'Name': image}}
    )
    for label in response['Labels']:
        if label['Name'] == 'Scooter':
        df.loc[index, 'img_scooter'] = 1
        break
Introduction to AWS Boto in Python

Detect scooter in image

service_request_id image img_scooter sentiment lat long public_description
93494 report_113439.jpg 1 NEGATIVE 32.723138 -117.128237 Electric scooter on sidewalk
101502 report_134938839.jpg 1 POSITIVE 32.7077658 -117.1281408 This scooter helped me move a mattress!
101520 report_272819.jpg 0 NEGATIVE 32.77605567 -117.100004 There is a scooter blocking the sidewalk
101576 report_3722938.jpg 1 NEGATIVE 32.68899358 -117.0584723 I tripped on a stupid scooter
Introduction to AWS Boto in Python

Final count!

Select only rows where there was a scooter image and that have negative sentiment 

pickups = df[((df.img_scooter == 1) & (df.sentiment == 'NEGATIVE'))]

num_pickups = len(pickups)

332 Scooters!

Introduction to AWS Boto in Python

Let's practice!

Introduction to AWS Boto in Python

Preparing Video For Download...