Studiu de caz: Trotinete!

Introducere în AWS Boto în Python

Maksim Pecherskiy

Data Engineer

Problema

Trotinete

Problema

Trotinete

Datele

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Hay un scooter electrico en sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

Produs final

Inițializare clienți boto3.

Inițializare client rekognition

rekog = boto3.client('rekognition',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Inițializare client comprehend

comprehend = boto3.client('comprehend',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Inițializare clienți boto3

Inițializare client translate

translate = boto3.client('translate', 
                         region_name='us-east-1', 
                         aws_access_key_id=AWS_KEY_ID, 
                         aws_secret_access_key=AWS_SECRET)

Traducere descrieri în engleză

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = translate_fake.translate_text(
          Text=desc, 
          SourceLanguageCode='auto', 
          TargetLanguageCode='en')
        df.loc[index, 'public_description'] = resp['TranslatedText']

Traducere descrieri în engleză

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

Detectare sentiment text

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = comprehend.detect_sentiment(
          Text=desc, 
          LanguageCode='en')
        df.loc[index, 'sentiment'] = resp['Sentiment']

Detectare sentiment text

service_request_id	image	lat	long	sentiment	public_description
93494	report_113439.jpg	32.723138	-117.128237	NEGATIVE	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	POSITIVE	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	NEGATIVE	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	NEGATIVE	I tripped on a stupid scooter

Detectare trotinete în imagini

df['img_scooter'] = 0
for index, row in df.iterrows():
    image = df.loc[index, 'image']
    response = rekog.detect_labels(
      # Specify the image as an S3Object
      Image={'S3Object': {'Bucket': 'gid-images', 'Name': image}}
    )
    for label in response['Labels']:
        if label['Name'] == 'Scooter':
        df.loc[index, 'img_scooter'] = 1
        break

Detectare trotinete în imagini

service_request_id	image	img_scooter	sentiment	lat	long	public_description
93494	report_113439.jpg	1	NEGATIVE	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	1	POSITIVE	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	0	NEGATIVE	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	1	NEGATIVE	32.68899358	-117.0584723	I tripped on a stupid scooter

Număr final!

Selectare rânduri cu imagine de trotinetă și sentiment negativ

pickups = df[((df.img_scooter == 1) & (df.sentiment == 'NEGATIVE'))]

num_pickups = len(pickups)

332 de trotinete!

Să exersăm!

Introducere în AWS Boto în Python