Studium przypadku: Na hulajnodze!

Wprowadzenie do AWS Boto w Pythonie

Maksim Pecherskiy

Data Engineer

Problem

Hulajnogi

Problem

Hulajnogi

Dane

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Hay un scooter electrico en sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

Produkt końcowy

Inicjalizacja klientów usług boto3

Inicjalizacja klienta rekognition

rekog = boto3.client('rekognition',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Inicjalizacja klienta comprehend

comprehend = boto3.client('comprehend',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Inicjalizacja klientów usług boto3

Inicjalizacja klienta translate

translate = boto3.client('translate', 
                         region_name='us-east-1', 
                         aws_access_key_id=AWS_KEY_ID, 
                         aws_secret_access_key=AWS_SECRET)

Tłumaczenie wszystkich opisów na angielski

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = translate_fake.translate_text(
          Text=desc, 
          SourceLanguageCode='auto', 
          TargetLanguageCode='en')
        df.loc[index, 'public_description'] = resp['TranslatedText']

Tłumaczenie wszystkich opisów na angielski

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

Wykrywanie sentymentu tekstu

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = comprehend.detect_sentiment(
          Text=desc, 
          LanguageCode='en')
        df.loc[index, 'sentiment'] = resp['Sentiment']

Wykrywanie sentymentu tekstu

service_request_id	image	lat	long	sentiment	public_description
93494	report_113439.jpg	32.723138	-117.128237	NEGATIVE	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	POSITIVE	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	NEGATIVE	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	NEGATIVE	I tripped on a stupid scooter

Wykrywanie hulajnogi na zdjęciu

df['img_scooter'] = 0
for index, row in df.iterrows():
    image = df.loc[index, 'image']
    response = rekog.detect_labels(
      # Specify the image as an S3Object
      Image={'S3Object': {'Bucket': 'gid-images', 'Name': image}}
    )
    for label in response['Labels']:
        if label['Name'] == 'Scooter':
        df.loc[index, 'img_scooter'] = 1
        break

Wykrywanie hulajnogi na zdjęciu

service_request_id	image	img_scooter	sentiment	lat	long	public_description
93494	report_113439.jpg	1	NEGATIVE	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	1	POSITIVE	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	0	NEGATIVE	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	1	NEGATIVE	32.68899358	-117.0584723	I tripped on a stupid scooter

Wynik końcowy!

Wybierz tylko wiersze ze zdjęciem hulajnogi i negatywnym sentymentem

pickups = df[((df.img_scooter == 1) & (df.sentiment == 'NEGATIVE'))]

num_pickups = len(pickups)

332 hulajnogi!

Czas na ćwiczenia!

Wprowadzenie do AWS Boto w Pythonie