案例研究：电滑车大作战

Python 中的 AWS Boto 入门

Maksim Pecherskiy

Data Engineer

难题所在

滑板车

难题所在

滑板车

数据

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Hay un scooter electrico en sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

最终成品

初始化 boto3 服务客户端

初始化 Rekognition 客户端

rekog = boto3.client('rekognition',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

初始化 Comprehend 客户端

comprehend = boto3.client('comprehend',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

初始化 boto3 服务客户端

初始化 Translate 客户端

translate = boto3.client('translate', 
                         region_name='us-east-1', 
                         aws_access_key_id=AWS_KEY_ID, 
                         aws_secret_access_key=AWS_SECRET)

将所有描述翻译为英文

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = translate_fake.translate_text(
          Text=desc, 
          SourceLanguageCode='auto', 
          TargetLanguageCode='en')
        df.loc[index, 'public_description'] = resp['TranslatedText']

将所有描述翻译为英文

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

检测文本情感

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = comprehend.detect_sentiment(
          Text=desc, 
          LanguageCode='en')
        df.loc[index, 'sentiment'] = resp['Sentiment']

检测文本情感

service_request_id	image	lat	long	sentiment	public_description
93494	report_113439.jpg	32.723138	-117.128237	NEGATIVE	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	POSITIVE	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	NEGATIVE	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	NEGATIVE	I tripped on a stupid scooter

检测图像中的滑板车

df['img_scooter'] = 0
for index, row in df.iterrows():
    image = df.loc[index, 'image']
    response = rekog.detect_labels(
      # Specify the image as an S3Object
      Image={'S3Object': {'Bucket': 'gid-images', 'Name': image}}
    )
    for label in response['Labels']:
        if label['Name'] == 'Scooter':
        df.loc[index, 'img_scooter'] = 1
        break

检测图像中的滑板车

service_request_id	image	img_scooter	sentiment	lat	long	public_description
93494	report_113439.jpg	1	NEGATIVE	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	1	POSITIVE	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	0	NEGATIVE	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	1	NEGATIVE	32.68899358	-117.0584723	I tripped on a stupid scooter

最终计数

仅选择含滑板车图像且情感为负面的行

pickups = df[((df.img_scooter == 1) & (df.sentiment == 'NEGATIVE'))]

num_pickups = len(pickups)

共 332 辆滑板车！

Passons à la pratique !

Python 中的 AWS Boto 入门