ケーススタディ：スクーター問題

Python で始める AWS Boto 入門

Maksim Pecherskiy

Data Engineer

課題設定

スクーター

課題設定

スクーター

データ

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Hay un scooter electrico en sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

最終成果物

boto3 サービスクライアントの初期化

Rekognition クライアントを初期化

rekog = boto3.client('rekognition',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

Comprehend クライアントを初期化

comprehend = boto3.client('comprehend',
                     region_name='us-east-1', 
                     aws_access_key_id=AWS_KEY_ID, 
                     aws_secret_access_key=AWS_SECRET)

boto3 サービスクライアントの初期化

Translate クライアントを初期化

translate = boto3.client('translate', 
                         region_name='us-east-1', 
                         aws_access_key_id=AWS_KEY_ID, 
                         aws_secret_access_key=AWS_SECRET)

記述を英語に翻訳する

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = translate_fake.translate_text(
          Text=desc, 
          SourceLanguageCode='auto', 
          TargetLanguageCode='en')
        df.loc[index, 'public_description'] = resp['TranslatedText']

記述を英語に翻訳する

service_request_id	image	lat	long	public_description
93494	report_113439.jpg	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	I tripped on a stupid scooter

テキストの感情を判定

for index, row in df.iterrows():
    desc = df.loc[index, 'public_description']
    if desc != '':
        resp = comprehend.detect_sentiment(
          Text=desc, 
          LanguageCode='en')
        df.loc[index, 'sentiment'] = resp['Sentiment']

テキストの感情を判定

service_request_id	image	lat	long	sentiment	public_description
93494	report_113439.jpg	32.723138	-117.128237	NEGATIVE	Electric scooter on sidewalk
101502	report_134938839.jpg	32.7077658	-117.1281408	POSITIVE	This scooter helped me move a mattress!
101520	report_272819.jpg	32.77605567	-117.100004	NEGATIVE	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	32.68899358	-117.0584723	NEGATIVE	I tripped on a stupid scooter

画像内のスクーターを検出

df['img_scooter'] = 0
for index, row in df.iterrows():
    image = df.loc[index, 'image']
    response = rekog.detect_labels(
      # Specify the image as an S3Object
      Image={'S3Object': {'Bucket': 'gid-images', 'Name': image}}
    )
    for label in response['Labels']:
        if label['Name'] == 'Scooter':
        df.loc[index, 'img_scooter'] = 1
        break

画像内のスクーターを検出

service_request_id	image	img_scooter	sentiment	lat	long	public_description
93494	report_113439.jpg	1	NEGATIVE	32.723138	-117.128237	Electric scooter on sidewalk
101502	report_134938839.jpg	1	POSITIVE	32.7077658	-117.1281408	This scooter helped me move a mattress!
101520	report_272819.jpg	0	NEGATIVE	32.77605567	-117.100004	There is a scooter blocking the sidewalk
101576	report_3722938.jpg	1	NEGATIVE	32.68899358	-117.0584723	I tripped on a stupid scooter

最終カウント

スクーター画像があり感情が NEGATIVE の行のみ抽出

pickups = df[((df.img_scooter == 1) & (df.sentiment == 'NEGATIVE'))]

num_pickups = len(pickups)

332台のスクーター！

Ayo berlatih!

Python で始める AWS Boto 入門