案例研究：构建通知系统

Python 中的 AWS Boto 入门

Maksim Pecherskiy

Data Engineer

最终产物

构建通知系统

主题设置

创建主题
下载联系人列表 CSV
为每个服务创建主题
将联系人订阅到各自主题

构建通知系统

获取汇总数据

下载月度"Get It Done"报告
获取坑洞数量
获取非法倾倒通知数量

发送警报

若坑洞超出 100，则发送警报
若非法倾倒超出 30，则发送警报

主题设置

初始化 SNS 客户端

sns = boto3.client('sns', 
                   region_name='us-east-1',
                   aws_access_key_id=AWS_KEY_ID, 
                   aws_secret_access_key=AWS_SECRET)

创建主题并保存其 ARN

trash_arn = sns.create_topic(Name="trash_notifications")['TopicArn']
streets_arn = sns.create_topic(Name="streets_notifications")['TopicArn']

主题设置

将用户订阅到主题

contacts = pd.read_csv('http://gid-staging.contacts.csv')

将用户订阅到主题

contacts.csv

Name	Email	Phone	Department
John Smith	[email protected]	+11224567890	trash
Fanny Mae	[email protected]	+11234597890	trash
Janessa Goldsmith	[email protected]	+11534567890	streets
Evelyn Monroe	[email protected]	+11234067890	streets
Max Pe	[email protected]	+11234517890	streets

将用户订阅到主题

创建 subscribe_user 方法

def subscribe_user(user_row):

    if user_row['Department'] == 'trash':
        sns.subscribe(TopicArn = trash_arn, Protocol='sms', Endpoint=str(user_row['Phone']))
        sns.subscribe(TopicArn = trash_arn, Protocol='email', Endpoint=user_row['Email'])

    else:
        sns.subscribe(TopicArn = streets_arn, Protocol='sms', Endpoint=str(user_row['Phone']))
        sns.subscribe(TopicArn = streets_arn, Protocol='email', Endpoint=user_row['Email'])

将 subscribe_user 应用于每一行

contacts.apply(subscribe_user, axis=1)

将用户订阅到主题

获取汇总数据

将一月报告加载为 DataFrame

df = pd.read_csv('http://gid-reports.2019/feb/final_report.csv')

获取汇总数据

service_name	count
Illegal Dumping	2580
Potential Missed Collection	150
Pothole	1170
Traffic Sign - Maintain	210
Traffic Signal Head Turned	60
Traffic Signal Light Out	120

获取汇总数据

设置索引，以服务名直接取数

df.set_index('service_name', inplace=True)

获取汇总数据

trash_violations_count = df.at['Illegal Dumping', 'count']
streets_violations_count = df.at['Pothole', 'count']

发送警报

if trash_violations_count > 100:

  # Construct the message to send
    message = "Trash violations count is now {}".format(trash_violations_count)

    # Send message
    sns.publish(TopicArn = trash_arn, 
                Message = message, 
                Subject = "Trash Alert")

发送警报

if streets_violations_count > 30:

    # Construct the message to send
    message = "Streets violations count is now {}".format(streets_violations_count)

    # Send message
    sns.publish(TopicArn = streets_arn, 
                Message = message, 
                Subject = "Streets Alert")

最终结果

Passons à la pratique !

Python 中的 AWS Boto 入门