Uploading and retrieving files

Introduction to AWS Boto in Python

Maksim Pecherskiy

Data engineer

Buckets and objects

A Bucket

• A bucket has a name
• Name is a string
• Unique name in all of S3.
• Contains many objects

An Object

• An object has a key
• Name is full path from bucket root
• Unique key in the bucket
• Can only be in one parent bucket

Creating the client

s3 = boto3.client(
  's3', 
  region_name='us-east-1', 
  aws_access_key_id=AWS_KEY_ID, 
  aws_secret_access_key=AWS_SECRET
)

Uploading files

s3.upload_file(
  Filename='gid_requests_2019_01_01.csv', 
  Bucket='gid-requests', 
  Key='gid_requests_2019_01_01.csv')

Uploading files

Uploading more objects

Listing objects in a bucket

response = s3.list_objects(
  Bucket='gid-requests', 
  MaxKeys=2,
  Prefix='gid_requests_2019_')

print(response)

Listing objects in a bucket

Listing objects in a bucket

Listing objects in a bucket

Getting object metadata

response = s3.head_object(
  Bucket='gid-requests', 
  Key='gid_requests_2018_12_30.csv')

print(response)

Getting object metadata

Downloading files

s3.download_file(
  Filename='gid_requests_downed.csv',
  Bucket='gid-requests', 
  Key='gid_requests_2018_12_30.csv')

Deleting objects

s3.delete_object(
  Bucket='gid-requests', 
  Key='gid_requests_2018_12_30.csv')

Summary

• Buckets are like folders
• Objects are like files
• boto3.client()
• s3.upload_file()
• s3.list_objects()
• s3.head_object()
• s3.download_file()
• s3.delete_object()

Let's make some objects!

Introduction to AWS Boto in Python