S3 deep dive: storage, versioning, and replication

Using Data Stores in AWS

Dunieski Otano

AWS Solutions Architect

The $3 million storage bill

  • Company stores 100TB of logs in S3 Standard
  • Files accessed once, then never again
  • Annual cost: $2,760,000
  • With lifecycle policy: $48,000 (98% savings)

cost

Using Data Stores in AWS

S3 storage classes explained

  • Standard: Frequent access, highest cost, instant retrieval
  • Standard-IA: Infrequent access, ~50% cheaper, retrieval fee
  • Glacier Flexible: Archive, ~80% cheaper, minutes-hours retrieval
  • Glacier Deep Archive: Long-term, lowest cost, 12-hour retrieval

classes

Using Data Stores in AWS

Lifecycle policies: automate transitions

  • Define rules based on object age
  • Automatic transitions between classes
  • Automatic expiration (deletion)
  • Apply to prefixes for different policies

policies

Using Data Stores in AWS

S3 versioning: protect against deletion

  • Versioning: Keep multiple versions of same object
  • Protects against accidental deletes and overwrites
  • Each version has unique ID
  • Can restore previous versions anytime

versioning

Using Data Stores in AWS

S3 replication: Cross-Region and Same-Region

  • Cross-Region Replication (CRR): Copy to different region

    • Disaster recovery, compliance, lower latency

  • Same-Region Replication (SRR): Copy within region

    • Aggregate logs, replicate between accounts
  • Both require versioning enabled

replication

Using Data Stores in AWS

S3 performance optimization

  • Multipart upload: Upload large files in parallel
  • Transfer Acceleration: Use CloudFront edge locations
  • S3 Select: Query data without downloading
  • Request rate: 3,500 PUT, 5,500 GET per prefix/second

optimized

Using Data Stores in AWS

S3 event notifications and Lambda triggers

  • Event types: Object created, deleted, restored
  • Destinations: Lambda, SQS, SNS, EventBridge
  • Lambda is a serverless compute service: runs code without servers
  • SQS is a message queue: buffers messages between services
  • SNS is a notification service: sends alerts via email or SMS
  • Filter by prefix and suffix

event

Using Data Stores in AWS

S3 security: bucket policies and encryption

  • Bucket policies: Resource-based access control
  • ACLs: Legacy, use bucket policies instead
  • Encryption: SSE-S3, SSE-KMS, SSE-C, client-side
  • Block Public Access: Prevent accidental exposure

security

Using Data Stores in AWS

Let's practice!

Using Data Stores in AWS

Preparing Video For Download...