Event capture, scaling, and partitions

Develop Azure Event-based and Message-based Solutions

Fiodar Sazanavets

Senior Software Engineer and Microsoft MVP

What you will learn

  • Archiving streams with Capture.
  • How to scale with:
    • Throughput units.
    • Partitions.
  • Resilience with:
    • Groups.
    • Checkpointing.
    • Geo-disaster recovery.

Event Hubs capabilities

Develop Azure Event-based and Message-based Solutions

Event capture

  • Event Hubs Capture:
    • Writes streams to Blob or Data Lake Storage.
    • Responsible for writing data to files.
  • Time- and sized-based batches.
  • Durable records for reprocessing, compliance, analytics.

Event Capture

Develop Azure Event-based and Message-based Solutions

Scaling Event Hubs

Throughput units

  • Controls the throughput capacity.
  • Namespace level.
  • Auto-inflate for dynamic scaling.

Partitions

  • Control parallelism.
  • More concurrent consumers.
  • Ordering only within partition.
  • Individual Event Hub level.

TUs

Partitions

Develop Azure Event-based and Message-based Solutions

Consumer groups

  • Consumer groups:
    • Independent views of the stream.
    • Different for different applications.
  • Checkpointing:
    • Records how far consumer has read.
    • Resumes from the last checkpoint.
    • Written periodically.
  • Frequent checkpoints reduce rework.
  • Infrequent checkpoints reduce storage writes.

Consumer groups

Develop Azure Event-based and Message-based Solutions

Geo-disaster recovery

Geo-Dr

  • Event Hubs supports geo-disaster recovery (Geo-DR).

  • You pair namespaces in different regions under a single alias.

  • Producers and consumers connect to the alias, not a specific region.

  • If the primary region fails, the alias can fail over to the secondary.

  • Geo-DR is a metadata-level failover for business continuity.

Develop Azure Event-based and Message-based Solutions

Putting it all together

Advanced Event Hubs

Develop Azure Event-based and Message-based Solutions

Let's practice!

Develop Azure Event-based and Message-based Solutions

Preparing Video For Download...