How AI Agents Scale (and Fail)

Building Scalable Agentic Systems

Korey Stegared-Pace

Senior AI Cloud Advocate, Microsoft

The confident start

The confident start

The confident start

The confident start

The confident start

The confident start

Trouble at scale

Failure mode #1 - Fragile evaluation

¹ nip over = British slang for a quick visit

Failure mode #1 - Fragile evaluation

¹ nip over = British slang for a quick visit

Failure mode #1 - Fragile evaluation

¹ nip over = British slang for a quick visit

Failure mode #2 - Intent drift

Failure mode #2 - Intent drift

Failure mode #3 - Undesirable feedback loops

Upvoting → jokes, sarcasm, etc.

Result → truth is deprioritized

Define clear metrics for all aspects of performance

• Truthfulness
• Clarity
• Tone

Failure mode #4 - Latency bottlenecks

Failure mode #4 - Latency bottlenecks

Failure mode #4 - Latency bottlenecks

Failure mode #4 - Latency bottlenecks

Failure mode #4 - Latency bottlenecks

Failure mode #4 - Latency bottlenecks

Failure mode #5 - Cost explosion

Failure mode #5 - Cost explosion

Let's practice!

Building Scalable Agentic Systems