From Proof-of-Concept to Production

Building Scalable Agentic Systems

Korey Stegared-Pace

Senior AI Cloud Advocate, Microsoft

The bigger picture

Step 1: Validate Real Interactions

 

Testing leads to failure, and failure leads to understanding.

— Burt Rutan

Step 1: Validate Real Interactions

 

¹ nip over = British slang for a quick visit

Step 2: Test Everything: Unit tests

Step 2: Test Everything: Integration tests

Step 2: Test Everything: Evaluate system prompts

Step 2: Test Everything: Subjective evaluations

Step 2: Test Everything: Subjective evaluations

Step 3: Guardrails and Observability

 

Step 3: Guardrails and Observability

 

Step 4: Shadow Deployments

 

Step 5: Deployment Strategies

Step 5: Deployment Strategies

Step 5: Deployment Strategies

Final Checklist

Let's practice!

Building Scalable Agentic Systems