Design Principles for Scalable Agents

Building Scalable Agentic Systems

Korey Stegared-Pace

Senior AI Cloud Advocate, Microsoft

Scalable or not scalable?

Scalable or not scalable?

Scalable or not scalable?

Scalable or not scalable?

The pillars of scalable agentic systems

Robust infrastructure and tooling

• Compute: Resources for running code

 

• Storage: Storing states, logs, and history

Robust infrastructure and tooling

Reliable Deployment Pipelines

Established Agent Tooling

Modularity

Designing systems as separate independent components to create a larger whole

• Enables independent design, development, and maintenance

Modularity

Designing systems as separate independent components to create a larger whole

• Enables independent design, development, and maintenance

Modularity

Designing systems as separate independent components to create a larger whole

• Enables independent design, development, and maintenance

Modularity in agents: Software modularity

Modularity in agents: Multi-agents

Modularity in agents: Multi-agents

Example: Customer support multi-agent

Sneak peak at Chapter 2

¹ https://langchain-ai.github.io/langgraph/concepts/multi_agent/#multi-agent-architectures

Continuous evaluation and feedback loops

 

• Comprehensive monitoring

 

Continuous evaluation and feedback loops

 

• Comprehensive monitoring
• Common metrics:
  • Success rate
  • Latency
  • Errors

 

Continuous evaluation and feedback loops

 

• Comprehensive monitoring
• Common metrics:
  • Success rate
  • Latency
  • Errors
• Collect user feedback

 

Let's practice!

Building Scalable Agentic Systems