Measuring the POC

Implementing AI Solutions in Business

Jacob H. Marquez

Data Scientist

How to define success?

$$

How to define success?

$$

How to define success?

$$

How to define success?

$$

Measuring business value

Measuring business value

Scheduling chat bot

Metrics

• Agent time spent scheduling
• Number of appointments scheduled
• Percentage of interested clients scheduled
• Time to booked appointment

OBJECTIVES

• Reduce customer service agent hours spent on scheduling
• More time for high-level tasks
• Increase the number of scheduled clients

• Percentage of canceled appointments rescheduled

Measuring the AI model

Measuring the AI model

• Accuracy: percentage of correct predictions
• Important metric for most AI solutions

Measuring the AI model

• Relevant results based on input
• Direct and indirect

Explainability of the AI

Explainability is the ability to understand the decision-making process of the AI solution.

• Fairness, transparency, bias, and safety

Fairness

Transparency

Measuring solution performance

Measuring solution performance

Measuring solution performance

Measuring solution performance

Metrics

• Load Time
• Time to appointment confirmation
• Customer satisfaction

Let's practice!

Implementing AI Solutions in Business