Model-agnostic explanations

Explainable Artificial Intelligence (XAI) Concepts

Folkert Stijnman

ML Engineer

Black box models

• We know what goes in
• We know what comes out
• In between are a lot of internal parameters

Global and local explanations

 

• SHAP (SHapley Additive exPlanations)
• LIME (Local Interpretable Model-agnostic Explanations)

 

SHAP (SHapley Additive exPlanations)

 

• Acts like a detective
• Look at each feature
• Helps to figure out the relative impact of each ingredient on the final outcome

Shapley values

• Concept from cooperative game theory
• Assess impact of features
• Which features matter for the model's prediction

SHAP image recognition

LIME (Local Interpretable Model-agnostic Explanations)

• Acts like a scientist
• Change input to see outcome
• What if we change some pixels in this image?

Building a self-driving car

SHAP

• Global understanding
• Impact of car features on overall performance

LIME

• Local understanding
• Impact of car features on a specific decision

Let's practice!

Explainable Artificial Intelligence (XAI) Concepts