Explainability and interpretability

Understanding Artificial Intelligence

Iason Prassides

Content Developer, DataCamp

Explainability and interpretability

Explainability: humans' ability to access and understand AI outputs, e.g. predictions, decisions

Explainability in an ML penguin classifier

Interpretability: understand AI systems' internal processes: algorithm, model, data workflow

Interpretability in a decision tree-based ML model

White-box vs black-box AI systems

White-box: transparent and easily interpretable models/systems

Linear regression as a white box model

White-box vs black-box AI systems

White-box: transparent and easily interpretable models/systems

Linear regression and decision tree

White-box vs black-box AI systems

White-box: transparent and easily interpretable models/systems

Linear regression and decision tree

White-box vs black-box AI systems

Black-box: higher complexity, little or no degree of understandability

Deep neural network as a black-box model

White-box vs black-box AI systems

Black-box: higher complexity, little or no degree of understandability

Deep neural network as a black-box model

Basic Explainable AI (XAI) tools

XAI: methods and tools to increase AI systems and models' transparency and explainability

Model introspection: examining internal model parameters to understand decisions

Basic Explainable AI (XAI) tools

XAI: methods and tools to increase AI systems and models' transparency and explainability

Model introspection: examining internal model parameters to understand decisions

Model documentation: shareable architecture and design considerations

Basic Explainable AI (XAI) tools

XAI: methods and tools to increase AI systems and models' transparency and explainability

Model introspection: examining internal model parameters to understand decisions

Model documentation: shareable architecture and design considerations

Model visualization: human-friendly representation of data features and model outputs

XAI tools: feature importance

Feature importance: impact or contribution of features (predictors) in model outputs

Understand how data-driven models (ML/DL) make decisions
Detect and mitigate issues, e.g. biases
Impact on model performance if a feature were removed

SHAP (SHapley Additive exPlanations)

Feature importance visualizations toolbox

SHAP visualizations for explainability

XAI tools: feature importance

SHAP (SHapley Additive exPlanations)

SHAP visualizations for explainability

XAI tools: feature importance

SHAP (SHapley Additive exPlanations)

SHAP visualizations for explainability

XAI tools: feature importance

SHAP (SHapley Additive exPanations)

SHAP visualizations for explainability

Practical implications of XAI

Algorithmic transparency:
- How algorithms process data and make decisions

Local and global interpretability:
- Understand system behavior for a specific prediction, vs
- Understand system overall behavior on a dataset or problem

Ethical considerations:
- XAI to address ethical AI concerns: biases, discrimination, compliance, etc.

Human-AI collaboration:
- Reliable collaboration based on trust and feedback

Let's practice!

Understanding Artificial Intelligence