Challenges of responsible AI

Responsible AI Data Management

Maria Prokofieva

Lead ML Engineer

Responsible data management practices in the real world

Complex

Involves trade-offs

Professional judgement

Responsible AI: blue circle with a hand holding an illustration of an ai chip

Common trade-offs

Trade offs

Responsible AI dimensions

Business factors

Profit over fairness and privacy
Revenue over testing and security

Pre-trained models

Reduce costs
Save time and resources
No need for data collection and training
Might have biased training data
Lack transparency

pretrained models

Using pre-trained models

Due diligence on model source
Good reputation
Credibility
Review model documentation
Additional tests for fairness and bias

Metrics trade off

Accuracy over fairness
Even in balanced datasets

Accuracy trade-offs

Lower accuracy for specific groups
No account for data quality or quantity for underrepresented groups
Privacy reduces accuracy

Facial recognction

Robustness trade-offs

Metrics trade off

Robustness versus bias
Robustness versus fairness

Professional conduct and duties of care

Code of ethics and conduct
Guidance varies by country and organization
Responsibility, non-harm, fairness
User privacy and confidentiality
Positive impact on society
Maintain high standards
Develop robust and secure systems
Inclusive and non-discriminatory

ACM

¹ https://www.acm.org/code-of-ethics

Let's practice!

Responsible AI Data Management

Preparing Video For Download...