XAI Goals

The DARPA team well articulated the goals of XAI:

XAI will enable human users to understand, appropriately trust, and effectively manage the emerging generation of artificially intelligent partners.


Black Box Problem

Often users are unable to understand, appropriately trust, and effectively manage machine learning driven systems because they suffer from:

… the black box problem. … able to make statistically sound decisions, but they can’t easily explain how they made them …
— Can A.I. Be Taught to Explain Itself?

Addressing this black box problem will improve machine learning in three dimensions:


The finest machine learning system in the world is of only academic value if users ignore its results.  The quality of explanation delivered with a result has a huge impact on real world adoption.  Read more …


Black boxes are brittle.  It is hard to make an opaque system robust and adaptable.  If you don’t understand how it works in the lab how can you fully project how it might fail in field.  Read more …


Machine learning algorithms create “models” or real world behavior.  Insights into how that reality operates enable us to build better models.  Building useful models generates new insights.  Maintaining this positive exchange between model and insight is only fully possible when the results from the model come with explanations that are meaningful to a domain expert.  Read more …

What data scientists get wrong about explainability

The median data scientist sees explainability as an “irrational constraint that limits my ability to get max predictive power”.  You have to go one sigma to the right before you run into data scientists who clearly “get” XAI.

Here are ten examples of what the median data scientist gets wrong about the black box problem and explainability?  Read more …