Takeaways from “Anchors” black box explanation announcement

Screenshot 2018-02-14 12.50.27

LIME (“Local Interpretable Model-Agnostic Explanations”) has been the highest profile Explainable AI technique that could be applied across a broad range of black box systems.  LIME was clearly a good early step towards XAI.

In his e-book “Interpretable Machine Learning”, Christoph Molnar provides a summary of LIME and points out some concerns with this approach:

“As always, the devil’s in the details. In a high-dimensional space, defining a neighbourhood is not trivial. Distance measures are quite arbitrary and distances in different dimensions (aka features) might not be comparable at all. How big should the neighbourhood be? If it is too small, then there might be no difference in the predictions of the machine learning model at all. LIME currently has a hard coded kernel and kernel width, which define the neighbourhood, and there is no answer how to figure out the best kernel or how to find the optimal width.”
— Christoph Molnar  “Interpretable Machine Learning

The team behind LIME presented their next generation technique “Anchors: High-Precision Model-Agnostic Explanations” at AAAI-2018.  This is another big step in maturing our options for model-agnostic explanation systems. 

Here are four key takeaways:

Good litmus test for explanatory power

What makes a good explanation is a matter of debate, it depends on the audience and the goal.  The explanation that will stand up in court might be different from the explanation that best helps you troubleshoot data issues.

In their paper the team proposes on a good litmus test for general purpose explanations aimed at layman end users:

A question at the core of interpretability is whether humans understand a model enough to make accurate predictions about its behavior on unseen instances … one can hardly say they understand a model if they consistently think they know what it will do, but are often mistaken.
— Anchors: High-Precision Model-Agnostic Explanations

Addresses biggest weaknesses in LIME

Their paper does a good job of articulating what we have considered to be the biggest concerns about LIME.

  • Inability to know how widely one can apply a “local” explanation.

… explanations are in some way local, it is not clear whether they apply to an unseen instance. In other words, their coverage (region where explanation applies) is unclear …
— Anchors: High-Precision Model-Agnostic Explanations

  • Dependence on assumptions of linearity.

… even the local behavior of a model may be extremely non-linear, leading to poor linear approximations and users being potentially misled as to how the model will behave … Linear explanations are also harder to parse and apply than simple rules, as they involve mental calculations.
— Anchors: High-Precision Model-Agnostic Explanations

  • Dependence on a concept of closeness/distance that was vague and ill-defined from end users point of view.

Finally, even if the blackbox model is approximately linear locally, humans may not be able to compute distance functions “correctly”, and may think that a local explanation applies when it does not – the “unclear coverage” problem.
— Anchors: High-Precision Model-Agnostic Explanations

The Anchors approach they propose is designed to address the above concerns.

Efficiency matters, good enough is likely good enough

It was clear from their paper that the team was thoughtful about achieving sufficient computational efficiency to make this approach practical.  However, as is often the case the practical approach requires probabilistic methods that produces results that are not exhaustively and provably correct.  However, we are optimistic that these results will be good enough for a wide range of real world applications.

The number of all possible anchors is exponential, and it is intractable to solve this problem exactly … Although this approach does not directly compute the coverage, and instead tries to find the shortest anchor, we note that short anchors are likely to have a higher coverage, and require less effort from the users to understand … Since we cannot compute the true precision, we rely on samples … This problem can be formulated as an instance of pure exploration multi-armed bandit problem …
— Anchors: High-Precision Model-Agnostic Explanations

Flexibility is a strength

The team maintains this technique will support any black box model including:

… variety of machine learning tasks (classification, structured prediction, text generation) on a diverse set of domains (tabular, text, and images) …
— Anchors: High-Precision Model-Agnostic Explanations

Given the range of techniques in use and how rapidly they are evolving it is a real advantage if we don’t have to define and validate new model specific explanation techniques for each machine learning algorithm.

Singh, Ribeiro and Guestrin summarize their “Anchors” work this way:

By construction, anchors are not only faithful to the original model, but communicate its behavior to the user in such a way that virtually guarantees correct understanding, and thus high precision. Anchors are also able to capture non-linear behavior, even locally. In sum, the anchor approach combines the benefits of local model-agnostic explanations with the interpretability of rules, constructed in a way to best support human understanding.
— Anchors: High-Precision Model-Agnostic Explanations

If their technique lives up to these claims it will prove valuable.

Leave a Reply