The performance of a machine learning models in the lab does not always translate to the utility of the target system in production.
Gurjeet Singh points out that for Enterprise use cases the trust challenge can stop a high performance algo from contributing to a high utility result.
In general, the better the machine learning system, the blacker the box – that is, the harder it is to decipher what is going on inside the model.
The performance of these models are outstanding, but the utility of the models can be severely limited. You cannot put a black box into production against mission critical tasks if you cannot explain what is going on. This is true across any industry – from hospitals to hedge funds. As a result, much of the recent work in machine learning works on perception based problems – where the errors of the model are not mission-critical. Imagine a machine learning model to recommend images of cats – the user won’t mind if a picture of a dog slips in. The bigger the stakes, however, the more important understanding the behavior becomes.
For AI to take hold in the enterprise it has to be explainable.
Christoph Molnar highlights how overly focusing on model performance can hurt the interpretability that enterprises need:
If you only focus on performance, you automatically will get more and more opaque models. Just have a look at interviews with winners on the kaggle.com machine learning competition platform: The winning models were mostly ensembles of models or very complex models like boosted trees or deep neural networks. – Interpretable Machine Learning
In the end you need a balance between max performance and max interpretability to achieve max enterprise utility.