For AI the archetypes that come to mind are the moonshot projects: autonomous driving, real time simultaneous translation, etc. These are decade long projects that consume billions in investment and reshape industries. They use breakthrough new technology to implement radical solutions to huge problems.
Because these projects are the easiest to bring to mind, when we talk about trust and explainability we often frame those issues in the context of these moonshots. That is a mistake.
Consider the distribution of machine learning projects over the next decade, a tiny sliver will be these ultra expensive long term ground breaking projects. The median project might be a 5-man month effort to optimize the test protocols on a manufacturing line or an effort to get a modest improvement on a narrow category of medical image recognition based on re-tuning an existing model. These projects use proven technologies in a modified context to implement incremental progress. In total these incremental everyday projects will drive progress as much as the moonshots. But our assumptions about what tools are required to efficiently execute them needs to be different.
With autonomous driving we will have a million miles of testing and a billion dollars of investment to build trust in that system. With the median machine learning project that just isn’t the case. We need to find techniques that can be applied to these everyday projects that deliver trust on modest budgets, without the huge brute force efforts applied to moonshots.
This is part of the reason that XAI matters, as we mature XAI techniques confidence, robustness and insights will be more readily available on reasonable budgets, which will broaden the range of machine learning projects that are truly practical.