AI Decision Support – Lawyer Edition

Screenshot 2018-02-28 10.46.04.png

More progress in moving AI to contribute to more complex and consequential tasks:

Competitors were given four hours to review five non-disclosure agreements (NDAs) and identify 30 legal issues, including arbitration, confidentiality of relationship, and indemnification. They were scored by how accurately they identified each issue.  Unfortunately for humanity, we lost the competition — badly.  The human lawyers achieved, on average, an 85 percent accuracy rate, while the AI achieved 95 percent accuracy. The AI also completed the task in 26 seconds, while the human lawyers took 92 minutes on average.
An AI just beat top lawyers at their own game

Note that the application here is really a decision support system rather than taking fully automated decision and actions.  The AI will guide the lawyer to what parts of an agreement need attention, but the lawyer will need to confirm if those are real issues and navigate the negotiation process required to get to an agreement.

These sorts of decision support systems generally require explainability of results.  In this case if the AI highlights a contract clause as problematic but does not explain why it is problematic than it is delivering only a fraction of the potential value.

Leave a Reply