Reviewing a standard contract for risk · Contested (since 2018)

In February 2018 a study run by the vendor LawGeex reported that its system spotted risky clauses in five unseen non-disclosure agreements with 94 percent accuracy against 85 percent for twenty United States trained corporate lawyers, in 26 seconds against an average of 92 minutes.

The task was narrow and the conditions favoured the machine. Five non-disclosure agreements, 153 paragraphs, a fixed list of issues to find, a system trained on tens of thousands of similar agreements, and a benchmark designed and paid for by the vendor. The best lawyer on the panel matched the machine at 94 percent, and the worst reached 67 percent, which says as much about the variance of tired human attention as about the software. What stays open is whether the result carries past the most standardised document in commercial practice. Nothing in the test touched negotiation, the decision to accept a residual risk in exchange for a deal, or the signature that makes the contract binding.