Measuring Commonality and Consistency—Do the Numbers Tell a Story?

Those who have read a few posts in the blog know that much of my research is based on pure statistics. And, of course, we know the adage: "lies, damn lies, and statistics." But, can they tell a story? Can they offer some insight into how lawyers use precedent and how precedent evolves over time?

Where clauses are repurposed from an earlier precedent, they remain very consistent. Is that because they are copied but not substantially changed, or is it because the language of a particular contractual provision can only be defined in a relatively limited number of ways?

Early research confirmed that the consistency of legal document can be measured. The results of the analysis uncovered an unexpected conclusion: the more sophisticated the underlying transaction, the more likely the document will be consistent. A merger agreement, for example, is more likely to be consistent than an employment agreement.

In order to further test these propositions, we must distinguish between: (a) the elements of a contract and (b) how each of those elements is defined. Mathematically we can measure these values by commonality and consistency. Commonality is calculated by the frequency of a particular clause. For example, we may find a governing law clause in 90% of the sample agreements, while a severability clause may appear in 50% of the documents. Consistency is calculated by the standard deviation of a factor itself calculated primarily by the commonality of the words in the clause.

I recently ran these statistical measures of commonality and consistency for merger agreements and intellectual property security agreements. The charts display the outline organization of the agreement and for each caption, the bar chart shows the commonality of the clause in the blue bar and its language consistency in the yellow bar. You can click on the images above or the links in this paragraph to see a larger (readable?) image.

In the case of the merger agreement, we see high commonality of clauses and high consistency of language. Indeed, the presence of these two factors indicates a market standard. However, in the case of the intellectual property security agreement we see only two common clauses: grant of security interest and rights and remedies. However, all clauses (whether common or uncommon) display a high degree of language consistency.

What observations can be draw? First, there is great opportunity to standardize (and enjoy the efficiencies thereof) in the reps, covenants and miscellaneous sections. It is truly remarkable the variation in the clause selected. Second, the greatest opportunities for standardization will likely come from building deal checklists, not just clause libraries. It appears that we generally agree as to the language of a given clause, but there is less agreement on what clauses should be included.