The Power of Statistics

We all know the catch phrase “lies, damned lies, and statistics.” It is typically used to mean that statistics can be manipulated to support any proposition and should therefore be viewed--presumably--with some degree of skepticism.

And that’s a concern for those of us seeking to harness the power of statistics to analyze contracts and other documents.

So I was intrigued when I came across Michael Lewis’s book Moneyball-The Art of Winning an Unfair Game (2003). I enjoyed Liar’s Poker (1989) and look forward to reading The Big Short: Inside the Doomsday Machine (2010). I confess to reading a synopsis of Moneyball on Wikipeadia. It summarizes the book:

“The central premise of Moneyball is that the collected wisdom of baseball insiders (including players, managers, coaches, scouts, and the front office) over the past century is subjective and often flawed. Statistics such as stolen bases, runs batted in, and batting average, typically used to gauge players, are relics of a 19th century view of the game and the statistics that were available at the time. The book argues that the Oakland A's' front office took advantage of more empirical gauges of player performance to field a team that could compete successfully against richer competitors in Major League Baseball.”

The application of math and statistics to literature, art, and even sports is often greeted with skepticism, doubt and sometimes nostalgia. For example, Bill Simmons of ESPN writes in Finally joining the revolution: “I longed for the old days when you could say things like, ‘I hate watching J.D. Drew -- when is that contract going to end?’ and there wasn't some dude lurking behind me with Drew's stellar OPS, VORP and WAR numbers saying, "Well, actually ... "

Simmons is now a convert and is convinced by the insights offered by sabermetrics.

Reading these articles also made me think of a piece in the Wall Street Journal titled, Contract Doesn't Let Board Fire Mann CEO for Jail Term, in which the authors appear outraged that “[u]nder the terms of the employment agreement for Louis Lower, the jailed chief executive of Horace Mann Educators Corp., serving a 60-day sentence on a drunken-driving charge isn't grounds for firing—or for losing any pay.”

If the authors harnessed the power of empirical evidence, they would discover that termination for substance abuse is rare, appearing in less than 5% of CEO contracts. Stewart J. Schwab and Randall S. Thomas report on their analysis in An Empirical Analysis of CEO Contracts: What Do Top Executives Bargain For?

Some contracts may include commission of a felony as cause for dismissal and termination of pay; but typically a DUI is a misdemeanor and in some states it carries a mandatory jail sentence.

While contract analysis is a very new field, it will likely offer new insights on precedent that we thought we understood thoroughly and it may question our long-held beliefs.

Creating Forms by Committee and Consensus

An earlier post—The Fastest Way to Create a Form—describes a process to identify standards using technology to find the most conforming document and the most conforming clauses to quickly establish a baseline and then bring it up to practice standards by supplementing the baseline with missing and divergent clauses.

In other cases, a consensus building approach might be preferred. Working with ABA Model Intellectual Property Task Force, we have proposed an efficient, technology-enabled approach to develop model forms based on a potentially diverse set of documents and a wide range of individual interests.

1. Create the reference set from a source set of documents. In this case kiiac examined 105 Intellectual Property Security Agreements. (If you are interested in viewing the set, please send me an email and I will provide you with access credentials).

2. Provide an overview of the process. John Murdock, partner at Bradley Arant, created a YouTube video explaining the goals of the project and how the group can achieve consensus and develop the model form. John created the video in PowerPoint and rendered it with Moyea PPT to Video Converter. (FYI: A good resource for ideas about putting PPT content on the web is: http://www.dvd-ppt-slideshow.com/ppt-to-dvd-tips/put-powerpoint-on-web.html.)

3. Gather feedback on drafting options. The consensus building approach needs to work both during in-person meetings (that may take occur twice a year) and between such dates. In order to move the process along more efficiently, the key drafting choices are identified and the group’s opinions gathered through a Survey Monkey.

The approach to identifying the drafting questions used the technology to examine where consensus exists and where there is a range of opinions. One of the goals of the MIPSA project is brevity. While it would be easy to use the technology to identify all possible variants and string together a comprehensive lists, the group determined to use the process to streamline and clarify contract language.

Where kiiac finds that all documents contain a particular provision, it is considered required. But where a particular provision is found in the some, but not all agreements, it is considered optional or deal specific. The group is then asked whether it should be included in the model. For example, 100% of the grant clauses contain a grant of a security interest, while a smaller percentage include language to “assign,” “pledge,” or “hypothecate” the collateral. Here is a screen shot of the Survey Monkey.

The first step to create a model for the grant clause will run through December. I’ll report back on the results and level of participation.

Contract Readability -- Part 1

What makes a document easier to read and understand by both humans and machines? Does readability—or lack thereof—affect document quality?

The next few blog posts will focus on drafting practices to assure quality and avoid mistakes. The purpose is to try and identify best—and worse practices—and discuss whether they have any significant impact on quality.

The golden rules of human readability are generally well understood. JoAnn Hackos and Dawn Stephens in Standards for Online Communication (1997) summarizes the rules:

• “Use short, simple, familiar words.
• Avoid jargon.
• Use culture-and-gender-neutral language.
• Use correct grammar, punctuation, and spelling.
• Use simple sentences, active voice, and present tense.
• Begin instructions in the imperative mode by starting sentences with an action verb.
• Use simple graphic elements such as bulleted lists and numbered steps to make information visually accessible.”

But, lawyers, as typified by a well-known Dilbert cartoon, are well known to violate each rule with reckless abandon.

There’s a fabulous site you can use to test the readability of your contracts.

Online Readability Test

It estimates the number of years of formal education required to understand text on first reading using the Gunning Fog index. The software reported that for one merger agreement I processed, you will need nearly 23 years of schooling. It will be interesting to discover the most incomprehensible document.

Contractual clarity has long been the goal of the Plain English movement. But, it has made only limited progress. In the next post, I’ll address whether readability matters.

The Fastest Way to Create a Model Form

We all know the most inefficient method: ask attorneys for their best samples and form a committee to review the documents. Virtually all firms have tried this approach: all with same result.

Now, with aid of technology, there are much more efficient methods that can produce a model document in days; sometimes hours.

1. Search the document management or file system for the desired document type. Vetting or editorial review is not required. The software should identify the required documents and parse the clauses to identify the core, non-negotiated language.

2. Analyze the set of documents and

· capture the document outline, indentifying the frequency of each provision;

· capture a library of clauses for each provision, identifying language consistency

3. Find the most conforming document. This is the document that contains all common terms and the most standard clause language. Most importantly, it will have been drafted so that all its clauses and defined terms work together. It is also useful to confirm that this model is also widely used at the firm (i.e. are there a number of similar documents). If so, it is not only the most conforming document, but also the de facto standard.

4. Find the clauses in the most conforming document that might be missing. These are the clauses that occur frequently in the document set, but are absent from the most conforming document. Determine whether to add the most conforming clause from the clause library or another clause example, if desired.

5. Find the clauses in the most conforming document that diverge from the standard core language. These are the clauses that may be missing some standard language or may contain deal-specific language. Determine whether to replace these clauses with the most conforming clauses or another clause example, if desired.

In a recent application of this approach, one top-level clause was indentified as missing and four clauses were identified as divergent. The lawyer review process to create the model form took 45 minutes.

A concise history of the use of forms in the legal profession

The earliest use of model forms can be traced back to 1392, when the Worshipful Company of Scriveners first employed scribes to beautifully copy one document to the next. WHEREAS, the aforementioned forms were penned in Ye 'Olde English, the innovation is remembered mainly for its use of hand-scrolled, script font.
	In these early years, the pace of innovation is swift. The next major milestone came in 1450, when Gutenberg invents the printing press, enabling the mass production of form books. In terms of drafting practice, the distinguishing feature of the innovation is the use of the blackletter font.
However, for nearly 400 years innovation is shunned as drafting enters the dark ages. It is not until 1829 that the next major innovation arrives with the invention of the typewriter, enabling lawyers to mass produce their own forms. The innovation is distinguished mainly by the eventual adoption of the courier font.
	Finally, in 1976, Jobs and Wosniak invent the personal computer, facilitating the automated mass production--and personalization--of legal forms. However, most contracts are still drafted by copying the last document and laboriously making the necessary changes for the next client. The key innovation is the wholesale transition from a fixed width to a proportional font with the introduction of Times New Roman.
One can only imagine what new discoveries will be made in the centuries to come.

ILTA-When will lawyers find the time to practice?

ILTA was a great experience: totally exhausting. Overall, I agree with Peter Krakaur’s (CKO Orrick) assessment that the presentations clearly showed we are embarking on a period of significant change, but also a renewed sense of energy or excitement.

For 10 years, there has been little new to talk about. There have many great ideas, but few instances of real change. For example, it was rare to see screen shots of examples of KM in practice (by which I mean not simply search and retrieval). Now there appears to be broad acceptance that change is happening, a few firms are embracing alternative billing and some are even moving towards performance based review for associates (not just number of hours).

To this new world, comes new technology such as profitability analysis and project management. Session after session highlighted new technology and placed some or all the burden of adapting to the “new normal” on the attorney. I began to wonder when lawyers will find time to practice. After 30 years of innovation, we require attorneys to be:

• Competent secretaries
• Skilled researchers
• Adept at numerous software applications
• Seasoned marketers, and
• Capable project managers

Lawyer circa. 1980	Lawyer 2010

We may need to start asking ourselves how many of these tasks can be offloaded to dedicated professional staff or handled by technology.

Contract Analysis—A Force for Contractual Bloat or Streamlining?

I would like to believe a significant event took place over the weekend in San Francisco—contract analysis moved beyond static surveys to serve as an interactive tool to evaluate and guide best practices.

In a session at the ABA Conference, John Murdock, a Partner at Bradley Arant Boult Cummings LLP and I presented an analysis of 105 Intellectual Property Security Agreements. I provided the statistical analysis; John presented the legal analysis of the documents. John titled his talk: "Did 29% of the lender's get it wrong; or did the rest of us blow out some IP." Here are links to John's and my presentations.

Our process as part of a Task Force to develop a model form was to:

(a) determine the structure of the form by identifying required and optional elements of the agreements;

(b) determine the content of each element by identifying the range contractual language; and

(c) establish a procedure for members of the task force to express their opinions as to why each optional provision and term should be included or excluded from the model.

For example, in the case of the grant clause, 100% of the agreements contain a grant of the security interest. Each optional additional elements is shown in italics and its frequency is displayed in parenthesis.

Borrower irrevocably grants to Secured Party a continuing security interest in and

pledges, (42%)
assigns, (23%)
hypothecates, (19%)
mortgages, (12%)
grants a lien upon, (5%)
and grants a right of setoff against, (9%)
the Collateral
with power of sale (8%)

The discussion raised the interesting question, will contract analysis tend towards producing ever longer agreements or will it be a force to consolidate language? I hope it is a means to rein in language bloat. Indeed, this is process we hope to put in place.

The mechanisms to push toward comprehensiveness and streamlined agreements both exist in the software. The tools to create the agreement structure will find all standard contract elements and all deal-specific variants. The tools to examine the contractual language for each provision identify the core, non-negotiated language and all variations. Whether the tool is used to find the model that seeks to address all possible contingencies or succinctly and clearly address the specific needs of the transaction is in the hands of the user (or committee).

Contract analysis tells you what provisions are required—they appear in 100% of the documents—and what provisions are optional. The Task Force plans to review each provision and all language and justify why each term needs to be in place, rather than simply take the approach that others have used the terms, therefore they must be important under some circumstances and should not be removed.

The capacity to review and analyze legal agreements has always existed. We can read large sample sets of documents, tabulate and publish the results. Automated systems can do it much faster (KIIAC can process the 105 agreements in less than 2 minutes). It can do it across all deal terms and in great detail. There is, however, a more subtle difference.

Computer analysis does not simply collect information on the frequency of occurrence, it gives us the capacity to easily examine the full range of alternatives and make judgments about which will deliver the better results. The observation might simply be that best practices are more easily identified when we are exposed to a dynamic and interactive medium compared to viewing static survey tabulations.