Empirical Analysis: What's Market?

In negotiating contracts, lawyers will often assert that certain terms are "market." On what basis can such a claim be made? How many documents are needed for a statistically relevant sample set? There is, in fact, only a limited amount of empirical analysis that is publically available. This post examines three academic studies.

An Empirical Analysis of CEO Employment Contracts: What Do Top Executives Bargain For?

Stewart Schwab and Randall Thomas of Cornell and Vanderbilt examine a set of 375 employment contracts, focusing on four primary data points: (a) cause, (b) good reason, (c) noncompete, (d) arbitration, and (e) stock option restrictions. The authors conclude that CEO employment agreements are weighted in favor of the CEO's. They "found evidence that CEOs have significant bargaining power in their negotiations over the terms of their employment contracts and change-of-control agreements. Furthermore, the differences between these CEO contracts and these of other corporate workers seem stark."

An Empirical Examination of Business Outsourcing Transactions

George S. Geis of University of Virginia School of Law analyzes 60 onshore and offshore outsourcing transactions in pursuit of two foundational questions: (1) how do parties write these deals; and (2) why do we observe major differences in governance terms? He concludes that outsourcing transactions are extremely diverse determining that the contracts are "[h]ighly variable, making it difficulty to draw any solid conclusions."

What’s in a Standard Form Contract? An Empirical Analysis of Software License Agreements

The most extensive study is an analysis by Florencia Marotta-Wurgler of New York University School of Law. She and her team analyzed 647 End User License Agreements (EULAs) from 598 companies. The study analyzes "23 common and important standard terms that allocate rights and risks between buyers and sellers." The study shows a significant bias in favor of the seller as shown in a Bias Distribution chart.



"An immediate conclusion is that the vast majority of the contracts in our sample are more pro-seller relative to the default rules of Article 2 of the UCC. Although EULA terms vary greatly across software markets, I find that larger and (controlling for size) younger firms tend to have more pro-seller terms than smaller and older companies. I find no evidence that firms offer worse terms with software targeted to the general public versus software targeted to larger business or corporate users."

Statistical Significance and Statistical Range

There are a number of tools on the web to calculate sample sizes. The key inputs are the size of the population (or total collection) size; sample size and answer range or spread. Using these calculators and assuming collection sizes ranging from 10,000 to 50,000 then sample size ranging from 250 to 500, as two of the studies used, give a margin of error of less than 5%.

I can confirm that when analyzing publicly available documents from EDGAR, typically drawn from a wide range of authors, a sample set of 250 to 500 documents appears to yield satisfactory levels of confidence. Much smaller sample sizes are needed, however, when analyzing documents from a single organization or documents that exhibit high consistency. In these cases, the sample size can be as few as 20 documents. It turns out that you need fewer merger agreements than employment contracts, because merger agreements are much more consistent. See, Measuring the Consistency of Transactional Documents; Ron Friedmann; It's Time To Rethink The Lawyer's Role In Dealmaking: Start By Facing Up To The New Realities, Robert A. Profusek and Lyle G. Ganske, Jones Day