Standardized measurement and its discontents

At the faculty lounge, Professor Bernard Burk of the University of North Carolina echoes questions raised earlier by Professor Merritt of Ohio State about whether it is unethical or misleading for law schools to report employment using the international standard definition of employment.   I have discussed these issues extensively before.*

Employment statistics are primarily useful for purposes of comparing alternatives.  Comparison requires standard measurements.  Standardization is efficient because it reduces the number of definitions that must be learned to use data.  The standard definition of employment is meaningful and useful because, notwithstanding preferences for particular kinds of work, a job of some kind is generally preferable to no job at all.  This does not mean that employment is the only measurement one should consider, but rather that it is a useful measurement.

Because international standards exist, it is not necessary to explain to a college graduate what a centimeter means when describing the length of an object.  Similarly it is not necessary to explain to college graduates contemplating law school what employment means when using the international standard definition of employment.**

College educated individuals who are unfamiliar with standard terminology can easily look up or inquire about the relevant definitions, and once they have learned, can begin to understand a world of data. The standard definitions of employment and unemployment can be quickly discovered through intuitive internet searches. (see searches for unemployment and employment definitions) These definitions are neither obscure nor technically challenging. 

In addition, many law schools disclose bar passage rates that are lower than their employment rates.  It seems doubtful that many college educated adults contemplating law school—in particular, the subset basing their decisions on outcome data such as employment and bar passage rates—would assume that every law graduate who is employed shortly after graduation is working as a lawyer when many of those graduates cannot legally practice law.

Critiquing international standardized measurements as inherently immoral is not without precedent.   

According to Martin Gardner, during the 1800s, a U.S. group attacked the French metric system as atheistic and immoral. 

“The president of the Ohio group, a civil engineer who prided himself on having an arm exactly one cubit in length, had this to say . .  . : "We believe our work to be of God; we are actuated by no selfish or mercenary motive. We depreciate personal antagonisms of every kind, but we proclaim a ceaseless antagonism to that great evil, the French Metric System. . .The jests of the ignorant and the ridicule of the prejudiced, fall harmless upon us and deserve no notice. . . It is the Battle of the Standards. May our banner be ever upheld in the cause of Truth, Freedom, and Universal Brotherhood, founded upon a just weight and a just measure, which alone are acceptable to the Lord." “

“A later issue printed the words and music of a song, the fourth verse of which ran:

        

        Then down with every "metric" scheme

        Taught by the foreign school,

        We'll worship still our Father's God!

        And keep our Father's "rule"!

        A perfect inch, a perfect pint,

        The Anglo's honest pound,

        Shall hold their place upon the earth,

        Till time's last trump shall sound!”

 

A catchy tune, although I’m not sure it’s one many scientists, mathematicians or engineers would appreciate!

Many thoughtful people believe the U.S.’s non-standard approach to measurement undermines U.S. competitiveness in science, math, engineering, and industry.  Time is wasted learning and converting to and from a redundant and inefficient measurement system.  This entails opportunity cost and leads to unnecessary and avoidable errors.

Law schools, the American Bar Association, and the National Association for Law Placement would be better served by using standard definitions for labor market measurements when standard definitions are available and widely in use elsewhere, or at least labeling non-standard definitions with names that will not be readily confused with standard definitions.

The ABA currently requires law schools to describe individuals as “Unemployed” who under standard definitions would be defined as either “Not in Labor Force” or “Unemployed.”  In other words, “unemployment” as reported under ABA definitions will be higher than unemployment under the standard and most widely used government definition.  A number of people have been confused by this, incorrectly claiming that “unemployment” for law graduates is unusually high in comparison to everyone else.  In fact, under consistent measurements, the fraction of recent law graduates who are employed is higher than the overall proportion of the population that is employed.   (Law graduates also do relatively well on the percent employed full-time).

I agree with Professor Burk that additional information about occupational categories could be useful to some users of data.  However, I do not agree that presenting standard summary statistics is inherently misleading or unethical, particularly for the sophisticated audience using the data —college educated, internet savvy adults.

 

 

* For previous posts on standard employment measurements and data reporting practices, see

http://leiterlawschool.typepad.com/leiter/2015/04/what-employment-and-unemployment-mean-michael-simkovic.html

http://leiterlawschool.typepad.com/leiter/2015/04/compared-to-what-michael-simkovic.html

http://leiterlawschool.typepad.com/leiter/2015/04/empower-the-census-bureau-to-collect-graduate-employment-data-michael-simkovic.html

http://leiterlawschool.typepad.com/leiter/2015/04/opportunities-college-majors-and-occupations-michael-simkovic.html

http://leiterlawschool.typepad.com/leiter/2015/04/information-overload-and-response-rates-michael-simkovic.html

http://leiterlawschool.typepad.com/leiter/guest-blogger-michael-simkovic/

The standard definition of employment has been promulgated by the International Labor Organization and adopted by the United States Census and the Bureau of Labor Statistics and other data collection agencies around the world.  The standard definition is used nearly universally by those who present employment statistics for other academic programs to which a law degree might be compared.

Under the standard definition of employment, graduates of engineering or history programs who are not working as engineers or historians (or in related fields) will be counted as employed.  They will also be counted as employed if they are working part time rather than full-time.  Similarly, law graduates will be counted as employed whether they are working as lawyers or not, and whether they are working full time or part time.  The standard definition is consistent across educational programs and therefore useful for purposes of comparison.

As I have noted before, a single measure may not provide all of the information that might be of interest.  One solution to this problem is to use multiple standardized measurements, such as measures of earnings and wages as well as employment.  Earnings and wages measurements convey important information about job quality, and can be used across fields.  When established standardized measurements are incapable of providing the information desired, researchers consider using non-standard measurements.

Additional information about occupational categories can be useful and interesting, but that does not mean that most users of data were previously misled by its absence. 

**In many contexts, additional information might be useful to at least a minority of the audience, but too much additional information could be distracting, unnecessary, or even patronizing for the majority.  There is a distinction between the potential for additional information to be useful and the stronger claim that summary information is inherently misleading.