Quantifying the Unquantifiable

First Dilbert and now college presidents… and it’s long past time. What am I talking about? As technology and instant calculation capabilities permeate U.S. society, Americans’ long-standing love of statistics and numbers has become an obsession, to the point that quantification crowds out qualification or anything that can’t be quantified, and we exalt these sacred numbers, from the Dow Jones to baseball batting averages to point spreads to SAT scores to cost-effectiveness in business and higher education to ranking colleges numerically… and thousands of other numbers. As a culture, we seem to be under the misapprehension that once we know the right numbers, we’ll somehow understand things better, be able to make the right decisions, set the right regulatory guidelines, pick the right college, determine whether there is or is not global warming…

Now, a number of college presidents are attacking the U.S. News and World Report collegiate ranking system. These aren’t the heads of fly-by-night diploma mills, but the presidents of rather prestigious institutions, some of which, heaven forbid, have already decided that SAT scores aren’t even a partial measure of a student’s abilities. With all the effort to quantify high school students to see who gets admitted where, is it any wonder that students respond with their own forms of quantification — adding community service projects, science projects, tutoring underprivileged students, athletics, internships, whatever they can find to boost that magic score that will get them accepted. And the treadmill continues in college as students and their parents press for higher grades, more prestigious summer jobs and internships, cram prep courses for the LSAT, MEDCAT, or GRE.

There is a place for numbers in society, and there are places where precision in such numbers is vital. I definitely want the components of my vehicle’s brakes machined precisely. I want medical measuring systems to be accurate and precise. I want architectural specifications, and the buildings constructed from them to match.

But there are more than a few places in society where numbers only provide a misleading and inaccurate assessment of quality. Nonetheless, numbers are being used there, despite the fact that the accuracy of the inputs is subject to such variance that the outputs, no matter what the mathematical models say, are less than statistically meaningless. There seems to be little understanding of the old GIGO model because of the continuing fascination with computer modeling and statistical analysis methodologies.

I’ve mentioned some examples before, such as rating the quality of the teaching of university professors on the basis of how many students they teach, or even worse, on anonymous student evaluations to which near-arbitrary numerical coefficients are attached. In certain disciplines, such as music, particularly on the undergraduate level, the competence and teaching expertise of the individual studio teacher is usually the most important single factor for the success of the individual student. In other disciplines, the laboratory facilities may be the most important, or the collective strength of a department faculty. Most of these factors are not accurately quantifiable, no matter what U.S. News and World Report claims.

Then, another set of examples comes from sports such as gymnastics and figure skating competitions, where the subjective analyses of judges are quantified in numerical terms, then calculated to hundredths or thousandths of a point. And how many times have audiences and seasoned professionals figuratively, if not literally, shaken their heads at the results.

In business, the emphasis on quantification has led to a short-term, profit-oriented mentality whose shortcomings are illustrated practically daily by the increasing revelations of back-dated stock options, misstated or incorrect financial statements, excessive executive pay, bookkeeping schemes to put expenses off the balance sheet. Yes, a business does have to make a profit to stay in operation, but emphasizing the quantifiable to the exclusion of all other factors leads inevitably to excess. What about fair-dealing, open communications, good service, reliability in products and services?

In entertainment, across the board, the issue is the ultimate in quantification — what sort of movies, games, programs, and music will appeal to the most people and bring in the most dollars, regardless of how crude, rude, and culturally repugnant they may be to tens of millions of Americans. Even more disturbing is the denigration of excellence as elitism and the pressure to exalt fame in any form increases, whether on one of the endless Survivor shows or on Fear Factor and various adaptations and clones. How does one quantify the loss of excellence, or the disillusionment with the United States occurring in other nations when people see what passes for entertainment in the USA? How does one quantify the loss of civility created by a media that increasingly seeks advertising revenues by fostering adversary journalism? How much crime and corruption results from this? What are the dollar costs in police and social remediation?

What about the areas of space and science? Yes, small unmanned satellites and space probes are certainly more cost-effective, but exactly how do they kindle a spirit of adventure and public support for space exploration? Aircraft design has now become a matter of how many bodies can be crammed into what space for maximum profit, a situation almost praised recently in the Wall Street Journal. Meanwhile, lost baggage claims, delayed flights, and customer complaints are at all-time highs and have been for the last three years.

There is a place for numbers and quantification, but let’s try to remember that not everything can be or should be quantified, because trying to quantify the unquantifiable, as Dilbert noted, is nothing more than a process of lying.