The 2017 James Madison Award Lecture: The Ethics of Counting

I was already working on a book about counting before I got the invitation to give the Madison Lecture, so the namesake could not have been better. After all, James Madison was the nation’s first quantitative political scientist. He is the one who came up with three-fifths as the correct number for valuing slaves in the federal census. ¹ He is the one who used that mere fraction to persuade northern and southern states to ratify the Constitution.

Madison even thought numbers could define political interests. In debates over the first census, he argued for counting the number of people engaged in commerce, agriculture, and manufacturing. How could legislators make policy to benefit the different parts of the community without knowing “the relative proportion of each, and the exact number of every division…?” If only legislators had more numbers at their disposal, Madison said, “they might rest their arguments on facts, instead of assertions and conjectures.” ²

Nothing could be more important to democracy than figuring out what interests are and how best to represent them. So today I want to talk about this mental leap from ideas about social reality to measures of social reality, and back again to ideas.

Whenever I begin to talk about this subject, people say something like, “So you mean how to lie with statistics?”

Yes and No. Darrell Huff, author of the 1957 classic How to Lie with Statistics, inspired a spate of books about numerical literacy. ³ The authors in this genre assume that numbers come into the world innocent. Unfortunately, wily people sometimes turn them to the dark side. Thus, Huff introduced his book as …

“. . . a sort of primer in ways to use statistics to deceive. It may seem altogether too much like a manual for swindlers. Perhaps I can justify it in the manner of the retired burglar whose published reminiscences amounted to a graduate course in how to pick a lock and muffle a footfall: The crooks already know these tricks; honest men must learn them in self defense.” ⁴

I will argue, instead, that statistics aren’t born with honest meanings that people later corrupt with deceptive packaging. Numbers are figments of our imagination, fictions really, no more true than poems or drawings. In this sense, all statistics are lies.

***

This lecture grew from a friendly challenge by Jens Blom Hansen, my colleague at Aarhus University in Denmark. After reading my draft book proposal, he wrote: “You’ve convinced me. Numbers are representations of underlying power structures and weapons in political fights. But,” he went on, “your enterprise is basically destructive. Can you move beyond destruction and give us something constructive?”

Jens posed a series of questions: “Are numbers only weapons? Can it make objective sense to count? Is there such a things as value-free counting? I realize that your answer is probably ‘no’ to all these questions, but I think it is fair to ask you say so explicitly. I guess that counting is here to stay, so how best do we live with it? Can you help us get counting right?”

This lecture is my first attempt at answering Jens. I can’t promise to get counting right, but I will try to get it better.

Here’s my plan:

Part 1. What does it mean to count?

Part 2. How do numbers get their meaning?

Part 3. How do numbers get their authority?

Part 4. How can counting change hearts and minds?

Part 5. Are there some things we shouldn’t count?

At the end of each section, I’ll think out loud about how can we count better. Jens was right: I don’t think counting can be value free, so my thoughts run to how we can make our values explicit, and to how we choose the values we incorporate in our numbers. I’m not sure “ethics” is the right word for this topic, but by the time we’re done, I hope you’ll understand why I use it.

PART 1: WHAT DOES IT MEAN TO COUNT?

“God made the integers and all else is the work of man.”

-Karl Kronecker, German mathematician (1828–1891) ⁵

The German anthropologist Karl Menninger defines counting as “assigning number words to things.” What do we do when we actually count? he asks.

“Before us lies a heap of peas which we wish to count . . . . We arrange them in a row, physically or mentally, touch the first one and say “one,” then touch the second one and say “two,” … [and so on until] the last one, and say “twenty two.” There are 22 peas in all. What have we actually done? We have assigned a word to each individual pea.” ⁶

This description of counting makes a certain sense because when we teach kids to count, the first thing we do is drill them on the number words in their language.

It’s easy enough to count the way Menninger says if some grown-up has already told you that everything in the heap is a pea (figure 1). But if I told you that this is a photo of ballots in the 2000 American presidential election, there’d be a question about “hanging chads” and whether those little runty things should be counted as votes.

Figure 1 Peas in Pods

Photo by Bill Ebbesen, https://en.wikipedia.org/wiki/Pea#/media/File:Peas_in_pods_-_Studio.jpg Creative Commons license: CC BY-SA 3.0

So how do we make decisions about what goes in the “heap” of things to be counted in the first place? To find out, I went around the corner to my local bookstore and headed straight for Dr. Seuss. For those of you who don’t know Dr. Seuss, his real name is Theodore Seuss Geisel and he was a physician better known for his fun, easy-to-read children’s books, full of kooky rhymes, made-up words, zany drawings, and irreverence for grown-ups.

When I found One Fish Two Fish Red Fish Blue Fish I was sure I’d found the best learn-to-count book and that it would explain how to count without a grown-up to get you started. ⁷ Here’s how it begins:

One fish, two fish, red fish, blue fish.

Black fish, blue fish old fish, new fish.

This one has a little star.

This one has a little car.

Say! What a lot of fish there are.

Did you notice anything peculiar? There are no number words after “two.” We get lots of attributes and wind up with the vague quantity word “a lot.” It’s as though Dr. Seuss has already given up counting after the first line because he’s captivated by something special about each fish.

But let’s continue. Maybe we’ll get beyond two.

Some are fast. Some are slow.

Some are high. And some are low.

Not one of them is like another.

Don’t ask us why.

Go ask your mother.

Well, no help from Dr. Seuss. He sends us back to the grown-ups.

I think Dr. Seuss meant this book as a message about celebrating differences and having fun together. But for me, One Fish Two Fish is also about the disintegration of counting.

Dr. Seuss never asks, “What makes them all fish? What is fish-ness?” That’s the unanswered question and the paradox we can’t resolve. He ignores differences to count them all as fish at the same time as he celebrates differences to count them all as fish. Here is the existential dilemma of counting: How can we possibly count things if not one of them is like another?

And here is where counting meets power. The only way to count is through intellectual coercion—to force things into categories by ignoring their differences and ultimately, their uniqueness.

By the time we enter graduate school, we’ve lost sight of the intellectual coercion entailed in counting. So let me refresh your memory by taking you back to pre-school. If you look at charts for teaching pre-school kids how to count, you’ll notice that in these elementary charts, the objects in a group are absolutely uniform in size, shape, color, adornments, and even their orientation on the page. Kids don’t have to ignore any differences to count these objects, because there aren’t any.

Once kids have mastered this much, we make it more complicated. We show them pictures of things that are very similar but have some variation. In the Sesame Street book, 1 2 3 Count with Elmo, we get the simple version and a more sophisticated version on the same page. ⁸ For the number ‘6,’ the Cookie Monster is shown with six glasses of milk and six cookies (figure 2). For the number ‘9,’ the Count points to nine lightning bolts and nine bats (figure 3). The milk glasses and lightening bolts are exactly the same in size, color, shape and orientation on the page, but no two cookies are alike and no two bats are alike. Some cookies have much more interesting frosting, with pink squiggles, chocolate dots or colored sprinkles, while others are more plain. The bats are different colors, their wings are in different positions, some have their mouths open and some have them closed. And some bats have teeth.

Figure 2 Counting with the Cookie Monster

© 2013, 2017 Sesame Workshop,® Sesame Street.® and associated characters, trademarks and design elements are owned and licensed by Sesame Workshop. All rights reserved. Figure title is the author’s own.

Figure 3 Counting with the Count

© 2013, 2017 Sesame Workshop,® Sesame Street.® and associated characters, trademarks and design elements are owned and licensed by Sesame Workshop. All rights reserved. Figure title is the author’s own.

To a kid, the differences in cookie frosting and bat mouths might be worth considering. “But wait!” the kid thinks to himself. “A grown-up lumped these different things together so I guess I’m supposed to consider them as the same.” Notice that when kids learn to count, they’re not just learning number words and symbols; they’re learning how adults see things.

Another classic type of counting worksheet shows several triangles, rectangles, circles and stars, with items of each shape appearing in different sizes and in different colors—small stars, big stars, red stars, blue stars, and so forth, rather like One Fish Two Fish. Somewhere on the page there are images of shapes and a blank space to fill in the number. Now it’s getting harder to count. There’s still an (imaginary) grown up in the room, but here the kid must make her own decisions. The grown up asks, “How many triangles are there?” or “ How many stars?” Somewhere along the way, the grown up has already taught the kid some clear-cut rules: “If it has three sides, it’s a triangle. Ignore color. Ignore size.”

So, counting isn’t just assigning number words to things. Counting requires classification. Only after classifying can we begin to count. Teaching kids to count is teaching them categorical thinking. “This is a that.”

Counting, to nail the point, is another way of making metaphors. Numbers—those things quantitative analysts love because they are supposedly precise and objective? We construct numbers by solidifying bits and pieces of the swirling miasma that is our real world. In the split second before the number was attached to the thing, the thing could go either way; it could be a this or it could be a that. Numbers are a magic wand that resolves ambiguity into one-ness.

Counting, to nail the point, is another way of making metaphors. Numbers—those things quantitative analysts love because they are supposedly precise and objective? We construct numbers by solidifying bits and pieces of the swirling miasma that is our real world.

If quantitative reasoning rests on poetic interpretation, on “seeing as,” then the quantitative / qualitative distinction vanishes. And quantitative thinking, like qualitative, is highly cultural. The hard part of counting isn’t memorizing “uno, dos, tres.” The hard part is learning to see the likenesses your culture deems important and to ignore the ones your culture deems incidental (figure 4).

Figure 4 Learning to See the Likenesses Your Culture Deems Important

Credit: Chuck Siler, Carrollton, Texas. February 18, 2011. Title is the author’s own.

Alright, enough with primary school. Let’s come back to the universities where we now sit. We do fancy statistics, regressions, and all that, but we’re just manipulating numbers that came into being by the same process that kids count bats or cookies: grouping things into piles and tagging them with numbers. If we have forgotten the human judgment and cultural conventions entailed in counting, that’s because counting was never taught to us that way. It was never taught to us that way because it was never taught to our teachers that way. Ad infinitum.

Now that we’ve learned how to count, it’s time to start our ethics lessons. Based on what I’ve said so far, here are four thoughts about how to get counting better:

Lesson One: Mixed Methods Aren’t the Answer.

Critics of quantitative methods would have us temper them with qualitative methods, and vice versa. But how can you cure the defects of a research method by combining it with another defective method? Two wrongs don’t make a right, as my mother used to say about our sibling squabbles. If you accept my argument that quantitative analysis rests on qualitative reasoning, then we need some kind of external standards for evaluating all methods. Those are what I’m reaching for. Not criteria about accuracy, rather criteria about rightness and wrongness, fairness and unfairness, justice and injustice.

Lesson Two: Does the Measure Actually Measure What It Purports to Measure?

Every course on research methods touches briefly on the topic of validity. Does the measure actually measure what you think it does? We’re taught to identify threats to validity and vanquish them. The authors of a popular textbook suggest “It’s easiest to maximize validity by adhering to the data and not allowing unobserved or unmeasurable concepts to get in the way.” ⁹ But, if you accept my interpretation of what it means to count, then the validity of a measure should always remain in question, because the raw data are already interpretations of a concept. When we use measures, and when we design them, we should acknowledge that a measure is only a representation of reality, and only one of many possible ones, at that. We’ll revisit validity at the end of Part 2.

Lesson Three: “Where Did the Idea for the Category or Concept Come From?”

Before you get too far into your counting enterprise, study the intellectual history of your concept. For example, gross domestic product, unemployment, immigrants, economic development, sustainability, or disability. When did people first start counting your concept, and why? Who did and who now does the counting? What are the counting rules, who made them, and how have they changed? All numbers have a social and intellectual history. Or several histories. We can’t ask every scholar or journalist who uses a number to rehearse the whole history, but we can ask for an acknowledgment that God didn’t create either numbers or categories.

Lesson Four: “What Did I Leave Out?”

When you create a proxy measure or rely on someone else’s, ask yourself: “What did I leave out?” Go back to Dr. Seuss. Remind yourself that “not one of them is like another.” What differences did you have to ignore in order to put things in the heap you counted? Don’t just toss off two or three things, wipe your hands, and declare, “I’m done with validity, let’s move on.” Force yourself to make a list, Seuss-like, of all the things that might be fish but didn’t make your cut.

Soon enough I’m going to ask you to justify your cuts, but before we get to that, we need to learn more about counting.

PART 2: HOW DO NUMBERS GET THEIR MEANING?

Numbers get their meaning partly from verbal and visual cues, the kinds of devices Darryl Huff unmasked in How to Lie With Statistics and Edward Tufte does in his books on infographics. ¹⁰ Jane Miller intends the Chicago Guide to Writing about Numbers to play the honest cop to Huff’s burglar, to teach people how to present their data accurately, leaving no room for misinterpretation. Here’s how she advises students to write about their quantitative data:

When writing about numbers, help your readers see where those numbers fit into the story you are telling—how they answer some question you have raised. A naked number sitting alone and uninterpreted is unlikely to accomplish its purpose. ¹¹

Oh, really? Do numbers have purposes? No. The people who design measures insert their purposes into the measures. This happens long before you or anyone else starts dressing them up in verbal and visual costumes.

Consider the United Nations indicator of violence against women. ¹² The expert committee charged with designing it decided to create lists of actions that constitute physical, sexual, psychological or economic violence. These lists would then be used to survey women around the globe. Surveyors would ask, “Have you experienced this or that action?” Add up the ‘yesses’ and we’ve got our measure of violence against women in each country.

At an expert committee meeting in 2009, members from Europe, North America, Australia, and New Zealand put forth their ideas about violence, based on gobs of feminist writing and victim surveys in their countries. Women from the Global South didn’t have as much writing or research to draw upon, but they did have plenty of experience with violence. Some Bangladeshi women on the committee put forth another list of items they wished to have included as psychological violence (see table 1).

Table 1 Defining Psychological Violence

Source: Sally Engles Merry, The Seductions of Quantification, University of Chicago Press 2013, pp. 77-78.

Some of these things are unimaginable to women who haven’t lived them.

“Expel from the house”: In Western Nepal, some women are forced to sleep outdoors on the wet ground while they’re menstruating, and for three months around the time they give birth.

“Rebuke for giving birth to a girl”: In many places in South Asia, women who don’t produce a male child might be rejected by the husband and his family; beaten, starved, and driven out of their village. A Pakistani woman I know told me she had been married off to a man who turned out to be impotent. To cover his humiliation, he beat her, and his family, of course, took his side and ostracized her. When she eventually fled back to her parents, they discouraged her from seeking divorce because it would bring shame on them, too. She’s now a PhD student at Brandeis.

“Marry other women in addition to existing wife”: Maybe you have an easier time relating to this one, but we’re not just talking affairs or serial marriages here. Dr. Sima Samar, who is now Head of the Afghanistan Independent Human Rights Commission, recalls how her childhood turned miserable when her father took a second wife. She and her mother were relegated to a back room and demoted in dignity, affection, and material things. It was bad enough to be female. Now they were second-class females. ¹³

As you can see, most of the Bangladeshi women’s suggestions didn’t make it into the final indicator (table 2). Of course, many if not all of the items in the United Nations guidelines apply to Bangladeshi women, too. Still, when the numbers come in, they will reveal little about the people whose ideas are not included in the heap of things to be counted.

Table 2 Defining Psychological Violence

Source: condensed and paraphrased from “Guidelines for Producing Statistics on Violence against Women—Statistical Surveys,” United Nations, Department of Economic and Social Affairs, Statistics Division, ST/ESA/STAT/SER.F/110, New York, 2014, pp. 16-17. Available at: https://unstats.un.org/unsd/gender/docs/Guidelines_Statistics_VAW.pdf.

What do numbers mean? To paraphrase Humpty Dumpty, the numbers mean exactly what the people who design them want them to mean.

Let’s stop here for a couple more ethics lessons:

Lesson Five: “Who Was in the Room Where It Happened?”

Substitute any indicator you want for psychological violence. Whatever it is, channel Lin-Manuel Miranda and ask: “Who was in the room where it happened?” ¹⁴ Who was asking the questions and what did they think to ask about? When we design measures, the people being measured and the people whose lives will be affected by the measure should have a voice in determining what gets counted. Perhaps instead of striving for universally applicable definitions, we should sacrifice comparability for verisimilitude, and for truly democratic representation of diverse experiences.

Lesson Six: If You’re Strong Enough to Be in a Position to Design a Measure, You Have an Obligation to Amplify the Voices of the Weak by Counting What Matters to Them.

PART 3: HOW DO NUMBERS GET THEIR AUTHORITY?

For all that’s missing from Menninger’s definition of counting as “assigning number words to things,” he gets one thing right. Assign. What comes into your head when you hear that word? I dare say, homework? Some kind of power? Or perhaps a sense of dread and powerlessness?

Numbers get their authority from people who are able to exert rhetorical and political power to assign words to things and things to categories.

When the delegates at the Constitutional Convention discussed how to count the population in the federal census, the most contentious issue was whether to count slaves as property, in which case their owners would be taxed on them; or to count them as people, in which case the states where they lived would get more representatives in Congress. In the infamous compromise ultimately written into the US Constitution, slaves were counted as three-fifths of a person. In The Federalist No. 54, James Madison defended the rightness of this way of counting:

“[T]he Federal Constitution therefore, decides with great propriety on the case of our slaves, when it views them in the mixt character of persons and property. This is in fact their true character.”

James Madison, The Federalist Papers, No. 54 ¹⁵

And yes, I know: Scholars aren’t 100% sure whether Federalist 54 was written by Madison or Hamilton, but most give it to Madison, and since this is the Madison lecture, I will, too.

Scholars often quote these two sentences as if they were Madison’s, and they are, but Madison is playing a cute rhetorical game here. He puts these words in the mouth of an imaginary southerner he creates to argue against the northern position. (Writers can do that sort of thing.) Madison has his imaginary southerner work through a legal analysis to prove that slaves “are considered by our laws in some respects as persons and in other respects as property.”

Federalist No. 54: Reasons Why Slaves Are Property

“In being compelled to labor not for himself, but for a master; in being vendible by one master to another master to another master; and in being subject at all times to be restrained in his liberty, and chastised in his body, by the capricious will of another, the slave may appear to be degraded from the human rank, and classed with those irrational animals, which fall under the legal denomination of property.”

Where does Madison’s alter ego look for evidence about the important attributes of slaves? ... he looks to laws. He looks to the political decisions that have been already been made by dominant white elites, and from these acts of raw power, he purports to find the standard of truth for how we should count slaves.

First, the imaginary southerner itemizes three legal characteristics of slaves that should make us count them as property:

• • Slaves must work for a master, not for themselves.

• • They can be sold.

• • They can be deprived of their freedom at any time and be punished at the whim others.

From these characteristics, says the southerner, you northerners might think slaves look just like animals, but you’d be wrong.

Federalist No. 54: Reasons Why Slaves Are People

“In being protected on the other hand in his life & in his limbs, against the violence of all others, even the master of his labor and his liberty; and in being punishable himself for all violence committed against others, the slave is no less evidently regarded by the law as a member of society . . ., as a moral person, not as a mere article of property.”

The southerner now itemizes two legal attributes of slaves that make them look like people:

• • Slaves, according to law, are protected from violence and murder.

• • And they can be punished for committing violence to others, just as free people can be.

Thus, says the southerner, the law regards the slave as a “moral person,” deserving of society’s protection and accountable for his actions. And then the imaginary southerner utters the famous assertion: “The mixt character or persons and property is in fact their true character.”

Notice the visual language in Madison’s text that I highlighted in italics: The words appear and regard carry the logic of the argument. We’re firmly in the land of metaphors, “seeing as.”

In the end, Madison returns to his own voice, pretending to be a neutral arbiter between North and South. (Somehow, we’re not supposed to notice that he was a southern slaveholder himself. Go ask your mother.) Madison concludes that although the imaginary southerner’s reasoning seems “a little strained on some points, yet on the whole, I must confess, that it fully reconciles me to the scale of representation which the Constitution have established.”

Where does Madison’s alter ego look for evidence about the important attributes of slaves? Does he look at slaves, or ask them what or who they think they are? No, he looks to laws. He looks to the political decisions that have already been made by dominant white elites, and from these acts of raw power, he purports to find the standard of truth for how we should make further political decisions (figure 5).

Figure 5 How Numbers Get Their Authority

ETHICS LESSONS

Lesson Seven: Justify Your Cuts.

When we left off talking about validity, I asked you to explain what you cut—what didn’t get into the heap of peas to be counted. Now I want to ask you to justify your cuts.

“It’s impossible to get data on that” is not an acceptable answer. What are the criteria for how you chose what to include and exclude? This may sound like the standard validity tests, but I’m asking for more. I’m asking for you to zoom in, metaphorically, on the laws of slavery. I’m asking you to identify the political decisions made by powerful people on which you now rely. To identify the cultural norms, practices and the unspoken assumptions everybody takes for granted. I’m asking you not to pretend that you made up your criteria yourself, out of your own original logic.

I’m asking you to crawl into the dark cave of history from whence your criteria came, interrogate the seemingly invincible monsters who live in there, and after you’ve found your way out again, convince me that we should let them tell us how to count.

Lesson Eight: Resist the Temptation to Think You Can Resolve Intractable Conflicts with Arithmetic.

Categorical, either/or decisions would seem to be cold, hard assignments, admitting no mercy. You’re either in or you’re out. But they can be modified, adjusted by quantity, and for that, arithmetic is oh-so-politically useful.

Perhaps this is the most important lesson of Federalist 54. Numbers can let you have your cake and eat it, too. Numbers enable politicians to gloss over irreconcilable political differences with fractions and averages. This is another sense in which Madison was the nation’s first quantitative social scientist. He taught us how to package qualitative thinking in quantitative imagery.

PART 4: HOW CAN COUNTING CHANGE HEARTS AND MINDS?

Much of our social science knowledge comes from asking people what they think, feel, or believe. Look for a minute at some of the questions that researchers have used to find out what whites think about blacks.

In one survey, the interviewers stated that blacks are not as well off as whites and then asked respondents why they thought this might be so. ¹⁶ Here are the possible answers.

• • “Blacks are born with less ability.”

• • “Most blacks who are on welfare could get a job if they really tried.”

• • “If blacks would only try harder, they would be just as well off as whites.”

• • “Black neighborhoods tend to be run down because blacks simply don’t take care of their property.”

• • “Most blacks have a chip on their shoulder.”

• • “Blacks are more violent than whites.”

Here is another question drawn from the National Election Studies. It is designed to get at white people’s sense of feeling threatened by blacks.

“What do you think are the chances these days that you or anyone in your family won’t get a job or a promotion while an equally or less qualified black employee receives one instead?” ¹⁷

I had been reading this scholarship on racism for years, disturbed by its findings but admiring its thoughtful, nuanced research strategies. Then I taught a course on American race politics in Denmark. One night before class, I wondered, as I always do, what my students would make of the readings I had assigned. Suddenly, seeing through young Danish eyes, I felt shocked and ashamed that academics would ask questions like these.

The questions themselves are racist. They presume that race groups are real and everybody knows what they are. They presume that characterizing the behavior and morality of entire groups is possible. The questions stereotype, and in doing so, they teach people that stereotyping is normal and acceptable.

Admittedly, I cherry-picked the most flagrantly racist questions I could find to show you. Most studies of racism include questions that permit respondents to pin the blame on discrimination or structural conditions instead of on blacks as individuals, and to say that blacks aren’t treated fairly.

I’m talking here about something deeper than framing effects, that is, how question wording can influence people’s answers. Some of these questions about race educate the public about how to think in the same way that children’s counting charts do, because they provide no opportunity to challenge the lumping assumptions behind them.

Nevertheless, asking people to answer questions like these encourages them to buy into stereotyping; they have to buy into it in order to answer. I think we’ve got an example of the Heisenberg Principle here: the measuring process influences the measurement. If counting doesn’t exactly create public opinion, it surely shapes it.

I’m talking here about something deeper than framing effects, that is, how question wording can influence people’s answers. Some of these questions about race educate the public about how to think in the same way that children’s counting charts do, because they provide no opportunity to challenge the lumping assumptions behind them. There is no possibility to say:

• • “I don’t think you can generalize like that.” Or,

• • “I think it’s just plain wrong to stereotype like you’re asking me to do.” Or,

• • “Get out of my face, you racist so-and-so.”

When we talk about these racism studies in my classes, my students at first share my dismay, but eventually, someone comes to their defense. Last semester, a Nigerian woman made the case with an impish twinkle in her eye and an unforgettable question: “I don’t mean any disrespect, Professor, but how else are we going to find out whether white people think Africans live in trees if we don’t ask them?”

I see the point, as the researchers certainly do, too. So let’s ask people if they think Africans live in trees. But let’s also give them opportunities to express incredulity and outrage, or at least disagreement with the premises of the questions. Better yet, by adding some different questions, we might get some white respondents to think about white privilege. For example: in addition to asking whites how likely it is that they’ll be scumped out of a job by a black person, we could ask:

• • Do you know anyone who you think probably got their job more because of their connections than their qualifications? Or,

• • In the place where you work, how much do you think personal relationships with higher-ups influence who gets raises and promotions?

Questions like these might put a chink in the belief that everyone earns whatever goodies they have solely by their own hard work.

We could also ask questions to illuminate how everyone gets help from government.

• • Have you or anyone in your family ever received help from government?

Most people will probably say “no,” so perhaps later probe:

• • Did you or anyone in your family go to public school?

• • Do you have running water in your home? If so, who provides it? How much do you pay for it?

• • Have you or anyone in your family ever received: Social Security, Medicare, Medicaid, or Disability?

While the Affordable Care Act was under debate in 2010, clueless protesters held signs demanding, “Keep Government Hands Off My Medicare,” and “Keep Government Out of Medicaid.” So I know it’s hard to crack deeply held beliefs, but if we want to get counting better, this is what we need to do. Here are three lessons about how.

ETHICS LESSONS

Lesson Nine: Could Your Measurement Instrument Influence the Size of Whatever You’re Measuring?

Put your number to the Heisenberg Test: Could your measurement instrument change the temperature of whatever you’re measuring? Asking questions, rewarding and punishing on the basis of measures, publicizing comparisons—all of that kind of public exposure makes people see themselves differently and behave differently. Social scientists used to call this “reactivity.” Now we call it “feedback.” Whatever you call it, look for it, and don’t kid yourself that there was a static “there” there before you measured it.

And yes, sometimes we want performance measures to goose people or governments to perform better, and that’s okay. What we don’t want is to have our measurement processes goose people to perform worse.

Lesson Ten: Examine the Hidden Curriculum of Your Questions.

If you are counting by doing surveys, opinion polls or interviews, examine the hidden curriculum of your questions. What are you conveying to respondents about morally and intellectually acceptable ways of thinking?

If you’re studying something you hope to reduce, like prejudice, consider how you can turn the mirror towards the people whose hearts and minds you want to change. Design some questions that plant seeds of empathy, questions that help people to identify with others. Write questions that teach people to see how much of their own success comes from the help of others.

In short, study conflict, but also design measures that help people come together.

PART 5: ARE THERE SOME THINGS WE SHOULDN’T COUNT?

Once when I was waiting my turn for the vet to see my dog, I overheard him talking with a woman whose dog he clearly thought was obese. They were arguing heatedly about the dog’s diet. “So what should I feed him,” the woman finally yelled in frustration. “Less,” the vet answered.

Sometimes when I’m contemplating Jens Blom Hansen’s question, “How can we get counting right?” I’m tempted to answer, “Don’t.” Two of my top candidates for “Don’t” are counting the value of a life and counting political will. Now, as Jens put it, counting them is here to stay, so take my musings as no-calorie food for thought.

Executive Order 13563 requires government agencies to quantify the expected costs and benefits of any regulation they are considering. ¹⁸ When the benefits include human health and safety, agencies measure them with a concept called “Value of a Statistical Life.” They calculate this value primarily by asking people how much they’d be willing to pay to avoid various risks of death. Mind you, they’re not putting a value on any real person’s life. They’re imputing a value to a human life in the abstract. They project—that means “guess”—the total number of people whose lives might be saved by a regulation and multiply that number by the dollar value of a statistical life. Out the other end of that equation comes the benefit of the regulation—in dollars.

There are many fine treatises for and against valuing lives as a way of deciding on government policy—some of them are no doubt sitting in this room. So here I’ll just note three reasons to say “don’t” based on what we know about counting.

First, in order to count intangibles, we have to fragment them into discrete, observable bits. Things like peas that you can point to and say “one,” “two,” “three.” For some things, such as life, cutting them into countable pieces destroys the meaning and value of the whole. I’m not saying, as many do, that life is “priceless.” Rather, the ways we have to measure its value to us don’t begin to capture what we value about it—never mind that we all value life differently. Not one of us is like another.

Second, the measure is infinitely malleable. The people in charge of an agency can make the analysis come out any way they want, simply by what they choose to include in the cost and benefit estimates, or whether they choose consultants with pro- or anti-government leanings. In 2008, the Environmental Protection Agency cranked some numbers and reduced its working number for the value of life by just under a million dollars. That devaluation rendered some proposed pollution regulations no longer worth their projected cost. ¹⁹ Lower the value of life and a regulation to save lives becomes less of bargain.

Third, as I’ve already said, arithmetic doesn’t resolve value disagreements, it only disguises them.

At the risk of being expelled from the APSA, I’m going to end by questioning what is perhaps the fundamental truth of our discipline (figure 6). Does it make sense to resolve political conflict with elections? To choose leaders and policies by counting votes?

Figure 6 Voting in Lempster, New Hampshire

Photo by Deborah Stone. November 4, 2008.

Barbara Kingsolver’s novel, The Poisonwood Bible, takes place in the Belgian Congo, in the period just before and after independence. ²⁰ In one scene, we get to see how the idea of majority rule looks to Anatole, a smart young village man. Anatole had been schooled by an American missionary, and he’s now one of the new leaders, responsible for helping manage the first national election in his village. In this passage, Anatole explains his point of view to the missionary’s daughter, who is now his girlfriend:

“It seems odd that if one man gets fifty votes and other gets forty-nine, the first one wins altogether and the second one plumb loses. That means almost half the people will be unhappy […] and in a village that’s left halfway unhappy, you haven’t heard the end of it. There is sure to be trouble somewhere down the line.” ²¹

So what went wrong with Madison’s idea in the Tenth Federalist that a large republic would cure the mischiefs of faction? ²² What went wrong with the grand hopes after World War II for the end of empire and genocide, and for development and democracy to flourish everywhere? They were all pinned on counting as the way to resolve conflict.

Kingsolver’s character Anatole contrasts tribal ways of making collective decisions with elections:

“The way it [works] here is that you need one hundred percent. It takes a good while to get there. They talk and make deals and argue until they are pretty much all in agreement on what ought to be done, and then Tata Ndu [the headman] makes sure it happens that way. If he does a good job, one of his sons will be chief after he dies. If he does a bad job, the women will chase Tata Ndu out of town with big sticks.” ²³

Anatole’s village is a simpler version of the deliberative democracy many of us wish for. It is, alas, a novelist’s fantasy, so it’s not the answer either. But I do know this: counting is no substitute for talking.