The challenges to scoring beauty and healthcare

by Glen Whitman:

... John scores each woman’s looks on a scale from 0 to 10. Then he interacts with each woman (from behind a screen, if you insist) and scores each woman’s personality on a scale from 0 to 10.

The scores John gives for looks range all over the map, from 0 to 10, while the scores he gives for personality are bunched together in the 6 to 8 range.

To calculate composite scores, the World Mating Association (WMA) decides to rescale the personality scores. It calculates each woman’s personality score as follows: personality = 10 x (raw score – 6) / (8 – 6). In other words, it measures a woman’s score as the percentage of the distance between the lowest-scoring and highest-scoring women. A woman John gave a 7 would be rescaled to a 5, because she’s halfway between 6 and 8. A woman he gave a 6 would now be a 0, and a woman he gave an 8 would now be a 10.

Does this method make sense? Well, let’s see. Take two women, Alma and Betsy. Alma got a 9 on looks and a 6 (now rescaled to 0) on personality. Betsy got a 6 on looks and a 7 (now rescaled to 5) on personality. So Alma’s and Betsy’s composite scores are 4.5 and 5.5 respectively.

“But wait a minute,” John objects. “I said looks and personality were equally important to me. Alma’s three whole points better looking than Betsy. And while her personality is not quite as nice, it’s not that different. I thought they were both nice enough. All things considered, I’d give Alma a 7.5 and Betsy a 6.5. What I’m trying to say, I like Alma better!”

The problem, obviously, is the rescaling. John said personality matters just as much as looks to him – but fortunately for him, he likes most women’s personalities. The WMA’s approach exaggerated the significance of personality to John by treating women whose personalities he liked somewhat (6’s) as women he didn’t like at all, and women whose personalities he liked a lot (8’s) as women he thought were flawless.

The punchline is that the method I’ve just described is the method the World Health Organization (WHO) used to make its composite scores of healthcare system performance. These are the scores used to create the widely-cited rankings of nations’ healthcare systems.

UPDATE: Don Surber holds up Giuliani's prostate as anecdotal evidence (no picture following, and Surber is holding it up figuratively).