Darrell Huff wrote a classic book in 1954: How to Lie With Statistics. Heather MacDonald’s argumentative column in the Wall Street Journal (May 29, 2015) predicting a new crime wave in our country would have been prime fodder for a chapter in Huff’s book.
Huff’s chapters offer object lessons in ways that journalists use statistics to confuse and deceive readers. MacDonald overstated the strength of the evidence she cited to support the purported crime wave. Worse, however, was her deliberate and systematic withholding of data, the erroneous conclusions based upon them, and the failure to note that researchers have not yet validated the effectiveness of the broken window theory (Kelling & Wilson, 1982).
Broken window theory assumes that crime and disorder, like ham and eggs, are inextricably linked. If a window in a house were broken and not fixed, it is assumed that more windows will be broken, whether the house is located in Watts or in West Palm Beach. The broken window represents a lack of caring, an invitation to break more windows. Thus, the failure of the police to arrest lawbreakers, regardless of the crime, invites committing more crimes. The strict enforcement of the law, however, subsequently results in fewer crimes. Similarly, the “Ferguson effect” refers to police officers “disengaging from discretionary law enforcement.”
MacDonald completely ignored the valuable lesson in Huff’s Chapter 3: “The Little Figures That Are Not There.” She provided multiple meaningless percentages; for example, “In New York, murder was up nearly 13%.” MacDonald repeated this same error 20 times—never providing “the little figures” corresponding to the given percentages.
What does it mean to state that burglaries in Walla Walla increased 25%? This percentage is meaningless without knowing the number of burglaries. Did burglaries increase from 8 to 10—or did they increase from 800 to 1,000, both increases of 25%? MacDonald offers readers only meaningless percentages—and conjectures instead of valid conclusions.
Chapter 3 in Huff’s book pointed out the inadequacy of using small samples to identify real (statistically significant) differences: “Sooner or later, by the operation of chance, a test group is going to show a big improvement worthy of a headline” (p. 38). Findings of the Rialto (California) study illustrate this flaw, research cited by advocates supporting the purchase of body cameras for police offices in Anaheim. This study reported > 50% reduction in the total number of incidences that included the use-of-force by Rialto police officers. The data: 17 incidences required the use of force were reported by police officers without cameras; 8 incidences were reported by police officers with cameras (p. 11). The Rialto study also reported that the number of annual homicides in the city were almost 50% more before Rialto police officers wore body cameras; During 2009-2011, homicides were “nearly 50% higher than the U.S. national rate per 100,000” (p. 5). In fact, during this period, the mean number of homicides was fewer than 7 per year.
MacDonald hedges her conclusion: “The nation’s two-decades-long crime decline may [emphasis added] be over.” This assertion is based upon spinning data, missing figures, and inventing implied facts; for example, “The most plausible explanation of the current surge in lawlessness is the intense agitation against American police departments over the past nine months.” Researchers have not yet established, as MacDonald asserted, that “broken windows policing, has saved thousands of Black lives, brought lawful commerce and jobs to once drug-infested neighborhoods and allowed millions to go about their daily lives without fear. “
Researchers (2014) at George Mason University reviewed the evidence for broken windows policing. The principal finding: “There is also no consensus on the existence of a link between disorder [broken windows theory] and crime, and how to properly measure such a link if it does indeed exist.” (A matrix that includes 130 studies of evidenced-based policing is here.)
MacDonald’s column is a must read—for figureless data, lack of evidence for assertions, and an object lesson in the deceptive use of statistics.
Disraeli was right: “There are three kinds of lies: lies, damned lies, and statistics.