Caveat Lector: Why You Should Always Look at the Data

By Mark Stonger

Whenever I’m scrolling through the online news of the day, I usually take the time to glance through the headlines of my chosen profession, psychology. I like to see what psychological topics have made headlines, as well as reading how the psychological information has been interpreted in an article. I was catching up on the news of the day when an article caught my eye: “Battling the Boys: Educators Grapple with Violent Play.” Alright, I’ll bite-what is this all about?

The news story leads with a seemingly factual statement, saying “four-year-old boys play superhero or enact mock fights much more frequently than girls, who seem to favor house or family themes for playtime.” On the surface, this is nothing new; it fits with common sense. However, the conclusions that are drawn should be examined.

The educational journal article itself is an interesting read. The authors create a positive narrative and detail their activities with the children and give very detailed accounts of how the children reacted and the steps the teachers ran through to arrive at their decision to allow superhero play in their class room. I laud their efforts to create a better environment in which children can grow and learn.

In fact, after reading the source journal article, the average reader might look the news story over and think, what’s wrong with it? It cites several knowledgable sources (professors in specialty fields) who have multiple publications and research credentials. This is true; in fact, the referencing of experts is an excellent step. Unfortunately, one should not simply take the word of those learned people. To gain a more complete understanding of what the author is trying to convey, the raw data should be examined. Yup, you gotta look at the numbers. Another critic might say, "well, who has time to do that?" My answer to that is simple: "everyone has time." Separating significant data from inconsequential dross is what allows human beings to draw meaning from information.

That same complainer might also say "the numbers look fine! Why are you making a big deal out of this?" The only reply I have to such an answer is this: how do you know what those numbers mean? That is the crux of the argument. As the famous quote goes, “There are lies, damn lies, and statistics.” In this article, the numbers are at the heart of the conclusions any reader should draw.

N=98. This simple phrase can speak volumes. In this case, it means that there were 98 teachers who were surveyed for the journal article. 98 female pre-school teachers were asked about how they felt regarding physically active, superhero-based play in children with whom they worked. These 98 women in Maine, USA, were asked how often they felt compelled to interfere in the play of boys and girls. The news article then says “nearly half” of the teachers had to stop the play of boys “several times a week or every day.” Nearly half is a big number! That’s almost 50%! If that were the world population, that would mean 3.5 billion people were involved in something! In this case, however, it means that 48 female teachers felt they had to redirect 4-year-old boys when their play became too violent or disruptive. The contrast with girls was equally statistically important: Only 29% of those very same 98 female teachers felt they had to interfere with girls play on a weekly basis. 29% is a little more than 1 in 4...that’s a tiny number...right? Here, it means 28 female teachers felt compelled to intervene in girl’s play on a weekly basis.

If you have read all of this and are still thinking, “What’s the big deal?”, let me put it this way: Imagine someone told you that one town’s pre-schools were going to change national educational policy because they had numbers that were too compelling to ignore. Imagine if they cited numbers like nearly half or barely more than a quarter and told you that your child would benefit from these same things too. Would you want to say all boys should act this way? Would you want 98 people to determine truth?

Thankfully, this article does not go so far as to say who is right; it merely presents several points of view. However, the article does present the opinions of 98 people as if it is fact. Hopefully, you’ll keep an eye on the numbers the next time you scroll through an article.