Statistics and critical thinking

Some quick thoughts on statistics, critical thinking, and foundation-level learners. 

Type one statistical thinking – Understanding how data is collected

Mark Twain noted ‘There are three kinds of lies: Lies, damned lies, and statistics’. He seemed to be implying that statistics were a tool used to manipulate the truth. Perhaps he was suggesting that politicians, advertisers, and others use statistics to mislead the public? What a concept.     

When teaching statistics to foundation-level learners it is useful to think of statistical critical thinking skills as two skills. The first is to understand ‘how data was collected’ and the second ‘how the statistics are represented’ to the reader. The first is closely related to typical critical skills, and the second more mathematically oriented. Foundation-level learners often enjoy the first and are not so familiar with the second.   

Understanding ‘how the data was collected’ is useful when presented with information such as this old advertisement for smoking.

Critical questions here might include:

Did they ask every single Doctor whether they smoked? How many did they ask, where were they from, how many answered, and how didn’t smoke and how many did? Of those who did smoke how many smoked Camel? Why did they smoke Camel? Were the Doctors given any incentives to promote Camel?

Question: Based on the evidence in the advertisement above is this scenario possible.

The researchers asked 50 Doctors whether they smoked. Five said they smoked. Three of these smoked Camels.

Could this be true based on the advertisements claims?

An old ‘good boy’ dog food scam.

A large company sold a popular brand of dogfood called ‘good boy’. The company was putting together a new advertisement and wanted some statistics to use in the advertising.

Here is how they produced the statistics.

An employee rang ten people and asked them what dog food they used to feed their dog. They recorded the amount of people out of ten who named the company.

They then rang another group of ten people. Then another. Then another. After doing this they simply selected the group of ten in which the highest number had selected their dog food. 

The statistic used was ‘six out of ten dog owners use good boy’. It was true, one of the groups of ten people had six members who used Good Boy. What wasn’t mentioned was that this was the highest of the ten groups. Some of the groups included only one or two people who used the dog food, others three, four or five. If the groups were combined the statistics would have read ’36 out of 100 dog owners use Good Boy’, or ’66 out of 100 dog owners do not use Good Boy’. Quite different. Yet the statistics are accurate – it’s the method needs to be critiqued.

However, be careful...

Despite these dubious methods of generating data, don’t allow learners to become cynical of all data. Most data collection methods are very good; understanding the strengths and limitations of the methods is the goal. Critical thinking when taken to the extreme means you can never know anything.  Don’t teach learners to be cynics, teach them to be critical evaluators.

Type two statistical thinking: Understand how statistics are represented to support certain ideas.     

Adults also need to know what statistical data analysis methods have been used and how the approach produces and represent findings. For example, if the term ‘average’ is used, does it refer to the mean, median or mode? Is the average even relevant? The ‘average NZ house price’ is frequently used in the NZ Herald. Would it not be better to know how many houses were sold in various price brackets? If two houses are for sale in a neighbourhood, one for 100,000 and one for 1,000,000, does the average of $550,000 fairly represent the situation? I don’t think so because an average of $550,000 suggests that people with one fifth of this amount cannot buy a house – and it isn’t true based on the situation.    

For example, the classic statistical scenario: There are two buckets, one full of boiling water and one full of freezing water. Stand with a foot in each. Stop screaming, because on average you feel warm.

If you only know the average, you don’t know enough to form an opinion.  

In sum, adults are bombarded with statistical information through politics, advertising, workplace statistics, dietary recommendations, safety courses, sports and health advice. In order to make sense of the information, adults require some knowledge of how data can be collected and how it is represented.