Statistics and critical thinking

Some quick thoughts on statistics, critical thinking, and foundation-level learners.

**Type one statistical thinking – Understanding how data is collected**

Mark Twain noted ‘There are three kinds of lies: Lies, damned
lies, and statistics’. He seemed to be implying that statistics were a tool used
to manipulate the truth. Perhaps he was suggesting that politicians,
advertisers, and others use statistics to mislead the public? What a concept.

When teaching statistics to foundation-level learners it is useful to think of statistical critical thinking
skills as two skills. The first is to understand ‘how data was collected’ and
the second ‘how the statistics are represented’ to the reader. The first is closely
related to typical critical skills, and the second more mathematically
oriented. Foundation-level learners often enjoy the first and are not so
familiar with the second.

Understanding ‘how the data was collected’ is useful when presented
with information such as this old advertisement for smoking.

**Critical questions here might include:**

Did they ask every single Doctor whether they smoked? How
many did they ask, where were they from, how many answered, and how didn’t
smoke and how many did? Of those who did smoke how many smoked Camel? Why did
they smoke Camel? Were the Doctors given any incentives to promote Camel?

**Question:**Based on the evidence in the advertisement above is this scenario possible.

The researchers asked 50 Doctors whether they smoked. Five said they
smoked. Three of these smoked Camels.

Could this be true based on the advertisements claims?

**An old ‘good boy’ dog food scam.**

A large company sold a popular brand of dogfood called ‘good
boy’. The company was putting together a new advertisement and wanted some
statistics to use in the advertising.

Here is how they produced the statistics.

An employee rang
ten people and asked them what dog food they used to feed their dog. They
recorded the amount of people out of ten who named the company.

They then rang another group of ten people. Then another.
Then another. After doing this they simply selected the group of ten in which
the highest number had selected their dog food.

The statistic used was ‘six out
of ten dog owners use good boy’. It was true, one of the groups of ten people
had six members who used Good Boy. What wasn’t mentioned was that this was the highest
of the ten groups. Some of the groups included only one or two people who used
the dog food, others three, four or five. If the groups were combined the
statistics would have read ’36 out of 100 dog owners use Good Boy’, or ’66 out
of 100 dog owners do not use Good Boy’. Quite different. Yet the statistics are
accurate – it’s the method needs to be critiqued.

**However, be careful...**

Despite these dubious methods of generating data, don’t
allow learners to become cynical of all data. Most data collection methods are
very good; understanding the strengths and limitations of the methods is the
goal. Critical thinking when taken to the extreme means you

*can never know anything.*Don’t teach learners to be cynics, teach them to be critical evaluators.**Type two statistical thinking: Understand how statistics are represented to support certain ideas.**

Adults also need to know what statistical data analysis methods
have been used and how the approach produces and represent findings. For
example, if the term ‘average’ is used, does it refer to the mean, median or
mode? Is the average even relevant? The ‘average NZ house price’
is frequently used in the NZ Herald. Would it not be better to know how many
houses were sold in various price brackets? If two houses are for sale in a
neighbourhood, one for 100,000 and one for 1,000,000, does the average of
$550,000 fairly represent the situation? I don’t think so because an average of
$550,000 suggests that people with one fifth of this amount cannot buy a house –
and it isn’t true based on the situation.

For example, the classic statistical scenario: There are two
buckets, one full of boiling water and one full of freezing water. Stand with a
foot in each. Stop screaming, because on average you feel warm.

If you only know the average, you don’t know enough to form
an opinion.

In sum, adults are bombarded with statistical information
through politics, advertising, workplace statistics, dietary recommendations,
safety courses, sports and health advice. In order to make sense of the
information, adults require some knowledge of how data can be collected and how
it is represented.