Every day we’re overwhelmed by an avalanche of figures and information.
Our tendency to believe in statistics is increasing, because we think they give us certainty. But numbers can and must be taken with a pinch of salt: statistics cannot make precise what isn’t precise.
When you see a statistic or figure somewhere, ask yourself the following six questions:
1. What does your gut say?
Do not just accept a statistic. Be critical and look for sources or studies that explain the story from a different angle.
2. Where does it come from?
Does a survey favour a certain party just before the elections? Or is a soft drink company telling you that sugar-free drinks are healthy? Doubt the source and look for additional studies.
3. How was it measured?
Which (moral) choices were made when measuring? Is the figure about a man-made concept, such as GDP or IQ? Then pay extra attention to the interpretation. Try to find numbers that measure the concept in other ways.
4. How was the data collected?
Was the sample random or not? If not, the results only apply to the specific group that was examined. Of equal importance: how were the questions and answers formulated?
Always be critical, because questioning can be suggestive.
5. How were the figures analysed?
Two events occurring together do not necessarily imply a causal relationship. Always ask yourself these 3 questions:
Could it be a coincidence?
Could other factors play a role?
Could the causal relationship be reversed?
6. How are the figures presented?
When interpreting figures, beware of:
- Risk: if you’re x% more likely to get a certain disease, then you need to know what that x is a percentage of. If the chance of getting a disease is insignificant, an increase of the percentage is equally insignificant.
- Graphs: always pay attention to the vertical axis. It’s the easiest way of distorting results.
- Average: if there are many outliers, an average loses its significance.
My final advice: don’t assume all the figures you hear in an interview are true. Observe them with an open, critical mind and pick out the nonsense.
Journalist and econometrist Sanne Blauw wrote the book The best-selling book ever - how numbers lead, seduce and mislead us.
Essential reading for 2019.
Need some help?
You want help in analysing statistics?
Don’t hesitate to reach out.
Written by Marc Goedefroy, Data Project Director
Filed under INSPIRATION