The Cognitive Reflection Test : A shortcut to measuring Intelligence
Can three innocent questions serve as a proxy for an intelligence test and predict cognitive ability? Even if not, it might be fun to play around with them, and reflect on what reflective thinking ability can teach us about our decision making and mental habits.
Are you smart enough to solve all three correctly, even if it is the first time you see them?
The CRT Test
The CRT is a short and simple intelligence test. It tries to estimate cognitive ability and mental skills by asking three short questions.
Let’s try this mental game – What are your answers to the following three questions:
There are only 4 possible scores: 0%, 33%, 66%, 100% … See the answers paragraph below to know your result.
The CRT is essentially a test of patience and the ability to reflect carefully (reflective thinking), and to reconsider an intuitive – seemingly ‘obvious’ answer.
Is the CRT a good measure of cognitive ability?
CRT is a ‘simple’ measure of cognitive ability. It has been used in different experiments to represent mental skills because it is quick and convenient … does it really work ?
A number of research experiments seem to indicate that it works as a ‘good enough’ estimate of cognitive ability. It has been compared to the WPT (Wonderlic Personnel Test, a cognitive ability test with 50 Items), and the ACT/SAT (Academic Achievement Tests). The results show correlations between good performance on the CRT and higher scores on these tests.
Interestingly, this short test, has been associated with a number of decision making variables, including some related to long-term thinking, risk-tolerance, and others. People with higher scores tended to think more rationally about the expected outcomes of uncertain situations.
However, the problem with the test is clear. If the subjects feel that there is a trick, or they need to consider, or if they have seen a similar question before, the test becomes useless.
Statistics on Results
If you feel discouraged by your results, don’t. The test is quite tricky.
The original article (referenced below) reports that of 3,428 people participating in the research, 33 percent missed all three questions. Most people–83 percent–missed at least one of the questions. Even very educated people made mistakes. Only 48 percent of MIT students sampled were able to answer all the questions correctly.
Note that test-takers in the sample where university students.
Reflecting on the ‘obvious’
Reflective thinking is an important mental skill and predictor of cognitive ability. It seems to correlate with patience and numeracy among others.
Observations and Thoughts:
- People who’ve seen do much better …
- As with everything in life, if you’re prepared and have experience, you’ll do better. Granted, the fact that the test is quite short makes this more obvious.
- Our brains have a great ability to improve and learn new things (neuro-plasticity). Repetition, and familiarity, are excellent learning tools.
- (re)Consideration makes a lot of difference.
- By looking at solutions and drafts, researchers concluded that whenever people reconsidered they were on track to solve correctly..
- Women do worse than men, even after controlling.
- This was quite an interesting observation. Both the test results, and the drafts, revealed a difference in performance. This is true even if the researchers control for SAT scores.
- Unknown Unknowns :
- Respondents who missed the problems thought they were easier than the respondents who solved them.
- People who don’t know that they don’t know, are likely to think they have done well.
Solutions and Your Score
So what about the answers ?
If you answered 0.1, 100, 24 …. bad news.
These are the intuitive answers that you’re supposed to question.
References
Original Article:
Frederick, S. (2005). Cognitive Reflection and Decision Making. Journal of Economic Perspectives, 19(4), 25–42. doi:10.1257/089533005775196732
Another Source:
Szaszi, B., Szollosi, A., Palfi, B., Aczél B., (2017) The cognitive reflection test revisited: exploring the ways individuals solve the test, Thinking and Reasoning, https://www.tandfonline.com/doi/abs/10.1080/13546783.2017.1292954
Relevant Resources:
Think Grow Games: Intelligent Play
The Think-Grow Games are mobile games that take simple game mechanics and blend them with elements that encourage thinking, exploration, and mental exercise.
Think-Grow Knowledge Modules
The Think-Grow Knowledge Modules are mini-libraries of book summaries, each giving a wide (wise) enough view of a specific field.
They offer a simpler, wiser, and more convenient approach to reading about different fields. The Better Brain Library in particular has summaries of books on intelligence, thinking, and mental training.