Cognitive Sophistication and the Development of Judgment and Decision-Making

By Maggie E. Toplak

Cognitive Sophistication and the Development of Judgment and Decision-Making reviews the existing literature on the development of reasoning, judgment and decision-making, with a primary focus on measures from the heuristics and biases tradition. The book presents a model based on cognitive sophistication to examine the development of judgment and decision-making, including age related differences in developmental samples, associations with intellectual abilities and executive functions, and associations with dispositional tendencies that support judgment and decision-making. Additional sections cover the empirical findings of a longitudinal study conducted over seven years that tie together the discussed aspects related to cognitive sophistication.

This book will provide a much-needed description of the theoretical and conceptual issues, a review of empirical findings, and an integrative summary of the implications for developmental models of reasoning, judgment and decision-making.

• Explores whether individual heuristics and biases are associated
• Reviews individual differences in cognitive abilities and thinking dispositions
• Examines reasoning from the lens of cognitive sophistication
• Discusses the implications for models, including dual process models
• Tests and elaborates using empirical findings from a longitudinal study

• Language: English
• Publisher: Academic Press
• Release date: Oct 27, 2021
• ISBN: 9780128166451

Maggie E. Toplak

Dr. Toplak’s research spans cognitive science and clinical research. She studies cognitive science models of rational thinking and decision-making in typically developing samples and in developmental psychopathology. She has published over 50 peer-reviewed papers and book chapters across the fields of decision-making and clinical research, including two books: Individual Differences in Judgement and Decision-Making: A Developmental Perspective (Psychology Press, 2016) and The Rationality Quotient: Toward a Test of Rational Thinking (MIT Press, 2018), the latter of which won the 2017 PROSE Award in Education Theory.

Book preview

9780128166451_FC

Cognitive Sophistication and the Development of Judgment and Decision-Making

First Edition

Maggie E. Toplak

Department of Psychology, LaMarsh Centre for Child and Youth Research, York University, Toronto, ON, Canada

Table of Contents

Cover image

Title page

Copyright

Dedication

Acknowledgments

1: Defining cognitive sophistication in the development of judgment and decision-making

Abstract

Measuring miserly information processing in adult samples: Implications for developmental samples

Laying the groundwork for studying the development of judgment and decision-making

The assessment of rational thinking in developmental samples: Using adult models and stimulus equivalence

Process and knowledge as key dimensions for a taxonomy of judgment and decision-making performance: Task level factors

Cognitive sophistication and the development of judgment and decision-making: Individual level factors

Longitudinal developmental study description

Summary and organization of book

References

2: Foundations for the development of judgment and decision-making: Cognitive abilities, thinking dispositions, and specific knowledge

Abstract

Processing that supports rational thinking: Cognitive abilities and thinking dispositions

Cognitive abilities: Intelligence and executive function task performance

Thinking dispositions

How cognitive abilities and thinking dispositions index sources of cognitive failure on judgment and decision-making tasks

Development of cognitive abilities: Intelligence and executive function task performance

Development of thinking dispositions

Specific knowledge: Facilitators and inhibitors

Knowledge facilitator: Probabilistic numeracy

Knowledge inhibitor: Superstitious thinking

Summary: Foundations for the development of judgment and decision-making

References

3: Development of the ability to detect and override miserly information processing

Abstract

Dual process models and miserly information processing in adult samples

Extending dual process models from adult samples to developmental samples

The developmental origins of miserly information processing

The assessment of resistance to miserly information processing in developmental samples: Ratio bias

The assessment of resistance to miserly information processing in developmental samples: Belief bias syllogisms

The assessment of resistance to miserly information processing in developmental samples: Cognitive reflection

Confidence ratings on miserly information processing tasks

Summary: Resistance to miserly information processing as an additional foundation to support the development of rational thinking performance

References

4: Recognizing the diagnosticity of statistical information in development: Base rate sensitivity

Abstract

Base rate sensitivity as a measure of rational thinking

Base rate sensitivity and development

Base rate sensitivity and correlations with individual differences

Data patterns from longitudinal developmental study

Summary: The development of base rate sensitivity

References

5: Preference for larger delayed rewards over smaller immediate rewards in development: Prudent temporal discounting

Abstract

Prudent temporal discounting as a measure of rational thinking

Temporal discounting and development

Temporal discounting correlations with individual differences

Data patterns from longitudinal developmental study

Summary: The development of prudent temporal discounting

References

6: Understanding descriptive invariance in development: Framing effects

Abstract

Resistance to framing as a direct measure of rational thinking

Framing effects and development

Data patterns from developmental longitudinal study

Summary: Framing effects and development

References

7: Correlations between judgment and decision-making tasks in developmental samples

Abstract

Correlations between judgment and decision-making paradigms in adult samples

Correlations between judgment and decision-making paradigms in developmental samples

Data patterns from developmental longitudinal study

Summary of Toplak and Flora (2020): The developmental trajectory of a Resistance to Cognitive Biases Composite

Separating rational thinking measures from resistance to miserly information processing: The Rational Thinking Composite

Summary: Associations between rational thinking tasks and predictors of performance in developmental samples

References

8: Real-world correlates of judgment and decision-making paradigms in developmental samples

Abstract

Correlations with real-world outcomes in adult samples

Correlations with real-world outcomes in developmental samples

Data patterns from developmental longitudinal study

Positive outcomes

Negative outcomes

Male/female differences in real-world behaviors

Summary: Considerations and promising next steps of real-world correlates and judgment and decision-making in developmental samples

References

9: The emergence of rational thinking in development: Conclusions and future directions

Abstract

Context for the operationalization and empirical study of judgment and decision-making in developmental samples

Cognitive sophistication: Age and individual differences in cognitive abilities and thinking dispositions as predictors of rational thinking in developmental samples

Resistance to miserly information processing as a measure of detection and override in developmental samples

Defining task characteristics for developmental samples: Process and knowledge in rational thinking tasks

Summary: Does rational thinking improve over the course of development?

Implications for models of the development of rational thinking

Real-world correlates and implications for training rational thinking in developmental samples

References

Appendix A: Tasks and measures used in developmental longitudinal study

Cognitive abilities (Chapter 2)

Thinking dispositions (Chapter 2)

Specific knowledge: Probabilistic numeracy (Chapter 2)

Miserly information processing tasks (Chapter 3)

Sample problem added at Time 3

Cognitive reflection test—Sample items

Calibration question following cognitive reflection task at Time 2

Belief bias syllogisms

Calibration question after belief bias syllogisms (Time 2 and Time 3)

Base rate sensitivity (Chapter 4)

Temporal discounting (Chapter 5)

Attribute framing problems (Chapter 6)

Other side thinking task

Procedure across all three timepoints

References

Appendix B: Cross-sectional age comparisons on cognitive ability, thinking disposition, probabilistic numeracy and superstitious thinking measures

Cognitive ability measures: Intelligence and executive function task performance

Thinking dispositions

Specific knowledge: Probabilistic numeracy

Superstitious Thinking Scale

Appendix C: Real-world outcomes questionnaire

Youth report (Time 2 and Time 3)

Parent report

References

References

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2022 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-12-816636-9

For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Image 1

Publisher: Nikki P. Levy

Acquisitions Editor: Joslyn Chaiprasert-Paguio

Editorial Project Manager: Sara Pianavilla

Production Project Manager: Omer Mukthar

Cover Designer: Miles Hitchen

Typeset by STRAIVE, India

Dedication

To Paul, Stephen, and Andrew—my constant sources of love, inspiration, and support.

Acknowledgments

This project technically started in 2008 when I submitted an application for a research grant to the Social Sciences and Humanities Research Council (SSHRC) with Keith Stanovich and Richard West. The grant was funded in 2009 for the Time 1 data collection occurring in 2010–11. I applied for a second research grant in 2012 to test two additional timepoints (2013–14 and 2016–17). I am grateful to the SSHRC for providing funding for these projects.

I thank Emily Ekle, the Senior Acquisitions Editor for Psychology at Elsevier, who approached me about writing this book in October 2017. Her interest and encouragement led to my book proposal that was accepted in February 2018. I had several editorial project managers who helped me along the way, including Sandra Harron, Anna Dubrow, and currently Sara Pianavilla. I thank Sandra for helping me get started, Anna for helping me design the cover of this book, and Sara for helping me finish this project. Shane Martin was a student in my research lab who brought creative energy to this work by designing my website, which led to the image that is on the cover of this book.

I thank all of the families involved in this project. Trying to explain the value of research on judgment and decision-making to children and youth can be challenging, so thanks to all of the parents and families for trusting and valuing this project by allowing their children to participate. For data collection at Time 1, some participants were tested in schools, but most participants were tested outside of school time. All of the families had busy schedules but worked with our research assistants to find times and places for testing. Our team has many interesting stories of how they collected the data reported in this book. From allowing us to conduct the research with their children in pajamas on New Year’s Eve to finding suitable spaces in hockey arenas to administer the paper and pencil battery of measures, our participants were always patient, flexible, and happily engaged with our team. I also thank the parents who went the extra mile to help us find additional participants and families. A special thank you to Stacey Lanzellotti and Marcelo Manfrini. If there was a prize for recruiting interest, they would both receive it.

I could not have done these projects without a dedicated team of research assistants. In particular, Geoff Sorge and Armita Hosseini (Time 1), Mohamed Al-Haj (Time 2), and Amanda Edwards (Time 3) were all hands on deck for the recruitment, data collection, data entry, and organizational aspects of the studies (including the longitudinal follow-ups). I am very grateful for the creative energy, persistence, and hard work that these research assistants provided during these studies. I have also had several undergraduate and graduate students who have also contributed, including conducting research projects with the data and participating in our lab meetings where this work evolved. These students include Saqina Abedi, Cameron Amini, Camelia Amiri, Alexandra Basile, Adam Burnett, Kaitlyn Butterfield, Andrew Chan, Stella Dentakos, Joshua Doidge, Tessah Dunn, Won Kim, Rachael Lyon, David Perez, Jala Rizeq, Wafa Saoud, Jill Shuster, and Elizabeth Wanstall. Thank you Cameron for pointing out that one of the most influential concepts you learned while spending time in my research lab was the importance of helping students learn "how to think, not what to think," a goal that we continue to have for all of our students.

My greatest source of intellectual inspiration has come from my longtime colleagues and mentors Keith Stanovich and Richard West. Thank you Keith for switching from the field of reading research to rationality and for writing books and papers that continue to shape my thinking and research. I cannot think of another scholar whose next book I await so eagerly, knowing that it will be filled with new ideas backed with a comprehensive survey of the literature to date. Your courage to adapt your ideas and models in the face of new data and information has been remarkable; you certainly "lead by example. You set high standards for your own research and work, something that I greatly appreciate and try to emulate. I also thank you Keith for providing comments on the chapters, including your assistance with writing Footnote b" in Chapter 3 on formative/reflective models. Richard, your collaboration and friendship throughout all of our work has ensured the successful completion of so many papers and chapters. I can always count on you to be patient and helpful. My interest in this field of research goes back to my graduate training when I developed some lifelong friendships with other trainees who shared a similar passion for this work, including Carol Kelley.

I have benefitted greatly from my reasoning colleagues who have attended my presentations and provided feedback on this work at conferences and meetings, including the London Reasoning Workshop (LRW), International Conference on Thinking (ICT), and the Society for Judgment and Decision Making (SJDM). In particular, Rakefet Ackerman, Linden Ball, Shira Elqayam, Jonathan Evans, David Over, Valerie Thompson, and Joshua Weller. Several of my colleagues at York University provided input and advice, including Christine Till, John Eastwood, Shayna Rosenbaum, and David Flora. David Flora is a co-author on the longitudinal paper for this developmental project (Toplak and Flora, 2020). His statistical expertise and understanding of the complex dataset helped to ensure our convergent findings across these manuscripts. David also provided the figures of the trajectory analyses for the cognitive ability measures in Chapter 2. My membership in the LaMarsh Centre for Child and Youth Research provided a network of colleagues who provided feedback on my presentations and progress on this work.

I also thank my students in the two sections of the new course I developed at York University, The Psychology of Reasoning, Judgment and Decision-Making (Psychology 3255). This course provided the opportunity to explain research in these fields in an understandable and meaningful manner to students, which helped me to constantly adapt and evaluate my own thinking and challenged me to communicate these concepts most clearly.

I also thank the Alliance for Decision Education (AfDE; https://alliancefordecisioneducation.org/), particularly Dave Lenowitz and Adriana Massara. While they were not part of the study described in this book, they have been integral in helping me think about the next steps with this work. I acknowledge their ongoing efforts in determining how to engage educators and schools in the importance of decision-making skills for students. In particular, they caught the interest of Ned Sherrill, the Head of School for Church Farm School (CFS), Grades 9–12. Ned’s team included Chris Seeley, Assistant Head of School and Director of Academics, Charles (Chuck) Keller, Director of Teaching and Learning, and Krista Peterson, Director of Studies/Registrar. Attempting to bring the science of decision-making together with education, the AfDE had the wisdom to offer fellowships to partner researchers and educators. I had the opportunity to work with Chuck to follow up on our research at the CFS during the 2020–21 academic year, to begin thinking about how to teach and design lessons for students at CFS. There is still considerable work to be done to bring the practical implications of this work to schools. Bringing this work into applied settings is very exciting, and I sincerely hope that this book provides some useful future directions.

The final part of writing this book occurred during the Covid lockdown in 2020–21. I truly believe that there is good and bad in every situation. While I could not see my parents very often during this time, our stay-at-home order allowed me to focus on my writing. I will always be grateful to my parents Veronika and Joseph for instilling the value of education in me and for their ongoing encouragement during this project. I re-connected with an old friend Glenn Becic, who provided regular messages of encouragement to help me stay on track with my writing.

I cannot thank my husband Paul and sons Stephen and Andrew enough for their patience during this entire project, but especially in this last year. Paul, I can always count on supportive encouragement from you. My older son Stephen’s reminder to finish strong always provided motivation to persist, and my younger son Andrew helped me stay calibrated and self-aware of what still needed to be done.

1: Defining cognitive sophistication in the development of judgment and decision-making

Abstract

Judgment and decision-making paradigms have been relatively well-studied in developmental samples. The measurement of these competencies in developmental samples has been of scientific interest. They have been recognized as having important implications for defining rational thinking in children and youth but also for teaching and training (such as, critical thinking in education). The origin of the theories and paradigms come from the adult literature, which has also undergone considerable progress in theoretical advancements and empirical studies over the last several years. The integration of our understanding from the work conducted in adults with consideration of developmental factors provides a way to advance our understanding of judgment and decision-making in children and youth. To accomplish this, establishing stimulus equivalence will be important given that these paradigms were first designed for adult samples. In addition, taking into account the rapid growth and change in cognitive capacities, that happen in development, are central for understanding performance on these paradigms. Using a working taxonomy of rational thinking based on adult samples, data from a longitudinal developmental study were used to empirically examine performance patterns on these paradigms.

Keywords

Judgment and decision-making; Children and youth; Development; Cognitive sophistication; Critical thinking; Rationality; Stimulus equivalence; Miserly processing

Judgment and decision-making skills are uniquely important competencies in children and adolescents. Assessing performance on these tasks requires an understanding of the rapid growth and change in cognitive abilities during childhood. An additional consideration is the measurement of judgment and decision-making skills in children and youth given that these paradigms were first designed for adult samples. We have referred to this issue as the stimulus equivalence problem (Stanovich, West, & Toplak, 2011a). This refers to whether the requisite knowledge and processing skills for generating both incorrect and correct responses on these tasks are available in samples of children.

One of the design features of the judgment and decision-making tasks originating from the heuristics and biases literature in adults was incorrect responses generated by miserly processing. The operationalization of miserly information processing in developmental samples is a key issue that interfaces with both stimulus equivalence and rapid growth in cognitive abilities. That is, how do we determine if children are doing the same task that was theoretically developed in adult samples and whether children have the skills and abilities to do such tasks?

Measuring miserly information processing in adult samples: Implications for developmental samples

The study of judgment and decision-making skills in adult samples has been based on a careful examination of response patterns on these tasks. A key insight from the heuristics and biases literature in adults was that individuals make systematic errors on these tasks (Gilovich, Griffin, & Kahneman, 2002; Kahneman, 2011; Kahneman, Slovic, Slovic, & Tversky, 1982). Similar observations were also occurring in the reasoning literature, such as understanding performance on the selection task (Evans, 2016, 2017; Evans, Newstead, & Byrne, 1993; Evans, Over, & Manktelow, 1993). These literatures described and empirically displayed systematic errors across several different judgment and decision-making tasks in adult samples (Evans & Frankish, 2009; Evans, Newstead, & Byrne, 1993; Evans, Over, & Manktelow, 1993; Gilovich et al., 2002; Kahneman, 2011; Kahneman et al., 1982; Stanovich, 1999, 2009a, 2011; Stanovich & West, 2000). Across these tasks, many incorrect responses were generated by miserly information processing (Stanovich, 2009a).

To illustrate this phenomenon, consider the following problem that I first studied as a PhD student (Toplak & Stanovich, 2002), that we called the Green Levels Problem.

There are five blocks in a stack, where the second one from the top is green, and the fourth is not green. Is there a green block directly on top of a non-green block?

(A) Yes,   (B) No,   (C) Cannot be determined

Looking at this image, try to imagine that these are a stack of five blocks. If the second block from the top is green and the fourth block is not green, is there a green block directly on top of a non-green block? After reading this question and reviewing the image, it likely becomes apparent that knowing the status of the third block would be helpful for determining the answer. Since this information is not provided, one may decide that an answer cannot be derived, as conveniently indicated by the Cannot be Determined option. However, using a disjunctive approach would be a useful strategy for deriving a better response (Shafir, 1994). A disjunctive approach involves an exhaustive consideration of the possibilities. The two different possibilities are that the third block is either green or not green. Considering the implications of these two possibilities, it becomes apparent that the correct answer to this question is Yes. If the middle third block is green, it sits directly on top of the not green fourth block, so the answer is Yes. If the middle third block is not green, it sits directly under the second block that is green, so the answer is still Yes. Either way, a green block is sitting directly on top of a nongreen block.

We found this disjunctive thinking problem in the artificial intelligence literature first described by Levesque (1986, 1989). All of my lab members were both enamored and puzzled at the same time by this problem. When we each first tried to solve this problem, many of us chose the incorrect response, Cannot be Determined. After choosing this response, I also remember having had a strong sense that this problem could not be solved and that this was likely the correct response. As students who were studying judgment and decision-making, we were somewhat embarrassed to find out that our first answer was incorrect. This embarrassment was further magnified when we found out the answer and how to solve this problem. We realized that the problem was not difficult, at least not from a computational perspective. Then I collected some data on this problem from a sample of university students. It was validating to find that only 9% of my participants (11 participants) got this problem correct and a whopping 84% of participants (105 participants) chose the Cannot be Determined incorrect response. Then, only 7% of participants (9 participants) chose the incorrect No response. The incorrect No response was chosen at a much lower rate than the incorrect Cannot be Determined response. There seemed to be something important about the large number of Cannot be Determined responders than the relatively low number of No responders. Both responses were incorrect, but there was something very interesting about the high prevalence of Cannot be Determined incorrect responders.

The Green Levels Problem was an important illustration of problems that seemed to have a unique characteristic, namely problems that are difficult because it is not apparent that an easily elicited initial response is incorrect. That is, the difficulty was recognizing that an alternative, better solution was available, not the complexity of the problem. When Frederick introduced our field to the Cognitive Reflection Test (CRT) with his paper published in 2005 (Frederick, 2005), this measure also seemed to capture some of the characteristics we identified with our Green Levels Problem. The most well-known item from the CRT is the Bat and Ball Problem:

A bat and a ball cost $1.10 in total. The bat costs a dollar more than the ball. How much does the ball cost? ____ cents

The most common response to this question is 10 cents, which is an incorrect response. However, if one sums 10 cents and $1.10, as the bat is $1 more than the ball, it immediately becomes apparent that 10 cents is not the correct answer. Calculating the correct answer of 5 cents then becomes relatively straight-forward. When we studied the CRT in a sample of 346 undergraduates, we found that 85 (24.6%) of our participants gave the correct response of 5 cents (Toplak, West, & Stanovich, 2011). Then, we found that 249 (72%) of our participants gave the answer of 10 cents, which was the miserly generated incorrect response, but only 12 participants (3.5%) gave other incorrect responses. Similar to the Green Levels Problem, this seemed to be an easy problem but one that elicited a high proportion of incorrect responses likely generated by miserly processing. These tasks are high on process dependence, requiring detection and override of miserly processing to derive a correct response (Stanovich, West, & Toplak, 2016). In these examples, the Bat and Ball Problem and the Green Levels Problem illustrate how miserly processing can lead to incorrect responses indicating issues related to detection. This was one type of cognitive failures attributable to miserly processing. In addition to detection, miserly processing can also lead to incorrect responses when there is an override failure and participants fail to generate the conflicting correct response, such as on belief bias syllogisms and framing tasks (Stanovich, 2018a; Stanovich et al., 2016).

An adapted version of the Green Levels Problem is now part of our Disjunctive Reasoning Subtest and cognitive reflection problems are included in the Reflection Versus Intuition Subtest of our Comprehensive Test of Rational Thinking (CART) for adults (Stanovich et al., 2016). Using this literature in adults as both a theoretical and empirical reference point, this book is about operationalizing and measuring these processing and knowledge requirements for successful performance on judgment and decision-making tasks in developmental samples.

Laying the groundwork for studying the development of judgment and decision-making

Judgment and decision-making tasks derive from the study of rationality in cognitive science. Stanovich (2011) describes how these tasks, collectively, assess components of both instrumental and epistemic rationality. The simplest definition of instrumental rationality is—Behaving in the world so that you get exactly what you most want, given the resources (physical and mental) available to you. Somewhat more technically, we could characterize instrumental rationality as the optimization of the individual’s goal fulfillment. Economists and cognitive scientists have refined the notion of optimization of goal fulfillment into the technical notion of expected utility. Epistemic rationality concerns how well beliefs map onto the actual structure of the world. Epistemic rationality is sometimes termed theoretical rationality or evidential rationality by philosophers. Likewise, instrumental rationality is sometimes termed practical rationality. The two types of rationality are related. In order to take actions that fulfill our goals, we need to base those actions on beliefs that are properly matched to the world. There is good evidence to suggest that children have both the capacity and inclination to seek truth (Fazio & Sherry, 2020; Koenig, Tiberius, & Hamlin, 2019; Moshman, 2009, 2011, 2015) and to pursue goals (Byrnes, 2002; Massey, Gebhardt, & Garnefski, 2008; Miller & Byrnes, 2001). Thus, this developmental work concerns both epistemic and instrumental rationality. Developmental researchers in the field of judgment and decision-making have also suggested that it is possible to evaluate the competence of youth to make rational decisions (Fischhoff, 2008), such as decisions that involve risk (Reyna & Farley, 2006).

Readers more immersed in educational psychology will be more familiar with the term critical thinking (Halpern, 1984, 1997; Kuhn, 1999). Critical reflective thinking in education was identified early on by John Dewey (1910) as the: active, persistent, and careful consideration of any belief or suppose it form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends, constitutes reflective thought. (p. 6). Since Dewey, the importance of fostering and teaching thinking skills has long been emphasized in the field of education (Byrnes & Dunbar, 2014; Kuhn, 1990, 1999). In the critical thinking literature, the ability to evaluate evidence and arguments independently of one’s prior beliefs and opinions is a skill that is strongly emphasized (Baron, 1991, 2000; Dole & Sinatra, 1998; Ennis, 1987, 1996; Perkins, 1995; Sternberg, 1997, 2001, 2003). These skills are captured in several measures of critical thinking (e.g., Ennis, Millman, & Tomko, 1985; Facione, 1990, 1992; Norris & Ennis, 1989; Watson & Glaser, 1980, 1991). The importance and value of critical thinking in education is reflected in the fact that these skills have been recommended to be taught across content areas and in an integrated manner across the curriculum (Angeli & Valanides, 2009; Ennis, 2018; Facione, 1990). The goals of critical thinking programs are to create habits of thinking that become a broad outcome of education. The broader goals of critical thinking in education are consistent with how cognitive scientists have conceptualized rational thinking (Toplak, West, & Stanovich, 2012). However, rational thinking is much more encompassing term that involves understanding response patterns and cognitive failures on paradigms from the cognitive science literature (Stanovich, 1999; Stanovich & West, 2000).

Just as the concept of critical thinking has had a longstanding history in the field of education, the study of reasoning in cognitive development has long historical roots in developmental psychology. These historical roots in the study of reasoning in children and youth provide insights into how developmentalists have framed these issues conceptually and methodologically. Piaget’s theory of cognitive development is a competency-based model, defined by whether children have reached certain cognitive milestones or abilities at each stage (Piaget, 1950; Siegler, 1991). For example, children who reach the period of concrete operations by 7–11 years of age can take the perspective of others, simultaneously hold multiple points of view and represent static and transformed situations relative to earlier stages of development. Each subsequent stage was proposed to bring with it the potential for solving many types of problems that children in earlier stages could not hope to conquer (p. 21, Siegler, 1991). According to this theory, development was thought to bring the gradual onset of logical thinking (Markovits, 2013a). Development was characterized by the acquisition of increasingly complex and sophisticated cognitive skills. There seemed to be an implicit assumption that developmental changes would translate into better performance on many tasks. This implicit assumption, perhaps not surprisingly, was reinforced by an empirical literature that has shown that children and youth display increases or improvements on several cognitive abilities (which will be elaborated on further in Chapter 2). Similarly, the observation of positive manifold in cognitive abilities, namely positive correlations between distinct cognitive abilities in adults (Stanovich, 1999) has also been described in developmental samples (Kievit, Hofman, & Nation, 2019; Kievit et al., 2017). These studies illustrated the consistent improvement in the efficiency and capacity of these abilities over the course of development.

Piaget’s competency-based model was based on the assumption or expectation of better performance on several cognitive developmental tasks. This assumption seemed to continue into Neo-Piagetian, information processing, and social-cultural (Vygotskian) perspectives on development (Flavell, Miller, & Miller, 1993). Neo-Piagetian researchers later reported that some children seemed to display higher competence than expected and that some adolescents and adults did not reach some of the cognitive developmental milestones at later stages (Flavell et al., 1993). These findings were interpreted as characterizing what seemed to be unexpected levels of competence early in development and a lack of competence later in development. Judgment and decision-making researchers have also remarked on what has been described as a paradox between the seeming competence by young children and surprising poor performance in adults. For example, developmentalists in the field of reasoning began raising questions about how cognitive development can give rise to better logical thinking in childhood but also how development may give rise to belief-biased thinking among adults (Barrouillet & Gauffroy, 2013; Markovits, 2013b). Such a framing among developmentalists reinforced a continuum of development starting from less competence and less rational to more competence and more rational. Others have suggested that there is no paradox given that development provides the opportunity for better reasoning and rationality, but that this development does not guarantee optimal reasoning performance (Moshman, 2015; Schlottmann & Wilkening, 2011).

If a conceptual separation is made between the capacity for comprehending and processing more complex information from rational or normative responding on judgment and decision-making tasks, there is no paradox in development that requires explanation. That is, the development of the capacity for comprehending and processing more complex information is conceptually separate from the development of rational or normative responding on judgment and decision-making tasks. This distinction has also been emphasized by dual-process researchers in the adult literature (Evans & Stanovich, 2013).

This conceptual separation between processing and responses is consistent with the distinction that has been made between measures of intelligence and rationality in adult samples (Stanovich, 2009a; Stanovich, Toplak, & West, 2020; Stanovich, West, & Toplak, 2011b). Cognitive processing may begin as less efficient and less complex in children and become increasingly efficient and complex with development. However, we would not describe processing as rational or irrational, but rather as efficient or inefficient or as having high or low capacity (Stanovich, 2011). Recall the definition of rational thinking previously described, where rationality concerns the actions of individuals in their environment that serve their goals and tracking truth in the world. Based on this definition, rational choices would not be defined by their efficiency or as a demonstration of complex thinking, but rather how well the choice serves the individuals goals and tracking of truth in the world. Thus, rational thinking becomes an evaluative endeavor about an individual’s judgment and choices (Stanovich, 1999; Stanovich & West, 2000; Stanovich et al., 2016), such as how well individuals follow certain axioms of choice and resist context effects.

The psychological processing underlying rational thinking can be understood using Stanovich’s tripartite model of the mind (Stanovich, 1999, 2009a, 2011). That model distinguishes between three levels, including biological, algorithmic, and intentional levels of analysis. The biological level refers to the hardware of the system that is less accessible to cognitive theorizing. The algorithmic level refers to the computational processing needed to carry out any cognitive task, such as cognitive abilities. Finally, the intentional level is concerned with the goals of the individual, the beliefs relevant to those goals and the choice of action that can be evaluated as rational or not given the goals and beliefs of the individual (Stanovich, 1999). Together, all of these levels contribute to the rationality of the individual (Stanovich, 2009a). The efficiency and capacity of computational processing is one component in the model that supports rational thinking, but do not explain all of the variance in rational thinking performance (Stanovich, 1999, 2009a). Extending this distinction to developmental samples, children may demonstrate more efficient and complex processing with emerging development, but these abilities are separable from rational responding. Thus, similar to the observation of smart people doing foolish things that has been well described in adult samples (Stanovich, 2009a), it is not surprising that children and youth with high cognitive abilities or processing capacities may not demonstrate high competency on rational thinking tasks.

The assessment of rational thinking in developmental samples: Using adult models and stimulus equivalence

While the study of judgment and decision-making in adults has served as an important reference point for advancing our understanding of the development of these skills and abilities, there are two main issues that need to be considered. One of these issues is the implications of using models of rational thinking from adult samples for understanding the development of these competencies. The other is the implications of importing the tasks and paradigms from adult samples for child and youth samples, which we have called the stimulus equivalence problem (Stanovich et al., 2011a).

The Great Rationality Debate in adults: Implications for developmental samples

The gap between descriptive and normative models has been used to evaluate performance on judgment and decision-making tasks in adults (Baron, 1988). Descriptive models refer to theories of how people typically think and make decisions. Normative models provide a standard for defining the type of thinking that is optimal for achieving one’s goals. In adult samples, there has been an extensive empirical literature documenting the gap between descriptive and normative models. Namely, many adults tend to provide a different response than indicated by a normative model and the interpretation of this gap has been called the Great Rationality Debate (Stanovich & West, 2000). The potential gap between these models (including how normative should be defined) have fueled these debates in defining human rationality in adult samples (Stanovich, 1999). Stanovich (1999) added prescriptive models to these theories, which refers to specifying how processes of belief formation and decision-making should be carried out, given the limitations of the human cognitive apparatus and the situational constraints (e.g., time pressure) with which the decision-maker must deal (p. 3). A focus on examining normative, descriptive and prescriptive aspects of behavior has also been suggested as providing an integrated structure for addressing adolescent judgment and decision-making, such as understanding, assessing and helping teens do better (Bruine de Bruin, 2012; Fischhoff, 2008).

One might have different standards for evaluating performance depending on whether one assumes any gaps between normative, descriptive and prescriptive models, or even the degree of gap between these different models. Three different positions have been described, including Meliorist, Panglossian and Apologist positions (Stanovich, 1999). Meliorist perspectives acknowledge that there are gaps between descriptive and normative models, and that prescriptive models should encompass striving to do better than descriptive models and aiming for normative models. Meliorist perspectives also recognize that computational and situational limits will only allow us to get closer to normative models, but that normative models may not be achieved. Panglossian positions suggest that there are no gaps between descriptive, prescriptive and normative models. Finally, Apologists acknowledge gaps between descriptive and normative models, but due to computational limitations, define normative as similar or close to descriptive models. These positions offer different implications for cognitive remediation, with the Meliorist position providing the most potential for cognitive remediation efforts and the opportunity for generating better responses than may be available at descriptive levels. The focus on critical thinking in education (Kuhn, 2005) and efforts to teach and train critical thinking skills in children and youth (Baron & Brown, 1991; Halpern, 1997; Halpern & Riggio, 2003; Kuhn, Hemberger, & Khait, 2016) are consistent with Meliorist perspectives.

Stanovich and West (Stanovich, 1999; Stanovich & West, 2000) provided an extensive analysis of the potential reasons to explain differences between descriptive and normative models based on participant’s performance across several reasoning and heuristics and biases tasks. In particular, they examined four major explanations: (1) performance errors, which refer to temporary lapses of attention, memory deactivation and other sporadic information processing mishaps; (2) stable and inherent computational limitations; (3) application of the wrong normative model; and (4) participant construal suggesting that the participant interpreted the problem differently and may be providing the normative answer to a different problem.

Performance errors refer to failures in applying rules and strategies because of a momentary or random lapse in attention or memory, including distractions (Stanovich & West, 2000). The fact that correlations between performance on several rational thinking tasks have been positive and statistically significant were interpreted as a lack of evidence for a strong version of the performance error explanation. Similar positive and significant correlations have also been observed in developmental samples (Klaczynski, 2001a; Kokis, Macpherson, Toplak, West, & Stanovich, 2002; Toplak & Flora, 2020; Toplak, West, & Stanovich, 2014a; Weller, Levin, Rose, & Bossard, 2012). Computational limitations refer to the limitations of cognitive resources that may explain the discrepancy between descriptive and normative models of rational thinking performance. In the adult literature, performance on several rational thinking tasks has been shown to be significantly correlated with cognitive abilities (Bruine de Bruin, Parker, & Fischhoff, 2007; Parker & Fischhoff, 2005; Stanovich & West, 2008a; Stanovich et al., 2016). Similar findings have also been reported in developmental samples (Chiesi, Primi, & Morsanyi, 2011; Primi, Morsanyi, Donati, Galli, & Chiesi, 2017; Toplak et al., 2014a). Thus, the first two explanations from adult models of rational thinking have displayed similar patterns of findings in developmental samples, suggesting that these explanations are also not controversial in child and youth samples.

Applying the wrong normative model and alternative task interpretation by the participant have been raised in interpretations of adult performance on several heuristics and biases tasks (Stanovich & West, 2000). Applying the wrong normative model suggests that the normative standards have been incorrectly applied to actual performance on a given task. The understanding/acceptance assumption, originally described by Slovic and Tversky (1974), suggests that reflective reasoners should adopt the appropriate normative model for a given problem. There has been some evidence to suggest that using instructions to induce understanding and acceptance significantly increased correct responding on several heuristics and biases tasks (Stanovich & West, 1999). However, the normative status of several reasoning problems has not been straightforward in the adult literature (Stanovich & West, 2000). Thus, one of the implications for the study of these paradigms in developmental samples is avoiding paradigms that have been contentious in the adult literature, or until there is more clarity in the adult literature regarding the normative status of these problems.

Finally, alternative task interpretation refers to the possibility that the participant may have an alternative interpretation of what is being asked in a given problem. This explanation suggests that the participant may be giving a normative response to a different problem than was intended by the experimenter. Task instructions have been shown to be extremely important for establishing that the participant has the correct task set in adult samples. This consideration is even