What is Scientific Reasoning and Why is it Important?
Science and mathematics education is emphasized worldwide. Reports from large-scale international studies such as TIMSS and PISA continually rank U.S. students behind many other nations. As a result, the U.S. has increased its emphasis on the implementation of a more extensive curriculum in K-12 education in science, technology, engineering, and mathematics (STEM). Educational reforms stress the need for a prepared 21st century workforce, which translates into students learning not only science content, but also acquiring advanced transferable reasoning skills (Bybee & Fuchs, 2006). The development of these skills will better enable students to handle open-ended novel situations and design their own investigations to solve scientific, engineering, and social problems in real world (Bao et al., 2009; Iyengar et al., 2008; Bloom, 1956; NRC 1996, 2002, 2005).
An important component of these abilities is scientific reasoning, which broadly defined, includes the thinking and reasoning skills involved in inquiry, experimentation, evidence evaluation, inference and argumentation that support the formation and modification of concepts and theories about the natural and social world (Zimmerman, 2007). Scientific reasoning is a general ability and methodology that is critical in enabling the successful management of real-world situations in professions beyond the classroom.
In K-12 education, the development of scientific reasoning skills has been shown to have a long-term impact on student academic achievement (Adey & Shayer, 1994). Positive correlations between student scientific reasoning abilities and measures of students’ gains in learning science content have been reported (Coletta & Phillips, 2005). These findings support the consensus of the science education community on the need for K-12 students to develop an adequate level of scientific reasoning skills along with a solid foundation of content knowledge.
Traditionally, it is often expected that rigorous content learning in science and mathematics will help develop students’ scientific reasoning abilities; however, recent studies have shown that the traditional style of STEM education has little impact on the development of students’ scientific reasoning abilities (Bao et al., 2009). It is not what we teach but rather how we teach that makes a difference in student learning of higher order abilities such as scientific reasoning.
On developing scientific reasoning, research has shown that inquiry based science instruction can promote scientific reasoning abilities (Adey & Shayer, 1990; Lawson, 1995; Marek & Cavallo, 1997; Benford & Lawson, 2001; Gerber, Cavallo & Marek, 2001). Controlled studies have shown that students had higher gains on scientific reasoning abilities in inquiry classrooms over non-inquiry classrooms (Bao et al., 2009). On the other hand, students’ and instructors’ levels of reasoning skills can significantly impact the effectiveness of using inquiry methods in teaching and learning science courses (Kuhn et al. 2000; Benford & Lawson, 2001). Therefore, in order to effectively implement inquiry based curricula, improving scientific reasoning abilities need to be highly emphasized K-12 education for both students and teachers.
Research on Assessment of Scientific Reasoning
There exists a large body of research on the multifaceted aspects of scientific reasoning. Zimmerman (2007) made a comprehensive review on the related work using the Klahr’s (2000, 2005) Scientific Discovery as Dual Search (SDDS) model as the general framework that organizes the main empirical findings in three areas including experiential skills, evidence evaluation skills, and integrated approaches in self-directed experimentation (Klahr, 2000, 2005; Zimmerman, 2007). Kuhn ( 2002) has argued that the defining feature of scientific thinking is the set of skills involved in differentiating and coordinating theory and evidence (Kuhn, 1989, 2002). The specific set of skills in scientific reasoning included the isolation and control of variables, producing the full set of factorial combinations in multivariable tasks, selecting an appropriate design or a conclusive test, generating experimental designs or conclusive tests, record keeping, the inductive skills implicated in generating a theory to account for a pattern of evidence, and general inference skills involved in reconciling existing beliefs with new evidence that either confirms or disconfirms those beliefs (Zimmerman, 2007). Elements concerning casual mechanisms (Koslowski, 1996) and epistemological understandings (Chinn & Malhotra, 2002) have also been carefully examined and debated.
In our current research on assessment of scientific reasoning, we focus on a set of basic reasoning skills that are commonly needed for students to systematically conduct scientific inquiry, which includes exploring a problem, formulating and testing hypotheses, manipulating and isolating variables, and observing and evaluating the consequences. The Lawson’s Test of Scientific Reasoning (LTSR) provides a solid starting point for assessing scientific reasoning skills (Lawson, 1978, 2000). The test is designed to examine a small set of dimensions including (1) conservation of matter and volume, (2) proportional reasoning, (3) control of variables, (4) probability reasoning, (5) correlation reasoning, and (6) hypothetical-deductive reasoning. These skills are important concrete components of the broadly defined scientific reasoning ability.
To fully assess students’ ability and provide fine tuned guidance for teachers, we have been working to expand the measurement capability of standardized assessment on scientific reasoning by incorporating sub-categories within the existing skill dimensions and new dimensions that are not included in the Lawson’s test. For example, we have developed questions on conditional probability and Bayesian statistics within the general category of probability reasoning as well as questions on an extended list of additional skill dimensions such as categorization, combinations, logical reasoning, causal reasoning, and advance hypothesis forming and testing. In addition, for each skill dimension, multiple questions are designed using a wide variety of scientific and social contexts and with different levels of complexity, so that we can measure students with different background and strengths from school age through college levels. These new dimensions and designs will improve the measurement capability to target students at a wider range of grade levels and backgrounds, and also provide more detailed information for researchers and teachers to address the development of scientific reasoning skills and the interactions of these skills with other aspects of learning is STEM education.
The new assessment instrument on scientific reasoning is called Inventory for Scientific Thinking and Reasoning (iSTAR – www.iSTARAssessment.org*), which includes approximately 150 questions covering a dozen SR dimensions. The development and validation of the instrument are supported by NIH and NSF grants and we expect to deliver, by the end of 2011, the final version of the instrument, which contains about 300 questions that are scaled and equated based on IRT method for using with students from third grade through senior college levels. Examples of the new questions will be released shortly.
Review of Scientific Reasoning
In general, we define scientific reasoning as domain general abilities along several skill dimensions. A developing list of such skill dimensions includes
- Control of Variables
- Proportions and Ratios
- Correlational Reasoning
- Basic Logical Reasoning
- Inductive reasoning
- Causal Reasoning
- Hypothetical-Deductive Reasoning
Adey, P. and M Shayer, Really Raising Standards: Cognitive Intervention and Academic Achievement (London: Routledge, 1994).
Adey, P. and M. Shayer. (1990). “Accelerating the development of formal thinking in middle and high school students,” Journal of Research in Science Teaching, 27, 267-285.
Bao, L., Cai, T., Koenig, K., Fang, K., Han, J., Wang, J., Liu, Q., Ding, L., Cui, L., Luo, Y., Wang,Y., Li, L., Wu, N. (2009). “Learning and Scientific Reasoning”, Science, Vol. 323. no. 5914, pp. 586 – 587.
Benford, R. & A. E. Lawson. (2001). “Relationships between Effective Inquiry Use and the Development of Scientific Reasoning Skills in College Biology Labs,” MS Thesis, Arizona State University. ERIC Accession Number: ED456157.
Bloom, B. S. (Ed.), Taxonomy of Educational Objectives: The Classification of Educational Goals, Handbook I: Cognitive Domain (David McKay, New York 1956).
Bybee, R., & B. Fuchs. (2006). Preparing the 21st century workforce: a new reform in science and technology education. Journal of Research in Science Teaching, 43(4), 349-352.
Chinn, C. A., & Malhotra, B. A. (2002). Epistemologically authentic inquiry in schools: A theoretical framework for evaluating inquiry tasks. Science Education, 86, 175-218.
Coletta, V. P. & J.A. Phillips. (2005). “Interpreting FCI scores: Normalized gain, reinstruction scores, and scientific reasoning ability,” Am. J. Phys., 73(12), 1172-1179.
Gerber, B.L., A.M. Cavallo, & E.A. Marek. (2001). “Relationships among informal learning environments, teaching procedures and scientific reasoning ability,” International Journal of Science Education, 23(5):535-549.
Iyengar, R. et al. (2008). “Inquiry Learning: Integrating Content Detail and Critical Reasoning by Peer Review”, Science 319, 1189.
Klahr, D. (2000). Exploring science: The cognition and development of discovery processes. Cambridge: MIT Press.
Klahr, D. (2005). A framework for cognitive studies of science and technology. In M. Gorman, R. D. Tweney, D.C. Gooding, & A. P. Kincannon (Eds.), ScientiWc and technological thinking (pp. 81-95). Mawah, NJ: Lawrence Erlbaum.
Koslowski, B. (1996). Theory and evidence: The development of scientific reasoning. Cambridge: MIT Press.
Kuhn, D. (1989). Children and adults as intuitive scientists. Psychological Review, 96, 674-689.
Kuhn, D. (2002). What is scientific thinking and how does it develop? In U. Goswami (Ed.), Blackwell handbook of childhood cognitive development (pp. 371-393). Oxford: Blackwell Publishing.
Kuhn, D., J. Black, A. Keselman, & D. Kaplan, (2000). “The Development of Cognitive Skills to Support Inquiry Learning,” Cognition and Instruction, 18 (4), 495-523.
Lawson, A. E. (1978). “The development and validation of a classroom test of formal reasoning,” Journal of Research in Science Teaching, 15(1), 11-24. Test used in study: Classroom Test of Scientific Reasoning, revised ed. (2000).
Lawson, A. E. (1995). Science Teaching and the Development of Thinking, Belmont, CA: Wadsworth Publishing Company.
Marek, E. A. & A. M. L. Cavallo. (1997). The Learning Cycle and Elementary School Science, Portsmouth, NH: Heinemann.
National Research Council (NRC), National Science Education Standards ((National Academies Press, Washington, DC, 1996).
NRC, Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools (National Academies Press, Washington, DC, 2002).
Singer, S. H., M. L. Hilton, H. A. Schweingruber, Eds., America’s Lab Report (National Academies Press, Washington, DC, 2005).
Zimmerman, C. (2007). The development of scientific thinking skills in elementary and middle school. Developmental Review, 27, 172-223.