Developing Rubrics for Evaluating Evidence Quality, Logical Soundness, and Cognitive Engagement in Debate and Argumentation-Based Training
Keywords:
debate, argumentation, assessment rubric, evidence quality, logical soundness, cognitive engagement, critical thinking, inter-rater reliability, higher educationAbstract
Debate and argumentation exercises are widely used in higher education and professional training to develop critical thinking, yet their outcomes are often assessed with general, subjective criteria. This study aims to develop and validate an analytic rubric that integrates three constructs—evidence quality, logical soundness, and cognitive engagement—for evaluating performance in debate and argumentation-based training. An integrative design was employed, comprising theoretical analysis of existing evaluation systems, construct and descriptor development, expert review by five specialists in education and logic, and a pilot study in which two independent raters assessed 30 university students during a debate exercise. The final rubric comprises three constructs and nine criteria, each with level-specific descriptors. Experts rated its content completeness, precision, and applicability highly (mean 4.6 on a 5-point scale). The pilot study showed high internal consistency (Cronbach’s α = 0.85) and substantial inter-rater agreement, confirming the rubric’s reliability and objectivity; the comparatively lower scores on logical soundness indicate a priority area for instruction. The rubric functions not only as an assessment instrument but also as a diagnostic and formative tool that clarifies expectations for learners and guides instructors in planning debate-based teaching. Rater training and further adaptation to different age groups, debate formats, and AI-assisted assessment are recommended.
References
[1] T. Govier, A Practical Study of Argument, 7th edn. Belmont, CA, USA: Wadsworth Cengage Learning, 2010.
[2] University of Cambridge, ‘Cambridge Assessment: Principles for Developing Rubrics’. 2020. [Online]. Available: https://www.cambridgeassessment.org.uk
[3] J. Arter and J. Chappuis, Creating and Recognizing Quality Rubrics. Portland, OR, USA: Pearson Assessment Training Institute, 2006.
[4] A. Fisher, Critical Thinking: An Introduction. Cambridge, U.K.: Cambridge University Press, 2001.
[5] R. Paul and L. Elder, Critical Thinking: Learn the Tools the Best Thinkers Use. Upper Saddle River, NJ, USA: Pearson Prentice Hall, 2006.
[6] International Debate Education Association (IDEA), ‘Debate and Argumentation Resources’. 2022. [Online]. Available: https://idebate.org/resources
[7] The Critical Thinking Community, ‘Defining Critical Thinking’. [Online]. Available: https://www.criticalthinking.org/pages/defining-critical-thinking/766
[8] A. Sh. Rahmonov, ‘Developing Students’ Logical Thinking Abilities’, Xalq Ta’limi, no. 5, pp. 28–33, 2019.
[9] Educational Testing Service (ETS), ‘ETS HEIghten Critical Thinking Assessment’. 2016. [Online]. Available: https://www.ets.org/heighten/about/critical_thinking
[10] D. N. Walton, Fundamentals of Critical Argumentation. Cambridge, U.K.: Cambridge University Press, 2006.
[11] S. M. Brookhart, How to Create and Use Rubrics for Formative Assessment and Grading. Alexandria, VA, USA: ASCD, 2013.
[12] J. Dewey, How We Think. Boston, MA, USA: D. C. Heath & Co., 1910.
[13] A. Tversky and D. Kahneman, ‘Judgment under Uncertainty: Heuristics and Biases’, Science, vol. 185, no. 4157, pp. 1124–1131, 1974, doi: 10.1126/science.185.4157.1124.
[14] D. M. Hasanova, ‘Pedagogical and Psychological Foundations of Developing Critical Thinking’, Pedagogik Mahorat, vol. 3, no. 12, pp. 45–51, 2020.
[15] Aristotle, Rhetoric. 350 AD.
[16] D. Kahneman, Thinking, Fast and Slow. New York, NY, USA: Farrar, Straus and Giroux, 2011.
[17] S. E. Toulmin, The Uses of Argument. Cambridge, U.K.: Cambridge University Press, 1958.
[18] D. F. Halpern, Thought and Knowledge: An Introduction to Critical Thinking, 5th edn. New York, NY, USA: Psychology Press, 2014.
