مطالعات اندازه گیری و ارزشیابی آموزشی

مطالعات اندازه گیری و ارزشیابی آموزشی

شایستگی‌های پایه و محتوایی ریاضی گروه انسانی: پیش‌بینی دشواری سوال‌های آزمون سراسری

نوع مقاله : مقاله پژوهشی

نویسندگان
1 استادیار سنجش و اندازه‌گیری، دانشگاه علامه طباطبائی، تهران، ایران
2 استادیار، سازمان سنجش آموزش کشور، تهران، ایران
3 دانشیار روش‌ها و برنامه‌های درسی و آموزشی، دانشگاه تهران، تهران، ایران
4 سازمان سنجش آموزش کشور، تهران، ایران.
10.22034/emes.2025.2058071.2648
چکیده
هدف: پژوهش حاضر با هدف شناسایی شایستگی‌های پایه و محتوایی مؤثر در پاسخگویی به سؤال‌های درس ریاضی کنکور سراسری گروه انسانی بر اساس رویکرد شناختی و تبیین میزان دشواری سؤال‌ها بر اساس این شایستگی‌ها است.
روش پژوهش: پژوهش به روش ترکیبی کیفی و کمی اجرا شد و چند جامعه داشت. جامعه نخست، دانشجویان سال اول دانشگاه بودند که 6 نفر انتخاب‌ شدند. جامعه دیگر شامل متخصصان موضوعی و طراحان سؤال بود که 8 نفر به‌صورت هدفمند و با رویکرد تنوع حداکثری انتخاب شدند. آخرین جامعه آماری، پاسخنامه شرکت‌کنندگان در دو نوبت آزمون سراسری 1402 بود. برای نوبت اول 71730 و نوبت دوم 79221 به صورت تصادفی انتخاب گردید.
یافته‌ها: با بررسی شیوه تفکر بلند دانشجویان، پیشینه پژوهش و کتاب‌های درسی ریاضی گروه انسانی توسط متخصصان موضوعی و طراحان، 13 شایستگی پایه و 6 شایستگی محتوایی شناسایی شدند. شایستگی‌های کاربرد قواعد جبری و مسائل چندمرحله‌ای پرکاربردترین شایستگی‌ها در نوبت اول بودند و در نوبت دوم درک پیچیدگی سوال پرکاربردترین شایستگی بود. شایستگی‌های پایه 56% ، شایستگی‌های محتوایی 31% و مجموع هر دو شایستگی 64% درصد واریانس دشواری سوالات را تبیین می‌کردند.
نتیجه‌گیری: شایستگی‌های پایه و محتوایی در مقایسه با روش‌های مرسوم در پیش‌بینی دشواری سوالات پیش از اجرا، عملکرد بهتری دارند و استفاده از آنها در ساخت آزمون سراسری می‌تواند به کمینه کردن تفاوت دشواری سوال‌های دو نوبت و عادلانه بودن آزمون کمک کند.
کلیدواژه‌ها

موضوعات


عنوان مقاله English

Basic and Content Competencies in Mathematics for the Humanities Group: Predicting the Difficulty of Items in the National Exam

نویسندگان English

Enayatollah Zamanpour 1
َAbdolkarim Shadmehr 2
Reza Mohammadi 3
Shoeayb Qasemi 4
1 Assistant Professor, Department of Assessment and Measurement, Allameh Tabataba'i University, Tehran, Iran
2 Assistant Professor, National Organization of Educational Testing, Tehran, Iran
3 Associate Professor, Faculty of Psychology and Educational Sciences, University of Tehran, Tehran, Iran
4 National Organization of Educational Testing, Tehran, Iran.
چکیده English

Objective: The present study aims to identify the basic and content competencies effective in answering the mathematics items of the national university entrance exam for the Humanities group based on a cognitive approach, and to predict the level of difficulty of the items according to these competencies.
Methods: The study was conducted using a mixed method. The first population consisted of first-year university students, from which 3 individuals were selected. Another population included subject matter experts and item writers, with 8 individuals purposefully selected. The final population comprised the participants in two sessions of the 2023 national exam, with 71,730 randomly selected for the first session and 79,221 for the second session.
Results: Through student think-aloud protocols, literature review, and analysis of humanities group mathematics textbooks by subject matter experts and item writers, 13 basic competencies and 6 content competencies were identified. The competencies of applying algebraic rules and solving multi-step problems were the most frequently used in the first test, while understanding question complexity was the most frequently used in the second test. Basic competencies accounted for 56%, content competencies for 31%, and the combination of both for 64% of the variance in question difficulties.
Conclusion: Basic and content competencies outperform conventional methods in predicting question difficulty prior to exam administration. Their use in constructing the national exam can help minimize differences in question difficulty between the two sessions and contribute to the fairness of the exam.

کلیدواژه‌ها English

Keywords: National Exam
Mathematics
Item Difficulty
Basic Competencies
Content Competencies
 

References

Attali, Y., Saldivia, L., Jackson, C., Schuppan, F., & Wanamaker, W. (2014). Estimating item difficulty with comparative judgments. ETS Research Report Series, 2014(2) , 1-8.
Benton, T. (2020). How Useful Is Comparative Judgement of Item Difficulty for Standard Maintaining?. Research Matters, 29, 27-35.
Chandía, E., Sanhueza, T., Mansilla, A., Morales, H., Huencho, A., & Cerda, G. (2023). Nonparametric cognitive diagnosis of profiles of mathematical knowledge of teacher education candidates. Current Psychology, 42(36), 32498-32511.
Corter, J. E., Tatsuoka, K., Guerrero, A., Dean, M., & Dogan, E. (2006). Revised coding manual for identifying item involvement of content, context, and process subskills for the TIMSS and TIMSS-R 8th grade mathematics tests. Technical Report MES-06-01, Department of Human Development, Teachers College, Columbia University.
Creswell, J. W., Clark, V. L. P., Gutmann, M. L., & Hanson, W. E. (2003). Advanced mixed. Handbook of mixed methods in social & behavioral research, 209, 209-240.
ElMasri, Y. H., Ferrara, S., Foltz, P. W., & Baird, J. A. (2017). Predicting item difficulty of science national curriculum tests: the case of key stage ۲ assessments. The Curriculum Journal, 28(۱) , 59-82.
Hamamoto Filho, P. T., Silva, E., Ribeiro, Z. M. T., Hafner, M. D. L. M. B., Cecilio-Fernandes, D., & Bicudo, A. M. (2020). Relationships between Bloom’s taxonomy, judges’ estimation of item difficulty and psychometric properties of items from a progress test: a prospective observational study. Sao Paulo Medical Journal, 138, 33-39.
Hambleton, R. K., & Jirka, S. J. (2014). Anchor-based methods for judgmentally estimating item statistics (Yadegarzadeh, Gh., Sharifi yeganeh, N., Khodaii, E, Trans). In Handbook of test development (pp. 413-434). Routledge. (Original work published 2006(
Minaei, A., Delavar, A., Falsafinezhad, M. R., Kiamanesh, A. R., & Mohajer, Y. (2014). Cgnitive diagnostic modeling of Iranian eight grade student to mathematics items of TIMSS 2007 using reduced noncompensatory reparameterized unified model and comparison between girls and boys. Quarterly of Educational Measurement, 5(16), 138-170.
OECD. (2013). PISA 2012 assessment and analytical framework: Mathematics, reading,  science, problem solving and financial literacy. OECD Publishing.
Pitoniak, M. J., & Cizek, G. J. (2016). Standard setting. In C. S. Wells & M. Faulkner-Bond (Eds.) , Educational measurement: From foundations to future (pp. 38–61). The Guilford Press.
Rezigalla, A. A. (2024). AI in medical education: uses of AI in construction type A MCQs. BMC medical education, 24(1), 247.
Saadati, S., Moghadamzadeh, A., Minaei, A., & Geramipour, M. (2020). Differential item functioning in the framework of cognitive diagnostic assessment: Questions related to the differential and integral calculus of the Iranian national university entrance examination 2018. Biquarterly Journal of Cognitive Strategies in Learning, 8(15), 19-35.
Shadmehr, A., Zamanpour, E., & Qasemi, S. (2024). The equating requirements of scores in alternative forms of high-stakes tests: the case study of the national entrance exam of the foreign language applicants. Quarterly of Educational Measurement15(57), 33-53.
Suri, H. (2011). Purposeful sampling in qualitative research synthesis. Qualitative research journal11(2), 63-75.
Tatsuoka, K. K. (2009). Cognitive assessment: An introduction to the rule space method. Routledge.
Tatsuoka, K. K., & Boodoo, G. M. (2000). Subgroup differences on the GRE quantitative test based on the underlying cognitive processes and knowledge. In Handbook of research design in mathematics and science education (pp. 821-857). Routledge.
Thorndike, R, L. (1996). Apllied psychometrics (Hooman, H,A, Trans). Houghton Mifflin School.(Original work published 1982(
Turner, R., & Adams, R. J. (2012). Some drivers of test item difficulty in mathematics: an analysis of the competency rubric.
Turner, R., Dossey, J., Blum, W., & Niss, M. (2013). Using mathematical competencies to predict item difficulty in PISA: A MEG study. In Research on PISA: Research outcomes of the PISA Research Conference 2009 (pp. 23-37). Springer Netherlands.
Van de Watering, G., & van der Rijt, J. (2006). Teachers’ and students’ perceptions of assessments: A review and a study into the ability and accuracy of estimating the difficulty levels of assessment items. Educational Research Review, 1(2), 133-147.
Van Onna, m., Lampe, t., & Crompvoets, E. (2019). Equating by pairwise comparison. Presentation at the 20th annual AEA-Europe conference, Lisbon, Portugal