Assessing Polytomous Cognitive Attributes of Mathematics Literacy of 9th Grade Students: Supplying PGDINA Model

Document Type : Original Article

Author

10.22034/emes.2019.36116

Abstract

Over the recent decades a new theoretical framework entitled cognitive diagnostic assessment (CDA) has gained a special status in the field of educational measurement as an approach to integrate cognitive theories with education. This framework provides formative diagnostic feedback through a fine-grained reporting of learners’ attribute mastery profiles which is necessary to respond to test items. The present study, through application of Polytomouse Deterministic Input, Noisy ‘‘and’’ gate [de la Torre & Douglas, 2013], investigated the degree to which the items of a mathematical literacy test can provide diagnostically useful information. Mathematics literacy test was designed based on PISA framework considering social and cultural characteristics. The construction of 20-item test was based on six cognitive competencies. Through expert rating, a Q-matrix including six fundamental cognitive attributes consisting of communication, mathematizing, representation, reasoning and argument, devising strategies for solving problems andusing symbolic, formal and technical language and operations was developed and data collected from 700 15-year-old students. Using PGDINA model, model fit indicators, latent Class Probabilities, items’ parameters, and student’s attributes profile were produced. Results indicated flexibilities of the PGDINA model and items can provide diagnostically useful information based on proposed model.

Keywords

References

محسن‌پور، مریم؛ گویا، زهرا؛ شکوهی‌یکتا، محسن؛ کیامنش، علیرضا و بازرگان، عباس(1393). طراحی و ساخت آزمونی برای صلاحیت‌های شناختی سواد ریاضی دانش‌آموزان ایرانی بر مبنای مطالعات پیزا. دوفصلنامهنظریهوعملدربرنامهدرسی، 2 (4)، 5 – 34.
Chen, J. & de la Torre, J. (2013). A general cognitive diagnosis model for expert-defined polytomous attributes. Applied Psychological Measurement, 37, 419-437.
De la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.doi:10.1007/s11336-011-9207-7
Di Bello, L. V.; Roussos, L. A. & Stout, W. (2007). Review of Cognitively Diagnostic Assessment and a Summary of Psychometric Models. In C. V. Rao & S. Sinharay (Eds.), Handbook of statistics (Vol. 26, Psychometrics, pp. 979-1027). Amsterdam, the Netherlands: Elsevier.
Haberman, S. J.; von Davier, M. & Lee, Y. (2008). Comparison of multidimensional item response models: multivariate normal ability distributions versus multivariate polytomous distributions (ETS Research Rep. No. RR-08-45). Princeton, NJ: ETS.
Jang, E. (2008). A Framework for Cognitive Diagnostic Assessment. In C. A. Chapelle, Y.‐R. Chung, & J. Xu (Eds.), towards adaptive CALL: Natural language processing for diagnostic language assessment (pp. 117‐131). Ames, IA: Iowa State University.
Karelitz, T. M. (2004). Ordered category attribute coding framework for cognitive assessments (Unpublished doctoral dissertation). University of Illinois at Urbana-Champaign.
Kunina‐Habenicht, Olga; Rupp, André A. & Wilhelm, Oliver (2012). The Impact of Model Misspecification on Parameter Estimation and Item‐Fit Assessment in Log‐Linear Diagnostic Classification Models. JEM, 49 (1), 59 – 81.
Leighton, J. P. Gierl, M. J. & Hunka, S. M. (2004). The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka’s Rule-Space Approach. Journal of Educational Measurement, 41 )3(, 205-237.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.
Niss, M. & Højgaard, T. (Ed.). (2011). Competencies and Mathematical Learning, Ideas and inspiration for the development of mathematics teaching and learning in Denmark, Roskilde: Roskilde University.
Niss, M. & Jensen, T. H. (eds) (2002). Kompetencer og matematiklæring –Ideer og inspiration til udvikling af matematik undervisning i Danmark, number 18 in Uddannelsesstyrelsens temahæfteserie. The Ministry of Education, Copenhagen, Denmark.
OECD (2009). Learning Mathematics for Life: A Perspective from PISA. Paris: OECD Publications.
OECD (2013). PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy. Paris: OECD Publications.
Ravand, H. (2015). Application of a Cognitive Diagnostic Model to a High-Stakes Reading Comprehension Test. Journal of Psychoeducational Assessment, 34 (8) 782–799.
Robitzsch, A.; Kiefer, T. George, A. & Uenlue, A. (2014). Package CDM, Date/Publication 2014-04-11 12:27:06 UTC
Rojas, G.; de la Torre, J. & Olea, J. (2012). Choosing between general and specific cognitive diagnosis models when the sample size is small. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Vancouver, British Columbia, Canada.
Roussos, L. A.; Templin, J. L. & Henson, R. A. (2007). Skills Diagnosis Using IRT-Based Latent Class Models. Journal of Educational Measurement, 44 (4), 293–311.
Rupp, A. A.; Templin, J. & Henson, R.A. (2010). Diagnostic Measurement, Theory, Methods, and Applications. New York: The Guilford Press.
Stacey, K. (2012).The International Assessment of Mathematical literacy: PISA 2012 Framework and Items. 12th International Congress on Mathematical Education. 8 July – 15 July, COEX, Seoul, Korea.
Turner, R.; Blum, W. & Niss, M. (2015). Using Competencies to Explain Mathematical Item Demand: A Work in Progress. In K. Stacey & R. Turner (Eds.), Assessing Mathematical Literacy, the PISA Experience (Vol. 26, Psychometrics, pp. 85-116. Amsterdam, the Netherlands: Springer.
Von Davier, M. (2005). mdltm: Software for the General Diagnostic Model and for Estimating Mixtures of Multidimensional Discrete Latent Traits Models. [Computer software]. Princeton, NJ: ETS
Educational Measurement and Evaluation Studies
Volume 9, Issue 26
August 2019
Pages 109-134

Files

Share

How to cite

Statistics