Choosing an Appropriate Cognitive Diagnostic Model for Reading Comprehension Tests: A Case Study of the Graduate Entrance Exam of the English language

Document Type : Original Article




Cognitive Diagnostic Assessment is a type of educational assessment that serves to identification and diagnosis of the learning disabilities through the application of psychometric models. Presence of numerous models, makes it crucial choose an appropriate Cognitive Diagnostic Model for data analysis. The purpose of this study was to determine a fitting model for reading comprehension tests. The examined data in this study were included responses from 3,000 participants in the Graduate Entrance Exam for English language field, which were selected randomly out of 16044 people. After Creation of matrix Q, two general models (G-DINA and LCDM), two compensatory models (ACDM and DINO) and two non-compensatory models (RRUM and DINA) were considered for comparison. In terms of Chi-squared maximum Fit Index, none of the models were fitted; but in contrast the results of SRMSR index showed that the fit of all data were acceptable with the data. A comparison of Relative Fit Indices cleared that G-DINA and ACDM models have a stronger fit and DINO and DINA have a poorer fit in comparison to the other models. Also, investigating the Question Fit index showed that only G-DINA and LCDM have a fit with all questions.


امبرتسون، سوزان ای؛ و رایس، استیون پی. (2000). نظریه‌های جدید روان‌سنجی برای روان شناسان؛ ترجمه حسن پاشا شریفی، ولی‌اله فرزاد، مجتبی حبیبی عسگرآباد و بلال ایزانلو (1388). تهران: انتشارات رشد.
سرمد، زهره؛ بازرگان، عباس؛ و حجازی، الهه.(1391). روش‌های تحقیق در علوم رفتاری. تهران: نشر آگاه.
Bandalos, D. L. (1993). Factors influencing cross-validation of confirmatory factor analysis models. Multivariate Behavioral Research, 28(3), 351-374.
Basokcu, T. O., Ogretmen, T., & Kelecioglu, H. (2013). Model data fit comparison between DINA and G-DINA in cognitive diagnostic models. Education Journal, 2(6), 256-262.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238. ‏
Bernhardt, E. (2005). Progress and procrastination in second language reading. Annual Review of Applied Linguistics, 25, 133-150. ‏
Cavanaugh, J. E. (2009). 171: 290 model selection, lecture VI: The Bayesian information criterion. Department of Biostatistics, Department of Statistics and Actuarial Science, University of Iowa.
Chen, J., Torre, J., & Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50(2), 123-140.
Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report Series, 2005(2), i-35. ‏
De La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179-199. ‏
García, P. E., Olea, J., & De la Torre, J. (2014). Application of cognitive diagnosis models to competency-based situational judgment tests. Psicothema, 26(3), 372-377.
George, A. C., & Robitzsch, A. (2015). Cognitive Diagnosis Models in R: A Didactic. The Quantitative Methods for Psychology, 11(3), 189-205.
Goldsmith-Phillips, J. (1989). Word and context in reading development: A test of the interactive-compensatory hypothesis. Journal of Educational Psychology, 81(3), 299. ‏
Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial & Special Education, 7(1), 6-10. ‏
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191-210. ‏
Hoover, W. A., & Gough, P. B. (1990). The simple view of reading. Reading & Writing, 2(2), 127-160. ‏
Kunina-Habenicht, O., Rupp, A. A., & Wilhelm, O. (2009). A practical illustration of multidimensional diagnostic skills profiling: Comparing results from confirmatory factor analysis and diagnostic classification models. Studies in Educational Evaluation, 35(2), 64-70. ‏
Leighton, J., & Gierl, M. (Eds.). (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge University Press. ‏
Li, H. (2011). A cognitive diagnostic analysis of the MELAB reading test. Spaan Fellow Working Papers in Second or Foreign Language Assessment, 9, 17-46.
Li, H., Hunter, C. V., & Lei, P. W. (2016). The selection of cognitive diagnostic models for a reading comprehension test. Language Testing, 33(3), 391-409.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research & Perspectives, 11(3), 71-101.
Ravand, H., & Robitzsch, A. (2015). Cognitive Diagnostic Modeling Using R. Practical Assessment, Research & Evaluation, 20.
Roussos, L. A., Templin, J. L., & Henson, R. A. (2007). Skills Diagnosis Using IRT‐Based Latent Class Models. Journal of Educational Measurement, 44(4), 293-311
Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitive diagnostic models, a case study. Educational & Psychological Measurement, 67(2), 239-257.
Sorrel, M. A., Olea, J., Abad, F. J., de la Torre, J., Aguado, D., & Lievens, F. (2016). Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19(3), 506-532.
Stanovich, K. E. (1980). Toward an interactive-compensatory model of individual differences in the development of reading fluency. Reading Research Quarterly, 16(1), 32-71. ‏
Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. Guilford Press.
Usó-Juan, E. (2006). The compensatory nature of discipline-related knowledge and English-language proficiency in reading English for academic purposes. Modern Language Journal, 90(2), 210-227. ‏