Diagnosis of science cognitive attributes of Iranian fourth grad students in TIMSS 2015

Document Type : Original Article


Assistant Professor of Research Institute for Education, Tehran, Iran



Objective: Nowadays, cognitive diagnostic assessment (CDA) is highly attended due to attain deeply information about examinees. In present study, CDA is used to provide more information about science responses of Iranian fourth graders who participated in TIMSS 2015.
Methods: 11 science attributes were identified and the Q matrix was created. Then, responses of 3823 students to 206 items was analyzed by G-DINA model.
Results: The results showed that the model can produce suitable diagnostic evidences. Also, three attributes including: Recognize, Providing examples, and Describe had higher attribute probabilities than others and their mastery probabilities were more than 0.50. Whereas, only probability of Recognize was higher than 0.65 and this is considered as mastered attribute. In other hand, Predict and Scientific inquiry had lowest attribute probabilities.
Conclusion: In general, the results revealed that there are sizable deference between knowledge-based attributes and higher-order thinking attributes in students, that can associated to distance between intended and implemented curricular.


Main Subjects

Afzali, F., Delavar, A., Falsafinezhad, M.R., Farrokhi, N.A., & Borjali, A. (2014). Using CDMs in determining the essence of gender differences of mathematic performance in first grade high school, Psychological Achievements, 21(4), 89-104. [in Persian]
Birenbaum, M., Tatsuoka, C., & Xin, T. (2005). Large-scale diagnostic assessment: comparison of eighth graders’ mathematics performance in the United States, Singapore and Israel. Assessment in Education, 12(2), 167-181.
Chen, Y. H., Gorin, J. S., Thompson, M. S., & Tatsuoka, K. K. (2008). Cross-cultural validity of the TIMSS-1999 mathematics test: Verification of a cognitive model. International Journal of Testing, 8(3), 251-271.
Chen, Y. H., Gorin, J. S., Thompson, M. S., & Tatsuoka, K. K. (2008). An alternative examination of Chinese Taipei mathematics achievement: Application of the rule-space method to TIMSS 1999 data. In M. v. Davier & D. Hastedt (Eds.), Issues and Methodologies in Large-Scale Assessments (Vol. 1, pp. 23-49). Hamburg/ Princeton: IEA-ETS Research Institute.
Chen, Y. H., Ferron, J. M., Thompson, M. S., Gorin, J. S., & Tatsuoka, K. K. (2010). Group comparisons of mathematics performance from a cognitive diagnostic perspective. Educational Research and Evaluation, 16(4), 325-343.
Chiu, C. Y., & Seo, M. (2009). Cluster analysis for cognitive diagnosis: An application to the 2001 PIRLS reading assessment. In M. v. Davier & D. Hastedt (Eds.), Issues and Methodologies in Large-Scale Assessments (Vol. 2, pp. 137-159). Hamburg: IEA-ETS Research Institute.
Curriculum Center in Iran. (2007). Guide of science curriculum in primary and middle high- school, Tehran: Organization of Educational Research and Planning. [in Persian]
DeCarlo, L. T. (2011). On the analysis of fraction subtraction data: The DINA model, classification, latent class sizes, and the Q-matrix. Applied Psychological Measurement, 35(1), 8-26.
de La Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76(2), 179-199.
de la Torre, J., & Minchen, N. D. (2019). The G-DINA Model Framework. In M. von Davier & Y. Lee (Eds.), Handbook of diagnostic classification models: Models and model extensions, applications, software packages (pp. 155-170). Switzerland, Cham: Springer.
Dogan, E., & Tatsuoka, K. (2008). An international comparison using a diagnostic testing model: Turkish students’ profile of mathematical skills on TIMSS-R. Educational Studies in Mathematics, 68(3), 263-272.
George, A. C., & Robitzsch, A. (2015). Cognitive diagnosis models in R: A didactic. The quantitative methods for psychology, 11(3), 189-205.
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74(2), 191-201.
Im, S., & Park, H. J. (2010). A comparison of US and Korean students' mathematics skills using a cognitive diagnostic testing method: Linkage to instruction. Educational Research and Evaluation, 16(3), 287-301.
Kabiri, M., Ghazi-Tabatabaei, M., Bazargan, A., Shokoohi-Yekta, M., & Kharrazi, K. (2017). Diagnosing competency mastery in science: An application of GDM to TIMSS 2011 data. Applied Measurement in Education, 30(1), 27-38.
Kabiri, M., Ghazi Tabatabae, M., & Bazargan, A. (2017). Specification of key competencies for eight graders in science: Conformity with of science curriculum requirements in Iran, Journal of Curriculum Studies, 44, 109-144. [in Persian]
Kim, Y. H. (2011). Diagnosing EAP writing ability using the reduced reparameterized unified model. Language Testing, 28(4), 509-541.
Javidanmehr, Z. & Anani Sarab, M.R. (2018). An investigation of the prevalence and difficulty of reading comprehension's sub-skills by the G-DINA model, Critical Language and Literary Studies, 19, 99-117. [in Persian]
LaRoche, S., Joncas, M., & Foy, P. (2016). Sample design in TIMSS 2015. In M. O. Martin, I. V. S. Mullis, & M. Hooper (Eds.), Methods and Procedures in TIMSS (Vol. 2016, pp. 3.1-3.37). Boston: TIMSS & PIRLS International Study Center.
Lee, Y. S., Park, Y. S., & Taylan, D. (2011). A cognitive diagnostic modeling of attribute mastery in Massachusetts, Minnesota, and the US national sample using the TIMSS 2007. International Journal of Testing, 11(2), 144-177.
Minaei, A., Delavar, A., Falsafinezhad, M., Kiamanesh, A. & Mohajer Y. (2014). Cgnitive diagnostic modeling of Iranian eight grade student to mathematics items of TIMSS 2007 using reduced noncompensatory reparameterized unified model and comparison between girls and boys, Quarterly of Educational Measurement, 16, 130-170. [in Persian]
Mohsenpour, M., Gooya, Z., Shokuhi Yekta, M., Kiamanesh, A., & Bazargan, A. (2015). A diagnostic test for math literacy cognitive competencies, Quarterly Journal of Educational Innovations, 53, 7-33. [in Persian]
Mohsenpour M. (2019). Assessing polytomous cognitive attributes of mathematics literacy of 9th grade students: Supplying PGDINA model, Educational Measurement and Evaluation Studies, 26, 109-134. [in Persian]
Moghadam, A., Falsafinejhad, M., Farokhi, N., & Estaji, M. (2016). Diagnostic analysis of general English reading comprehension's items of PhD entrance exam using non-compensatory fusion model, Quarterly of Educational Measurement, 22, 41-68. [in Persian]
Mullis, I. V. S., & Martin, M. O. (2013). TIMSS 2015 assessment frameworks. Amsterdam, the Netherlands: International Association for the Evaluation of Educational Achievement (IEA).
NRC. (1996). National science education standards. Washington, DC: National Academies Press.
Rahimi, R., Younesi, J., & mokarami, M. (2018). Application of a cognitive diagnostic assessment to analysis English comprehension's items of master's degree, Quarterly of Educational Measurement, 32, 17-40. [in Persian]
Rahemei, R., Delavar, A., Younesei, J., & Naserei, Z. (2020). Choosing an appropriate cognitive diagnostic model for reading comprehension tests: A case study of the graduate entrance exam of the English language, Educational Measurement and Evaluation Studies, 29, 205-227. [in Persian]
Ranjbaran, F., & Alavi, S. M. (2017). Cognitive diagnostic assessment of a reading comprehension test battery for formative diagnostic feedback, Journal of Foreign Language Research, 6(3), 321-342. [in Persian]
Ratcliffe, M. (1998). The purpose of science education. In M. Ratcliffe (Ed.), ASE guide to secondary science education. London: Association for Science Education.
Roussos, L. A., Templin, J. L., & Henson, R. A. (2007). Skills diagnosis using IRT based latent class models. Journal of Educational Measurement, 44(4), 293-311.
Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. New York: The Guilford Press.
Rupp, A. A., & van Rijn, P. W. (2018). GDINA and CDM packages in R. Measurement: Interdisciplinary Research and Perspectives, 16(1), 71-77.
Saadati, S., Moghadamzadeh, A., Minaei, A. & Geramipour, M. (2020). Differential item functioning in the framework of cognitive diagnostic assessment: Questions related to the differential and integral calculus of the Iranian national university entrance examination 2018, Biquarterly Journal of Cognitive Strategies in Learning, 15, 19-35. [in Persian]
Shahmirzadi, N., Siyyari, M., Marashi H., & Geramipour, M. (2020). Test fairness analysis in reading comprehension PhD nationwide admission test items under CDA, Journal of Foreign Language Research, 10(1), 152-165. [in Persian]
Sinharay, S., & Almond, R. G. (2007). Assessing fit of cognitive diagnostic models: A case study. Educational and Psychological Measurement, 67(2), 239-257.
Skaggs, G., Wilkins, J. L. M., & Hein, S. F. (2016): Grain Size and Parameter Recovery with TIMSS and the General Diagnostic Model, International Journal of Testing, 16(4), 310-330.
Taghiyan, H., Khodayee, E., Bazergan, A., Moghaddamzadeh, A., & Kabiri, M. (2018). Developing a reading test for sixth grade students using cognitive diagnostic assessment model, Journal of Teaching Persian to Speakers from Other Languages, 7(1), 3-30. [in Persian]
Tatsuoka, K. K., Corter, J. E., & Tatsuoka, C. (2004). Patterns of diagnosed mathematical content and process skills in TIMSS-R across a sample of 20 countries. American Educational Research Journal, 41(4), 901-926.
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61(2), 287-307.
von Davier, M. (2009). Using the general diagnostic model to measure learning and change in a longitudinal large-scale assessment (RR-09-28). Retrieved from Princeston, NJ:
Yamaguchi, K., & Okada, K. (2018) Comparison among cognitive diagnostic models for the TIMSS 2007 fourth grade mathematics assessment. PLoS ONE, 13(2): e0188691.