Educational Measurement and Evaluation Studies

Educational Measurement and Evaluation Studies

Title Differential Item Functioning of Fourth Grade Mathematics Test of the TIMSS 2019 Based on Gender

Document Type : Original Article

Authors
Ensie Khalilnejadi 1
Maryam Mohsenpour 2
Azam Moghadam 3
1 Master of Educational Psychology, Alzahra University, Tehran, Iran
2 Assistant Professor at Departmant of Educational Psychology, Faculty of Education and Psychology, Alzahra University, Tehran, Iran
3 Ph.D of Measurement and Assessment, Tehran, Iran
10.22034/EMES.2024.2012558.2511
Abstract
Objective: The present study was conducted with the aim of investigating the differential item functioning of the fourth-grade mathematics items in the 2019 TIMSS according to gender.
Methods: Given that the analysis is based on the data collected from Iran  in aforementioned study, this research is descriptive and secondary analysis.
Results: To investigate the differential item functioning (DIF), among the 14 booklets, two booklets (one and seven booklet)were randomly selected and the research sample consisted of 861 student. To analyzing DIF, two traditional and modern methods were used, respectively, "logistic regression method" and "Chi-2 Lord method based on Rush model".The findings showed that the results of DIF were the same with both traditional and modern methods, and out of 32 items in booklet one, four items had differential functioning which two items were in favor of girls and two items were in favor of boys. Also, out of 30 items in booklet seven, three items had a differential functioning, two items were in favor of boys and one item was in favor of girls.
Conclusion: In examining the structure of questions with DIF, it was found that, boys were more likely than girls to correctly answer numbers items and girls were more likely than boys to correctly answer geometry items. Furthermore, boys were more likely than girls to correctly answer in applying and reasoning items, and they were more likely than girls to correctly answer life related problems. Thus, it is suggested that items with differential functioning be investigated more deeply through task-based interview.
Keywords
Keywords: TIMSS study
Differential Item Functioning
Mathematics Achievement Test - Gender - Fourth grade students

Subjects

References
Alimirzaie, M., Mogadam, Z. A., Minaei, A., Ezanloo, B., & Salehi, K. (2019). Sources of the differential item functioning and its application in education. journal of Research in Teaching. 7(1), 129-149.
Ali moussa, M. (2019). Item bias approaches comparison of TIMSS test among Egyption environment. International journal of research in educational sciences, 2(4), 501-540.
Arim, R. G., & Ercikan, K. (2014). Comparability between the American and Turkish Versions of the TIMSS Mathematics Test Results. Education & Science/Egitim ve Bilim, 39(172), 33-48.
Askari Turzen, D. (2021). Comparison and application of the mixed method of rush, question-oriented tree and lasso in identifying the differential item functioning of mathematical questions of the 2015 TIMSS 8th grade test. Unpublished Master's thesis, Faculty of Psychology and Education, Allameh Tabatabai University.
Bolt, D. M. (2000). A SIBTEST approach to testing DIF hypotheses using experimentally designed test items. Journal of Educational Measurement, 37(4), 307-327.
Doolittle, A. E. (1989). Gender differences in performance on mathematics achievement items.
Applied Measurement in Education, 2, 161-177.
Gökçe, S., Berberoğlu, G., Wells, C. S., & Sireci, S. G. (2021). Linguistic Distance and Translation Differential Item Functioning on Trends in International Mathematics and Science Study Mathematics Assessment Items. Journal of Psychoeducational Assessment, 39(6), 728-745.
Hambleton, R., & Rogers, J. (1994). Item bias review. Practical Assessment, Research, and Evaluation, 4(1), Article 6. Available at: https://scholarworks.umass.edu/pare/vol4/iss1/6
Hashemi, M. (2010). Examining the bias of the 2007 TIMSS test questions in the third grade of middle school in the science course with regard to gender and ethnicity variables, as well as comparing classical theories and question-answer. Unpublished Master's Thesis, Faculty of Psychology and Education, Allameh Tabatabai University.
Holland, P. W., & Thayer, D. T. (1985). An alternate definition of the ETS delta scale of item difficulty. ETS Research Report Series, 1985(2), 1-10.
Innabi, H., & Dodeen, H. (2018). Gender differences in mathematics achievement in Jordan: A differential item functioning analysis of the 2015 TIMSS. School Science and Mathematics, 118(3–4), 127–137.
Jodoin, M. G. and Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329-349.
Kiamanesh, A., Mohsenpour, M., Safarkhani, M., Aghdasi, S. (2012). The trend of change in mathematics performance of grade 8 Iranian students between 1999 to 2007 based on the TIMSS findings in Iran and other countries of the region considering the 20 years vision document of Iran, Journal of curriculum studies, 6(24), 59-82.
Lan, M. C. (2014). Exploring gender differential item functioning (DIF) on eighth grade mathematics items for the United States and Taiwan (Doctoral dissertation).
Magis, D., Sebastien Beland, S., and Raiche, G. (2022). Collection of Methods to Detect Dichotomous Differential Item Functioning (DIF), difR package, Version 5.1.
Minaei, A. (2012). Measuring the comparability of the structure and analyzing the differential item functioning of questions (DIF) and blocks (DTF) of the 2007 TIMSS 8th grade science test among Iranian and American students. Educational Measurement.4(11), 113-151.
Minaei, A. and Ghaffari, Z. (2014). The differential item functioning of the eighth grade questions of the 2011 TIMSS math test among male and female students using the question-answer theory (IRT) approach. Educational measurement. 6(21), 21-40.
Minaei, A., Younesi, J. and Kazemi Dana, B. (2019). The application of cognitive diagnostic models (CDM) in the study of the differential item functioning of 2007 TIMSS 8th grade mathematics questions among boys and girls. Educational Measurement and Studies. 10(32), 205-234.
Mullis, I. V. S., & Martin, M. O. (Eds.). (2017). TIMSS 2019 Assessment Frameworks. Retrieved from Boston College, TIMSS & PIRLS International Study Center.
Mullis, I. V. S., Martin, M. O., Foy, P., Kelly, D. L., & Fishbein, B. (2020). TIMSS 2019 International Results in Mathematics and Science. Retrieved from Boston College, TIMSS & PIRLS International Study Center.
Payne, D., A. (2003). Applied Educational Assessment, 2nd ed, Wadsworth.
Sadeghi, M. (2012). Differential Item Functioning (DIF) of TIMSS 2011 4th and 8th grade math test questions using IRT and classical models. Unpublished Master's Thesis, Faculty of Social Sciences, Islamic Azad University.
Sobhani Sinab, L. (2021). The application of logistic and multinomial regression in identifying the differential Item Functioning (DIF) and options (DOF) of the 2019 TIMSS 8th grade math test between girls and boys in Iran. Unpublished Master's Thesis, Faculty of Psychology and Education. Allameh Tabatabai University.
Zumbo, B. D. (1999). A Handbook on the Theory and Methods of Differential Item Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Item Scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense.
Educational Measurement and Evaluation Studies
Volume 14, Issue 45
Spring 2024
Pages 7-26