The Application of Multiple Choice Model (MCM) in Item Analysis and the Comparison of its Fit and Information with Three Parametric model (3PM): Case Study: The Biology Test of University’s Entrance Exam 2012

Document Type : Original Article

Authors

Abstract

The main objective of the present study is to apply multiple-choice model (MCM) for analyzing multiple choice items and comparing its fit and information with the three parametric model (3PM). To this end, among all the participants in biology test (contained 50 four-choice items) of University’s entrance exam in the field of Natural Sciences in 2012(1391), 5000 participants were randomly selected as the sample by SPSS software. In order to analyze the test and its items based on the multiple choice model, the assumptions of unidimensionality were firstly investigated by NOHARM software. Then, the test items were analyzed by MULTILOG program. Three-parameter model, which is the closest one to multiple-choice model in terms of parametric structure, was selected as the most appropriate binary model for comparing its information and fitting with multiple-choice model. The results indicated the efficacy of multiple-choice model for analyzing distractors. In comparing the fit of 3PM with MCM, the results showed better fitness of 3PM to the whole test than multiple-choice model. In addition, results indicated the appropriateness of multiple-choice model in accuracy of estimating the ability of subjects with lower-than-average ability, while 3PM estimates the ability of higher than average subjects more accurately. Accordingly, it is recommended that MCM model can be applied for farther knowledge about items’ distractors performance along with other methods.

Keywords

References

ایزانلو، بلال؛ بازرگان، عباس؛ فرزاد، ولی‌اله؛ صادقی، ناهید و کاوسی، امیر (1393). تفکیک ابعاد متعامد از خوشه‌های سؤال بر اساس هشت روش تعیین بعد در داده‌های دوارزشی: مورد، آزمون ریاضی رشته ریاضی فیزیک کنکور 92-91.اندازه­گیری تربیتی، 5 (18)، 207-240.
معلمی اوره، مهرناز (1387). مقایسه­ دقت برآورد توانایی در سؤالات چندگزینه­ای با به‌کارگیری مدل‌های سؤال پاسخ دو و چندارزشی. پایان­نامه­ کارشناسی ارشد رشته­ سنجش و اندازه­گیری دانشگاه علامه طباطبایی.
 
Abad, F. J.; Olea, J., & Ponsoda, V. (2009). The Multiple-Choice Model Some Solutions for Estimation of Parameters in the Presence of Omitted Responses. Applied Psychological Measurement, 33 (3), 200-221.
Bock, R. D. (1972). Estimating item parameters and latent ability when response are scored in two or more­ nominal categories. Psychometrika, 37 (1), 29- 51.
Bock, R. D. (1997). The nominal categories model. In Handbook of modern item response theory (pp. 33-49). New York: Springer.
De Ayala, R. J. (1993). An introduction to polytomous Item Response Theory Models.Measurement and Evaluation in counseling Development, 25 (4).
Drasgow, F.; Levine, M. V.; Tsien, S.; Williams, B., & Mead, A. D. (1995). Fitting polytomous item response theory models to multiple-choice tests. Applied Psychological Measurement, 19 (2), 143-166.
Hambleton, R. k. (1989). Principles and selected applications of item Response Theory. In R. L. Linn (Ed.), Educational measurement (3rd ed.). (pp. 147–200).
Levine, M. V. (1993). Orthogonal functions and the niteness of continuous item response theories. Unpublished manuscript.
Levine, M. V., & Drasgow, F. (1983). The relation between incorrect option choice and estimated ability. Educational and Psychological Measurement, 43 (3), 675-685.
Masters, G. N. (1988). An analysis of partial credit scoring. Applied Measurement in Education, 1, 279–297.
Ning, R.; Waters, A. E.; Studer, C., & Baraniuk, R. G. (2015). SPRITE: A Response Model for Multiple Choice Testing. ArXiv preprint arXiv: 1501.02844.
Ostini, R. & Nering, L. (2006). Polytomous Item Response Theory Models. Sage Publications.
Samejima, F. (1979). A New Family of Models for the Multiple-Choice Item (No. RR-79-4). Tennessee Univ Knoxville Dept of Psychology.
Smilaycoff, M. P. (1989). An application of the Bock_Samejima model for multiple category scoring to test items in which distractors contains information related to latent ability. Dissertation Abstracts International, (12-A), 4100, (University Microfilms No. 9112783).
Thissen, D. & Steinberg, L. (1984). A response model for multiple- choice items. Psychometrika, 49 (4) 567-577.
Thissen, D. & Steinberg, L. & Fitzpatrick, A. R. (1989). Multiple-Choice Models; the Distracters are also part of Item. Journal of Educational Measurement. 26 (2), 161-176.
Thissen, D. (1976).Information in wrong responses to the Raven Progressive Matrices. Jounal of Educational Measurement, 13 (3), 201-214.
Yutong, Yin (2007). Using Beaton fit inices to assess goodness-of-fit of IRT models. PHD thesis.
Educational Measurement and Evaluation Studies
Volume 6, Issue 14
October 2016
Pages 11-39

Files

Share

How to cite

Statistics