Optimum Weighting to Entrance Subtests and Their Items to Make Composite Score

Document Type : Original Article

Authors

Abstract

This research has been accomplished to weight national subtests and their items to make composite score. The aim of the project was to reduce measurement error associated with composite score in classical test theory framework. weighting procedure has been done in three levels; first, 30 multiple-choice-item test in differential calculus course with 3409 sample size was weighted at item choice level (choices popularity percent and formula score), and item level (simple total without weight or item effect weight, item factor weight and item difficulty weight). At subtest level, also, a test battery of medical assistance national test with 5 equally sized subtests, 6 items, which administered on 3572 candidates, has been weighted in different ways (average Pearson product-moment correlation coefficients weights, factor weights and regression coefficients). Another test battery of medical assistance national test with 5 unequal subtest length, 45, 26, 24,6, and 6 items which administered on 3638 candidates were studied without weighting just on their effective weights. This research revealed that formula score method produces more error than other procedures. Only weighting by item difficulty could rearrange examinees ranking in favor of qualified examinees. Other weighting methods are not satisfactory to enhance reliability coefficients and reliability coefficients, ab initio, are affected by appropriate items and well-made subtests with optimum length. 

Keywords

References

-     پرند، کوروش و ذوالفقارنسب، سلیمان (1389). گزارش تحلیلی آزمون دستیاری پزشکی13/12/1388. وزارت بهداشت، درمان و آموزش پزشکی. انتشار نیافته.
-     آلن، مری جی. و ین، وندی ام. (1979). مقدمه‌ای بر نظریه‌های اندازه‌گیری (روانسنجی). ترجمه  علی دلاور (1374). سازمان مطالعه و تدوین کتب علوم انسانی دانشگاه‌ها.
 
-     Brennan, Robert L. (2004). Some Perspectives on Inconsistencies among Measurement Models. Measurement and Assessment (CASMA) College of Education. University of Iowa City, IA 52242. Tel: 319-335-5439. Fax: 319-384-0505. Web: www. uiowa. edu/˜casma.
-     Childs, Ruth A., Susan Elgie, Tahany Gadalla, Ross Traub, Andrew P. Jaciw (2004). IRT-Linked Standard Errors of Weighted Composites. Practical Assessment, Research & valuation, 9 (13). Retrieved May 17, 2010 from http: //PAREonline. net/getvn. asp?v=9&n=13
-     De Klerk, G. (2008). Classical test theory (CTT). In M. Born, C. D. Foxcroft & R. Butter (Eds.), Online Readings in Testing and Assessment, International Test Commission, http: //www. intestcom. org/Publications/ORTA. php.
-     He, Qing ping (2009). Estimating the Reliability of Composite Scores. Office of Qualifications and Examinations Regulation Spring Place Coventry Business Park Herald Avenue Coventry CV5 6UB. www. ofqual. gov. uk
-     Hendrickson, A., Patterson, B. & Ewing, M. (2010). Developing Form Assembly Specifications for Exams with Multiple Choice and Constructed Response Items: Balancing reliability and validity concerns. Paper presented at the Annual Conference of the National Council for Measurement in Education, May 1, 2010, Denver.
-     Nedelsky, L. (1954). Absolute Grading Standards for Objective Tests.Educational and Psychological Measurement. 1954, v. 14 no. 1.
-     Nelson, L. R. (2001). Item Analysis for Tests and Surveys Using Lertap 5. University of Technology Perth, Western Australia. www. lertap. curtin. edu. au/HTMLHelp/LRTP5HHelp. pdf
-     Roid, G. H., & Carson, A. D. (2003). Special Composite Scores for the SB5. (Stanford-Binet Intelligence Scales, Fifth Edition Assessment Service Bulletin No. 4). Itasca, IL: Riverside Publishing.
-     Shun-Wen Chang (2008). Effects of Gaps-Minimizing Approaches on the Raw-to-Scale Score Conversions When Forms Vary in Difficulty. Bulletin of Educational Psychology, 2008, 39, Special Issue on Test and Measurement, 151-174 National Taiwan Normal University, Taipei, Taiwan, R. O. C.
-     Shun-Wen Chang (2009). Choice of Weighting Scheme in Forming the Composite. Department of Educational Psychology and Counseling. National Taiwan Normal University. Bulletin of Educational Psychology, 2009, 40 (3), 489-510
-     Stanley, J. C., & Wang, M. D. (1968). Differential weighting a survey of methods and empirical studies. New York/10027
-     Webb, N. M., Shavelson, R. J., & Haertel E. H. (2006). Reliability Coefficients and Generalizability Theory. Handbook of Statistics, Vol. 26. ISSN: 0169-7161. © 2006 Elsevier B. V. DOI: 10. 1016/S0169-7161 (06) 26004-8.
-     Yu, Min-Ning (1991). The Assessment of partial Knowledge. The Journal of National Chengchi University, Vol. 63, 1991.

Files

Share

How to cite

Statistics