Recognizing and prioritizing fair classroom assessment themes using Fuzzy Analytical Hierarchy Process (FAHP) method

Document Type : Original Article

Authors

1 Ph.D. Student, University of Tehran, Faculty of Psychology and Education, Tehran, Iran

2 Assistant Professor, University of Tehran, Faculty of Psychology and Education, Tehran, Iran

3 Associate Professor, University of Tehran, Faculty of Psychology and Education, Tehran, Iran

4 Professor, University of Tehran, Faculty of Psychology and Education, Tehran, Iran

5 Associate Professor, University of Kharazmi, Faculty of Psychology and Education , Tehran, Iran

10.22034/emes.2022.527579.2241

Abstract

Objective: The aim of this study was to identify and weigh the themes and sub- themes of fair classroom assessment.
Methods: To achieve this aim, an interpretive phenomenology study was carried out. Semi-structured qualitative interviews were used to collect data. Using criterion sampling, after conducting 29 individual interviews, one group interview and one focus group interview with students who had at least two semesters of experience in the University of Tehran, inductive thematic saturation was obtained. Fuzzy analytical hierarchy process method was used to weigh these themes. Data for this stage were collected by a pairwise comparison questionnaire that included 53 comparisons from a sample of 9 experts in the field of assessment.   
Results: Students' point of view analysis was performed using the Ricoeur method. Extraction of meaning units led to the identification of 20 sub-themes in 5 main themes of "educational fairness", "procedural fairness", "Nature of assessment", "Interactional fairness", and "fairness in paper and pencil test". The results of fuzzy analytical hierarchy process showed that among the 5 main themes, "educational fairness" with a weight of 0.318 was in the first place, followed by the themes of "procedural fairness", "nature of assessment", "interactional fairness", and "fairness in paper and pencil test" respectively. The final weights of the sub- themes were obtained by multiplying the weight of the main themes by the relative weight of the sub- themes; Accordingly, the sub- theme of "teaching quality" with a weight of 0.22 in the first place and the sub- themes of "informational fairness" and "professionalism in teaching" with weights of 0.11 and 0.09 were in the next positions, respectively.
Conclusion: Students' perception of fairness in assessment is a combination of the concepts of measurement theories and organizational justice, and the high weight of the theme of "educational fairness" among the all themes and sub- theme of "teaching quality" of this theme among all sub- themes obtained showed that Violation of educational fairness in the dimension of teaching quality will lead to an unfair perception of assessment. Conversely, if people feel that the teacher's teaching is of good quality and that the procedures used to assign the results are fair, even if it is not desirable for them, they will accept the results and understand them more fairly.

Keywords

References

Aldian, A., Taylor, M. A. (2005). A consistent method to determine flexible criteria weights for multicriteria transport project evaluation in developing countries. Journal of the Eastern Asia Society for Transportation Studies, 6, 3948-3963.
Azar, A. (2010). Applied decision making (MADM approach). Tehran: Negah Danesh. (Persian)
 Allal, L. (2013). Teachers’ professional judgement in assessment: A cognitive act and a socially situated practice. Assessment in Education: Principles, Policy & Practice, 20(1), 20–34.
Alm, F., & Colnerud, G. (2015). Teachers’ experiences of unfair grading. Educational Assessment, 20(2), 132–150.
Baniasadi, A., Salehi, K. (2019). Introduction on the Principles and Process of Construction and Validation of the Interview Protocol. Higher Education Letter, 12(46), 177-203. (Persian)
Baniasadi, A., Salehi, K., Khodaie, E., Bagheri Noaparast, K., Izanloo, B. (2021). Students’ perceptions of fair classroom assessment: A qualitative study. Educational Measurement and Evaluation Studies, 11(35), 68-88. (Persian)
Bazargan, A. (2013). An Introduction to Qualitative and Mixed Methods Research. Tehran: Didar. (Persian)
Bempechat, J., Ronfard, S., Li, J., Mirny, A., & Holloway, S. D. (2013). “ SHE ALWAYS GIVES GRADES LOWER THAN ONE DESERVES:” A QUALITATIVE STUDY OF RUSSIAN ADOLESCENTS’PERCEPTIONS OF FAIRNESS IN THE CLASSROOM. Journal of Ethnographic & Qualitative Research, 7(4), 168-187.
Brookhart, S. M. (2003). Developing measurement theory for classroom assessment purposes and uses. Educational Measurement: Issues and Practice, 22(4), 5–12.
Brown, G. T. L., Pishghadam, R., & Sadafian, S. S. (2014). Iranian university students’ conceptions of assessment: Using assessment to self-improve. Assessment Matters, 6, 5–33.
Čiuladienė, G., & Račelytė, D. (2016). Perceived unfairness in teacher-student conflict situations: students’ point of view. Polish Journal of Applied Psychology, 14(1), 49–66.
DeLuca, C. (2012). Preparing teachers for the age of accountability: Toward a framework for assessment education. Action in Teacher Education, 34(5–6), 576–591.
Ghanbari, S., Ardalan, MR., Karimi, I. (2015). Effect of the Challenges of Student Learnings Evaluation
on Deliberate Practice Study Approach. Education Strategies Medical Sciences, 8(2), 105-113. (Persian)
Houston, M. B., & Bettencourt, L. A. (1999). But that’s not fair! An exploratory study of student perceptions of instructor fairness. Journal of Marketing Education, 21(2), 84–96.
Hsieh, T.Y., Lu, S.T. and Tzeng, G.H., 2004. Fuzzy MCDM approach for planning and design tenders selection in public office buildings. International journal of project management, 22(7), 573-584.
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational
consequences. Educational research review, 2(2), 130–144.
Kosko, B. (1994). Fuzzy Thinking: The New Science of Fuzzy Logic (A. Ghaffari, Trans.). Tehran: K. N. Toosi University. (Persian)
Liu, J., Johnson, R., & Fan, X. (2016). A comparative study of Chinese and United States pre-service teachers’ perceptions about ethical issues in classroom assessment. Studies in Educational Evaluation, 48, 57–66.
McMillan, J. H. (2017). Classroom assessment: Principles and practice that enhance student learning and motivation. Pearson.
Murillo, F. J., & Hidalgo, N. (2017). Students’ conceptions about a fair assessment of their learning. Studies in Educational Evaluation, 53, 10–16.
Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.
Pornaderi, p., Nasr, A, R., Nili. M, R., & Bagheri, Z. (2013). Evaluation of the Implementing Different Methods of Measurement and Sustainable Learning Based Measurements. Technology of Education Journal, 8 (1), 45-54. (Persian)
Ricoeur, P. (1976). Interpretation theory: Discourse and the surplus of meaning. TCU press.
Rodabaugh, R. C. (1994). College students’ perceptions of unfairness in the classroom. To Improve the Academy, 13(1), 269–282.
Seraji, F., Maroofi, Y., & Razeqi, T. (2013). Identifying the challenges of evaluating what students have
learned in the Iranian higher education system. Educational Measurement and Evaluation Studies, 4 (5), 33-54. (Persian)
Solomonidou, G., & Michaelides, M. (2017). Students’ conceptions of assessment purposes in a low stakes secondary-school context: A mixed methodology approach. Studies in Educational Evaluation, 52, 35-41.
Speziale, H. S., Streubert, H. J., & Carpenter, D. R. (2011). Qualitative research in nursing: Advancing the humanistic imperative. Lippincott Williams & Wilkins.
Supreme Council for Cultural Revolution. (2012). Comprehensive scientific map of the country. Tehran: Secretariat of the Supreme Council of the Cultural Revolution. (Persian)
Tata, J. (2005). The influence of national culture on the perceived fairness of grading procedures: A comparison of the United States and China. The Journal of Psychology, 139(5), 401–412.
Tierney, R. D. (2013). Fairness in classroom assessment. SAGE Handbook of Research on Classroom Assessment, November, 125–144. https://doi.org/10.4135/9781452218649.n8
Tierney, R. D. (2014). Fairness as a multifaceted quality in classroom assessment. Studies in Educational Evaluation, 43, 55–69.
Tillema, H., Leenknech, M., & Segers, M. (2011). Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning – A review of research studies. Studies in Educational Evaluation, 37, 25-34.
Twenge, J. M. (2009). Generational changes and their impact in the classroom: teaching Generation Me. Medical Education, 43(5), 398–405.
Vinodh, S., Prasanna, M., & Prakash, N. H. (2014). Integrated Fuzzy AHP–TOPSIS for selecting the best plastic recycling method: A case study. Applied Mathematical Modelling, 38(19–20), 4662–4672.
Yamtim, V., & Wongwanich, S. (2014). A study of classroom assessment literacy of primary school teachers. Procedia-Social and Behavioral Sciences, 116, 2998–3004.
Educational Measurement and Evaluation Studies
Volume 12, Issue 37
April 2022
Pages 19-38

Files

Share

How to cite

Statistics