Assessing the Quality of Academic Research Outcomes: A Reflection on the Existing Studies and Views of the Knowledge Production Actors in Academic System

Document Type : Original Article




The present research was conducted to evaluate the quality of academic research outcomes from the perspective of knowledge production actors in one of the major universities in Iran with qualitative approach, case study method and triangulation technique. The study method was qualitative case study. For this purpose, in the first stage, we examined and studied the research background on the quality of research outcomes and in the second phase we conducted in-depth interviews with faculty members of Shiraz University. Statistical population were 610 faculty members of Shiraz University. We selected 24 members in four groups of humanities, engineering, agriculture and veterinary sciences and basic sciences that have at least a degree of associated professor and have been selected as the top researcher in the last five years with criteria sampling method. The results of this research explored the quality of academic research in two components of "quality of publications" and "quality of presentation and maturity of research" and, respectively, in five indicators of "quality of the publication", "alignment with the indicators of scientometrics","Research credibility "," Innovation and efficiency "and" Applicability "were identified in a total of 32 indicators.


سهیلی، فرامرز؛ شریف‌مقدم، هادی؛ موسوی چلک، افشین و خاصه، علی‌اکبر (1394). ارزیابی پژوهش‌های آی‌متریکس با استفاده از مدل نفوذ علمی. پژوهشنامه پردازش و مدیریت اطلاعات. بازیابی شده در 3/6/1395، از http;;//
طاهری، بهجت؛ قضاوی، رقیه؛ زاهد، آرش و سلیمان‌زاده نجفی، نیره‌السادات (1394). تأثیر خوداستنادی بر شاخص‏های کمی و کیفی سنجش برون‌داد و دستاورد پژوهشی اعضای هیئت علمی دانشگاه علوم پزشکی اصفهان. مجله علم‌سنجی کاسپین، 2 (2)، 28- 35.
عطاپور، هاشم؛ نوروزی چاکلی، عبدالرضا و حسن‌زاده، محمد (1388). بررسی و تحلیل عوامل مؤثر بر ضریب تأثیر نشریات حوزه اقتصاد مورد تأیید وزارت علوم، تحقیقات و فناوری. سیاست علم و فناوری، 2 (1)، 53- 64.
کرامت‌فر، عبدالصمد و رفیعی خشنود، محدثه (1395). ارزیابی برون‌داد و دستاوردهای علمی، پژوهشگاه رویان، 3 (1)، 36- 44.
محمدی، سید مهرداد؛ عاملی، امید و محمدی، سید فرزاد (1377). استانداردهای کیفیت پژوهش، استانداردهای کیفیت- سازمان پژوهشی. اولین کنفرانس بین‌المللی مدیریت کیفیت. تهران: دانشگاه علامه طباطبایی.
مختاریان، فرانک و محمدی، رضا (1392). بررسی مفهوم قدرت در نظام ارزشیابی کیفیت آموزش عالی. مجموعه مقالات نخستین کنفرانس ارزشیابی و تضمین کیفیت در نظام‌های آموزشی. تهران: اداره کل چاپ و انتشارات سازمان سنجش کشور.
Bergmann, M.; Brohmann, B.; Hoffmann, E.; Loibl, M. C.; Rehaag, R.; Schramm, E. & Voß, J. P. (2005). Quality criteria of transdisciplinary research. A guide for the formative evaluation of research projects. ISOE-Studientexte, (13).
Belcher, B. M.; Rasmussen, K. E.; Kemshaw, M. R. & Zornes, D. A. (2016). Defining and assessing research quality in a transdisciplinary context. Research Evaluation25 (1), 1-17.
Boaz, A. & Ashby, D. (2003). Fit for purpose? assessing research quality for evidence based policy and practice. London: ESRC UK Centre for Evidence Based Policy and Practice.
Byl, L.; Carson, J.; Feltracco, A.; Gooch, S.; Gordon, S.; Kenyon, T. ... & Stirling, P. (2016). White Paper: Measuring Research Outputs through Bibliometrics. Doi: 10.13140/rg.2.1.3302.5680
Christenson, James A. & Sigelman, Lee (1987). Accrediting knowldge: journal stature and citation impact in social science. Essay of an information scientists, 10 (38), 265-271.
Clyde, L. A. (2004). Evaluating the Quality of Research Publications: A Pilot Study of School Librarianship. Journal of the American Society for Information Science and Technology, 55 (13), 1119–1130.
De Jong, S. P.; Van Arensbergen, P.; Daemen, F.; Van Der Meulen, B. & Van Den Besselaar, P. (2011). Evaluation of research in context: an approach and two cases. Research Evaluation20 (1), 61-72.
Fitzgerald, L.; Ruth, S.; Jan, Cilliers. (1991). Quality and the research assessment exercise: just one aspect of performance. Quality Assurance in Education, 9 (1), 5-13.
Furlong, J. & Oancea, A. (2008). Assessing Quality in Applied and Practice-based Educational Research, A Framework for Discussion. Oxford University Department of Educational Studies.
Hassanain, M.; Anil, S. & Abdo, A. (2016). Institutional Research Evaluation Model (IREM): A framework for measuring organizational research trends and impact and its application in medical academia in Saudi Arabia. Journal of Epidemiology & Global Health, 6 (4), 249-256.doi: 10.1016/j.jegh.2016.03.002.
Hu, Y.; Liang, W. & Tang, Y. (2017). Evaluating Research Efficiency of Chinese Universities. Springer.
Jaroonkhongdach, W.; Todd, R. W.; Hall, D. & Keyuravong, S. (2011). Three Dimensions of Research Quality. Proceedings of the International Conference: Doing Research in Applied Linguistics
Lahtinen, E.; Koskinen-Ollonqvist, P.; Rouvinen-Wilenius, P.; Tuominen, P. & Mittelmark, M. B. (2005). The development of quality criteria for research: a Finnish approach. Health promotion international20 (3), 306-315.
Lee, K. P.; Schotland, M.; Bacchetti, P. & Bero, L.A. (2002). Association of journal quality indicators with methodological quality of clinical research articles. The Journal of the American Medical Association, 287 (21), 2805–2808.
Leite, D. & Pinho, I. (2016). Evaluating collaboration networks in higher education research: drivers of excellence. Springer.
Litman, T. (2012). Evaluating Research Quality: Guidelines for Scholarship. International Electronic Symposium on Knowledge Communication and Peer Reviewing, International Institute of Informatics and Systemics (
Luoma, P.; Raivio, T.; Tommila, P.; Lunabba, J.; Halme, K.; Viljamaa, K. & Lahtinen, H. (2011). Better Results, More Value: A Framework for Analysing the Societal Impact of Research and Innovation. TEKES review, 288, 2011.
Mårtensson, P.; Fors, U.; Wallin, S. B.; Zander, U. & Nilsson, G. H. (2016). Evaluating research: A multidisciplinary approach to assessing research practice and quality. Research Policy45 (3), 593-603.
Moed, H. F. & Halevi, G. (2015). Multidimensional assessment of scholarly research impact. Journal of the Association for Information Science and Technology66 (10), 1988-2002.
National Research Council. (2008). Evaluating research efficiency in the US Environmental Protection Agency. National Academies Press.
National Research Foundation (NRF). (2004). Key Research Areas and Types of Research Outputs. Draft 5. (A working document for the Assessment Panels.)
Ochsner, M.; Hug, S. E. & Daniel, H. D. (2012). Four types of research in the humanities: Setting the stage for research quality criteria in the humanities. Research Evaluation22 (2), 79-92.
Ochsner, M.; Hug, S. E. & Daniel, H. D. (2014). Setting the stage for the assessment of research quality in the humanities. Consolidating the results of four empirical studies. Zeitschrift für Erziehungswissenschaft17 (6), 111-132.
Quality Indicators in the Social Sciences Committee (2013). Towards a Framework for the Quality Assessment of Social Science Research. Royal Netherlands Academy of Arts and Sciences.
REF 02.2011 (2011). REF 2014: Assessment framework and guidance on submissions. Available at
REF 01.2012 (2012). Panel criteria and working methods. Available at
Slack, N. et al (1998). Operation management. 2nd. London: Pitman.Quoted in Boaden, Ruth; Jan, Cilliers.(2001). Quality and the research assessmentexercise: just one aspect of performance". Quality Assurance in Education, 9(1), 5-13.
Smith, A. (2001). Never mind the width, feel the quality: improving VET research in Australia. British Medical Journal, 26 (6), retrieved from