Designing the Return on Expectations Training Evaluating Model (ROE): A Qualitative Research

Document Type : Original Article

Authors

10.22034/emes.2020.39882

Abstract

Regarding the evolution of training evaluation, this paper seeks to examine the new concept of return on expectations in the field of training with quality approach and documentary research method. The method for data analysis was thematic analysis. Searching in scientific databases include Irandoc, Magiran, Sivilica, Elsevier, Emerald, Google Scholar, Amazon, Science Direct, IEEE, Springer, SAGE, obtained 92 documents and sources, that 19 of them were selected in the process of reviewing and selecting documents. The triangulation method was used to increase the validity and reliability coefficient calculated using the Holsti method was 85.9. The process of coding was carried out by MAXQDA software. In this process, 632 codes were obtained that were categorized in the form of 55 basic themes, 11 organizing themes and one global theme. In the end, the thematic network of return on expectations in training evaluation was provided, and based on the relationship between the organizing themes in the literature and through semi-structured interviews with 5 training evaluation experts the return on expectations training evaluating model (ROE) was designed. The components of this process model include: attributes, requirements, identifying stakeholders, determining the expectations, indexing, designing the training, delivering training, evaluating, supporting actions, monitoring, feedback, and consequences. This model can show and increase the effectiveness of training programs based on indicators derived from stakeholder expectations.

Keywords

References

بازرگان، عباس (1379). ارزشیابی آموزشی مفاهیم الگوها و فرایند عملیاتی. تهران: انتشارات سمت.
حسینی انجدانی، مریم (1395) روش تحلیل مضمون. تهران: مرکز پژوهش و سنجش صدا و سیما.
خراسانی، اباصلت؛ و دوستی، هومن (1391). ارزیابی اثربخشی آموزش‌های سازمانی (راهنمای عملیاتی). تهران: مرکز آموزش و تحقیقات صنعتی ایران.
صادقی فسایی، سهیلا؛ و عرفان‌منش، ایمان (1394). مبانی روش‌شناسی پژوهش اسنادی در علوم اجتماعی. فصلنامه راهبرد فرهنگ، 29، 61-91.
عابدی جعفری، حسن؛ تسلیمی، محمدسعید؛ فقیهی، ابوالحسن؛ و شیخ‌زاده، محمد (1390). تحلیل مضمون و شبکه مضامین: روشی ساده و کارآمد برای تبیین الگوهای موجود در داده‌های کیفی. دوفصلنامهاندیشهمدیریتراهبردی، 5(2)، 198-151.
کمالی، یحیی (1397). روش‌شناسی تحلیل مضمون و کاربرد آن در مطالعات سیاست‌گذاری عمومی. فصلنامه سیاست‌گذاری عمومی، 4(2)، 189 – 208.
گال، مردیت؛ بورگ، والتر؛ و گال، جویس (1996). روش‌های تحقیق کمی و کیفی در علوم تربیتی و روان‌شناسی؛ ترجمه احمدرضا نصر و دیگران (1393). تهران: سمت و دانشگاه شهید بهشتی.
نیک‌نشان، شقایق؛ نوروزی، رضاعلی؛ و نصر اصفهانی، احمدرضا (1389). تحلیلی بر رویکردهای روایی در پژوهش کیفی. فصلنامهروش‌شناسیعلومانسانی، 16(62)، ۱۴۱ -160.
Abd Rahman, A., Imm Ng, S., Sambasivan, M., & Wong, F. (2013). Training and Organizational Effectiveness: Moderating Role of Knowledge Management Process. European Journal of Training & Development, 37(5), 472 – 488.
Ahmed, J. (2010). Documentary Research Method: New Dimensions. Indus Journal of Management & Social Sciences, 4(1),1-14.
Anderson, V. (2007a). The Value of Learning: From Return on Investment to Return on Expectation. London: Chartered Institute of Personnel and Development.
Anderson, V. (2007b). The value of learning, A new model of value and evaluation. London: Chartered Institute of Personnel and Developmen (CIPD). http://www.voced.edu.au/content/ngv%3A7127
Attride-Stirling, J. (2001). Thematic Networks: An Analytic Tool for Qualitative Research. Qualitative Research, 1(3), 385-405.
Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77-101.
Clarke, V. & Braun, V. (2014). Thematic analysis. In T. Teo (Ed.), Encyclopedia of Critical Psychology (1947-1952). New York: Springer.
Fierceinc (2015). The Big Picture of ROE a Supplementary White Paper to “The ROI of Skillful Conversation”. https://www.fierceinc.com
Ford, D. J. (2005) Bottom Line Training: Performance Based Result. USA: Training Education Management LLC.
Hannes, K., Lockwood, C., & Pearson, A. (2010). A Comparative Analysis of Three Online Appraisal Instruments’ Ability to Assess Validity in Qualitative Research. Qualitative Health Research, 20(12),1736-1743.
Husaini Usman, S., & Jaedun, A. (2016). Impact-Based Training Evaluation Model (IBTEM) For School Supervisors in Indonesia. International Education Studies. 9, (2), 149-160.
Larsen, T. A., & Schultz, M. A. (2014). Transforming Simulation PracticesdA Quest for Return on Expectations. Clinical Simulation in Nursing, 10, 626-629.
Kirkpatrick, D. L., & Kirkpatrick, J. D. (2006). Evaluating Training Programs, the Four Levels (3h ed).  San Francisco: Berrett Koehler.
Kirkpatrick, D. L., & Kirkpatrick. J. D (2007). Implementing the Four Levels. San Francisco, California. Berrett-Koehler Publishers
Kirkpatrick, J. D. (2009). Return on Expectations: The ultimate demonstration of training value. https://www.trainingzone.co.uk/develop
Kirkpatrick, J. D., & Kirkpatrick, W. K. (2010a). ROE’s Rising Star Why Return on Expectations is getting so Much Attention. T+D, 64(8), 34-38.
Kirkpatrick J. D., & Kirkpatrick W. K. (2010b). Training and Trial How workplace learning must reinvent itself to remain relevant. USA: American Management Association.

Kirkpatrick, J. D., & Kirkpatrick W. K. (2011). Creating ROE: The End Is the Beginning. ATD, December 01.

 https://www.td.org/Publications/Magazines/TD/TD-Archive/2011/11/Creating-ROE-the-End-Is-the-Beginning

Kirkpatrick J. D., & Kirkpatrick W. K. (2015). The Foure Levels of Evaluation – An Update. ATD Learning and Development, 15, 1502, December 01.

Kucherov, D., & Manokhina, D. (2017). Evaluation of training programs in Russian manufacturing companies. European Journal of Training & Development, 41(2), 119 – 143.
McCoy, M., & Hargie, O. D. W. (2001). Evaluating Evalouation: Impliciations for Assessing Quality. International Jornal of Health Cear Quality Assessment, 14(7), 317-327.
McLinden, D., & Trochim, M. K. (1998). Getting to Parallel Assessing the Return on Expectations of Training. Performance Improvement, 37, 21-26.
Moorhous, C. (2013). A Strategic Conversation Model to Optimise Retern on Occupational Training Expetations. (Doctoral dissertatio). University of South Africa.
Murray, L. W., & Efendioglu, A. M. (2007). Valuing the investment in organizational training. Industrial & Commercial Training, 39(7), 372 – 379.
Nickols, F. (2005). Why a stakeholder approach to ivaluating training. Advances in Developing Human Resources, 7(1), 121 – 134.
Nickols, F. (2004). A Stakeholder Approach to Evaluating Training. https://www.nickols.us/stakeholder.pdf
Padilla Rodriguez, B. C. (2014). Instructional interactions and online course effectiveness at a large Mexican organization. (Doctoral dissertatio), School of Education: University of Leicester.
Porkodi, S., & Jahan, U. (2015). Bank Employees’ Expectations of Training Programs in Oman - An Exploratory Study. Research Journal of Commerce & Behavioral Science, 4(3), 92-100.
Scott, J. )1990(. A Matter of Record: Documentary Sources in Social Research. Cambridge, UK: Polity Press.
Stuffelbeam, D. L., & Zhang, G. (2017). The CIPP Evaluation Model, how to Evaluate for Improvement and Accountability. New York: The Guilford press
Throgmorton, C., Mitchell, T., Morley, T. & Snyder. M. (2016). Evaluating a Physician Leadership Development Program – a Mixed Methods Approach. Journal of Health Organization & Management, 30(3),1-31.
United Stade Office of Personnel Management. (2011). Field Guide to Training Evaluation. https://www.opm.gov/policy-data-oversight/training-and-development/reference-materials/training_evaluation.pdf
Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & Health Sciences, 15, 398–405.
Verschuren, P. J. M., & Zsolnai, L. (1998). Norms, Goals, and Stakeholders in Program Evaluation, Human Systems Management, 2, 155-160.
Werner, J. M. & DeSimone, R. L. (2009). Human Resource Development, 5 Ed., USA: South-Western.
Educational Measurement and Evaluation Studies
Volume 10, Issue 29
May 2020
Pages 7-34

Files

Share

How to cite

Statistics