nvestigating the Generalizability and Dependability Coefficients of Admission Interview Scores of Ph.D. in Psychology

Document Type : Original Article



The scores obtained on the interviews can be the basis for decision making provided that they are embedded with low measurement error. The purpose of this study was to the estimate the generalizability and dependability coefficients of the admission interview scores of Ph.D. at Shahid Beheshti University. To do this end, interview scores of 27 applicants in the field of psychology at Shahid Beheshti University were analyzed using a two-dimensional cross-sectional design. The real variance, which was related to the “purpose of measurement”, accounted for about one-third of variance of observed score.
Besides, unsystematic measurement error and systematic measurement error accounted for about one-fifth and half of the variance of the observed score, respectively. The findings showed that the generalizability and dependability coefficients of the interview scores were acceptable and they were close to the coefficients estimated for the medical fields. However, the interaction between interviewers and applicants had the greatest contribution to the systematic measurement error, that can lower the credibility of the scores.
According to the results, it is suggested that a workshop is organized for psychological training groups so that they can carry out reliability analyses to obtain a better understanding of the quality of the interview scores and to determine which changes in the concerned dimensions can lower the systematic error and boost the reliability of the interview scores.


Arce-Ferrer, A. J. & Castillo, I. B. (2007). Investigating postgraduate college admission interviews: generalizability theory reliability and incremental predictive validity. Journal of Hispanic Higher Education, 6 (2), 118-134. Doi: 10.1177/1538192707299287.
Barrick, M. R.; Shaffer, J. & DeGrassi, S. D. (2009). What you see may not be what you get: Relationships among self-presentation tactics and ratings of interview and job performance. Journal of Applied Psychology, 94 (6), 1394–1412. Doi: 10.1037/a0016532.
Brennan, R. L. (2001). Generalizability Theory. New York: Springer-Verlag.
Cardinet, J.; Johnson, S. & Pini, G. (2010). Applying generalizability theory using Edu G. Routledge.
Eva, K. W.; Reiter, H. I.; Rosenfeld, J. & Norman, G. R. (2004). The relationship between interviewers’ characteristics and ratings assigned during a multiple mini-interview. Academic Medicine, 79 (6), 602-609. Doi: 10.1097/00001888-200406000-00021.
Fauber, R. L. (2006). Graduate admissions in clinical psychology: Observations on the present and thoughts on the future. Clinical Psychology: Science and Practice, 13, 227–234. Doi: 10.1111/j.1468-2850.2006.00029.x.
Goho, J. & Blackman, A. (2006). The effectiveness of academic admission interviews: an exploratory meta-analysis. Medical teacher, 28 (4), 335-340. Doi: 10.1080/01421590600603418.
Hanson, M. D. Kulasegaram, K. M. Woods, N. N. Fechtig, L. & Anderson, G. (2012). Modified personal interviews: resurrecting reliable personal interviews for admissions? Academic Medicine, 87 (10), 1330-1334.
       Doi: 10.1097/ACM.0b013e318267630f.
Helmes, E. & Pachana, N. A. (2008). Value of interviews for admission to clinical training programs: Perspective of program directors. Australian Psychologist, 43 (4), 249-256.
     Doi: 10.1080/00050060802413362.
Jones, P. E. & Forister, J. G. (2011). A comparison of behavioral and multiple mini-interview formats in physician assistant program admissions. Journal of Physician Assistant Education, 22 (1), 36-40. Doi: 10.1097/01367895-201122010-00006.
Kim, K. J.; Nam, K. S. & Kwon, B. S. (2017). The utility of multiple mini-interviews: experience of a medical school. Korean Journal of Medical Education, 29 (1), 7-14. Doi: 10.3946/kjme.2017.48.
Lamadrid-Figueroa, H.; Castillo-Castillo, L.; Fritz-Hernández, J. & Magaña-Valladares, L. (2012). Admissions criteria as predictors of students’ academic success in master’s degree programs at the national institute of public health of Mexico. Public Health Reports, 127 (6), 605-611. Doi: 10.1177/003335491212700612.
Lemay, J. F.; Lockyer, J. M.; Collin, V. T. & Brownell, A. K. W. (2007). Assessment of non‐cognitive traits through the admissions multiple mini‐interview. Medical Education, 41 (6), 573-579. Doi:10.1111/j.1365-2923.2007.02767.x.
McAndrew, R.; Ellis, J. & Valentine, R. A. (2017). Does a selection interview predict year 1 performance in dental school? European Journal of Dental Education, 21 (2), 108-112. Doi: 10.1111/eje.12188.
Oranye, N. O. (2016). The Validity of Standardized Interviews Used for University Admission into Health Professional Programs: A Rasch Analysis. SAGE Open, 6 (3), 1-10. Doi: 10.1177/2158244016659119
Patterson, F.; Knight, A.; Dowell, J.; Nicholson, S.; Cousans, F. & Cleland, J. (2016). How effective are selection methods in medical education? A systematic review. Medical Education, 50 (1), 36-60. Doi: 10.1111/medu.12817.
Rosenfeld, J. M.; Reiter, H. I.; Trinh, K. & Eva, K. W. (2008). A cost efficiency comparison between the multiple mini-interview and traditional admissions interviews. Advances in Health Sciences Education, 13 (1), 43-58. Doi: 10.1007/s10459-006-9029-z.
Sebok, S. S.; Luu, K. & Klinger, D. A. (2014). Psychometric properties of the multiple mini-interview used for medical admissions: findings from generalizability and Rasch analyses. Advances in Health Sciences Education, 19 (1), 71-84. Doi: 10.1007/s10459-013-9463-7.
Shavelson, R.; Webb, N., & Rowley, G. (1989, June). Generalizability theory. American Psychologist, 44 (6), 922-932.
Stewart, G. L.; Dustin, S. L.; Barrick, M. R. & Darnold, T. C. (2008). Exploring the handshake in employment interviews. Journal of Applied Psychology, 93 (5), 1139–1146. Doi: 10.1037/0021-9010.93.5.1139.
Taylor, C. A.; Green, K. E. & Spruce, A. (2015). Evaluation of the effect of socio-economic status on performance in a multiple mini interview for admission to medical school. Medical Teacher, 37 (1), 59-63. Doi: 10.3109/0142159X.2014.923562.
Till, H.; Myford, C. & Dowell, J. (2013). Improving student selection using multiple mini-interviews with multifaceted Rasch modeling. Academic Medicine, 88 (2), 216-223. Doi: 10.1097/ACM.0b013e31827c0c5d.
Timer, J. E. & Clauson, M. I. (2011). The use of selective admissions tools to predict students’ success in an advanced standing baccalaureate nursing program. Nurse Education Today, 31 (6), 601-606. Doi: 10.1016/j.
Zaidi, N. B.; Swoboda, C.; Wang, L. L. & Manuel, R. S. (2014). Variance in attributes assessed by the multiple mini-interview. Medical teacher, 36 (9), 794-798. Doi: 10.3109/0142159X.2014.909587.