A qualitative assessment of behavioral interview method among anesthesiology residency applicants
Background
The objective of residency recruitment is to select the most appropriate candidate. While cognitive skills are identified before an interview and can be measured objectively, non-cognitive skills can be harder to discern. These non-cognitive skills though are a good predictors of future residency performance. A structured behavioral interview is better at identifying noncognitive skills compared to a traditional interview.
Objective
Compare the noncognitive traits identified in the interviews with those identified in resident evaluations
Methods
Using the semi-annual evaluations 6 residents were split between satisfactory group and excellent group. Behavioral-based interviews and traditional unstructured interviews conducted on the same individual were compared and the results of the interview were compared to the semi-annual and annual evaluations submitted by the teaching faculty and the program directors. The interviews were analyzed for non-cognitive skills.
Results
Qualitative analysis of behavioral interview narratives and the narrative part of the semiannual evaluations independently identified the non-cognitive characteristics of adaptability, decisiveness, time management, judgment, and the ability to work in a team. It also identified other essential non-cognitive skills necessary for an anesthesiologist, such as ability to prioritize, study techniques, ability to destress.
Conclusion
The noncognitive traits found in the excellent group response to the structured interview match those found in the semi-annual evaluations. In this case individuals who are suited to the field of anesthesiology are adaptable, decisive, team-players with excellent judgment.
1. Stephenson-Famy A, Houmard BS, Oberoi S, Manyak A, Chiang S, Kim S. Use of the Interview in Resident Candidate Selection: A Review of the Literature. J Grad Med Educ. 2015;7(4):539-548. doi:10.4300/JGME-D-14-00236.1
2. National Resident Matching Program. Results and Data: 2024 Main Residency Match®.; 2024.
3. Brummond A, Sefcik S, Halvorsen AJ, et al. Resident Recruitment Costs: A National Survey of Internal Medicine Program Directors. Am J Med. 2013;126(7):646-653. doi:10.1016/j.amjmed.2013.03.018
4. Haas MRC, He S, Sternberg K, et al. Reimagining Residency Selection: Part 1-A Practical Guide to Recruitment in the Post-COVID-19 Era. J Grad Med Educ. 2020;12(5):539-544. doi:10.4300/JGME-D-20-00907.1
5. Marcus-Blank B, Dahlke JA, Braman JP, et al. Predicting Performance of First-Year Residents: Correlations Between Structured Interview, Licensure Exam, and Competency Scores in a Multi-Institutional Study. Academic Medicine. 2019;94(3):378-387. doi:10.1097/ACM.0000000000002429
6. Neitzschman HR, Neitzschman LH, Dowling A. Key Component of Resident Selection. Acad Radiol. 2002;9(12):1423-1429. doi:10.1016/S1076-6332(03)80670-0
7. Luedi MM, Doll D, Boggs SD, Stueber F. Successful Personalities in Anesthesiology and Acute Care Medicine: Are We Selecting, Training, and Supporting the Best? Anesth Analg. 2017;124(1):359-361. doi:10.1213/ANE.0000000000001714
8. Dirschl DR, Campion ER, Gilliam K. Resident Selection and Predictors of Performance. Clin Orthop Relat Res. 2006;449:44-49. doi:10.1097/01.blo.0000224036.46721.d6
9. Swanson WS, Catherine Harris M, Master C, Gallagher PR, Mauro AE, Ludwig S. The Impact of the Interview in Pediatric Residency Selection. Ambulatory Pediatrics. 2005;5(4):216-220. doi:10.1367/A04-149R1.1
10. Mytna Kureková L, Beblavy M, Haita C, Thum AE. Employers' skill preferences across Europe: between cognitive and non-cognitive skills. Journal of Education and Work. 2016;29(6):662-687. doi:10.1080/13639080.2015.1024641
11. Holtzman KZ, Swanson DB, Ouyang W, Dillon GF, Boulet JR. International Variation in Performance by Clinical Discipline and Task on the United States Medical Licensing Examination Step 2 Clinical Knowledge Component. Academic Medicine. 2014;89(11):1558-1562. doi:10.1097/ACM.0000000000000488
12. Powis DA. Selecting medical students. Med Educ. 1994;28(5):443-469. doi:10.1111/j.1365-2923.1994.tb02555.x
13. Boyse TD, Patterson SK, Cohan RH, et al. Does Medical School Performance Predict Radiology Resident Performance? Acad Radiol. 2002;9(4):437-445. doi:10.1016/S1076-6332(03)80189-7
14. Lee AG, Golnik KC, Oetting TA, et al. Re-engineering the Resident Applicant Selection Process in Ophthalmology: A Literature Review and Recommendations for Improvement. Surv Ophthalmol. 2008;53(2):164-176. doi:10.1016/j.survophthal.2007.12.007
15. Fortune JB. The content and value of letters of recommendation in the resident candidate evaluative process. Curr Surg. 2002;59(1):79-83. doi:10.1016/S0149-7944(01)00538-4
16. Bernstein AD, Jazrawi LM, Elbeshbeshy B, Valle CJD, Zuckerman JD. ORTHOPAEDIC RESIDENT-SELECTION CRITERIA. The Journal of Bone and Joint Surgery-American Volume. 2002;84(11):2090-2096. doi:10.2106/00004623-200211000-00026
17. Wang X, Zhang W, Sheng S, et al. Residency Attrition and Associated Characteristics, a 10-Year Cross Specialty Comparative Study. Journal of Brain and Neurological Disorders. 2022;5(4).
18. Kremer TR, Kremer MJ, Kremer KP, Mihalic A. Predictors of getting a residency interview: Differences by medical specialty. Med Educ. 2021;55(2):198-212. doi:10.1111/medu.14303
19. Vinagre R, Tanaka P, Park YS, Macario A. Red Flags, Geography, Exam Scores, and Other Factors Used by Program Directors in Determining Which Applicants Are Offered an Interview for Anesthesiology Residency. Cureus. Published online November 18, 2020. doi:10.7759/cureus.11550
20. Shah S, Con J, Mercado L, Smiley A, Weber G, Abramowicz AE. Predictors of Matching into Anesthesiology and Surgery: Analysis of One Program's Results. J Surg Educ. 2023;80(9):1231-1241. doi:10.1016/j.jsurg.2023.06.021
21. Cohn MR, Bigach SD, Bernstein DN, et al. Resident Selection in the Wake of United States Medical Licensing Examination Step 1 Transition to Pass/Fail Scoring. Journal of the American Academy of Orthopaedic Surgeons. 2020;28(21):865-873. doi:10.5435/JAAOS-D-20-00359
22. Chen F, Arora H, Martinelli SM, et al. The predictive value of pre-recruitment achievement on resident performance in anesthesiology. J Clin Anesth. 2017;39:139-144. doi:10.1016/j.jclinane.2017.03.052
23. Blouin D. Reliability of a Structured Interview for Admission to an Emergency Medicine Residency Program. Teach Learn Med. 2010;22(4):246-250. doi:10.1080/10401334.2010.511979
24. Janz T, Hellervik L, Gilmore D. Behavior Description Interviewing. Allyn and Bacon Publishers; 1986.
25. McDaniel MA, Whetzel DL, Schmidt FL, Maurer SD. The validity of employment interviews: A comprehensive review and meta-analysis. Journal of Applied Psychology. 1994;79(4):599-616. doi:10.1037/0021-9010.79.4.599
26. Altmaier EM, From RP, Pearson KS, Gorbatenko-Roth KG, Ugolini KA. A prospective study to select and evaluate anesthesiology residents: Phase I, the critical incident technique. J Clin Anesth. 1997;9(8):629-636. doi:10.1016/S0952-8180(97)00195-5
27. Giannantonio CM. Book Review: Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology (2nd ed.). Thousand Oaks, CA: Sage. Organ Res Methods. 2010;13(2):392-394. doi:10.1177/1094428108324513
28. Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105-112. doi:10.1016/j.nedt.2003.10.001
29. Gunzburger LK, Frazier RG, Yang YM, Rainey ML, Wronski T. Premedical and medical school performance in predicting first year residency performance. J Med Educ. 1987;62:379-384. doi:10.1097/00001888-198705000-00002
30. Erlandson EE, Calhoun JG, Barrack FM, et al. Resident selection: applicant selection criteria compared with performance. Surgery. 1982;92:270-275.
31. Dirschl DR, Dahners LE, Adams GL, Crouch JH, Wilson FC. Correlating selection criteria with subsequent performance as residents. Clin Orthop Relat Res. 2002;265(399). doi:10.1097/00003086-200206000-00034
32. Turner NS, Shaughnessy WJ, Berg EJ, Larson DR, Hanssen AD. A quantitative composite scoring tool for orthopaedic residency screening and selection. Clin Orthop Relat Res. 2006;449(50). doi:10.1097/01.blo.0000224042.84839.44
33. Calhoun KH, Hokanson JA, Bailey BJ. Predictors of residency performance: a follow-up study. Otolaryngol Head Neck Surg. 1997;116:647-651. doi:10.1016/S0194-5998(97)70242-0
34. Gardner B. A multivariate computer analysis of students' performance as a predictor of performance as a surgical intern. J Surg Res. 1972;12:216-219. doi:10.1016/0022-4804(72)90111-4
35. Takayama H, Grinsell R, Brock D, Foy H, Pellegrini C, Horvath K. Is it appropriate to use core clerkship grades in the selection of residents? Curr Surg. 2006;63:391-396.
36. Gardner B. A multivariate computer analysis of students' performance as a predictor of performance as a surgical intern. J Surg Res. 1972;12:216-219. doi:10.1016/0022-4804(72)90111-4
37. Borowitz SM, Saulsbury FT, Wilson WG. Information collected during the residency match does not predict clinical performance. Arch Pediatr Adolesc Med. 2000;154:256-260. doi:10.1001/archpedi.154.3.256
38. Janis JE, Hatef DA. Resident selection protocols in plastic surgery: a national survey of plastic surgery program directors. Plast Reconstr Surg. 2008;122(6):1929-1941. doi:10.1097/PRS.0b013e31818d20ae
39. Korman M, Stubblefield RL. Medical school evaluation and internship performance. J Med Educ. 1971;46:670-673.
40. Lee AG, Golnik KC, Oetting TA. Re-engineering the resident applicant selection process in ophthalmology: a literature review and recommendations for improvement. Surv Ophthalmol. 2008;53:164-176. doi:10.1016/j.survophthal.2007.12.007
41. Komives E, Weiss ST, Rosa RM. The applicant interview as a predictor of resident performance. J Med Educ. 1984;59:425-426. doi:10.1097/00001888-198405000-00009
42. Altmaier EM, Smith WL, O'Halloran CM, Franken EA. The predictive utility of behavior-based interviewing compared with traditional interviewing in the selection of radiology residents. Invest Radiol. 1992;27:385-389. doi:10.1097/00004424-199205000-00013
43. Campion MA, Campion JE, Hudson JP Jr. Structured interviewing: a note on incremental validity and alternative question types. J Appl Psychol. 1994;79:998-1002. doi:10.1037/0021-9010.79.6.998
44. Orpen C. Patterned behavior interviews versus unstructured interviews: a comparative validity study. J Appl Psychol. 1985;70:774-776. doi:10.1037/0021-9010.70.4.774
45. Andrades M, Bhanji S, Kausar S, Majeed F, Pinjani S. Multiple Mini-Interviews (MMI) and Semistructured Interviews for the Selection of Family Medicine Residents: A Comparative Analysis. Int Sch Res Notices. 2014;2014:1-6. doi:10.1155/2014/747168
46. Long T, Dodd S, Licatino L, Rose S. Factors important to anesthesiology residency applicants during recruitment. J Educ Perioper Med. 2017;19(2):E604. doi:10.46374/volxix-issue2-dodd