To standardize the residency rank list across different interviewers and interview dates. 

The process of residency selection presents inherent challenges in determining which applicants will become high-performing residents. Even with a rigorous selection process, numerous factors influence final decisions, including individual interviewer preferences. Due to the large number of interviews required and limited interviewer availability, not every applicant is interviewed by all the same interviewers. In our program, interviewers assign scores to each applicant. Some interviewers demonstrated a tendency to give overall higher or lower scores.  This represents potential for interviewer leniency or strictness biases skewing the rank list. On dates where multiple interviewers gave higher scores, this significantly impacted the rank list. 

This is a single program study of the neurology residency interview scoring process.  Applicants were given a score from each interviewer, which were averaged to create a composite score for each applicant. The scores from individual interviewers were converted to Z scores for standardization across each interviewer. The adjusted Z scores were used to create the comparative rank list. 

Prior to converting to Z scores, the rank list demonstrated evidence of bias based on which interviewers were present each day with some interviewers giving overall higher average scores (leniency basis). Once scores were normalized, the rank list changed significantly with substantial shifts in the upper to midrange of the list. The selection committee judged the adjusted list to be more representative of applicants.

Standardizing interview scores across residency interviewers reduces rank list bias, which is critical in a process as important as selecting residents. Our neurology program selects 4 residents per cycle, but this may be even more important in larger residency programs with even more interviewers who only interact with a portion of applicants. Additionally, this information provides opportunities for feedback and program interview process development.