HowÂ good are our existing measures for assessing the ‘value-added’ of differentÂ university lecturers? According to a new Canadian paper, those much-maligned student ratings do a better job of predicting value-added than faculty rank.
Professor Qualities and Student Achievement
by Florian Hoffmann, Philip Oreopoulos
This paper uses a new administrative dataset of students at a large university matched to courses and instructors to analyze the importance of teacher quality at the postsecondary level. Instructors are matched to both objective and subjective characteristics of teacher quality to estimate the impact of rank, salary, and perceived effectiveness on grade, dropout and subject interest outcomes.Â Student fixed effects, time of day and week controls, and the fact that first year students have little information about instructors when choosing courses helps minimize selection biases.Â We also estimate each instructor’s value added and the variance of these effects to determine the extent to which any teacher difference matters to short-term academic outcomes.Â The findings suggest that subjective teacher evaluations perform well in reflecting an instructor’s influence on students while objective characteristics such as rank and salary do not.Â Whether an instructor teaches full-time or part-time, does research, has tenure, or is highly paid has no influence on a college student’s grade, likelihood of dropping a course or taking more subsequent courses in the same subject.Â However, replacing one instructor with another ranked one standard deviation higher in perceived effectiveness increases average grades by 0.5 percentage points, decreases the likelihood of dropping a class by 1.3 percentage points and increases in the number of same-subject courses taken in second and third year by about 4 percent.Â The overall importance of instructor differences at the university level is smaller than that implied in earlier research at the elementary and secondary school level, but important outliers exist.
Maybe we should be paying more attention to the way ANU academics perform on Rate My Professors?