The Point: Course Evaluations and Overconfidence Bias
5/11/2020
Colleagues,
The FSU core bargaining team recently circulated a document that outlines tentative agreements made with the UMB administration with respect to urgent COVID-related concerns (see the document here; see here for more on FSU’s Covid-19 response). The team proposed to the administration team that we forgo using a course evaluation instrument designed for face-to-face learning as we moved to the remote environment. The tentative agreement reached represents a compromise position—to wit, that the evaluations will be used as part of a “holistic” approach to review and that “Spring 2020 teaching evaluations may not be singled out as uniquely persuasive evidence of poor or inadequate teaching for any faculty member.”
As many of you know, there is a growing literature in higher education circles challenging the use of such evaluations for personnel matters in the best of times: our own colleagues in the Philosophy Department prepared a convincing memo in 2018 to capture this paradigm shift as it emerged (https://bit.ly/3bePVI3)
As they wrote, the data collected in course evaluations add discriminatory bias, do not add information about teaching effectiveness, and actively detract from an appropriately critical evaluation of relevant information. The writers pointed to numerous types of bias baked into course evaluations, including “overconfidence bias”—the flawed notion that “we are not susceptible” to all the other manifest biases, and are more than capable of “making holistic decisions in which all data points are weighed appropriately.” It is impossible, they explain, to draw unbiased conclusions from a compromised data set.
In the immediate wake of going remote, university leaders around the country suspended the use of course evaluations (see here, for instance: https://bit.ly/2YJHLox). The amount of work required to go online, the reality that some fields--those that are performance- and lab-based, just for instance--are not easily adaptable, and the discomfiting truth that few among us are trained to do this kind of teaching, led administrators elsewhere to press pause on doing course evaluations this term.
Perhaps most troubling for the FSU Executive Committee is hearing numerous accounts that support the notion that the most vulnerable among us—instructors with subpar internet connections driving to coffee shop parking lots to teach from their cars, single-parent instructors balancing homeschooling of their own children with UMB teaching responsibilities, and so on—are experiencing pedagogical challenges that cannot be satisfactorily accounted for by the existing evaluator instrument. There is little doubt that existing inequities will only be amplified by the current crisis. Non-tenure track colleagues who are teaching without continuing contracts will be particularly at risk.
But to get to the real point: the tentative impact bargaining agreement for Spring 2020 was made under the assumption that course evaluations would be implemented with consistency, professionalism, and faculty input at the department level. There is mounting evidence that this is not the case. Faculty are not being notified when evaluations are going out, students who have not been taking certain courses in any meaningful sense are being invited to fill out the evaluations, some students report not being able to fill out the evaluations for more than one class they are enrolled for, and so on.
Given the sloppiness of the remote course evaluation rollout, it seems that the data gathered from Spring 2020 course evaluations might need to be put into quarantine with the rest of us.
Please keep us posted about your experience with respect to course evaluations this term by emailing fsu@umb.edu.
And let us know more generally what you are thinking about and how you want to be involved.
This is your union.
Sincerely,
Jeffrey Melnick
Graduate Program Director, American Studies Department
Communications Director, Faculty Staff Union Executive Committee
Check out the FSU Blog
Please “like” the FSU Facebook Page
Follow us on Twitter at @FSU_UMB