Student evaluations of professors and teachers

[Clutch Munny]

Inspired by liv's "teaching to the test" thread...


Are they reliable? What, if anything, do they measure? What is your experience with them?

At my institution we have long used a 9-question thingy that seems to have been written by monkeys. Its instructions are contradictory, its questions are hard to parse, and it sometimes asks double- or even quadruple-barreled questions ("Was the course F and G with respect to X and Y?").

Fortunately, students rarely read instructions and are well accustomed to responding reflexively to keywords when parsing looks like it will be a chore. But the upshot is predictable. After years of concern from faculty over the fact that various faculty assessment procedures rely on the average score of these nine questions, hence losing fine-grained details about how students regard different elements of a prof's teaching, we recently ran a factor analysis that suggested that the nine questions really encode just one factor. (That is, the different questions aren't really sampling different opinions about different facets of the course/instructor. All the answers reflect one big, messy, schmozzled opinion about the instructor/course/text/room/timeofday/hottiesittingnexttome/TA/mygrade/smurf.) Which you might get no matter how well you devised and asked the questions, but which you will certainly get when the questions and instructions are obscure beyond the general perception of students that they are being asked to somehow evaluate this thing/event/person/experience/smurf.

Anyhoo. I think it's good to give students a chance to be heard. Everyone I work with takes the written comments section really seriously, taking great (and poorly concealed) pleasure in the positive ones and being very sensitive (typically in the best sense of the term) to any negative ones. It's unclear how well these evaluations measure student learning rather than student satisfaction (or students' perceptions of learning, which is another thing again). But evaluating teaching quality is important, and it's really difficult to do in any way that's fair (= consistent) between instructors, so the seeming objectivity of numerical ratings has huge administrative appeal.

Also, because you should always have a link in an OP, here's (http://www.informaworld.com/smpp/section?content=a782926822&fulltext=713240928) something about ratemyprofessors.

So what's your opinion or experience as a student or an instructor?

[Sock Puppet]

I always saw them as a chance to vent my spleen at instructors I disliked, and to express gratitude for those I enjoyed. As I recall, I don't think I ever panned a professor based on the grade I received, although I probably complained about a grading method I thought was unfair or stupid. The majority of my comments tended to address how engaging the instructor was, and how involved he/she seemed to be in both the material and the teaching thereof. If the professors seemed to love what they taught, they'd usually get favorable comments from me.

Probably the most unfair I ever got was when an instructor's focus was on areas I didn't give a shit about; I was primarily into language, so in lit classes, if they disdained language studies and focused on entirely thematic concerns, I tended to clobber them. This was immature, but what the hell, that's what undergrads are for. Beats saying "He gave me a D so he sux lol."

I remember barely any quantitative measurements (on a scale of 1 to 6, with one being least ... zzzz ... and 6 being most ... *yawn*), so when assessment forms had that crap I'd either ignore it or blast through it fairly thoughtlessly.

As for how it's applied, before Null's post in the other thread, I don't think I ever thought about the surveys being used by administration. I think when I was in college (UC Berkeley, 80s), they made it explicit that the forms were only for use by students and teachers. One could go to a big binder of sample assessments when deciding whether to take a prof's course (when one had a choice in the matter, obviously). I tended not to use them much, favoring asking other students and reading a prof's stated interests.

[Clutch Munny]

As I recall, I don't think I ever panned a professor based on the grade I received, although I probably complained about a grading method I thought was unfair or stupid.

Fair enough. But this sort of influence is typically unconscious or implicit; and if it does become conscious, it is easily rationalized post hoc (by, e.g., viewing it as a beef about method, etc. -- just sayin'). I doubt that grade is ever the whole story, but there is a good whack of evidence suggesting that evaluation score vary with student grade. Naturally this is confounded by the prospect that genuinely lousy teachers ipso facto do not teach students very well, leading to the students' getting lower marks on average and then accurately characterizing their instructor's performance as poor. So it is likely that at least some component of the covariation is consistent with the proper function of the evaluations.

[Sock Puppet]

Oh, I don't doubt that I could have rationalized the hell out of my assessments -- although I rarely had grades low enough to bitch about in the first place.

[Clutch Munny]

I rarely had grades low enough to bitch about in the first place.
:smugnod:

...more like.

[Sock Puppet]

I totally thought about adding the smugly! But I was thinking of this one: :pleased:

[Janet]

I hate, hate, hate filling out course evaluations. I always thought it was a stupid hoop to jump through at the end of the class and worthless. I have never, as far as I recall, put anything in the comments section because I just wanted to get the blasted thing over with. Actually, I take that back, in grad school I said the Research Methods class was too easy to be our culminating class. I remember doing that because another student got mad at me for it. But it was! My undergraduate Research Methods class was much harder than that one AND we actually had to do the research to get our honors degree. For the MLS all we had to do was write a research proposal. Weak!

[Dragar]

They are indeed horribly designed documents, just as you describe, Clutch. It seems insane that people who write these things as part of their job seem never to have been informed how to do so (and there is a great deal of research that has gone into how to write questionnaires).

When I filled them out as a student, I would leave fairly blunt feedback. Some of my lecturers were astonishingly wonderful paragons of pedagogy; others seemed to only have a vague sort of grasp of the material, or served merely to read aloud the notes and the derivations while we all squinted at the poor handwriting and tried to work out what was attempted to be explained. Some forgot they were supposed to be teaching us at all...

Our course evaluations were done before grades were handed out, incidentally. And grades came from exams, so ultimately it's not like the lecturer could have much to say in regards to what we scored.

[Clutch Munny]

We do our evals before final exams, but there has almost always been a fair bit of "continuous assessment" under the bridge by that point.

[wildernesse]

I loved evaluations. But, then I love surveys and giving my opinion--even sometimes when it is unsolicited! :gasp:

In law and grad school, I would occasionally take notes of what I wanted to add to my evaluation at the end of the year. I didn't always make comments, but if I thought that there were things that could be improved or that my comments might have some actual weight, then I was sure to write comments. I hope most of my comments were in the constructive criticism vein. I tried to specify the instance that I did/did not like, and why.

My master's level research methods class was also very weak, Janet. I think I learned more about research methods in fourth grade. I kinda think master's degrees should be abolished.

[Clutch Munny]

I hate, hate, hate filling out course evaluations. I always thought it was a stupid hoop to jump through at the end of the class and worthless. I have never, as far as I recall, put anything in the comments section because I just wanted to get the blasted thing over with. Actually, I take that back, in grad school I said the Research Methods class was too easy to be our culminating class. I remember doing that because another student got mad at me for it. But it was! My undergraduate Research Methods class was much harder than that one AND we actually had to do the research to get our honors degree. For the MLS all we had to do was write a research proposal. Weak!

Before I leave the room, I always give a little speech about how rare it is for a student to be asked their opinion about their learning experience, so they should take the chance to let someone know. (And I assure them that the evaluations are taken seriously.) It does seem to elicit a fairly high return of commenters from the class.

[Angakuk]

So, what you are saying is that you routinely lie to your students. That is awesome!

[beyelzu]

I just looked up stanton on ratemyprofessor and I have decided that the students rating him for the most part are precious crybaby snowflakes that deserved their hopefully failing grades.


I know its a derail.

[Clutch Munny]

Who is stanton?