Why Students Rate Courses High / Low
Interview outline
Question 1: What were the most prominent features of the professional course you rated lowest?
Question 2: From the following, choose 1-3 that left the deepest impression (no need to answer all). Your impression of this course's teacher? Your impression of this course's TAs? Your impression of this course's workload? Your impression of the classroom teaching? Your impression of the exam? Your impression of the assignments? Your impression of the documentation? Your impression of the grading?
Question 3: What were the most prominent features of the professional course you rated highest?
Question 4: For Question 3's course, repeat Question 2.
Interview result statistics (sample size: 15)
Positive features
| Category | Feature | Count |
|---|---|---|
| Assignments | Clear requirements / detailed docs | 5 |
| Assignments | Fit course content | 2 |
| Assignments | Moderate difficulty | 2 |
| Assignments | Optional harder challenges | 1 |
| TAs | TAs responsible and conscientious | 3 |
| TAs | TAs actively answer questions, prompt and guide | 8 |
| Teaching | Slides/lecture clear | 7 |
| Teaching | Teacher lively and interesting | 2 |
| Teaching | Good interaction atmosphere | 2 |
| Exams | Moderate difficulty | 3 |
| Communication | Cares about students' learning and feedback | 2 |
| Grading | TA grading careful (timely, detailed comments) | 2 |
| Grading | Good grading, high excellence rate | 2 |
| Grading | Grading reasonable | 2 |
| Grading | Grading doesn't encourage involution | 1 |
Negative features
| Category | Feature | Count |
|---|---|---|
| Assignments | Unclear requirements | 5 |
| Assignments | Grading without detailed comments | 1 |
| Assignments | Grading not timely | 1 |
| TAs | TAs don't provide high-quality materials (docs/tutoring/recitations) | 5 |
| TAs | TAs themselves unfamiliar with assignments | 1 |
| Overall | Teaching, assignments, exams not closely related | 5 |
| Overall | Workload too heavy vs. credits | 1 |
| Overall | Self-study burden too heavy | 2 |
| Teaching | Fast pace, reading slides | 2 |
| Teaching | No slides provided | 1 |
| Teaching | Slides/lecture unclear | 4 |
| Teaching | Slides/content outdated | 2 |
| Teaching | Content deviates from course theme | 2 |
| Exams | Too many questions | 2 |
| Exams | High proportion of past questions | 1 |
| Exams | Lack of review materials (no answers published) | 1 |
| Communication | Little teacher-student contact | 1 |
| Grading | Exams dominate the final grade | 1 |
| Grading | Grading too lenient | 2 |
| Grading | Grading vague, explained afterward | 1 |
| Grading | Grading polarizing | 1 |
Students point to a few concentrated pain points and favorites; optimizing these may improve their experience.
Limitations of this survey
Of course, what students care about changes yearly. The survey has limitations; below we discuss them:
- Students' reasons mix teaching quality with personal bias. (Special case - general)
- Even seemingly general reasons can't directly yield solutions. (Problem - solution)
- Can't discover "things no current course has done." (What's missing)
Limitation 1: Some reasons relate to teaching quality, some to personal bias.
Teachers need to distinguish "bad experience from personal bias" vs. "bad experience from course gaps." If you treat personal bias as course deficiency, you only improve a few students' experience, possibly lowering it for most. Time/energy is limited; spending it on a minority reduces it for the majority. For extreme opinions, the silent majority is often on the other side.
An example is "grading quality": Suppose course A gives 4.0 to 30%, course B to 40%. Someone ranks similarly in both but doesn't get 4.0 in A, does in B, so complains A's grading is bad. Is that representative?
Raising A's 4.0 rate from 30% to 40%: does it help the former top 30%? No — they feel their grade's worth dropped. The bottom 60%? They never get 4.0 anyway.
Many people's rankings differ greatly between A and B, feelings are more complex. Someone rank 50% in A, 35% in B, complains A's grading, but even if A matched B, they still wouldn't get A's 4.0.
So under the grade-point system, adjusting "grading quality" basically only affects "students on the boundary sensitive to GPA." Most students don't rate highly just because of "good grading." Conversely, if grading is too lenient, diligent students see slack students also get 4.0, and develop negative feelings.
Even if grading quality truly affected ratings, within a few years it'd converge to everyone grading maximally; then students judge only on actual teaching quality.
Supplement
Rather than mere leniency, students care more about fairness and reasonableness. We can still think about more reasonable grading to help students learn. Later chapters cover this.
Here we just say that, under an unchanged grading mechanism, simply "raising grades" has limited positive effect on ratings.
Limitation 2: Even general reasons can't directly yield solutions.
Some students say "lecture too fast to digest." That may be general. But is slowing down the solution? Might not finish the syllabus, and some feel it's too slow.
If the real problem is "can't digest in class," can we help digestion outside class? E.g. add preview segments, recitations to review, encourage asking in the course group. This raises concrete questions: how to encourage preview? Assign preview homework / worksheets? Who runs recitations — TA capacity? No one answers in the group? ... We discuss some in later chapters.
Also, specifically which points aren't digested? If many students struggle at similar points, spend more time there — needs more specific research.
If digestion difficulty is a prerequisite the teacher assumed was learned but wasn't/forgotten, provide docs / review in class?
More causes: unclear figures / small font in slides? Students busy taking notes and not listening? Doing homework in class? An earlier conceptual misunderstanding? International students slower due to language? Stayed up for a deadline and sleepy in the morning class?
Even: does everything taught need digesting? Some frontier/intro material needn't be understood deeply; should the teacher say "this is for reference, no need to fully understand"?
Solving these needs more specific understanding of students, i.e., specific research for timely feedback.
Limitation 3: Can't discover "what no current course has done."
Collecting views, compiling "features of the highest-rated courses," and designing accordingly, even if perfectly accurate, only fits "the currently-best-rated courses." A good start. But if students have needs none of the current courses address, this method can't find corresponding solutions.
Cross-disciplinary Inspiration: shortening the feedback/sampling cycle in a course
In control theory, feedback means feeding the output signal back to the input, controlling the system to reduce the gap between actual and ideal output.
In biology, the body senses temperature/humidity and maintains homeostasis via hormones/neural regulation; a feedback mechanism.
In autonomous driving, the vehicle constantly senses the lane and adjusts speed/direction; feedback.
Network load balancing uses feedback: periodically get state, adjust traffic.
Teaching evaluation / course research adjusting teaching is feedback.
Biology's homeostasis, autonomous driving's control are continuous-time systems, sampling/regulating constantly.
Network load balancing, course feedback are discrete-time: periodic sampling, adjustment.
For discrete systems, "sampling period" heavily affects feedback-control stability. In short, shorter sampling → performance closer to continuous, better stability, faster convergence. Longer sampling → more unstable. Intuitive: a racecar driver who can see the road only every X seconds and adjusts then; larger X, harder to drive.
For a system as complex as teaching, control theory doesn't always apply. Feedback sampling is complex and costly. Students' energy for homework/questionnaires/interviews is limited; teacher/TA energy for questionnaires/analysis/adjustment is limited.
A simple idea: the school's evaluation sampling period is one semester/year, delaying improvement. Try low-cost, short-cycle in-semester feedback: e.g. after each class, email a short questionnaire to a few (<10) students; or send slides before class to a few (<10) to ask about hard points, and adjust. If students know their feedback affects this semester's learning, they may respond more eagerly. (End-of-semester feedback only benefits later cohorts.)
But if the adjustment period is too short, students may feel unsettled? Weigh it: students also need time to adapt to teaching style; don't keep changing.
Cross-disciplinary Inspiration: Users, Needs, and Products
Product managers hear: "listen to users, but don't do what they say."
Ford: before I made cars, people wanted faster horses.
Jobs: consumers don't know what they want until we show them.
Perhaps we need "listen to students, but don't do what they say" — maybe not as extreme as Jobs, but at least like Ford: from "want a faster horse," analyze the essential need for "faster transportation," and satisfy it in ways users didn't imagine.