Home » Teaching » Assessment » Prestige, Measurement, and What’s Left Out that Matters

Prestige, Measurement, and What’s Left Out that Matters

Fifth in a Series.  First one is here.

When I was at MIT, it seemed like hardly anyone there was interested in teaching.  My advisor told me that mathematicians viewed teaching as akin to golf: an unrelated side-interest.  The teaching award was known as the “kiss of death,” because faculty who got it did not get tenure.  Graduate students who showed too much interest in teaching were presumed not sufficiently serious about research.

But with MOOCs, MIT and its peers Harvard, Stanford, and the like, are now interested in education, and due to their prestige, are the presumed leaders.  A bit after I enrolled in the physics MOOC from MIT there was an article about MOOCs in the Boston Globe that discussed the course, and where another MIT instructor talked about how from MOOC data he learned that students often start the problems before they watch lectures, which gave him the idea to sometimes intentionally start with problems, something that many of us (including some in the dreaded ed schools) have known for years (and some of us skip lecturing almost entirely).

Apologies if I sound a bit bitter; having wealthy prestigious schools care about education is huge, and I am for it.  But there is an infuriating trend in education reform (a phrase that I used to see as mostly positive and no longer do) where those with money and prestige jump to the head of the line whenever they decide to take a break from the pursuits that led to wealth and prestige to take an interest in education.   Since almost everything about existing education is presumed to be a complete failure, the voices of those with a long-term interest in education are the voices of failure, not worth heeding.

So Bill Gates now gets to decide which education programs are worthy and which aren’t, as he funds them based on Return on Investment metrics modeled on those of Microsoft.  He funds Sal Khan, who goes from being an investment banker to perhaps the “best math teacher in America,” based on his friendly voice and innovate delivery model, even though his videos are riddled with the most common beginning math teacher mistakes.  Teach for America sends graduates of prestigious universities into communities that have suffered the most from racism and poverty, with only a little bit of training, often displacing existing teachers, and expects them to fix the schools and often become principals and other leaders within a few years.

The big data that MOOCs are supposed to provide to revolutionize education are only as good as their metrics, and flaws in metrics are present throughout the current version of education reform, which consistently favors what is easiest to measure, which is not necessarily what is most important.

Here are a few things that are critical, but hard to measure:

Retention: What will the student know of the course five or ten or thirty years from now?   When I think about how I came to understand, say, fractions, a topic that many college students don’t understand well, it is impossible to sort out what percentage of my understanding can be attributed to which teacher, parent, or book, as well as how much I just figured out myself.   Many metrics for measuring teacher or course effectiveness fail because in order to isolate the “value added” of the teacher or course, the metrics need to be very short term, and teaching for short term gain can be in opposition to teaching for long term understanding.  Frequently students warn me on the first day of class that they haven’t had math in a year or two, and thus cannot be expected to know very much.

Perhaps MOOC  instructors could follow up by asking some similar questions several years later, but the response rate would likely be low, and it’d be difficult to attribute successes to the MOOC (although if there was widespread failure, as in students who need to take remedial math classes after years of high school math, that might be some information).

Joy: Joy is a word that is not often mentioned in the context of education reform, yet at the moment, part of what’s wonderful about MOOCs is that so many people enroll in them for the joy of learning – and by joy, I mean something deeper than non-stop fun and amusement.  MOOCs at the moment are full of self-selected students who are there because they want to learn and can leave when they don’t want to be there anymore.  Many already have deep backgrounds in the subject.  Most of us who teach in other contexts are thrilled when we get a class full of well-prepared, highly-motivated students: this is not the norm.   If MOOCs become more mainstream, full of people taking them because the class is a requirement, perhaps resentful because they can’t afford the smaller, more personal experience they would prefer, then they will become something very different, and it’s not clear that data currently gathered will be relevant.

Other ed reform efforts seem designed to suck joy out of learning, with boring scripted curricula and frequent tedious testing.  In math, it’s far too common for students to do well enough to pass the tests, forget most of the material, and then avoid math for the rest of their lives because the most important mathematics lesson they learned in school is that there’s no joy in math.

Originality/Creativity: When I think about computer graded essays, as many are proposing (or already using) for MOOCs in the humanities and other subjects, my response is on the horror continuum.  The point of writing is to communicate with humans (even writing for oneself in a journal).  Often in school, I felt like I was writing papers I didn’t particularly want to write for teachers who didn’t particularly want to read them, and having my reader be a computer would cement the pointlessness of such exercises.    In the past few years, I’ve been conscious of crafting the writing assignments I give so that I will be excited about reading them: I give students a choice of problems to solve, choose problems with many different entry points, and ask students to choose their own ways to extend the problems.    On every such assignment, I am delighted by students’ originality and creativity (and often they take great joy in it themselves, even though they might have initially complained about having to write papers in math class).

The Creative Programming MOOC was so wonderful because we were encouraged to do something original, and we got to share in others’ creativity.  There’s no good way to quantify the value added of the course to our creativity, so half the grade of the class was based on quizzes that were focused on what’s easy to measure.  If students took this course to meet a requirement, then they might skip the creative part, focusing on what they needed to do to get the grade they wanted.

What else besides retention, joy, and originality/creativity would you add to the list of things missing from the education reform debate?  Relationships is next on my list, but I’m out of time for today.

<– Prev (Learning to Dislike Programming

Leave a Reply

Your email address will not be published. Required fields are marked *