The lie that my high school teacher told me

In my previous post I blogged about how there's no real mystery to academic achievement, how my teachers told me I'd be on my own in college so I studied accordingly.  Now I want to talk about a lie that they told me:

One of my high school English teachers said that in college you're expected to read books that aren't explicitly assigned.  She said that if a literature professor assigned a novel it was understood that besides reading the assigned novel you should be independently scouring the library for critiques of the novel and background on the writer, so that you could come to class prepared to contribute to the discussion.

I took that to heart.  The summer before my freshman year of college I was given the reading list for the humanities class that I would take that fall.  The class had a pretty long list of classic works, and I decided that I would read the thickest book on that list during the summer, so that when all hell broke loose in fall I would have the worst assignment out of the way.  So I read The Brothers Karamazov.  And then I did one better: I read a biography of Dostoyevsky, because my teacher said that you're supposed to be proactive like that.

Well, in the middle of that first semester we were reading Dostoyevsky (and chemistry was hard as hell and sucking up all my attention, so I was very glad that I was ahead in my humanities class), and during one of the discussions a classmate started offering assertions about the author's life and views and how they related to whatever scene in the novel.  I honestly don't remember what her point was, but I do remember that her assertion about Dostoyevsky's life was 100% wrong (remember, I'd actually read a biography of him, though I don't remember much of it 23 years later), and when I pointed that out the professor and class were rather uninterested.

So much for the idea that you have to be on top of things in college.  Mrs. Sadowsky lied to me!

Why I am what I am

I went to a decent high school.  Not fancy, not lousy.  Science labs were OK but we weren't putting out national science fair competitors.  Most kids weren't poor but most weren't upper-middle-class either.  Not particularly disadvantaged but not a lot of affluent kids either.  Pretty much in the middle.  Which is fine.  We had college-prep classes but we also had a lot of vocational classes.  It was a decent school in the middle of the class hierarchy.  I was more privileged than a lot of the US but less privileged than a lot of the people I later met in college.

One thing that was repeatedly driven home to me by my teachers was that we would be on our own in college, and that there would be no hand-holding.  It was repeatedly said that we would have to figure it out, that they wouldn't patiently explain to us how much we should be reading and reminding us to study and reminding us to do our homework and turn it in on time.  It would be hard and we would be on our own.  This was repeatedly said to me and my middle-class friends.

And, to a large extent, that was true.  College was a bit more structured and supportive than my high school teachers made it sound, but only a bit.  Ultimately, it was on me.  I accepted that from day one, and it felt utterly unsurprising to me that I was doing well because I studied all the time.  It felt utterly unsurprising when friends and dorm-mates who studied less didn't do as well.  It's the natural order of the world.  Likewise, in grad school I felt a bit more burned out, a bit more interested in my own life, and I didn't do as well as I did in college.  I still got through, but I was not on top, and that was no surprise to me because I didn't put as much in.  Then I became a professor and buckled down more and published more and got decent student evaluations because I put in a ton of prep and none of this seemed at all surprising to me.

So you can imagine how strange it seems to me when people keep saying that we need to do more for students who don't know how to study, how to take notes, how to manage their time, etc.  It baffles me that we are supposed to be responsible for their success.  Why would we be?  Why would it be my problem that people who never put much effort into mastering freshman material are now doing poorly with advanced material?  Why is it my duty to fix this?  It's not like I could actually do anything about it (you only get the benefit of years of effort by putting in years of effort--there are no secret tricks), so why is it my responsibility?

Likewise, it seems strange to me when people say that it's so unfair that we attach weight to grades and test scores.  I got into a college with people who were way above me in the class hierarchy, people whose parents had more money and more degrees, people who had attended prep schools and magnet schools.  I was there because I had done at least as well as they had done on the SAT (among other things).  I got better grades than them in college.  I out-studied them, out-performed them in the academic arena, and moved ahead academically.  Isn't that how it's supposed to be?

I've spent more than two years reading and blogging about the cultural currents that underlie the notions that seem so strange to me, but I'll be honest:  To this day there's still a part of me that thinks "Well, duh, I studied my ass off because my teachers all told me that that's what I'd have to do to succeed in college.  What else were you expecting?"

Project idea: Incentives, measurement, and fairness

Among the problems that I find most fascinating and most frustrating is the that of people who want to systematize education and incentives.  We all agree that what we really want is meaningful, intensive, in-depth discussion of important topics, creative assignments, and lots of attention to students as individuals.  We all agree that this is hard to systematize and measure, even if some approaches are clearly less bad than others.  We all agree that the things that are easiest to measure are not quite what we desire. We all agree that the more you incentivize something the more that people will focus on hitting the metric rather than doing what the metric is supposed to be a proxy for (sometimes called Campbell's Law).  And we all agree that without incentives of SOME sort people will get lazy.  So we have a hard problem.

My thought is that if a simple metric isn't TOO distant from what you really want, and if the competition isn't TOO intense, then being measured will keep people from being lazy, but because you still have some flexibility (the competition isn't so intense as to push you into a single-minded focus on that metric) and because you yourself actually value the thing that REALLY matters (and the metric isn't completely decoupled from it) then you'll show up to work and split your effort between hitting the metric and doing the more meaningful thing that everyone REALLY values.

But if the competition is more intense then you have to hit that number no matter what.

As to the people measuring you, on some level they know that the simple metric is flawed, but they face two other pressures;

1) Measuring something closer to what they want would be more expensive.

2) Because those measurements are of limited precision and may be subjective, it would be seen as unfair to focus so much on them.  Indeed, I sometimes think that the current focus on the findings of bias research plays into this:  The output of the bias research community has demonstrated that everything we do can be unfairly biased.  It's thus tempting to seek simpler and more transparent technical evaluations instead of subjective appraisals of nebulous quality.

So here's my idea for a model:

An administrator has a pot of money to allocate and people.  The funder assigns money based on two measures, and easy one and a hard one.

The researchers getting the money have to allocate their efforts among three things: Leisure (bad!), hitting an easy target (which generates some utility, but quickly reaches the point of diminishing marginal returns), and hitting a hard target (which takes more effort but yields more satisfaction).  

The administrator's utility payoff is based on 3 things:

1) How much effort people allocate to doing hard things.  The payoffs here are huge, because the administrator gets to take credit for good things that happen in the system that he/she oversees.

2) How fair and transparent the administrator is.  The more the administrator rewards easy things, the more that political bosses and/or the public will perceive him/her as fair and transparent.

3) How much time the administrator invests in measuring the hard thing.  The payoffs here are negative because it's hard to measure.

The researchers' utility payoff is based on 3 things:

1) How much time they spend on leisure.  There are diminishing marginal returns here.  Zero vacation days will kill a person, but after too much vacation they want to get back to the lab.

2) How much time they spend on hard things.  There are increasing returns on scale here, because you don't get anywhere until you've invested a lot of effort.

3) How much money they get from the administrator.

What can we say about the Nash Equilibria of this problem?  It strikes me as posing issues similar to Holmstrom's Theorem.

On some level I've basically outlined the issue to the point where a key idea can be presented:  The easy measure is useful to the extent that it satisfies the public's need to see that the system is honest, and also keeps researchers from being lazy.  The easy measure is parasitic to the extent that it diverts time away from what is meaningful.  You can say "Just improve the easy measure so it's more aligned with what you value!" but the key points are:

1) Measuring valuable things is hard (by assumption).

2) Aligning the easy measure more closely with what you really value is great, but if the competitive pressures are high enough you'll eventually see the difference between the easy measure and what  you really value.  When competitive effects are weak then people will still allocate a lot of time to hard but satisfying efforts.  When competitive effects are strong then people will devote much more time to easy things.

Of course, it's nice to make these points, but the system that I work in doesn't reward blog posts.  The system that I work in rewards peer-reviewed journal articles, and journals reward mathematical formalism.  I could just sit down and do it but I have lots of other projects on my plate, so I need a collaborator to prod me.  If anyone wants to help please let me know!