Sein und Zeitgeist

In Chapter 10, Lee Jussim gets to the question of why inaccuracies and bias in human perception get more attention than accuracy.  On page 153 he addresses an aspect of the issue that I think is most salient to my concerns as an educator: If we believe that a person is unskilled, under-qualified, under-accomplished, or whatever because our perceptions are inaccurate, then "all" that we need to do is change our perceptions.  That isn't always easy to accomplish as a seamless thing, it often requires conscious effort, but, in the end, bringing yourself to say "You know, he/she is actually pretty talented; I'll hire them/admit them to college/etc." is still easier than saying "Uh-oh, this person is way under-prepared, we have work to do to fix that."  Any research that tells us that problems can be fixed by thinking differently is going to attract more support from the intellectual class than research that tells us that we'll have to do a lot more work on problems that stubbornly defy easy solutions.

On page 158, Jussim also notes that social psychology is more fun if you find bias and inaccuracy.  If people generally perceive things pretty accurately then what is there to say?  What need is there for a consultant?  People will figure things out and, um, yeah.  But if bias and misperception are everywhere then there's something to work on.  This is akin to issues of publication bias in other areas of science.  Perhaps bias research is biased in favor of bias research?

If the concerns in the first paragraph are largely social and political, and the concerns in the second paragraph are technical and pecuniary, then I think there's also a third contributor to the intellectual zeitgeist: Negative results from Western intellectuals in the 20th century.  From them, we learned that measurements of distance and time are not absolute, but rather depend on how you are moving relative to someone else.  (Einstein)  We learned that mathematics is incomplete, and contains statements that cannot be proved by pure logic.  (Godel)  We learned that matter is not composed of solid particles with fixed positions and motions, but rather is "fuzzy" and smeared out, with uncertain positions and velocities. (Schrodinger, Heisenberg) Then Turing taught us that computers, the defining technology of the modern era, would have limits to what they could compute.  Subsequently, Arrow taught us that it is impossible to even construct reasonable and egalitarian procedures for picking a winner in an election!*  All of this set the stage for post-modernism and numerous other ideas from French intellectuals, and ultimately led to classmates in the 1990's telling me that scientific truth is just, like, a social construction, man.**

Mind you, I would happily argue that relativity, quantum mechanics, formal logic, computer science, and election theory should not be taken as death knells for ideas of objective knowledge and accurate perception.  I would be happy to argue that all of these ideas are misinterpreted.  However, if we are discussing how these topics are viewed by the educated classes rather than how they ought to be viewed then I think it's clear that the zeitgeist is favorable to the notion that humans can't really, like, know anything.  And, having just graded a stack of final exams, I am also favorable to the notions that most humans are unable to, like, know anything!  :)

*What Arrow's Theorem really says is a bit more subtle than that, but let's leave that aside for now.  Some other time I'll argue that the Gibbard-Satterthwaite Theorem can be interpreted as a theorem about conflicts of interest.
**However, the "science wars" of the 1990's, in which some humanities scholars went way too far in their critiques of science, became irrelevant after the 2000 election.  The academic community found a new common cultural enemy, and recently I read an article (can't find the link now) in which one combatant on the left wing of the "science wars" now finds himself on the side of objective scientific fact when facing right-wingers who argue that global warming is just, like, your opinion, man.

Several chapters into Jussim: Is zeitgeist a self-fulfilling prophecy?

I'm about a quarter of the way through Jussim's book, and a lot of it is fairly technical.  I'd rather not wade through his critiques of studies on bias and expectations and self-fulfilling prophecies.  Rather, I'll just note that if we take the findings of those studies seriously, and take the narrative around those finding seriously, then amidst an intellectual zeitgeist of "People are way more biased and subject to influence than they realize!" isn't it inevitable that psychologists will find bias and influence and self-fulfilling prophecies everywhere?

Perception of control, reality of complexity

I'm in the middle of Chapter 3 of Jussim's book.  Right now he is covering the Pygmalion study by Rosenthal and Jacobson (1968), which found that if teachers were told that student 1 had strong potential to improve and student 2 had less potential to improve then there would be a difference in improvement, even if the descriptions were attached to the students at random, without any evidentiary basis.  Jussim has his critiques of the study and its limitations, as well as the complexity in follow-up studies, and I am not really qualified to sort through the detailed strengths and weaknesses of a myriad of psychology experiments and meta-analyses. Instead, I want to muse on the attraction of theses studies in the wider academic community outside of the psychology department.

First, let me note that it is indeed attractive to academics.  When I read his description of the study it vaguely rang a bell in my mind, but I could easily be confusing it with some other study that got a lot of attention.  My wife, however, had definitely heard of it when she was studying for her elementary school teaching credential.  Indeed, given the potential implications of a study like this for how people teach, one would hope that prospective teachers do learn about this work, provided that it is reliable.

However, Jussim notes that the follow-up work painted a more complicated picture than the original study.  This should not be surprising to anybody who has ever engaged in any sort of scholarly activity.  Academic research is complicated enough, and real-world phenomena even more so.  The maze of studies that did and didn't replicate at varying levels of statistical significance and under varying conditions tells me that this is a complicated phenomenon of human behavior, a question that is worth getting to the bottom of.  As a non-specialist, I want to read the final review article on efforts to sort ouf this phenomenon, not the first high-profile study.  I know that in my own line of work, on optics in biology, the first impressive deployment of a new imaging technique always relies on a particular apparatus looking at a particular specimen, and achieving the promised benefits in a robust manner always takes a lot of hard work by a lot of people in a lot of different specialties, sometimes collaborating and sometimes competing to push each other harder.

Of course, nobody wants to hear "First there was a promising idea, then hundreds of people worked for years to make it work reliably."  Especially not for human interactions, where we tell ourselves that since we don't need expensive equipment it really ought to be simple and reduce to some short script, so just tell us what to do and make it easy.  We all know about the original stereotype threat study, so let's just move some questions to the end of the test and change the description at the beginning of the test.  We know about the study of Rosenthal and Jacobson in the 1960's, so let's just get teachers psyched up to believe that all of their students will improve, and BOOM! Everybody learns more! (And the people who are most receptive to This One Study seem to be the ones who most enjoy the "Let's get the audience fired up!" part of a workshop.) We all heard about the values affirmation study (.pdf), so let's just give everybody a short essay assignment and erase the achievement gap.  It's all so seductive:  Simple hack, big results!  Small intervention, big improvement!  Eat this one food, lose weight!  (My belly could serve as This One Piece Of Evidence showing that a kitchen full of fruit doesn't automatically lead to weight loss.)  Indeed, Jussim cites a good follow-up article showing how the social zeitgeist was thirsty for the results of the Pygmalion study, and people were willing to overlook the flaws.

Well, first of all, follow-up efforts almost always show that the original dramatic result is not the whole story, and not everybody gets the same effect.  If your original study is done in, say, the environment of a particularly progressive academic department, where everyone believes that doing particular tricks will work, even if you really do blind the participants to the significance of that essay assignment or putting that question at the end vs. the beginning of the test, a lot of students may just generally be primed to respond to those things.  Put them in a warm, fuzzy environment, and maybe a warm, fuzzy writing assignment is meaningful to them.  This is why replication is so important:  It doesn't matter if you have hundreds of students in your study.  If the control and experimental groups are both taught in an otherwise identical environment by the same person (which they should be) then you may have a large sample for the purpose of asking "Was this effect significant for this experiment?" but you have n=1 for the purpose of asking "Is this intervention robust and useful for the typical instructor?"  That's not a criticism of the study, that's just a reinforcement of the basic scientific principle that replication is everything.

Also, a lot of these studies get discussed in the context of racial and gender inequities, where there are centuries or millenia of cultural baggage attached to both the students and their teachers and the entire environment surrounding them.  I'm sorry, but I simply don't believe that simple little hacks can make giant dents in these problems outside of rarified environments.  I don't care how big your sample of students is, until you sample a wide variety of environments, beyond just instructors who are already hyper-sensitized, I won't believe your claim that the effects of millenia of male dominance in Western culture and centuries of racial inequality in the US can be meaningfully blunted in the classroom by reordering a few questions and assigning a short essay or whatever.  Call me anti-science if you will, but that's my prior.

Mind you, I don't really object to moving the demographic questions to the end of the test, and if you want to assign an essay on values affirmation then knock yourself out.  These interventions are cheap and at the very least harmless, but I am unconvinced that simply knowing and following the recommendations of This One Study can make a huge dent in timeless problems.  It's too good to be true.  Next you'll be telling me that real estate prices never fall.

Speaking of real estate prices, self-fulfilling prophecies are definitely a real phenomenon in social science.  Economists and marketers alike know that with enough hype the price of a good can indeed rise, at least for a while.  (And if you're smart enough, do you care if it later drops?  After all, you already cashed in! You did cash in your chips, right?  Um, right?)  But even hype tends to be expensive and unpredictable.  Not every ad campaign succeeds, but when they do they often cost a lot of money.  An expensive ad campaign is, like, work and stuff, whereas just doing This One Thing is easy.

Next book: Social Perception and Social Reality by Lee Jussim

The next book that I'll be blogging is Social Perception and Social Reality by Lee Jussim.  The thing that's most attractive to me about the book is that it is contrarian, and something of a corrective to the pop psychology that gets the most circulation in educated circles.  Spend any significant time among academics and you'll hear about famous studies of bias (not just about socially and morally significant topics like race and gender, also about prior assumptions on smaller matters), fooling yourself, false perceptions, etc.  Academics collect tidbits about bias in the same way that geeks collect Star Wars paraphernalia.  All of these things are real, all of them need to be kept in mind, and having the humility to recognize that you don't always perceive accurately is an important thing for personal growth.  Moreover, from a scholarly perspective, finding the mismatches between perspective and reality is surely essential to understanding the human mind (one of the fundamental goals of psychological science).

Jussim's point is that focusing solely on misperception and ignoring accurate perception would be like medical researchers who only study disease and never study how a healthy body works.  It isn't good for the science, and it wouldn't be good for society if all we ever heard reported from the medical journals was "There are millions of different ways that you can get sick" and never "Exercise is healthy."

My own annoyance is that the narrative on bias, misperception, etc. is almost a bias of its own.  What gets the most circulation in nerdy, educated circles is umpteen different studies on how wrong you are, followed by a suggested "life hack" or quick fix to correct it.  My prior is that if we are prone to fooling ourselves, and if bias and delusion are everywhere, surely the embrace of quick fixes is itself a delusion.  We probably get a lot of things right, and I suspect that fixing the things that we often get wrong is a bit harder than just doing This One Thing From An Article.

Anyway, enough of my rant.  Let's dive into the book.