Statistics and Standardized Tests

I have posted a paper on the arXiv (a website for physics articles) addressing statistical issues in interpreting standardized tests.  This is a volatile topic, and my argument is long and nuanced, so I don't want to retype the entire thing in a blog post.  The short version is that a couple years ago somebody noticed that there are many successful scientists who did poorly on the GRE.  One possible interpretation is that the GRE has no predictive power for performance in a PhD program.  Another possibility is that performance is predicted by a combination of several variables, and a low score on any one of them might easily be compensated for by a high score on another.  Moreover, admissions processes typically enforce such a condition, so the only people getting in with low GRE scores probably did well by some other measure (e.g. research experience, relevant work experience, lab skills, etc.), while people who did poorly by some other measure must have had something to compensate (e.g. the GRE).  Consequently, when you compare people with high and low GRE scores, you aren't actually holding everything constant.  I use computer simulations to illustrate these points.

I also include a digression on "ought" vs. "is" statements, illustrated with two characters (Helena and Cosima) from Orphan Black, my current favorite science fiction show.  It may be that Helena is less likely to succeed in graduate school than Cosima is, but if Helena had a disadvantaged upbringing (and believe me, she did) then we ought to give her a chance to exceed our expectations.  This is an important point, given the diversity implications of standardized tests.