Latest Article: "Studies Have Shown" Is Not Enough

 My latest opinion piece has been published in Discourse Magazine. It's titled "'Studies Have Shown' Is Not Enough." It's on the problem of people wanting to jump on edufad bandwagons as soon as a few studies supposedly show something.

In other news, the Pope is Catholic

I came across this meta-analysis of studies of the effect of class attendance on college grades.  A meta-analysis is not a single study, but rather a statistical analysis of a large body of studies.  Any one study might find a given result for any number of reasons.  However, by comparing many studies one can get an idea of whether there are any consistent findings, and put outliers in context.  That's what a meta-analysis is.  In short, these authors find that class attendance is an excellent (not to be confused with "perfect") predictor of college grades, even better than test scores and high school GPA.  Also of interest, attendance is a better predictor than personality traits (aka "non-cognitive traits"), throwing cold water on occasional claims that track record matters less than potential as evidenced by non-cognitive traits.

Higher Superstition, first two chapters

The first chapter is an introduction to the general controversy that they're addressing.  I already outlined that in the previous post.  The second chapter is mostly a historical survey of politics and ideas relevant to their concerns.  I will not recap the whole thing.  Instead, I will just highlight a few particularly interesting tidbits from these chapters:

On page 3 they clarify that the "Academic Left", as they use the term, is not merely a group of people who happen to have day jobs in academia and happen to support policies pursued by the more liberal members of the Democratic Party.  Rather it is a collection of people pursuing a particular vision of cultural transformation, the overthrow of old categories and labels.  This is certainly in keeping with the diversity vision of today's STEM reformers; one could say that the critique shifted from scientific knowledge to scientists.

On page 4 they mention scientists having encounters with "literary critics waxing sentatious over the uncertainty principle or Godel's theorem."  It's almost as though they witnessed my freshman year of college!

On page 7 they issue a too-optimistic prophecy concerning education:

"We expect little early change in the teaching and learning of science on the basis of these politicized critiques (although proposals in that direction, including some from people who should know better, pop up now with regularity)."

Indeed.  Transformation of STEM education is now front-and-center in the agenda of technocrats, and they speak endlessly of identity issues when they promote their agenda.  It's rarely as simple as "This group learns this way, that group learns that way"; they are mostly too smart to say something that explicitly bigoted and factually incorrect.  But there is much "Your students are different from you, demographics are changing, science is more diverse, we must teach differently."  Certain links are never made explicit (since stating such things explicitly would be both unpopular and factually wrong, a rare instance of a grossly false idea being unpopular).  But diversity and identity are certainly close to everyone's minds, and are invoked to push people in warmer-and-fuzzier directions.

On page 19, they make the interesting observation that the Enlightenment is the ancestor both of free market ideas and Communist ones.  Though Communism obviously failed very badly at liberating or enlightening people, it came from a rejection of traditional authority, including feudalism.  Market capitalism has similar elements in common.  I note this not to make any sort of special plea for communism, but simply to note that the Enlightenment was prodigious with offspring.

On page 20 they attribute the rise of Romanticism to, among other things, the horrific failures of the French Revolution.  Romanticism, in my primitive understanding (supplemented with what little Gross and Levitt have to say), is an idea of things beyond reason mattering in human affairs, or even superseding reason in praiseworthy ways.  Truthfully, I'm not wholly unsympathetic, having remarked many times on the impossibility of deriving "ought" from "is."  Liberal technocrats always want to elide that point.  I don't know that it's a terribly Romantic point, since it doesn't celebrate any particular alternative to empiricism, nor does it even reject reason.  One could draw some "ought" postulates from non-empirical sources but then use those postulates, in combination with empirical evidence, to reason logically, e.g. "When spending money as the fiduciary of another person's interests I ought to spend thriftily, and Acme makes the cheapest widgets, therefore I will buy Acme widgets in this role."  So one can be suspicious of those who are excessive in their pretenses to logic and empiricism (because they never make their arbitrary "ought" postulates transparent) but still support logic and empiricism.

So I guess that one of my favorite topics here is hardly "Romantic."

On page 23 they note that while many illiberal ideas sprang up under the guise of science (e.g. social Darwinism, racial ideologies), the scientific notions that have stood the tests of time and experimentation have fortuitously been mostly friendly to egalitarian and emancipatory projects.

On page 24 they note that science has remained congenial to the project of scholarship as a liberating force because science pushes back on superstition, which I take to mean religion.  However, the study of humanities, which prepares one to interrogate texts in context, seems a more powerful force for inducing skepticism of the Bible than science class.  Miracles aren't the main point of the Bible.  Moral codes binding communities and bringing meaning to life are the main point of the Bible.  The study of humanities will prompt much more searching questions about that than anything a physics professor might say.

On page 25 they say that American academic leftists rebeled against science to signal rejection of the Enlightenment and its failure to make further progress on racism, poverty, etc.  They judge that it comes from restlessness and discontent with modern society.  I find it interesting that modern academic leftists share the same restlessness, but approach it with a renewed faith in technocracy.  "Studies have shown" that we are biased, irrational, etc., so if people can be persuaded of these points and induced to accept remedies then they will become more rational and (presumably) more supportive of projects for the liberation and betterment of humanity.

Or at least that seems to be the theory.

Finally, on page 33, we get to something that has most definitely carried forward:  The idea that the oppressed have unique epistemological standing, access to more knowledge than the advantaged.  However, whereas that was used to argue for rejecting the science espoused by the dominant power structures back in the "Science Wars", nowadays we have people seeking to do social science about microaggressions, biases clouding the minds of the privileged, etc.  Whether or not one has a favorable judgment of the soundness of such research, it is indisputably an attempt to use social science to prove the epistemological limitations of the privileged and epistemological standing of the under-privileged.  There is no romanticism here, it is thoroughly technocratic and accepting of science.  The scientific validity of these research programs can only be evaluated through empirical work, but from a cultural perspective it is clear that it is not an anti-science program.

Chapters 1-2 of _The Counter-Revolution in Science_

Hayek spends much of the first chapter arguing that bringing science to social questions has done little to bring insight.  I already gave my general critique of Hayek as an ideologue in the previous post; here I want to focus on the nature of social science, and what I think Hayek gets right and wrong.

The biggest difference between natural science and social science is that natural science is more about discovery while social science is more about testing.  Yes, natural and social scientists both make discoveries and they both test hypotheses, but human experience provides us plenty of ideas about social issues, and in some sense most social ideas are right within a certain domain of applicability.  You don't need a double-blinded, properly-controlled, statistically rigorous study to know that standardized tests cannot tell you the entirety of a person's abilities, that humans have biases, that people often manage to rise above their prejudices, that people also fall short of their ideals, and that small changes in policy can have large or small effects in difference circumstances.  As basic ideas, I think that casual observation of the world is sufficient support for all of them.

What social science can do at its best is answer questions like "When? Under what circumstances?"  Social science can take us from "People have many biases, but they don't always control our behavior" to "Unconscious biases measured by this instrument have demonstrable effects in these settings but not those other settings."  Social science can take us from "Sometimes policy changes matter, sometimes they don't" to "Raising the minimum wage by $0.25/hour from a baseline of $10/hour will have little effect on unemployment in a city with a high cost of living, but raising it by $2/hour in a locale with a low cost of living could have measurable effects on unemployment."  Social science can take us from "Standardized tests don't tell you everything" to "Due to the narrow range of scores, the quantitative section of the GRE has little predictive power for PhD students in such-and-such field, but the relevant subject test explains X% of the variance in outcomes as measured by the following instrument."

So social science does not necessarily tell us much that we don't know on SOME level, but it tells us much about when our intuitions are valid and what their limitations are.  Hayek, being interested in grand social questions and large "Ought" issues missed that, on some level.  Ironically, though, Hayek's grandest hypothesis was falsifiable by casual observations (Western Europe is not a hellhole) without fancy statistical methods.  He railed against a harder enemy than he was actually taken down by.

Natural science operates in domains where we have much less intuition, and much less prior experience from which to develop ideas.  By the time we know enough to even frame a precise hypothesis about certain topics, we're often on our way to testing it.  That's not to say that wrong ideas never take off, and that there's never a contest of hypotheses, but we're definitely operating in realms where we have (comparatively) fewer preconceptions to cling to.

Ironically, the people who most abuse social science are probably closest in practice to Hayek's approach to social questions.  They run with a few findings that flatter their preconceptions and ignore contrary findings.  They love "ought" more than "is."  The only difference is that they flatter themselves with a pretense of empiricism, whereas he is openly critical of the excesses of empiricism.

Next reading: Hayek's _The Counter-Revolution of Science_

I couldn't bring myself to read Dewey.  He's just too boring.  Now I understand progressive educators a bit better:  Reading Dewey would be enough to make anyone hate reading.  I'll try again, after I've had a bit more summer vacation to recover, but for now I need a break.

Instead I'm reading The Counter-Revolution of Science by F. A. Hayek.  I picked it up on a whim, while browsing the science section of a used bookstore.  Honestly, two chapters in, I think I'm going to be pretty critical of this book, but since it's a book that says a lot of things I'm inclined to agree with I suppose that my negative commentary on it will be a useful corrective.

This book is quite critical of social science as science, arguing that it's a mistake to approach social questions with the methods of natural science.  On the surface, I may be primed to agree, for a number of reasons.  First, as I've noted many times, the technocrats of the modern managerial classes like to derive their "ought" statements from news reports in the "According to a recent study..." genre.  They want the world to be simple. They want neat theories that avoid the complexities of human nature, and they want policies that people will comply with rather than either game it or push back.   Well, people aren't like that, but I work in a system where many try to pretend that people are or ought to be like that. So I ought to like a Hayekian critique.  Also, Hayek was very much a libertarian, and I lean libertarian in my non-academic politics, and to a certain extent that bleeds over into my academic politics. (Though only to an extent; in the end I still like the authority of the sage on the stage, and a true libertarian has to distrust authority.  In my defense, I like the authority of the sage who analyzes complex questions from many angles, not the authority of the well-funded technocrat who holds forth on Best Practices.)

In order to understand my ambivalence about Hayek's writings here, we first have to understand a bit about where Hayek was coming from.  Hayek wrote much on economics and politics, and won a Nobel Prize for his economics work.  However, while he did economics work that earned praise from the wider profession (I'm not qualified to judge it myself, I can only surmise that it must have been respected if he won a Nobel), he also produced a lot of political and ideological commentary that was much more controversial.  He argued quite forcefully that the modern Western administrative state will lead to tyranny as surely as the 20th century Marxist states did.  Indeed, among non-academic audiences The Road To Serfdom is arguably his most popular work, and in there he argues that the complexity of managing an economy will inevitably require a more and more comprehensive administrative state that ultimately takes away human freedom.

As much as I disdain managerial liberals, it is an empirical fact that Western Europe didn't turn into Eastern Europe.  That's just a fact.  There are numerous reasons for that, reasons that I won't pick apart in gory detail here, but surely we must include among those reasons the fact that managerial liberals tend to balance their well-meaning obsessions with a bit of selfish laziness.  They might want to make the world into a particular image, but they also want to see good things happen while carving out a comfy and self-flattering niche for themselves.  One easy way to do that is to let market-driven processes do their part in society (and thereby do some good), while also administering some programs that can be made to look like they are doing some good.  It's a win-win for everyone.  Western managerial liberals don't need the total domination that Russian rulers (and their viceroys) need.  Culture matters, and Western managerial liberals have ways of making themselves feel like they are doing good without the total domination that Eastern Europeans require.  So we simply haven't seen Western societies with mixed economies turn into Eastern European tyrannies.  I certainly won't defend everything that Western social engineers have done, but Hayek's prophecies failed badly.

Also, managerial liberals are ill-suited for true tyranny, because true tyranny requires that somebody be able to say "You!  Against the wall!  Now!"  Western liberals would  never do that.  They'd say "You!  Report to this place with the following forms filled out!  Within 60 to 90 days or as summoned!"  There have been plenty of abuses by Western authorities, plenty of individual-scale human rights violations that are as appalling as anything out of the East, but it isn't the managerial liberals who want to scale it up.  When they see those things they feel terrible and say "This calls for an immediate investigation and evaluation process, followed by a top-to-bottom review and retrospective analysis for the formulation of reform guidelines!"  That bureaucratic process might not do anybody any good, but it also won't entrench the abuses that they're responding to.  Mostly it will entrench ineffectual responses to abuses, while sending a message that abuses need to be kept on a level of "plausible deniability."

There's a ton that's wrong with it, but it doesn't give you Eastern Europe.  It gives you a modern US city, where the police can get away with a lot but they also can't scale it up.  What scales it up is political cover for the police, and that (mostly) comes from a different segment of the political spectrum than managerial liberalism.

Mind you, the tools created by managerial liberals can and will be abused by illiberal authorities of other mindsets, but the point is that the modern mixed economy isn't enough to give you the sort of illiberal regime that Hayek feared.  You need other elements.  Venezuela is as close as we've seen to a left-liberal state sliding into tyranny gradually rather than overnight, and Venezuela had elements beyond left-liberal administrative types overseeing a mixed economy.  It had an openly Marxist and populist demagogue.

But I'm getting far afield from The Counter-Revolution of Science.  I think I'll just stop this post here, with my critique of Hayek's most famous prophecy, and take up the current reading in the next post.

The two cultures on display: Gender studies

In the past few weeks two articles from gender studies have gotten some attention.  The first was not actually from gender studies as such, but was from two people attempting a hoax against the field of gender studies.  Whatever you think of the hoax and what it does or doesn't teach us, it certainly put gender studies in the spotlight.  The second one, "Assembled Bodies: Reconfiguring Quantum Identities" got less attention outside of physics, but is arguably more significant because it was offered sincerely.

I tried reading the second one.  I gather that it is using quantum mechanics as a metaphor for ideas in feminist theory, but I lack the background to put it in context.  It is a very, very dense read, impenetrable to an outsider.  That is not necessarily an unforgivable sin for a scholarly work; much of scientific writing is impenetrable to the non-expert.  However, C.P. Snow offered a partial defense of the impenetrable nature of scientific writing, arguing that science is the more cumulative of our "Two Cultures", and hence it is impossible to make scientific progress (or understand such progress) without understanding a body of existing knowledge.  On the other hand, he also noted that a scientist need not have a detailed knowledge of literature from the distant past, as long as the scientist knows which specific ideas from the past remain valid after replication efforts from the past.  In humanities, conversely, engagement with the primary text is everything, but the nature of innovation is such that new ideas and new works of art and literature can be generated without reference to prior work.  Engagement with a primary text written yesterday can be as intellectually satisfying as engagement with a primary text written in 4,000 years ago.  In light of that, I've argued that social science sits somewhere between humanities and natural science, being akin to natural science in many of its goals and standards, but engaging with human issues like humanities and re-fighting many battles in each generation (not always for ill).

So, where does gender studies sit?

An exhaustive answer to that question would require far more engagement with the field than I can lay claim to, but I can examine the article on "Quantum Identities" and ask where it sits in terms of requiring a reader with cumulative knowledge.  In that respect, gender studies sits quite far along the cumulative end of the spectrum.

I then decided to do comparisons with other fields.  I first picked two very recent articles from two respected psychology journals, Journal of Experimental Social Psychology (whose editor seems to be a reasonably productive academic) and Psychological Science (which also seems to have a pretty respectable editorial board).  I can't claim to understand the full context, import, or meaning of the work in either article as well as a psychologist would, but I can certainly get an idea of what they're  trying to do.  They are much easier on the reader than the gender studies article.  I can come away with some idea of what I get and what I'm unsure about and questions beyond "Um, wtf are they saying?"

Of course, I'm a natural scientist, which means I understand statistics and am accustomed to reading experimental articles, and I have done a fair amount of reading of educational literature, which has some overlap with the psychological literature.  Perhaps "harder on this reader than an experimental psychology article" is an unfair standard for me to use in evaluating gender studies.  With that in mind, I googled for "top journals in history" and all of the links sent me to American Historical Review.  I picked an article from a little more than a year ago (since my institution does not have access to the most recent year) with a title that is not particularly interesting to me.   Again, I can read it.  It is accessible.

Gender studies thus seems to make more demands on the reader than many humanities and social science fields.  The exceptions that come most immediately to mind are philosophy (generally considered a humanities field, though formal logic overlaps strongly with mathematics and computer science) and economics (definitely a social science field, but strongly influenced by mathematics).  Given the leaning of gender studies literature towards ideas and experience rather than statistics (that is not meant as a criticism), I think we'd have to situate it in the humanities, while acknowledging areas of interest shared with social science.

It's worth asking is what purpose dense jargon serves.  In the natural sciences it enables us to speak with precision, both so that we can make claims that are testable in quantitative experiments and so that we can reference specific elements from a large body of cumulative knowledge.  In philosophy, my understanding is that the purpose of dense jargon is precision in the drawing of fine distinctions.  It doesn't lend itself to clarity of exposition for the non-expert, but it does lend itself to clarity of distinctions for insiders.

In light of these considerations, my conclusion is that gender studies presents itself in a manner most similar to philosophy.  If other academics wish to critique the field, it would be most reasonable to do so in comparison to philosophy.

Back to book blogging: The Philosophy of John Dewey

I'm reading a collection of essays by John Dewey.  About 2/3 of the volume is on philosophy, and largely out of my depth, so I've just skimmed those parts.  But now I'm at a 1931 essay titled "Science and Society."  The editor of the volume indicates that this essay deviates somewhat from Dewey's usual views on the nature of knowledge--he treats science as impersonal, objective knowledge, whereas usually he stressed the contextual nature of knowledge and learning.  That is actually unsurprising to me, because the second half of this essay reads as technocratic, and technocrats have a soft spot for the opportunities offered by objective knowledge.  Of course, he spends the first half of the essay urging would-be technocrats to be less smug (and good for that!) but Dewey was neither the first nor last technocrat to think that if they just rethought the project they could totes make it work.

He starts by noting that the societal changes wrought by technology had not actually changed human nature:

In its effect upon men's external habits, dominant interests, the conditions under which they work and associate, whether in the family, the factory, the state, or internationally, science is by far the most potent social factor in the modern world. It operates, however, through its undesigned effects rather than as a transforming influence of men's thoughts and purposes. This contrast between outer and inner operation is the great contradiction in our lives. Habits of thought and desire remain in substance what they were before the rise of science, while the conditions under which they take effect have been radically altered by science.

Maybe that's because technology isn't actually new to human beings.  Indeed, we control the planet precisely because we are the best tool-users on the planet.  So a technology-reliant existence isn't a new existence for humans, it's actually all that humans have known for at least a few tens of thousands of years.  We fought off predators by using spears rather than superior muscles and speed, we survived the ice age with fire and blankets rather than fur and insulating fat, and we acquired food with tools rather than claws.  And while I freely concede that the pace of technological change today is greater than in ancient times, when I read the classics I do not notice any substantial differences in human nature.  The Bible, the Epic of Gilgamesh, the Greek philosophers, the scribes of ancient Egypt, they all show people who are remarkably similar to today's people.  The educated classes of today feel compelled to visibly celebrate diversity, but similarity is a much bigger fact of human existence.

Having just conceded that humans of 1931 are remarkably similar to those of the past, Dewey goes on to note that information technology has not transformed society:

No sooner do we begin to understand the meaning of one such change than another comes and displaces the former. Our minds are dulled by the sudden and repeated impacts. Externally, science through its applications is manufacturing the conditions of our institutions at such a speed that we are too bewildered to know what sort of civilization is in process of making.

Because of this confusion, we cannot even draw up a ledger account of social gains and losses due to the operation of science. But at least we know that the earlier optimism which thought that the advance of natural science was to dispel superstition, ignorance, and oppression, by placing reason on the throne, was unjustified. Some superstitions have given way, but the mechanical devices due to science have made it possible to spread new kinds of error and delusion among a larger multitude.

... 

The airplane binds men at a distance in closer bonds of intercourse and understanding, or it rains missiles of death upon hapless populations.

Honestly, this reads like an account of the hopes and subsequent disappointments among those who envisioned the internet as transforming society and freeing people.  Yes, the internet has enabled great things, and spread information to be used for good, but it has also been used for dull entertainment and malignant propaganda.  Let us remember that Wikileaks was originally founded as a site for undermining totalitarian regimes via the cleansing light of truth, but in 2016 it was used to advance the political machinations of a Russian dictator and his billionaire puppet.

Also, if we were to change the dated term "mechanical devices due to science" to "information technologies enabled by science" we would have something that any reader in 2017 would think is a critique of the internet.  It's been noted before that the rise of cinema was accompanied by predictions of the demise of universities (classes would be delivered on film reels), as was the rise of television.  It seems that every new information technology elicits the exact same cycle of euphoria and frustration.  All of this has happened before and will happen again.

Beyond information technology, Dewey's basic point is not so different from Kentaro Toyama's Law of Amplification: Technology doesn't so much level or transform human society as allow us to do what we were doing before only moreso.  The advantaged can derive more advantages from technology (though they can also fall if they don't keep up with competitors), the disadvantaged can be left behind in the new economy as they were in the old (though some can also discover a new path forward), people who seek to do good with technology can do more good with new technology, and people who seek to do ill with technology can do the same.

Having noted the ways that science has enabled both good and evil, merely amplifying on human nature rather than transforming it, Dewey goes on to note a point that has my most enthusiastic agreement (though not his):

Shall we try to improve the hearts of men regard without to the new methods which science puts at our disposal? There are those, men in position in church and state, who urge this course. They trust to a transforming influence of a morals and religion which have not been affected by science to change human desire and purpose so that they will employ science and machine technology for beneficent social ends. The recent Encyclical of the Pope is a classic document in expression of a point of view which would rely wholly upon inner regeneration to protect society from the injurious uses to which science may be put. Quite apart from any ecclesiastical connection, there are many "intellectuals" who appeal to inner "spiritual" concepts, totally divorced from scientific intelligence, to effect the needed work.

Indeed, my own belief is that character matters in every age.  Dewey will go on to urge that we apply the methods of social science to solve the problems of human society.  However, the 20th century saw the formulation of theorems (e.g. Holmstrom's Theorem, Sen's Theorem) that demonstrated the limits of what can be accomplished via institutional designs.  There's no way to totally automate decision-making.  And central planning failed.  In my own job I am seeing more and more evidence that "best practices" only take you so far, and in the end you need to hire and retain people with character and values that drive them to do the job within a reasonable incentive structure; process only gets you so far.

Anyway, Dewey does name two areas where the application of science to social problems has been beneficial:  Insurance (where we use statistics to price and mitigate risk, so that people can take the risks that inevitably accompany attempts at great things) and the germ theory of disease/hygiene.  However, the first example involves something that people can choose to buy, and that can be priced without subjective value judgments, and the second ties quite closely to facts of the natural world.  Neither relies too much on the steering of behavior.

He goes on to consider education as an arena for the application of science to social problems.  Despite the exasperation I often display with educational fads, I actually have a lot of sympathy for the idea of applying science to educational problems.  However, I believe that the passions aroused by the cultural, ethical, and political dimensions of education often leak into educational studies, and people believe that their subjective choices about which question to ask shape the answers that they get and (more importantly) the ways in which those answers are applied.  For instance, it is only very recently that people looked at whether "reformed" physics classes were as good as "traditional" classes for improving student understanding of the topics and skills that get greater emphasis in "traditional" classes (since "reformed" classes often emphasis different aspects of the subject).  For decades, many researchers were emphasizing "conceptual understanding" over quantitative problem-solving, for a number of reasons.  I have argued before that this is in part because of the cultural baggage that gets attached to mathematical reasoning (at least in the US), but most of the people who study physics education in the US are natural scientists who have not done a lot of reading on American cultural history, so they are blind to their own assumptions.  It is interesting that the lead author in the study I linked here is a psychologist, not a physicist, and is probably better-trained at unpacking cultural baggage and accompanying assumptions than most physicists would be.

(Then again, Dewey was trained as a philosopher, a field that specializes in skewering assumptions since at least the days of Socrates, yet he also fell into the technocrats' trap.)

So, as far as my own views go, I'm a big fan of social science as science, but skeptical of social engineering.  It's the difference between pure and applied science.  As far as Dewey, I find it fascinating that he could start an essay by noting the ways in which the technological environment experienced by humans has failed to change thought and behavior, but then express great confidence that science can be used to shape social environments in such a way as to steer human behavior.  In that sense the essay falls flat.  On the other hand, I give him full marks for understanding the distinction between changes in the technological environment and changes in human nature.  He even managed to note ways in which the information technology of his era failed to live up to hype about transforming society.  Too bad he didn't carry that humility forward in thinking about social engineering.

The hand-wringer test

I have a casual interest in linguistics--I enjoy browsing an Indo-European dictionary and occasionally tweet about what I learn.  I thus came across this discussion of whether linguistics qualifies as a STEM field.  If the question is whether linguists approach questions in their field with a scientific mindset, or draw upon approaches akin to those in the natural sciences, the answer is an unambiguous yes.  Moreover, if that is enough to qualify a field as STEM then most/all social science fields are either STEM fields or at least have sub-fields that count as STEM.  Linguistics may have elements that are more akin to humanities (qualitative and descriptive analysis of texts and behavior), but it also has social science and even natural science (e.g. neuroscience) components.  It is, a minimum, a good fit for the stated description of STEAM, if not STEM.

However, I mostly approach the definitions of words from a descriptivist perspective, not a prescriptivist perspective.  STEM sits on a pedestal, and the descriptivist's question is not whether linguistics belongs on the pedestal but whether the gatekeepers will recognize its right to stand on the pedestal.  Linguistics may fit the gatekeepers' stated criteria for inclusion in STEM, but people are rarely honest about the criteria that they actually employ to determine admission to a pedestal.  You have to watch their actions, not just listen to their words.  And from my experience in a university where the local culture perceives its moral legitimacy as deriving from our work on bringing students into STEM, social science is only STEM when we're trying to be collegial with social science faculty, or when a social scientist is studying issues of STEM equity and the STEM workforce.

As I said in my post about STEAM, the way to figure out if a field is STEM is to do a thought experiment involving students changing majors.  Suppose that two twins, Alice and Bob, start off as electrical engineering majors.  Alice then changes majors to physics, while Bob changes to a social science field.  Which decision would elicit more hand-wringing among the people who worry about the STEM Pipeline?

A tempting rejoinder is that we shouldn't care about the hand-wringers, we should just look at the intellectual rigor of the field, and we'd have to agree that there are plenty of things in linguistics that qualify as science.  I don't deny that, but I would note that (1) there are plenty of people whose work is definitely not science but is nonetheless intellectually rigorous (e.g. good scholars in the humanities) so why is intellectual rigor a sufficient criterion for inclusion in STEM? and (2) if we go down that road then most departments on a university campus would have STEM components (e.g. there's plenty of chemistry in art, plenty of acoustic science and technology in music, plenty of behavioral science in marketing, etc.) and STEM becomes so broad that it papers over the distinctions that make for intellectual diversity.  If STEM is the arbiter of good then everything is STEM and everything is good and everything belongs on campus, but we already agreed that the art and literature and business faculty should work on the same campus as the physicists and biologists and mathematicians, so what was the point of this label again?  Oh, right, STEM is on the pedestal.  Well, maybe instead of putting everything onto the pedestal we should point out how silly the pedestal is, and how ultimately destructive it is to the notion of intellectual diversity.

Anyway, I have a casual hobbyist interest in linguistics, and I certainly respect the rigor and value of the field, but I think that instead of including everything in STEM we should question why inclusion in STEM is considered so valuable.

Social vs. Natural Science: The cumulative difference

I've been reading other things, things that I simply haven't had the inclination to blog, but whilst reading about some controversies in social science I realized that one of the things that Snow used to demarcate the line between science and humanities can also demarcate a line between social and natural science:  The cumulative nature of knowledge.

I should state at the outset that I do, in fact, acknowledge that social scientists have cumulative knowledge, that they do, over time, develop and refine a body of generalizable knowledge about human beings.  That's not to say that every theory or every finding turns out to have wide applicability (or even replicability), but the process of social science can indeed select, filter, and refine the body of knowledge, just as natural scientists do.

In the sense that social science can produce a refined, tested body of cumulative knowledge, social science is akin to natural science, and distinct from arts and humanities in the sense that Snow articulated.

On the other hand, because social scientists study humans they have to confront human prejudices and cultural inertia, and thus they have to re-fight certain battles in every generation. Topics like, say, gender differences, get re-fought in each generation, and the cumulative weight of data will not dampen the appeal of gender essentialism as a way of legitimizing inequality. (One could easily come up with other perennial battles; I simply picked that one for the sake of easy illustration.) In that sense, there is a human limit on the extent to which social sciences can, in practice, be as cumulative as natural science, or at least a limit to the rate of accumulation, because of the need to re-fight battles in each generation.  There is much less of that in natural science.  OK, evolutionary biologists have to re-fight battles in the public sphere, but not within academia.  And biologists studying development, cognition, and gender will have to join social scientists in re-fighting gender battles in each generation, but that reflects their position at the intersection of natural and social science.  Closer to (my) home, we physicists have to help each cohort of freshmen overcome Aristotelian intuitions about motion, but that battle really only takes a semester.  By the end of the semester they know that they ought to be Newtonians; they might still have Aristotelian impulses, but they know that they're supposed to check those impulses.

So, if we take the cumulative nature of natural science as a line of demarcation between Snow's "Two Cultures" then I think a tri-partite division of liberal arts is appropriate:  The humanists can generate new works of art and inspiration and analysis in each generation of a changing world, without the constraints of the past, the natural scientists develop a cumulative understanding of nature, and the social scientists seek to build a cumulative understanding of people but do so while re-fighting perennial battles of human culture and prejudice.

The false ecumenicalism of STEAM

There's an editorial that I half-agree with in the latest issue of Scientific American, and like many things that get at half of an issue it's almost as dangerous as simply ignoring the issue.  The editors make the completely valid point that our society need more than just people who are trained in natural science and engineering, that we also need people who study the humanities and social sciences.  I agree completely, as you might gather from the fact that most of the books I've blogged about here are written by people who aren't trained in math, natural science, or engineering, and are primarily about non-technical issues.  I denounce the short-sightedness of politicians who want to focus our higher education system primarily on STEM.  Partly because (as I've said before) putting STEM on a pedestal is actually quite bad for STEM* and partly because the rest of society (i.e. the world beyond the walls of the science buildings on campus) also needs more than STEM.

The editors of SciAm get that society needs more than STEM, but they can't help but couch this need in terms of the economic competitiveness of high-tech industries:

The need to teach both music theory and string theory is a necessity for the U.S. economy to continue as the preeminent leader in technological innovation. The unparalleled dynamism of Silicon Valley and Hollywood requires intimate ties that unite what scientist and novelist C. P. Snow called the “two cultures” of the arts and sciences.

Steve Jobs, who reigned for decades as a tech hero, was neither a coder nor a hardware engineer. He stood out among the tech elite because he brought an artistic sensibility to the redesign of clunky mobile phones and desktop computers. Jobs once declared: “It's in Apple's DNA that technology alone is not enough—that it's technology married with liberal arts, married with the humanities, that yields us the result that makes our hearts sing.” 

A seeming link between innovation and the liberal arts now intrigues countries where broad-based education is less prevalent. In most of the world, university curricula still emphasize learning skills oriented toward a specific profession or trade. The ebullience of the U.S. economy, which boasted in 2014 the highest percentage of high-tech outfits among all its public companies—has spurred countries such as Singapore to create schools fashioned after the U.S. liberal arts model.  

My problem with this is that it still has STEM on the pedestal:  There's nothing about the value of the humanities and social sciences** in their own right, only their significance for high-tech industries. Personally, my reason for reading and blogging about so many books on history, social issues, and so forth is that I want to understand people, not that people often work in STEM.  Teaching is an activity in which one can benefit from knowledge of history, psychology, etc. irrespective of what discipline one is in.  The value of humanities and social science for my work can thus be measured by how much my work involves people, not by how much humanities and social science help STEM.  More importantly, the value of humanities and social science should be measured according to more than just the extent to which they matter for my work (or anyone's "work", as opposed to the rest of life).

Now, in academia I can sense a backlash starting to stir against the current STEM craze.  However concerned I might be about how this pedestal will affect STEM, I can hardly expect sympathy from people who aren't in STEM.  The pedestal might have its downsides, but not being on the pedestal has even more downsides.  Consequently, some administrative types now talk about "STEAM", which stands for "Science, Technology, Engineering, Arts, and Mathematics."  Nobody is entirely sure whether the "Arts" only include visual and performing arts, or also the liberal arts*** more broadly.  I'm less interested in definitions listed in some particular document and more interested in usage. When I hear administrators talking about the "A" in "STEAM" I've heard several different points, some of which could be roughly summed up as:

1. "Designing technology requires attention to aesthetics."  This is a completely valid point that has been known for millenia, hence the field known as "architecture."
2. "Designing technology requires attention to human factors more broadly."  Again, a completely valid point.
3. "Scientists can learn a lot about their field from historians and philosophers of science and technology."  This is a point with which I also agree completely, and it informs much of my blogging here.  Indeed, I strongly encourage students to make connections between their major and their general education by taking classes on the history of science.
4. "Solving workforce issues in the sciences requires the involvement of people who have considered the cultural factors and problems of inequality that affect the pipeline of talent."  While I might dissent from some of the most common narratives offered in regard to these topics, I completely agree that these are important areas for inquiry.  I would like to see more and broader inquiry on these issues.

However, do you notice that in each case the "Arts" (however construed) are examined only in regard to how they interact with STEM?  That's fine if the specific topic on the table is whether to include some non-STEM people in a STEM-focused endeavor, but not if the topic on the table is the purpose and future direction of a comprehensive university.  The STEAM buzzword could probably be invoked to justify hiring an art historian who emphasizes architecture, an English professor who is an expert on teaching technical writing, or an ethnic studies professor who studies equity issues in STEM.  However, I'm not sure that it could be invoked to justify hiring a historian who studies the cultural history of China, an English professor who's interested in 18th century American poetry, or an ethnic studies scholar who's interested in representations of ethnicity in cinema.  These people might fit under the Arts (at least in the sense of Liberal Arts) but I doubt that they would fit under any but the most bland (and thus pointless) definitions of STEAM.  STEAM is false ecumenicalism, a way of offering University Strategic Initiatives that are nominally inclusive of disciplines beyond STEM while still measuring those disciplines by the yardstick of STEM.  As long as the paramount yardstick of academic inquiry and teaching is relevance to STEM the Academy is going to suffer.
____________

*In a society that is democratic in the way described by de Tocqueville, those academic fields that are deemed most important will have an obligation to take in the huddled masses, however unprepared and untalented they may be.  This may provide some benefit for members of the masses as individuals, at least initially, but it tends to impede the pursuit of excellence, and eventually the phenomenon of credential inflation will rear its head, to the detriment of all, but disproportionately hurting the least privileged.

**Whether or not the social sciences are included in STEM depends on whether the "S" in STEM is implicitly "NS" (Natural Science).  I personally exclude the social sciences from STEM, not because I disrespect the intellectual rigor of those fields but because I eschew linguistic prescriptivism in favor of looking at how people actually use words.  To wit, consider the following thought experiment:  We have two students, both of whom start off majoring in chemistry.  One of them then switches majors to physics, while the other switches majors to economics.  Would a typical "STEM crisis" hand-wringer react to both with equal nonchalance, saying "They're still in STEM", or would they evince more concern over the new economics major than the new physics major?

***We could also ask which disciplines get counted as "liberal arts", or the extent to which the liberal arts overlap STEM, but let's set that aside for now.

Blogging the countervailing currents

I'm going to blog two social science articles from a high-profile journal.  I should state up-front that such articles often contain results that are hard to replicate, so take them with a grain of salt.  However, these two articles deviate from the current zeitgeist, and hence might have faced slightly more scrutiny in peer review than other articles.  That doesn't guarantee that the results will stand up to replication efforts, but it does make them worth examining.

First, this is a nice study showing that placebo effects can be substantial in "cognitive training" activities.  It's not my area of expertise, but they clearly put a lot of effort into constructing good control and placebo groups, more care than I see in a lot of educational studies.

Second, here's a large-scale study from Denmark, in which the number of instructional hours devoted to reading was increased in thousands of elementary school classrooms, but in half of the classrooms the teachers were told to do whatever they wanted while in the other half they were given a very strict reading curriculum to adhere to in those extra hours.  The teachers with flexibility had statistically significant improvements in their classrooms.  The teachers with the prescribed curriculum had gains that were larger than zero but not as large as those of the teachers given flexibility.  The interpretation is slightly tricky because the error bars on the second gain overlap both zero and the teachers with flexibility.  Still, what we can say confidently is that there's no evidence that giving teachers some amount of flexibility is bad in this context.  The authors interpret it as evidence that teachers respond with instruction tailored to the needs of their particular class.  Not a shining moment for technocracy.

So much for the latest panacea

In a recent interview, psychologist Carol Dweck reminds everyone that "growth mindset", while a valid and important concept, is not the be-all-and-end-all of success, and is hardly a panacea.  I mention this because I've heard quite a few progressive-minded people tell me that "growth mindset" is what REALLY matters for success, more than anything else.  I'm glad that she's adding some nuance.  It seems to be the destiny of every Great Big Educational Theory to go through a cycle of discovery, promotion, hype that exceeds the original intent of the founder, and an eventual disclaimer by the founder.  This pattern goes back at least as far as Dewey, according to Hofstadter, and is probably even older than that.

All of this has happened before and will happen again.

"Ought" can stand irrespective of "Is"

One will often hear it asserted that diversity improves team performance and even corporate performance.  Surely there are situations where this is true--an ad campaign designed by a team of white guys from the Midwest would probably not do terribly well in New York or Los Angeles--but it is often presented as a universal truism.  Well, a recent meta-analysis argues that diversity probably does not, as a general rule, improve team performance, though it may very well be advantageous for certain types of tasks.

Some readers* may feel threatened by this finding, or may be tempted to argue with it, but there is no moral content to that finding. The statement "People ought to be inclusive in their recruitment and hiring practices" can be defended on simple moral grounds, irrespective of the truth value of the statement "A diverse team is more productive than a homogeneous team." An "Ought" statement cannot be derived from an "Is" statement.  I am quite comfortable believing that fairness and inclusion are morally defensible irrespective of whether they are, According To A Recent Study, profitable.  Morality isn't always profitable, and doesn't need to be profitable.  If it were profitable we wouldn't call it "morality", we'd call it "self-interest."  A stable society will certainly try to align morality and self-interest as much as possible, but they needn't be 100% in alignment.

The fact that this type of social science might discomfit people is part of the modern zeitgeist.  On the right, "morality" seems to be about judging other people's sex lives; in all other aspects of life the right believes that the only morality that matters is the morality (such as it is) of the market.  Consequently, the right needs to be sold on the market value of diversity.  On the left, I think that the belief in some concept of inclusion and diversity is sincere, but they aren't really comfortable with moral language.  The Boomers rebelled against the morality of their elders, the educated members of Generation X (my generation) were taught to embrace subjectivity, and the Millenials seem to be afraid of having their values challenged, so everyone is ill-equipped to deal with "is/ought" distinctions from a left-leaning perspective.  The Boomers would feel stodgy saying "ought", we Xers would feel anti-intellectual saying "ought", and the Millenials would be fine with "ought" but would need to see a trauma counselor if somebody disagreed with them.  So we're all more comfortable if social science simply tells us that our value preferences are also perfectly congruent with everyone's self-interest, because then there's no need to debate "ought" statements.

If, however, we recognize that there's no "is" statement that can refute or prove an "ought" statement, and if we are comfortable with the divide between morality and self-interest, then a meta-analysis questioning the direct material benefits of diversity needn't threaten our value system.  But this would require us to have confidence in our value system, rather than to be timid about it.

EDIT:  I just realized that a few months ago I had blogged about a related point made by de Tocqueville.  All of this has happened before and will happen again.

*Here I generously assume that I have readers.

The Journals of Irreproducibly Irreproducible Results

A while back I blogged about a study of reproducibility in experimental psychology.  A critique of the methodology has just been published.  I haven't had a chance to chew on it and form a judgment, but since I blogged the original study I should blog the critique.

Quick tidbits

The Bureau of Labor Statistics has found that there are shortages of STEM workers in some fields and at some levels and not at others.  It's almost as if skilled labor were a differentiated good being sold to heterogeneous buyers.  If only there were a social science discipline that could handle this concept.

Meanwhile, a new article shows that the ancient Babylonians were working out the beginnings of calculus.  Now if we could just teach calculus to California public school graduates we'd be making some progress.

The Journals of Irreproducible Results

A group of psychologists recently attempted to replicate 100 studies published in psychology journals in 2008.  The results are in, and they aren't great.  Roughly a third of results were replicated at a statistically significant level.  Before I unpack some of the implications, I should put in an important qualifier, namely that this does not mean that those studies should not have been published or that the investigators did something wrong!  A study isn't conducted and published in order to provide the final word, but rather to invite the rest of the community to scrutinize the work and attempt to replicate it.  If a conscientiously-conducted and well-designed experiment gives a statistically significant result (positive or negative), that result is fair game for dissemination, discussion, and further scrutiny.  This replication effort is not evidence that science is not being done, but rather evidence that science is being done right.  We need more replication efforts, not fewer studies.

That said, it should give pause to people who point to a study and make prescriptions.  I've written before about the cult of This One Study, and the challenges of actually reproducing so many psychology studies point up the follies of that cult.  Moreover, as challenging as it is to get reproducible results in psychology, many educational research studies that I read do an even worse job on blinding participants and maintaining proper controls,  Some of that is inevitable, given the nature of classroom work, but some of it is a result of people uncaringly playing with too many variables at once.  If the person teaching the class is enthused about trying a new educational method and transmits that enthusiasm, and also varies the topics and assignments, while the person in the "control" group is not doing anything to get fired up and refreshed, you can't treat this as a single-variable experiment.  Moreover, often educational experiments are tapping into deep wells of cultural baggage; will the experiment get the same effect if it is conducted with people from the opposite side of that cultural divide?  Finally, it has been pointed out to me by at least one person trained in statistics and educational psychology that when you have one class section taught by New Method and another class section taught by Old Method, even if you have hundreds of students in each section you still have n=1 in the experimental and control groups, not n=hundreds, if your goal is to study methods rather than students.  "n=1" is a fancy statistical term for "anecdote."  Despite all of this, educational research tends to lead to a lot of teaching and prescription.

Strangely enough (or maybe not strangely at all, depending on your degree of cynicism), I suspect that Right-Thinking People will delightedly cite this finding that psychology experiments are not always reproducible as a way of reminding people that they don't necessarily know what they think they know, but will also continue to cite their favorite studies as rhetorical cudgels.  Consistency conshmistency!

Science vs. Engineering, "This one study" vs. Practice

Any reader of this blog (assuming that I actually have readers...) can tell that I'm a curmudgeon who is skeptical of reports of Great New Ways Of Teaching.  My default is that teaching is largely a work of timeless issues and problems, and there are no easy substitutes for lots of time and attention and study and patient attempts to solve a problem many different ways, to understand something from many different perspectives.  I believe that the acquisition of skills and knowledge and insights is difficult, time-consuming, and painful, while also having sweet rewards and attractions that keep us coming back for more.  The difficulty of finding genuinely new ideas, genuine innovations, is not a new problem.

This tends to get me into a lot of arguments at lunch with colleagues.  One genuinely good thing about the people whom I tend to argue with is that they are optimistic and open-minded, and since I believe in balance I believe that it is important to have their perspective around.  I'm right and they're wrong and we thereby get balance :)  More seriously, my reason for dissenting from their optimism and open-mindedness, even while understanding its virtues, is that I think many of our disagreements ultimately come down to the difference between science and engineering.  (Bear with me for a few paragraphs before I get back to the science vs. engineering point.)

A typical lunch argument will involve somebody enthusiastically noting some recent study whose results they hope to incorporate into their own practice of teaching or mentoring in the near future (though we have also argued over studies of things like diets and lifestyle, perennial topics of news reports that begin with "According to a new study...").  I am generally skeptical that going from a small study to large adoption will yield the promised improvements.  Part Most of my skepticism is possibly obviously rooted in my own personality and prejudices, but there's more to it.  Studies of new techniques for solving some problem in human interaction are almost always either small-scale experiments undertaken by teams putting a lot of effort attention into it (notoriously vulnerable to the Hawthorne Effect), or else studies of cohorts that are larger but composed primarily of early adopters, who will tend to adopt things differently than the wider population.  When you are talking about teaching, where the human interaction is everything, the difference between an early adopter and a more cautious person is crucial to implementation.

One famous example from physics is the 1999 Hake Study (journal link) (free .pdf) which compared 62 different introductory physics courses and found that the more interactive courses tended to show better improvements on the Force Concepts Inventory (FCI).  That basic finding has stood the test of time, and I will note that I incorporate many interactive elements into many of my courses.  However, the study also found gains as high as 60% or better in some interactive courses (on a scale that is described in the study; read it for more details).  My understanding from talking to people in the field of physics education research is that those gains are not terribly common today, and anything above 40% is really good and even above 30% is decent.  My hypothesis for the difference is that if you were using interactive teaching techniques in the 1990's you were an enthusiastic early adopter, and maybe even willing to plow ahead in the face of resistance.  If you are using interactive teaching techniques today you are following peer pressure, doing what is expected of you (in many but not all institutions) and not necessarily an enthusiast.  If the human element matters in the classroom, then you either restrict your hiring to early adopter types (assuming you can find enough of them) or you look for principles and practices that have a more essential, timeless element to them, fundamentals that will work even if the instructor is not possessed of an early adopter mindset.

What does this have to do with science vs. engineering?  The purpose of a good scientific study is to generate knowledge and understanding.  The purpose of engineering is to translate ideas into practice.  If I went to my lunch group and said "I just read this one study and the researchers made a photovoltaic with [insert great performance numbers here]!  Why don't companies just make that and sell it?" people would look at me funny.  If I went to my colleagues and said "I just read this one study and the researchers successfully treated tumors in preliminary trials, so why don't all doctors just do that?" they would wonder if I have a brain tumor.  The road from a scientific study showing proof of concept to something that can be adopted and work in widespread practice is a long one.  In the hypotheticals that I outlined, an engineer who said "Well, it's just a problem of consumer attitudes, we don't need to examine whether this [product, practice, drug, whatever] is robust" would quickly find himself at the unemployment office.  Making something robust is the essence of engineering.  However, if you look at This One Study and wonder if it will scale well outside of special environments or early adopter groups or whatever, people say that you have the wrong attitude.

I have a deep respect for social science.  I respect it enough to understand that the road from genuine scholarly insight to human practice in the field can be just as long as the road from laboratory physics innovation to product in your home.