This was originally published on my old blog, Social Geometer, and is reproduced here with slight revision.
A while back I did a guest post for Lee Jussim’s blog at Psychology Today where I compared social psychology’s growing pile of unreplicable findings to belief in magic. Many of the casualties of the replication crisis were claims that some tiny and seemingly irrelevant intervention would produce major important effects through the work of invisible and mysterious subconscious forces.
I’m certainly not the first to make that observation; Scott Alexander had a lengthy post at Slate Star Codex (the old, pre-NYT-doxxing one) comparing such claims to voodoo. But as someone long interested in the comparative sociology of supernatural beliefs, I wanted to investigate the analogy in light of things like Bronislaw Malinowski’s theory of magic or Donald Black’s theory of witchcraft accusations.
It later occurred to me to challenge my own argument with a counter example: disinfection.
Okay, says Ignaz Semmelweiss, I can cut mortality in the maternity ward by more than half by having my attending physicians wash their hands in a solution of chlorinated lime. Mortality during childbirth is serious, long-standing problem, and this extremely brief, very cheap intervention will solve it. And it will do so by acting through mysterious means — I speculate that there’s some tiny invisible particles of “corruption” on the hands that cause disease, and that they are being destroyed by this “disinfection” process.
If we’re going to mock trivial interventions that miraculously solve major problems by invisible means, it seems like we ought to be mocking Semmelweiss. Which is, as I understand, just what his colleagues did, as they continued causing women to die in childbirth by prodding them with their infected hands.
The main difference between this and whatever goofy two-minute intervention researchers are using to boost kids’ test scores is that Semmelweiss’s method was reliable. It works week after week. The death rates stay down.
When an intervention is that reliable it doesn’t seem so magical anymore. It seems more like technology. Saving lives through disinfection is as predictable as ending them through stabbing or shooting. The latter doesn’t seem terribly magical, even though your average person probably doesn’t understand the atomic forces that cause the spear to part the flesh. It’s just how things work.
Arthur C. Clarke said any sufficiently advanced technology is indistinguishable from magic. Not really. Magic is a concept we reserve for things outside of that which is so common and reliable that it’s part of the mundane world. An arrow is a far more advanced military technology than a supernatural curse, and even in societies that believe in curses they probably recognize arrows as a different sort of weapon and the one you want to reach for when the enemy raiding party comes in the night.
Corrected formulation: Any sufficiently advanced technology is indistinguishable from nature.
I’ve seen people lament that modern normies have no more idea how their light switch works than would some preliterate tribesman. They say its just magic for the ignorant masses (including the educated ignorant masses). But I don’t think that’s quite right: It’s not set apart and special, not viewed with fear and awe, not something reached for to cover the gaps in our practical technology. It’s just nature.
For those who criticize how little moderns appreciate the systems they depend on, this might actually be worse.
All this makes me consider the way social scientists think of the experimental method and the concept of replication. We tend to have a model that involves sorting out the statistical differences between two study groups and looking at whether the results pass a significance test.
Contrast that with physical scientists building some experimental set-up and showing its reliability by repeatedly producing the same result and teaching others to do so as well. I think of something like the development of the laser. The core workings were proposed in theory in the early twentieth century, and the building of the first working laser can be understood as a test of those principles. What control group? What statistics? We know the laser works because we can turn it on and see effects. Turn it off and they go away.
Not that making something like that reliable is easy. When another team first tried to replicate the laser, they couldn’t get theirs to work, and had to have the original developers visit and communicate whatever tacit knowledge wasn’t coming across in the published instructions.
Still, one wonders what sociological or psychological analogues there are to this sort of thing. And I wonder if our most replicated findings are the ones we don’t even think of as experiments or findings or sociological technologies because we rely on them every day. For over a decade now, I’ve started the first day of my courses by asking the students to stand up and line themselves up around the room so I can sort them into teams. And they reliably comply! It’s just nature.
typo: "cut morality" should be "cut mortality"