A guest post by Harvey Whitehouse and Pieter Francois

This is an interesting moment in the development of history as an academic discipline. We stand on the brink of a sea change, not necessarily in the way historical evidence is gathered and documented, but in the way the resulting data can then also be compared across space and time. For those who are interested in theories about how human societies have evolved, these are exciting times. But they are also turbulent times because many of those theories will turn out to be wrong. One of the first casualties appears to be the hypothesis that big societies require moralizing gods.

Our Nature paper is just a first step towards adjudicating on the moralizing gods hypothesis. But it is an important step because it demonstrates that even using very lenient criteria for the presence of beliefs in supernatural punishment, such beliefs appear late in the rise of social complexity. Advocates of the view that such beliefs occur much earlier include distinguished academics whose work we respect. But the way some have recently gone about defending their cherished hypothesis is problematic.

In the first of two papers posted online our critics have argued that they can reverse our results by systematically changing the data to adjust for what they call ‘forward bias’. Unfortunately, half the adjustments they propose are indefensible on factual grounds effectively beyond dispute. Even if we adjust all the remaining data in their favour exactly as they propose, this doesn’t reverse our main finding, as claimed.

The second paper challenges the quality of our data and will be published in the Journal of Cognitive Historiography alongside a rebuttal that we are currently working on. The fact that both critiques have been ‘pre-published’ online, and that considerable effort has been invested to disseminate them to the widest possible audience, means we can no longer restrict our rebuttal to academic journals and the pressure is on to summarise key points at a much faster pace on more informal platforms, such as blogging sites. This situation has its limitations but it also affords novel opportunities.

A limitation of this informal online approach to debating scientific findings is that it is hard to coordinate critique and response. The pre-published attack on our work submitted to the JCH includes a substantial appendix, the contents of which we need to rebut at length (and will do so). But in the meantime, it could look to some as if a valid pre-published critique stands while the main finding of our Nature paper, which underwent rigorous peer review, can simply be dismissed. It would arguably have been better for science if critique and rebuttal had appeared side by side, as the journal editors in this case intended.

More positively, though, online debate allows us the license to step back more informally and consider bigger-picture issues. For example, are we really at a turning point in the history of history? Potentially yes. The likes of Marx, Spencer, Tylor, Frazer, and Durkheim – among other big-thinking Victorians – dreamt of establishing generalizable theories of history but they were held back by the ‘cherry picking problem’. That is, theories of history – from grandiose visions of economic and technological determinism through to the idea that the division of labour in society evolves through discernible stages – have always rested on evidence selected because it supported the theory, while less congenial evidence was rejected or overlooked. What is radically new about the approach adopted in our Nature paper is that it tests theories of history based on a serious effort to avoid bias in the selection of data by coding for features of social complexity, religion, and ritual, in exactly the same way across hundreds of polities. The data itself and the methods used to gather and analyse it are all publicly available so that colleagues can inspect it, replicate and criticize our efforts, and run analyses of their own. As we have seen, they can even run analyses that explicitly bias the data to fit their own theories if they so wish – but at least we can see clearly that this is what they are doing.

Seshat: Global History Databank allows us for the first time to address the problem of selection bias convincingly in our efforts to test theories of world history empirically. Fully realizing this vision requires the input of very large numbers of experts from fields as diverse as history, archaeology, classics, anthropology, comparative religion, and others. Many scholars in these fields, however, are wary of scientific methods so it is no mean feat to have attracted such large numbers (around 100 or so currently) to the Seshat enterprise. Can we continue to do so?

The very public attack on our data, analysis, and methods launched online, albeit using material that has not been peer reviewed, has the potential to undermine confidence in Seshat. Those leading the criticisms against us are closely associated with a rival database which is at a much earlier stage of development but which may hope to catch up if only we can be slowed down. Attacking Seshat could, however, hamper everyone’s efforts in this new field and not just our own.

If our new approach to the study of global history survives, this will be very good news for the humanities. It will not change the fundamental methods of historical enquiry but will complement them. Existing historical research will become more thoroughly integrated with many areas of the social sciences and attract more resources. On the other hand, it will be mostly bad news for theories.

Few theories will survive unscathed. But that is a desirable situation scientifically. What is undesirable is to try to smother the latest prodigies of science before they are old enough to speak or loud enough to be heard.

Harvey Whitehouse and Pieter Francois are both at the University of Oxford, and (together with Peter Turchin) are Seshat Databank Founding Directors