Eleven years ago we launched the Seshat project with the goal of collecting data that would enable us to test the many theories aiming to explain the evolution of large-scale complex human societies over the past 10,000 years. We now have a bunch of papers, some already published, others accepted and soon to be published, that report on the results of these tests. A major article is coming out next month in Science Advances, in which we tested 17 specific hypotheses from five general classes of theories, and I will talk about it here when it is published.

We are also (finally!) coming to a closure on the controversy about the role of moralizing religion in the rise of complex societies. I wrote about it in these posts:

Do “Big Societies” Need “Big Gods”?

What Came First: Big Gods or Big Societies? Round Two

This debate is the subject of a special feature in the journal Religion, Brain, & Behavior. The Feature has been finalized and should be soon published on line. It starts with the “target” article written by our team (see here the SocArxiv preprint); then there are six commentaries on our article, followed by our response and, finally, a “retake” article on the history of the controversy.

What are the main conclusions? Our analysis confirms that there is a strong and positive correlation between socio-political complexity (SPC) and moralizing supernatural punishment (MSP). However, this correlation arises not because there are causal effects in any direction (that is, either SPC is an evolutionary driver for MSP, or MSP drives SPC, or both), but because both variables respond to a similar set of other evolutionary forces (namely, agriculture and warfare). Thus, the correlation between SPC and MSP is not causal, but “spurious.” This amounts to a decisive rejection of the Big Gods theory (in all its variants).

A summary of causal influences affecting the rise of MSP, suggested by the dynamic regression analysis of the Seshat data (Figure 3 in the Target Article)

Of the six commentaries that were written, all but one are quite positive. They are at times critical of various aspects of theory, data, and methodologies in our article, but such constructive critique is welcome. We do not claim that our article is the final word; clearly, there is a lot of work ahead before we can claim that we really understand the role of religion in the evolution of complex societies. I look forward to this future research.

The one exception is the comment by Purzycki et al. (also published as a SocArxiv preprint). It is a bizarrely intransigent commentary that finds fault with “each step of its [our target article] workflow—from data production and theory to modeling and reporting”. We fully rebut these criticisms in our response, which will be published as part of the special feature, but here I wanted to address the main critique from Purzycki et al, on the causal inference.

If you read their claims carefully, it becomes apparent that this particular team of scientists simply lacks sufficient data-science expertise to evaluate the statistical approaches used in our article. Purzycki et al. are like the proverbial boy with a hammer, to whom every problem looks like a nail. They know one particular approach to causality, which uses directed acyclic graphs (DAG). However, our statistical approach to causality, as explained in the target article, uses a different conceptual framework, dynamic regressions (DR), which is based on the ideas of Norbert Wiener in 1956, later developed by Clive Granger (thus, economists know about this approach as the “Granger causality”).

Possible causal scenarios giving rise to correlation between X and Y

The most important difference between DR and DAG is that the latter does not explicitly include the time dimension. Thus, instead of causal links in DR, such as X_t→ Y_(t+1), in DAG causal connections are denoted without time subscripts, as X → Y. Because of this difference, DAGs have to be acyclic. In other words, scenarios of mutual causation cannot be investigated; only one-directional causality is allowed. Furthermore, the main goal in DAG is estimation of the causal effect. This approach is appropriate if we need to know, for example, by how many years a particular drug would increase life expectancy, when we only have observational data. To answer this question, an analyst must assume a particular DAG—its form is under-determined by (time-unresolved) data. In other words, rather than asking what are causes and what are effects, we assert the direction of causality and then use the data to estimate its strength.

The goal of the DR approach is different. We aim to use data to adjudicate between different theories of social evolution, each proposing a different causal graph. This is generally impossible to do with static (time-unresolved) data, which is why the goal of the Seshat project from its inception was to collect time-series data. Unlike with DAGs, the main goal of the DR approach is model selection, choosing among different models each proposing a different causal graph. We are also interested in estimation, because we want to compare the numerical strengths of different factors, but this goal is secondary to model selection, as we first need to determine which causal graph should be used for coefficient estimation. Thus, the DAG approach, excellent as it is, differs in goals and methods from the DR approach.

The DR approach was designed to resolve questions of causation in evolutionary processes (“descent with modification”) that unfold slowly in time. A model’s ability to predict data is interesting not in itself, but as a tool for adjudicating between different theories. The DR analysis attempts to distinguishes correlation from causation by estimating what influence potential causal factors at a previous time have on the response variable at a later time.

Purzycki et al. did not understand this crucial point, which is why they make statements such as “the direction of causality cannot be determined by data (or regression coefficients) alone.” This statement is correct for an analysis of static data, but not true for time-resolved data. If a theory predicts that a factor X causes Y, but we find that change in Y precedes change in X in time, then clearly the theory cannot be correct. This is how we can use time-resolved data to falsify theories. Of course, there are many complicating factors that require caution in analyzing time-resolved data, but generally a temporal dimension allows us to approach questions of causality much closer than it is possible with static data.

Purzycki et al.’s lack of statistical expertise is particularly glaring in the bizarre simulation model that they use to illustrate their claim that our approach cannot yield accurate causal inferences. They start with this DAG (see their Figure 3, based on the one in McElreath 2020: 515):

Their R script then implements the DAG in a mechanical way that actually does not make sense. Thus, “Missingness” is simply a random variable that is positively correlated with “Writing” (although we would expect that Missingness would decrease in the presence of texts). Worse, Missingness then has positive effects on observed Social Complexity and Moral Gods. In other words, high Missingness increases the value of observed Moral Gods. This is nonsense, because what Missingness actually does is convert some values of observed Moral Gods into missing values. Instead, in the Purzycki et al. model, Missingness grows Moral Gods! Naturally, the “results” from a model with such meaningless assumptions can have no bearing on the validity of the DR approach.

Returning to the Special Feature on Moralizing Religion in RBB, what I find strange is that none of the supporters of the Big Gods theory have chosen to write a commentary on our target article. Most of them have been actively criticizing us on Twitter over the past three years, but when the time came to respond in a substantive way—and they were given ample opportunity to do so—they fell strangely silent. Does this mean they now accept our conclusion? Is the theory dead?