Eleven years ago we launched the Seshat project with the goal of collecting data that would enable us to test the many theories aiming to explain the evolution of large-scale complex human societies over the past 10,000 years. We now have a bunch of papers, some already published, others accepted and soon to be published, that report on the results of these tests. A major article is coming out next month in Science Advances, in which we tested 17 specific hypotheses from five general classes of theories, and I will talk about it here when it is published.
We are also (finally!) coming to a closure on the controversy about the role of moralizing religion in the rise of complex societies. I wrote about it in these posts:
Do “Big Societies” Need “Big Gods”?
What Came First: Big Gods or Big Societies? Round Two
This debate is the subject of a special feature in the journal Religion, Brain, & Behavior. The Feature has been finalized and should be soon published on line. It starts with the “target” article written by our team (see here the SocArxiv preprint); then there are six commentaries on our article, followed by our response and, finally, a “retake” article on the history of the controversy.
What are the main conclusions? Our analysis confirms that there is a strong and positive correlation between socio-political complexity (SPC) and moralizing supernatural punishment (MSP). However, this correlation arises not because there are causal effects in any direction (that is, either SPC is an evolutionary driver for MSP, or MSP drives SPC, or both), but because both variables respond to a similar set of other evolutionary forces (namely, agriculture and warfare). Thus, the correlation between SPC and MSP is not causal, but “spurious.” This amounts to a decisive rejection of the Big Gods theory (in all its variants).
A summary of causal influences affecting the rise of MSP, suggested by the dynamic regression analysis of the Seshat data (Figure 3 in the Target Article)
Of the six commentaries that were written, all but one are quite positive. They are at times critical of various aspects of theory, data, and methodologies in our article, but such constructive critique is welcome. We do not claim that our article is the final word; clearly, there is a lot of work ahead before we can claim that we really understand the role of religion in the evolution of complex societies. I look forward to this future research.
The one exception is the comment by Purzycki et al. (also published as a SocArxiv preprint). It is a bizarrely intransigent commentary that finds fault with “each step of its [our target article] workflow—from data production and theory to modeling and reporting”. We fully rebut these criticisms in our response, which will be published as part of the special feature, but here I wanted to address the main critique from Purzycki et al, on the causal inference.
If you read their claims carefully, it becomes apparent that this particular team of scientists simply lacks sufficient data-science expertise to evaluate the statistical approaches used in our article. Purzycki et al. are like the proverbial boy with a hammer, to whom every problem looks like a nail. They know one particular approach to causality, which uses directed acyclic graphs (DAG). However, our statistical approach to causality, as explained in the target article, uses a different conceptual framework, dynamic regressions (DR), which is based on the ideas of Norbert Wiener in 1956, later developed by Clive Granger (thus, economists know about this approach as the “Granger causality”).
Possible causal scenarios giving rise to correlation between X and Y
The most important difference between DR and DAG is that the latter does not explicitly include the time dimension. Thus, instead of causal links in DR, such as X_t→ Y_(t+1), in DAG causal connections are denoted without time subscripts, as X → Y. Because of this difference, DAGs have to be acyclic. In other words, scenarios of mutual causation cannot be investigated; only one-directional causality is allowed. Furthermore, the main goal in DAG is estimation of the causal effect. This approach is appropriate if we need to know, for example, by how many years a particular drug would increase life expectancy, when we only have observational data. To answer this question, an analyst must assume a particular DAG—its form is under-determined by (time-unresolved) data. In other words, rather than asking what are causes and what are effects, we assert the direction of causality and then use the data to estimate its strength.
The goal of the DR approach is different. We aim to use data to adjudicate between different theories of social evolution, each proposing a different causal graph. This is generally impossible to do with static (time-unresolved) data, which is why the goal of the Seshat project from its inception was to collect time-series data. Unlike with DAGs, the main goal of the DR approach is model selection, choosing among different models each proposing a different causal graph. We are also interested in estimation, because we want to compare the numerical strengths of different factors, but this goal is secondary to model selection, as we first need to determine which causal graph should be used for coefficient estimation. Thus, the DAG approach, excellent as it is, differs in goals and methods from the DR approach.
The DR approach was designed to resolve questions of causation in evolutionary processes (“descent with modification”) that unfold slowly in time. A model’s ability to predict data is interesting not in itself, but as a tool for adjudicating between different theories. The DR analysis attempts to distinguishes correlation from causation by estimating what influence potential causal factors at a previous time have on the response variable at a later time.
Purzycki et al. did not understand this crucial point, which is why they make statements such as “the direction of causality cannot be determined by data (or regression coefficients) alone.” This statement is correct for an analysis of static data, but not true for time-resolved data. If a theory predicts that a factor X causes Y, but we find that change in Y precedes change in X in time, then clearly the theory cannot be correct. This is how we can use time-resolved data to falsify theories. Of course, there are many complicating factors that require caution in analyzing time-resolved data, but generally a temporal dimension allows us to approach questions of causality much closer than it is possible with static data.
Purzycki et al.’s lack of statistical expertise is particularly glaring in the bizarre simulation model that they use to illustrate their claim that our approach cannot yield accurate causal inferences. They start with this DAG (see their Figure 3, based on the one in McElreath 2020: 515):
Their R script then implements the DAG in a mechanical way that actually does not make sense. Thus, “Missingness” is simply a random variable that is positively correlated with “Writing” (although we would expect that Missingness would decrease in the presence of texts). Worse, Missingness then has positive effects on observed Social Complexity and Moral Gods. In other words, high Missingness increases the value of observed Moral Gods. This is nonsense, because what Missingness actually does is convert some values of observed Moral Gods into missing values. Instead, in the Purzycki et al. model, Missingness grows Moral Gods! Naturally, the “results” from a model with such meaningless assumptions can have no bearing on the validity of the DR approach.
Returning to the Special Feature on Moralizing Religion in RBB, what I find strange is that none of the supporters of the Big Gods theory have chosen to write a commentary on our target article. Most of them have been actively criticizing us on Twitter over the past three years, but when the time came to respond in a substantive way—and they were given ample opportunity to do so—they fell strangely silent. Does this mean they now accept our conclusion? Is the theory dead?
Joseph Henrich and Jonathan Schulz argue that the medieval Catholic church has effects on psychology and society today. Is their research incorrect or is it compatible with the results of your study?
That’s a separate theory. I’d like to test it also, but unlike the Big Gods theory it’s not about a general mechanism of cultural macroevolution, but about a specific event in human history (the rise of Europe). So, it’s not clear to me whether it is testable with Seshat.
Your argument seems to say that their toy model is nonsense because it has positive coefficients from writing to “missingness” and from “missingness” to observations. This is purely a semantic problem since renaming “missingness” as “non-missingness” gives the same net effects among observables without those anti-intuitive signs. Did I miss something?
As for DAGs and time-dependence, time-dependent variables can be unrolled and are often used in DAGs. There’s nothing about your formalism or DAG methods that should require more than a little translational work to see the equivalence. Work on time-dependent variables using modern causal techniques goes back to at least Robins 1986 paper.
There are two main reasons why their toy model is nonsense:
1. The effect of missingness is not to increase or decrease the value of another state variable. It is to increase the number of missing values. To implement this process, one would convert somve values from observed into missing. This is not what they did.
2. As I explained in the post, the time-resolved nature of data is a key feature enabling us to approach causality more closely than is possible with synchronic data. But throwing away the time dimension, their method is not comparable to what we did.
We don’t want to “unroll” time-dependent variables. By throwing the time dimension away you severely limit the information you can extract from data. I don’t care about equivalence. Once again, the goals and methodology of the DR and DAG approaches are different. I want to make inferences about which DAG to use, not to assume it.
I think you have the meaning of “unroll” backwards. It means to include the time-dependence of the variables. If i understand correctly, the point of their DAG was not that it would be anything like a complete representation. It was just a compact way to show an important thing you missed- the dependence of the data censoring on the variables of interest. That could give a major bias to your analysis and in fuzzy cases like this could lead to mis-identification of the main causal pattern.
Have you tried enlisting anyone from the modern causal community? Your general goals are too exciting to use flimsy old analysis techniques.
Maybe google G-formula, etc.?
Or go to a standard text, that does not at all claim that DAGs are the whole story.
p.s. You do agree now that the two peculiar positive coefficients have no effect on the analysis, since that depends only on the product?
Unnecessarily derogatory comments of a personal nature directed against anyone only detracts from your excellent work and never helps improve scientific discord. Your rebuttal of their comments should speak for itself though logic alone.
It would be helpful to quote the “unnecessarily derogatory comments of a personal nature…”
As is, the implication that somebody has been unnecessarily derogated is a personal derogation of Peter Turchin. Perhaps you should confine your comments to “logic” alone?
Your comment that we would expect Missingness to decrease in the presence of texts I fear needs a caveat. In “religions” associated with state power, we should expect censorship to positively correlate Missingness and Writing. Consider the suppression of Marcionite writings even within the Christian tradition, much less the suppression of anti-Christian writings. Plus, subsidized speech, as today, effects drowns out dissidence, leaving the actual censorship to the erosion of time. (My guess this is the strategy of the Big Gods proponents to deal with the scientific challenge of the Seshat analysis. The real issue is the visceral objection to the very notion of a science of man. The anti-Marxist impulse in creating a genuine social science is still rejected because the notion of social science is rejected, as in principle beyond the limits of science, so far as I can tell.)
But the DAG showed above seems to have its own problems. In examining the question of whether the idea of moralizing gods was a cause of complex societies, another DAG that has an arrow from Unobserved Moral Gods to Social Complexity seems to me to be required for comparison? Also, even ignoring the time dimension, I can’t even guess how “Military” fits in here, starting with the problem that militaries and military technology are not obviously distinct from Social Complexity.
That is pretty good!
BTW, mee too I happened to have preprinted, these days, something about religion on SocArxiv (Joe GANIO-MEGO). It is a drop-off of a hobby that has been generated by reading several of your books 10 years or more ago. The hobby lasted too long. Now I plan to rest for a year or so. But then I will be glad to generate some discussions about these theories.
In the meantime, I’ll reflect on the concept of big (size?) in divination.
a strong and positive correlation between socio-political complexity (SPC) and moralizing supernatural punishment (MSP).?
The Chinese have managed perfectly well without MSP for 2200 years.
Is this actually true? Yes, some readings of Confucian classics portray it as a nontheistic philosophy. Maybe even just a morality/moralizing.. But as near as I can tell, even those variants are functionally like Stoicism in the Roman Empire. They ae an esoteric philosophy for some of the ruling class and its aspirants, meant from the beginning to be completely compatible with traditional religion. Both are fundamentally conservative, endorsing the social order, even if it pleads for humanity in the treatment of the lower orders. Further, the inclusion of the book of Rites and the I Ching in the Five Classics, and the Great Learning and the Doctrine of the Mean smell of the supernatural. The term so often translated “sage” seems to me to be very similar in usage to “saint.”
Confucianism like Stoicism, wasn’t the only game in town either. Taoism and Buddhism also blended with Confucianism and traditional mythology. I’m watching on Netflix right now a Korean series where the Jade Emperor looks and acts very much like a supernatural moralizing god.
The causality implicit in DAGs reminds me of one of my pet peeves: that “educating women” will result in reducing fecundity and subsequent stabilization of world population by mid-century.
My assertion, equally guilty of implied causality, is that it is increased access to energy that causes women in developing nations to both reduce their fecundity and to become more educated.
If this hypothesis is true, then that throws a big wrench in the works for those planning for a demographic transition from continuous growth to a steady state!
I assert that, as fossil sunlight continues its inevitable, irrevocable decline, women will have less access to energy, and will once again have to employ the time-honoured means of obtaining both a slave labour force and a retirement plan.
Might DR be used to tease out the causal relationships between energy, women’s education, and population growth?
(I’ve been accused of being against “educating women.” I assure you, that is completely wrong, and I support educating women to the greatest degree possible, while arguing that will not change civilization’s catabolic descent. I would risk producing “excess elites” for no good reason, then. 🙂