The last few days were very hectic. I gave more than ten interviews, about half by phone or Skype and the rest by answering e-mailed question. I prefer the second approach: I am a better writer than speaker.
I am glad that there is so much interest in our results in the popular press. And in this case, I can’t help feeling that this level of publicity is well deserved. Whether or not Cliodynamics can become a fully-fledged scientific discipline has been a recurrent theme of this blog. The PNAS article is an unequivocal proof that it can. It represents a completely new way of doing history. We start with a general theory (cultural evolution and multilevel selection), develop a detailed mathematical model based on this theoretical framework, collect a large dataset on the appearance and spread of large-scale complex societies across three millennia in the Old World, and test the detailed, quantitative model predictions on this dataset. And the model ends up predicting the data very well. It just doesn’t get any better than this. In fact, if you were to ask me 10 years ago whether such a striking result was possible, I would say ‘no’.
There were several recurrent questions that came up in conversations with reporters. I and my co-authors have been thinking about these issues for years, so we take certain connections for granted, connections that are not obvious to people who are just learning about this study. So hopefully it will be helpful if I list these recurrent questions together with answers. This may help to flesh out the details of our study.
What is new about this model?
The novelty is putting together enough general mechanisms in the model (but no more than necessary) and predicting a very large dataset on the rise and spread of macrostates in history – and then succeeding in doing a very good job of prediction. Such a systematic approach, which starts with general theory (cultural multilevel selection) translates it into a specific agent-based model, extracts detailed predictions from the model on where and when in Afroeurasia large states arose and spread, and then compares to actual quantitative data has never been applied to the question of what explains the evolution of states.
Why does a question such as this need a computer model to answer?
Although our theory is relatively simple, it’s too complex to reason through using verbal arguments. There are important nonlinear feedback loops that can be captured only mathematically. Furthermore, the heart of our approach is getting detailed, quantitative predictions that can be compared to a large dataset on historical evolution of states in Afroeurasia. This can be done only with a quantitative, dynamical model.
Does this model tell us anything new, or is it a way to confirm what we already know?
The model results in new knowledge. Before we went through this exercise we did not know whether competition between societies, taking the form of warfare, was really an important driver in the evolution of large complex societies. Now we know that it is the main factor, with the presence of agriculture as a necessary condition, and various environmental effects (e.g., rugged terrain) also playing a role. Undoubtedly, cultural peculiarities are also important, although they were not included in the model. But since the model predicts 65 percent of variance in the data, such other factors must be of lesser importance than those included. At best, they provide the remaining 35 percent of the explanation. Our main result, that patterns of warfare are the most important factor explaining the rise and spread of large states is quite novel, and it will not be immediately accepted by most anthropologists and historians (although I expect that political scientists will be more sympathetic). Our results are likely to generate much controversy, which is why we plan to continue with this research program to address various criticisms that people will be bringing forth.
What are “large complex societies”?
In the simplest terms, societies counting a million of individuals or more. Such societies are invariably organized as states, have many complex institutions that are designed to prevent them from breaking up, extensive division of labor, complex internal organization, and so on. So they are ‘complex’ in many different ways
Why do you call these societies “anonymous”?
For most of our evolutionary history humans lived in small-scale societies – numbering just hundreds of people. These human groups were integrated by face-to-face interactions. In other words, everybody knew everybody else. Then there was a transition to large-scale societies, starting 10,000 years ago. In large societies of today each of us knows only a tiny proportion of people – the huge majority is strangers. In this sense our societies are anonymous. We interact all the time with people who are not personally known to us (think of taking a subway in New York City, or shopping in a supermarket).
How did you go about developing this model? How did you decide which data inputs to include?
Our model was guided by a general theoretical framework – cultural multilevel selection (CMLS). This theory predicts that competition between societies is the main driver of evolution of complex societies. Thus, emphasis on warfare. But then we needed to ‘operationalize’ such quantities as ‘warfare intensity’. What does it mean? It turned out that for the period of history we focused on, 1500 BC – 1500 AD, we could capitalize on the spread of warhorse-related technologies as a proxy for intense warfare. The importance of rugged terrain was also suggested by the CMLS theory. It is easier to defend mountainous areas.
And it is clear that agriculture is a necessary condition for the rise of complex societies. That was already well known. However, our model shows that just the spread of agriculture does a not-so-great job explaining where and when large-scale societies arise. It’s a necessary, but far from sufficient condition. Warfare patterns do the bulk of explanation – that’s what allowed such a remarkably good match between model results and data.
Why do you think intense warfare and the spread of war technology turned out to be so important in deciding which large states would form?
That comes out from the CMLS theory, as I explained earlier. To evolve to a large size, societies need special institutions that are needed for holding them together; preventing them for splitting along the seams. But such institutions have large internal costs and, without constant competition from other societies, they collapse. Only constant competition between societies ensures that ultrasocial norms and institutions will persist and spread. So it really was war that made the state.
Were you at all surprised by these results?
I was certainly surprised by how well the model predicted the data. Even with first-guess parameters, the ones I tried during the early phase of the work, the model output looked very much like data. Quantitatively the model explained >50% of variance. Moderate adjustments of just 4 parameters increased prediction to 65%. This is much better than anyone, including myself, had thought can be done with historical data. Even though history is very complex, it turns out that simple models can capture very well many of its patterns.
What are the limitations of this sort of approach? Are there any nuances or particular cultures’ idiosyncrasies that affect their expansion and can’t be incorporated into mathematical models?
Of course, differences in culture, environmental factors, and thousands of other variables not included in the model all have effect. If you look carefully at Figure 1, you will see that there are lots of differences in detail between data and the model. That’s as it should be. A simple general model should not be able to capture actual history in all its glorious complexity. Both general processes and cultural idiosyncrasies play a role. But most historians and the lay public don’t realize that general processes can be very powerful in shaping history. Our model proves that they are wrong.
Are you planning any further work in this area? Do you hope to develop a model that’s even more accurate?
This paper is only a part of a long-term research program. It was preceded by two other, more technical and less spectacular papers. Now we are taking this research in several directions. One is to make a more detailed characterization of the environment and of agricultural productivity (rather than just presence/absence as in the current model). Another direction is to apply the general model to history after 1500 AD, with a different set of military technologies, naturally enough. We are also planning to develop a model for pre-Columbian North America, using the spread of bow-and-arrow MilTech as a proxy for warfare intensity.
Do you think the same parameters used in this study could be used to define the spread of civilizations today? Or, in other words, if 1,000 years in the future this study was replicated, what criteria would people use to track the spread of civilization?
No, we need to use different approaches for the study of other historical periods. For example, we have a plan to develop a model for post-1500 history, but of course instead of cavalry and archery we will use gunpowder weapons and ocean-going ships. More recently, military competition between societies has become of much less importance, compared to the period we studied (this is actually a good development). So we need to focus on economic and perhaps ideological competition.
What will happen in the next thousand years? Who knows, perhaps we will encounter intelligent aliens from Tau Centauri?
So of course parameters and many processes have changed across human history and will continue changing in the future. However, at a more fundamental and abstract level, multilevel selection, the processes bringing about increased social (and biological) complexity have been the same and are likely to continue to be the same. So competition between human groups and societies, hopefully nonviolent, will probably continue to be the main engine of social evolution.
A little non-academic feedback, if you will. I have been following your work since your presentation in Hartford at Real Art Ways. This blog did a magnificent job of clarifying, in layman’s terms the meat and potatoes of your research. Thank you. You have made this an exciting topic. I will continue to follow and understand, I hope, more of your work.
Thank you, Bill!
How much of the “high” R-squared values are driven by the large swaths of low probability (green) territory at high latitudes?
Based on a cursory reading of your paper, it seems like you can predict where large empires aren’t relatively well, but you have a harder time predicting where they are (i.e. the yellow/red areas at low latitudes are).
Do you think additional parameters might allow more accurate modelling of this?
I am not sure I agree. It’s not the question of high/low latitudes. If you include latitude as a predictor variable, you won’t get much predictability out of it.
Sorry, I don’t think I articulated that very well, I wasn’t suggesting that you include latitude as a variable. I was pointing out that the large swathes of green areas on both maps line up pretty nicely and might be driving a large part of the correlation.
By contrast, the squares with higher probabilities (yellow and red) of large societies are in the same general area, but (in so far as I can tell from the heat map) don’t match up so well in their details (although their outlines are reasonably similar).
I guess what I’m saying is, what would happen if you looked at the correlations of these two subsets independently?
Re: More recently, military competition between societies has become of much less importance, compared to the period we studied (this is actually a good development). So we need to focus on economic and perhaps ideological competition.
Of course the enormous scale of cooperative modern societies – combined with the modern peace – weighs pretty heavily against warfare-driven models of cooperation. Evidently, there’s something else going on.
The model is on pre-1500 period of human history, when essentially all intersocietal competition was warfare-related If not fighting wars, then preparing for them). I have repeatedly stressed that the model only applies to Afroeurasia and 1500 BCE – 1500 CE. So the observation that military competition is of less importance today cannot “weigh pretty heavily against warfare-driven models of cooperation”.
I’m not so sure that military competition between societies has become all that less of a driving force in social evolution in modern times. For example, it is highly unlikely that Russia or China would have embarked on such a painfully quick and unsettling process of rapid industrialisation had it not been for the knowledge that there were factories across the border pumping out tanks.
p.s. congrats on another landmark publication – I look forward to reading it.
Looks like the model handles some regions better than others. Poor Mongols, in particular, get greened over all the time. I wonder why this happens.
Also, some parts of the “data” side look weird. No complex societies in Ethiopia before 1500?
We did not model steppe polities, only agrarian empires. The Mongols turn will come, in one of the next iterations.
How did you avoid overfitting to the data? I’ve seen a lot of discussion from historical and social point of view, but purely statistically, it is easy to construct a model that explains past data well.
We used the standard approaches, in this case, the Akaike Information Criterion. See the supporting information file. But, hey, you cannot overfit an empirical dataset of 7941 data points with a theoretical model that has 4 adjustable parameters. The result is very solid.
I challenge you to construct a mechanistic model that makes sense (whose assumptions are reasonable) and has a dozen or fewer parameters, and that explains our dataset as well. If you do, I’ll publish your paper in Cliodynamics, the journal.
Thanks, I need to check the supporting info!
I am not an expert in the field so no chance of beating your model 🙂 I was just curious because I had seen no discussion about that statistical aspects of your work.