As readers of End Times (published last week) know, the theoretical foundations of my explanation of why we in America are in crisis is provided by the structural-demographic theory (SDT). The essentials of this theory were proposed by Jack Goldstone in a 1991 book (be sure to read about his thorny road to publication in Chapter A1).

The theory has come a long way from Goldstone’s original formulation to where we are today. On the mathematical and conceptual side, the verbal (but quantitative) original formulation was translated into dynamical models, expressed both as differential equations and as agent-based simulations.

But especially much progress came on the empirical side. Goldstone developed and applied his theory to several revolutions and state collapses in the early modern world. The two periods that he focused on were the General Crisis of the Seventeenth Century and the Age of Revolutions during the “long” nineteenth century (which started in the 1780s). Together with Sergey Nefedov we extended the list of case-studies to medieval and early-modern England, France, and Russia, as well as Roman Republic and Empire (in Secular Cycles).

Other cliodynamicists doubled and tripled the number of empirical examples. And now we are building CrisisDB, a historical database of past societies sliding into crisis and then emerging from it. In End Times I describe our preliminary results from the first 100 cases. That was the state of things back in August of 2022, when I turned the book in to the publisher. Now we are approaching two hundred crisis cases, and in a year’s time, hopefully, 300 cases. However, we are not waiting until all those data are gathered—as we learned with Seshat, gathering data is a neverending story! There are several analysis papers submitted to journals or nearing that state (although don’t expect them to be out soon—academic publishing is glacially slow).

The philosophy of science underlying these empirical tests is retrodiction (an innovation of our project). Let me explain. There are many explanations for why declines and falls of empires and other states happen. One German historian counted more than 200 theories about just one such collaps, the end of the Roman Empire. But we are interested in learning why empires fall in general.

In science, we weigh relative merits of different theories by using scientific prediction. We translate different explanations into specific mathematical models, use the models to make predictions about novel data, and then determine which theory makes better—more accurate—predictions. This is not guaranteed to give us the “one and only true theory,” but it enables us reject theories that make worse predictions in favor of those that make better ones. This is how science advances, one dead theory at a time. And this is why we need CrisisDB.

Note that there is nothing wrong with predicting the past (retrodiction). As long as testing data are not known when the model was built, comparing model predictions with such data is a true test. In the jargon of complexity science, it is an “out-of-sample” prediction.

Nevertheless, there is something particularly satisfying about predicting the future. We have one such successful case, my prediction, made in 2010, about an outbreak of political instability in the USA around 2020. But I will be the first to admit that a single success could just be due to luck. In order to test the theory more thoroughly, we need to see how will predictions about, say, ten countries fare 10 years from now. More importantly, while our theory is clearly yielding valid, and valuable insights, it is equally clear that it’s still quite young, and there is a lot more work needed. Undoubtedly, there will be many ways in which the actual future deviate from theoretical predictions, and our intent is to learn from such failures so that we can develop better models. And also to better define the limits of prediction.

So, this is a call for those interested in joining in on the fun to develop a predictive science of societies. Dan Hoyer and I have just posted a preprint on SocArxiv, Empirically Testing and Refining Structural Demographic Theory: A Methodological Guide, in which we describe the logic and the methodology of the approach in detail. Note that, because in each society the ruling elites are recruited and reproduced in different ways, the general SDT model needs to be tailored to take such details into account.

As we wrote at the end of this article,

we still need to explore how well SDT can predict future crises, how it compares to, or fits together with, other prevailing causal theories about the drivers of societal conflict, and most importantly whether improving our ability to see what lies ahead can also help us to identify ways we can avoid the more disastrous outcomes. The best way to answer these questions is to compile as many studies utilizing SDT to track dynamics of unrest in as many societies as possible, drawn from every corner of the globe and spanning time from the ancient past to the present day.             

This chapter offers a road-map for researchers to take up this call and extend the body of SDT cases. The pay-off of these efforts will be two-fold: firstly, these studies can provide novel insight into periods of social strife for any number of cases that require further understanding; secondly, and more impactfully, compiling these cases will allow us and others to refine the theory and add new analytical techniques. The overall goal of all these efforts is to employ SDT on all current societies, identify “trouble spots” and reveal possible interventions that may be able to head off trouble before it starts, or at least to mitigate the severity of crises that do strike. This is work that we, along with our colleagues at the Seshat Databank, will be engaging in for the foreseeable future. We hope that others will join us. We are always eager to collaborate or consult in any such efforts.