Structural-Demographic Theory: What’s Next?

Peter Turchin

20 Comments

Join 36.7K other subscribers

As readers of End Times (published last week) know, the theoretical foundations of my explanation of why we in America are in crisis is provided by the structural-demographic theory (SDT). The essentials of this theory were proposed by Jack Goldstone in a 1991 book (be sure to read about his thorny road to publication in Chapter A1).

The theory has come a long way from Goldstone’s original formulation to where we are today. On the mathematical and conceptual side, the verbal (but quantitative) original formulation was translated into dynamical models, expressed both as differential equations and as agent-based simulations.

But especially much progress came on the empirical side. Goldstone developed and applied his theory to several revolutions and state collapses in the early modern world. The two periods that he focused on were the General Crisis of the Seventeenth Century and the Age of Revolutions during the “long” nineteenth century (which started in the 1780s). Together with Sergey Nefedov we extended the list of case-studies to medieval and early-modern England, France, and Russia, as well as Roman Republic and Empire (in Secular Cycles).

Other cliodynamicists doubled and tripled the number of empirical examples. And now we are building CrisisDB, a historical database of past societies sliding into crisis and then emerging from it. In End Times I describe our preliminary results from the first 100 cases. That was the state of things back in August of 2022, when I turned the book in to the publisher. Now we are approaching two hundred crisis cases, and in a year’s time, hopefully, 300 cases. However, we are not waiting until all those data are gathered—as we learned with Seshat, gathering data is a neverending story! There are several analysis papers submitted to journals or nearing that state (although don’t expect them to be out soon—academic publishing is glacially slow).

The philosophy of science underlying these empirical tests is retrodiction (an innovation of our project). Let me explain. There are many explanations for why declines and falls of empires and other states happen. One German historian counted more than 200 theories about just one such collaps, the end of the Roman Empire. But we are interested in learning why empires fall in general.

In science, we weigh relative merits of different theories by using scientific prediction. We translate different explanations into specific mathematical models, use the models to make predictions about novel data, and then determine which theory makes better—more accurate—predictions. This is not guaranteed to give us the “one and only true theory,” but it enables us reject theories that make worse predictions in favor of those that make better ones. This is how science advances, one dead theory at a time. And this is why we need CrisisDB.

Note that there is nothing wrong with predicting the past (retrodiction). As long as testing data are not known when the model was built, comparing model predictions with such data is a true test. In the jargon of complexity science, it is an “out-of-sample” prediction.

Nevertheless, there is something particularly satisfying about predicting the future. We have one such successful case, my prediction, made in 2010, about an outbreak of political instability in the USA around 2020. But I will be the first to admit that a single success could just be due to luck. In order to test the theory more thoroughly, we need to see how will predictions about, say, ten countries fare 10 years from now. More importantly, while our theory is clearly yielding valid, and valuable insights, it is equally clear that it’s still quite young, and there is a lot more work needed. Undoubtedly, there will be many ways in which the actual future deviate from theoretical predictions, and our intent is to learn from such failures so that we can develop better models. And also to better define the limits of prediction.

So, this is a call for those interested in joining in on the fun to develop a predictive science of societies. Dan Hoyer and I have just posted a preprint on SocArxiv, Empirically Testing and Refining Structural Demographic Theory: A Methodological Guide, in which we describe the logic and the methodology of the approach in detail. Note that, because in each society the ruling elites are recruited and reproduced in different ways, the general SDT model needs to be tailored to take such details into account.

As we wrote at the end of this article,

we still need to explore how well SDT can predict future crises, how it compares to, or fits together with, other prevailing causal theories about the drivers of societal conflict, and most importantly whether improving our ability to see what lies ahead can also help us to identify ways we can avoid the more disastrous outcomes. The best way to answer these questions is to compile as many studies utilizing SDT to track dynamics of unrest in as many societies as possible, drawn from every corner of the globe and spanning time from the ancient past to the present day.             

This chapter offers a road-map for researchers to take up this call and extend the body of SDT cases. The pay-off of these efforts will be two-fold: firstly, these studies can provide novel insight into periods of social strife for any number of cases that require further understanding; secondly, and more impactfully, compiling these cases will allow us and others to refine the theory and add new analytical techniques. The overall goal of all these efforts is to employ SDT on all current societies, identify “trouble spots” and reveal possible interventions that may be able to head off trouble before it starts, or at least to mitigate the severity of crises that do strike. This is work that we, along with our colleagues at the Seshat Databank, will be engaging in for the foreseeable future. We hope that others will join us. We are always eager to collaborate or consult in any such efforts.

Subscribe
Notify of
guest
20 Comments
Most Voted
Newest Oldest
Inline Feedbacks
View all comments
Sally

Reading your work has made me consider if essentially immigration into western countries (such as Canada, USA, Australia, UK) is essentially one way (major way?) countries such as India, say, are getting their aspirational elites out of their countries and thus avoiding the problems you have been describing that occur from the overproduction of aspirational elites. Would you please consider this and maybe write a blog post on it?

James Jackson

What should be next is modeling the impact of psychopathy on the wealth pump. If the structural-demographic theory already includes psychopathy within the elites, my limited reading has missed it. Ascribing the wealth pump’s ever-increasing volume to “human nature = that’s how people are,” ignores the fact that psychopaths are more frequent in elite professions as well as the likelihood that psychopathy is naturally selected within the elite when certain conditions obtain, e.g., in a long-lasting dynasty. If elite psychopathy is a stronger force later in the immiseration phase, that would be useful to know for purposes of avoiding the worst societal outcomes.

Harrison Koehli’s blog “Political Ponerology” focuses on the origin of evil in political contexts.

Dave Evans

There’s no mention of any interdependency between the MMP, EMP and SFD variables – I assume you have thought about that, as it almost certainly exists? Does it matter?

Dennis

Intra-elite competition occurs in many contexts below the societal level. It occurs in organizations, within social and political movements, in municipal politics, in academic departments, …. I wonder to what extent SDT might apply in situations like these.

Navaneethan Santhanam

I’m not sure I follow this point –

Note that there is nothing wrong with predicting the past (retrodiction). As long as testing data are not known when the model was built, comparing model predictions with such data is a true test.

Let’s consider an example where we use data from a society dating from 1200 to 1400 CE to predict what would happen in 1420. Given that we (the modellers) presumably know what happened to this society in 1420, wouldn’t that bias towards features (in the ML sense of the term) that produce higher accuracies and therefore validate SDT? Meaning, even though the test set is out-of-sample, the predictors can be curated based on what the modeller knows of the test set. This might push the accuracy upwards.

Am I misunderstanding something here? I am yet to read your paper, so it’s possible this concern is addressed there.

Steven Johnson

How is it possible to compensate for the limitation of the sample data to sets already well-defined and continuous? Structural-demographic factors that according to the theory should be part of the model’s retrodiction can be ignored. For example in models of French society, at various times “France” had a significant North/South divide. Are SDT factors modeled including or excluding those limited to Provence etc.? At other times, are SDT factors modeled assuming a de facto continuity between “France” and the lowlands, especially “Belgium?” How are intersections with English SDT factors incorporated in the phase of the model retrodicting the epoch of the so-called Hundred Years’ War?

I see an objection to trying to (in effect) use SDT to model alternative histories, unless you’re writing SF. The value of such a project is doubtful. But it’s not very clear how SDT models exclude exogenous and endogenous factors. To be a science of history, it’s not enough to discover the internal dynamics of states, whether they are the necessary units of analysis or simply the most relevant and manageable. You also need some way to handle the question of why, for instance, the US history doesn’t include SDT factors in Canada and Mexico. Or so it seems to me. Socially, economically not just politically geography changes more rapidly than the physical geology.

And in addition to the geographical issues, there are the temporal issues raised by immigration for the role of the “father-son cycle?” Shouldn’t there be some sort of regularity in the actual working out of the cycle in accordance to immigration/emigration? And significant population declines/rises seem to need to incorporate other factors than the classic SDT cycles. How can you do a SDT model of Ming China without incorporating the effects of American food crops on population, yet how does such a process fit into SDT? How do you incorporate the Price Revolution of the Sixteenth Century into SDT models for the various societies?

Perhaps I just misunderstand, but at this moment SDT seems to be something like a theory of normal development, normal meaning, dominated by endogenous factors, and abnormal development is when outside development like American gold and food crops and immigration/emigration or epidemic population crashes are to be seen as re-set. But in many respects, the truly significant dynamics of history may be those “exogenous” events (as you might call them.) Is the goal to scientifically analyze the forces giving regularity to history, rather than the intrinsically unpredictable?

Loren Petrich

Here’s more detail on my PCA of the data released so far, finding correlations between the various kinds of calamity.

The strongest component, by nearly a factor of 2 in axis length over the others, was essentially what fraction of total calamities each kind of calamity was. So they are all correlated together.

The next two are about 30% longer than the remaining ones, ones which form an approximately smooth curve, and which likely reflect uncorrelated variation.

The first one has in one direction population decline, epidemic, and political fragmentation, and to a lesser extent, downward mobility of the elite and population collapse, and in the opposite direction uprising against the leaders, deposing them, and assassinating them — population weakening vs. rebellion.

The second one has in one direction conquest by foreign powers, fragmentation, and rather weakly, deposing of leaders, and in the other direction population decline, epidemics, uprisings, and civil war — loss of control of territory vs. rebellion and population weakening.

Alan Patrick

I’ve been doing similar modelling for years, how do I get involved?

Alan Patrick

I’ve started collecting data for UK. I had a look at France this last weekend as well, noted a higher propensity for civil action over many decades (centuries!) despite relatively better levels of immiseration, wealth gap and elite conflict vs UK/US.

Alan Patrick

OK, will do that too. There are a lot of European elections coming up next year, the results are going to be interesting I think.

Alan Patrick

Peter – I am finishing up some initial data for France, where do I send it? In summary though, they are doing better than many contemporary societies – but there is more unrest than nearly all. Have you found that different societies start unrest at different levels of stress, or are there any specific / tell-tale drivers to dig more into for this factor?

Last edited 7 months ago by Alan Patrick
Alan Patrick

Done

TJ Butler

Thank you Peter for your great work. I read your “End Times” book and plan to read more of them soon.

I had a few questions about the political instability model, specifically regarding (Median worker salary / GDP per capita) I believe it was.

If I’m understanding right, this is a dimensionless ratio, aka multiplier of the average GDP per citizen compare to a median worker’s wage. I am wondering if it might be better to include the size of the GDP in the equation somehow? I’m thinking for example that in a GDP as large as the USA, the ratio could get quite a bit worse than a country or political structure with a much smaller GDP because dividing up a very large pie can still give people a somewhat livable slice, even if they are very upset at the inequality of the pie’s division. I am thinking that absolute terms might be helpful to include, although it is not everything because we do judge relative wealth and relative inequality very importantly.

Another question I had was using PPP instead of GDP? I notice for example that the PPP of China is currently higher than that of the USA, despite the fact that USA GDP is higher: https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(PPP)

Could that mean that China, despite having a low GDP, uses the GDP efficiently to provide the goods and services needed for its citizens in a way that the higher GDP of the USA fails to do?

Last, I have read a lot of Michael Hudson’s work about the inflation of GDP using debt and other financial measures of wealth that are pretty much fake because financial services like debt are actually a drain on a productive economy, it is just a wealth transfer from those who do not own money to those that do (maybe a source of the wealth pump you describe in your book?). Is there an effort to re-calculate GDP using more material measures of the economy, by excluding debt and financial services for example and focus on food, industrials, healthcare (though insurance might be a confounder)? For example, if workers recieve a high portion of a GDP that is mostly illusory, say bitcoin or access to debt-repayments, but there isn’t a productive economy behind the financial one to back up the ‘wealth’ that is being produced, might that trigger political instability once enough counter-elites and commoners realize that our wealth is fake and there isn’t enough material goods to go around, despite a high GDP calculation?

  1. Home
  2. /
  3. Cliodynamica
  4. /
  5. Regular Posts
  6. /
  7. Structural-Demographic Theory: What’s Next?

© Peter Turchin 2023 All rights reserved

Privacy Policy