Publications

Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities. However, benchmarks are not keeping pace in difficulty: LLMs now achieve more than 90% accuracy on popular benchmarks such as Measuring Massive Multitask Language Understanding1, limiting informed measurement of state-of-the-art LLM capabilities. Here, in response, we introduce Humanity’s Last Exam (HLE), a multi-modal benchmark at the frontier of human knowledge, designed to be an expert-level closed-ended academic benchmark with broad subject coverage. HLE consists of 2,500 questions across dozens of subjects, including mathematics, humanities and the natural sciences. HLE is developed globally by subject-matter experts and consists of multiple-choice and short-answer questions suitable for automated grading. Each question has a known solution that is unambiguous and easily verifiable but cannot be quickly answered by internet retrieval. State-of-the-art LLMs demonstrate low accuracy and calibration on HLE, highlighting a marked gap between current LLM capabilities and the expert human frontier on closed-ended academic questions. To inform research and policymaking upon a clear understanding of model capabilities, we publicly release HLE at https://lastexam.ai.

The scientific understanding of the complex dynamics of global history – from the rise and spread of states to their declines and falls, from their peaceful interactions with economic or diplomatic exchanges to violent confrontations – requires, at its core, a consistent and explicit encoding of historical political entities, their locations, extents and durations. Numerous attempts have been made to produce digital geographical compendia of polities with different time depths and resolutions. Most have been limited in scope and many of the more comprehensive geospatial datasets must either be licensed or are stored in proprietary formats, making access for scholarly analysis difficult. To address these issues we have developed Cliopatria, a comprehensive open-source geospatial dataset of worldwide states from 3400BCE to 2024CE. Presently it comprises over 1600 political entities sampled at varying timesteps and spatial scales. Here, we discuss its construction, its scope, and its current limitations.

Soil fertility depletion presents a negative feedback mechanism that could have impacted early adopters of agriculture. We consider whether such feedback can lead to population cycles among early agriculturalists, such as the boom-and-bust patterns suggested by an increasing amount of evidence for Neolithic Europe. Using general mathematical arguments, we show that this is unlikely, due to the interplay of two factors. First, there is an important mathematical difference between biotic (i.e., logistic) and abiotic resource replenishment; soil nutrients are better modeled by the abiotic case, which leads to more stable dynamics. Second, under realistic conditions, the resource replenishment process operates on fast time scales compared to attainable population growth rates, reinforcing the tendency towards stable dynamics. Both these factors are relevant for early agricultural societies and imply that nutrient depletion is likely not the main contributing factor to boom-and-bust cycles observed in the archaeological record.

The impact of inter-group conflict on population dynamics has long been debated, especially for prehistoric and non-state societies. In this work, we consider that beyond direct battle casualties, conflicts can also create a ‘landscape of fear’ in which many non-combatants near theatres of conflict abandon their homes and migrate away. This process causes population decline in the abandoned regions and increased stress on local resources in better-protected areas that are targeted by refugees. By applying analytical and computational modelling, we demonstrate that these indirect effects of conflict are sufficient to produce substantial, long-term population boom-and-bust patterns in non-state societies, such as the case of Mid-Holocene Europe. We also demonstrate that greater availability of defensible locations act to protect and maintain the supply of combatants, increasing the permanence of the landscape of fear and the likelihood of endemic warfare.

Climate variability and natural hazards like floods and earthquakes can act as environmental shocks or socioecological stressors leading to instability and suffering throughout human history. Yet, societies experience a wide range of outcomes when facing such challenges: some suffer from social unrest, civil violence or complete collapse; others prove more resilient and maintain key social functions. We currently lack a clear, generally agreed-upon conceptual framework and evidentiary base to explore what causes these divergent outcomes. Here, we discuss efforts to develop such a framework through the Crisis Database (CrisisDB) programme. We illustrate that the impact of environmental stressors is mediated through extant cultural, political and economic structures that evolve over extended timescales (decades to centuries). These structures can generate high resilience to major shocks, facilitate positive adaptation, or, alternatively, undermine collective action and lead to unrest, violence and even societal collapse. By exposing the ways that different societies have reacted to crises over their lifetime, this framework can help identify the factors and complex social–ecological interactions that either bolster or undermine resilience to contemporary climate shocks.

This paper analyzes the collapse of the Qing dynasty (1644–1912) through the lens of the Structural Demographic Theory (SDT), a general framework for understanding the drivers of socio-political instability in state-level societies. Although a number of competing ideas for the collapse have been proposed, none provide a comprehensive explanation that incorporates the interaction of all the multiple drivers involved. We argue that the four-fold population explosion peaking in the 19th century, the growing competition for a stagnant number of elite positions, and increasing state fiscal stress combined to produce an increasingly disgruntled populace and elite, leading to significant internal rebellions. We find that while neither the ecological disasters nor the foreign incursions during the 19th century were sufficient on their own to bring down the Qing, when coupled with the rising internal socio-political stresses, they produced a rapid succession of triggering events that culminated in the Qing collapse.

Archaeological evidence suggests that the population dynamics of Mid-Holocene (Late Mesolithic to Initial Bronze Age, ca. 7000-3000 BCE) Europe are characterized by recurrent booms and busts of regional settlement and occupation density. These boom-bust patterns are documented in the temporal distribution of 14C dates and in archaeological settlement data from regional studies. We test two competing hypotheses attempting to explain these dynamics: climate forcing and social dynamics leading to inter-group conflict. Using the framework of spatially-explicit agent-based models, we translated these hypotheses into a suite of explicit computational models, derived quantitative predictions for population fluctuations, and compared these predictions to data. We demonstrate that climate variation during the European Mid-Holocene is unable to explain the quantitative features (average periodicities and amplitudes) of observed boom-bust dynamics. In contrast, scenarios with social dynamics encompassing density-dependent conflict produce population patterns with time scales and amplitudes similar to those observed in the data. These results suggest that social processes, including violent conflict, played a crucial role in the shaping of population dynamics of European Mid-Holocene societies.

The causes, consequences, and timing of the rise of moralizing religions in world history have been the focus of intense debate. Progress has been limited by the availability of quantitative data to test competing theories, by divergent ideas regarding both predictor and outcomes variables, and by differences of opinion over methodology. To address all these problems, we utilize Seshat: Global History Databank, a large storehouse of information designed to test theories concerning the evolutionary drivers of social complexity. In addition to the Big Gods hypothesis, which proposes that moralizing religion contributed to the success of increasingly large-scale complex societies, we consider the role of warfare, animal husbandry, and agricultural productivity in the rise of moralizing religions. Using a broad range of new measures of belief in moralizing supernatural punishment, we find strong support for previous research showing that such beliefs did not drive the rise of social complexity. By contrast, our analyses indicate that intergroup warfare, supported by resource availability, played a major role in the evolution of both social complexity and moralizing religions. Thus, the correlation between social complexity and moralizing religion seems to result from shared evolutionary drivers, rather than from direct causal relationships between these two variables.

Our target article empirically tested the Big Gods Hypothesis which proposes that beliefs in moralizing supernatural punishment (MSP) contributed to the evolution of socio-political complexity (SPC) in world history. We tested this hypothesis using a suite of measures of MSP, SPC, and other potential evolutionary drivers coded in Seshat: Global History Databank. Our analyses indicate that intensity of warfare and productivity of agriculture were major drivers in the evolution of both SPC and MSP. The correlation between social complexity and moralizing religion resulted from shared evolutionary drivers, rather than from direct causal relationships between these two variables. Most commentaries on the target article broadly accept our conclusions, but some argue that alternative measures might be used in future studies before the Big Gods Hypothesis can be conclusively rejected. In this response, we argue that while some of these alternative measures should be developed, they are closely related to the ones we have already adopted. Thus, it seems unlikely that further research will give rise to substantially different outcomes. A particularly fruitful aspect of the discussion is that it illustrates both the pitfalls and productive affordances of transdisciplinary research that seeks to bridge the “two cultures” of the humanities and sciences.

During the Holocene, the scale and complexity of human societies increased markedly. Generations of scholars have proposed different theories explaining this expansion, which range from broadly functionalist explanations, focusing on the provision of public goods, to conflict theories, emphasizing the role of class struggle or warfare. To quantitatively test these theories, we develop a general dynamical model based on the theoretical framework of cultural macroevolution. Using this model and Seshat: Global History Databank, we test 17 potential predictor variables proxying mechanisms suggested by major theories of sociopolitical complexity (and >100,000 combinations of these predictors). The best-supported model indicates a strong causal role played by a combination of increasing agricultural productivity and invention/adoption of military technologies (most notably, iron weapons and cavalry in the first millennium BCE).

Evolutionary scientists studying social and cultural evolution have proposed a multitude of mechanisms by which cultural change can be effected. In this article we discuss two influential ideas from the theory of biological evolution that can inform this debate: the contrast between the micro- and macro-evolution, and the distinction between the tempo and mode of evolution. We add the empirical depth to these ideas by summarizing recent results from the analyses of data on past societies in Seshat: Global History Databank. Our review of these results suggests that the tempo (rates of change, including their acceleration and deceleration) of cultural macroevolution is characterized by periods of apparent stasis interspersed by rapid change. Furthermore, when we focus on large-scale changes in cultural traits of whole groups, the most important macroevolutionary mode involves inter-polity interactions, including competition and warfare, but also cultural exchange and selective imitation; mechanisms that are key components of cultural multilevel selection (CMLS) theory.

What have been the causes and consequences of technological evolution in world history? In particular, what propels innovation and diffusion of military technologies, details of which are comparatively well preserved and which are often seen as drivers of broad socio-cultural processes? Here we analyze the evolution of key military technologies in a sample of pre-industrial societies world-wide covering almost 10,000 years of history using Seshat: Global History Databank. We empirically test previously speculative theories that proposed world population size, connectivity between geographical areas of innovation and adoption, and critical enabling technological advances, such as iron metallurgy and horse riding, as central drivers of military technological evolution. We find that all of these factors are strong predictors of change in military technology, whereas state-level factors such as polity population, territorial size, or governance sophistication play no major role. We discuss how our approach can be extended to explore technological change more generally, and how our results carry important ramifications for understanding major drivers of evolution of social complexity.

This Retake article presents a corrected and extended version of a Letter published in Nature (Whitehouse et al., 2019) which set out to test the Big Gods hypothesis proposing that beliefs in moralizing punitive deities drove the evolution of sociopolitical complexity in world history. The Letter was retracted by the authors in response to a critique by Beheim et al. (2021). Correction of errors in the coding and analysis of missing data to address this critique does not, however, significantly change the main findings of the original Nature Letter. We report the results of a major reanalysis of Seshat data following expansion of the codebook and database and substantial improvements to our data management methods. We also employ a more direct statistical methodology to test theories of evolutionary causality. Together, these results show a compellingly convergent picture, confirming the headline finding of the original Letter in Nature, which shows that the largest increases in social complexity do indeed precede Big Gods in world history and that Big Gods did not contribute to the evolution of sociopolitical complexity as predicted by the Big Gods hypothesis.

This article reports the results of a collaborative effort to estimate agricultural productivities in past societies using Seshat: Global History Databank. We focus on 30 Natural Geographic Areas (NGAs) distributed over 10 major world regions (Europe, Africa, Southwest Asia, South Asia, Southeast Asia, East Asia, Central Eurasia, North America, South America, and Oceania). The conceptual framework that we use to obtain these estimates combines the influences of the production technologies (and how they change with time), climate change, and effects of artificial selection into a Relative Yield Coefficient, indicating how agricultural productivity changed over time in each NGA between the Neolithic and the 20th century. We then use estimates of historical yield in each NGA to translate the Relative Yield Coefficient into an Estimated Yield (tonnes per hectare per year) trajectory. We tested the proposed methodology in two ways. For eight NGAs, in which we had more than one historical yield estimate, we used the earliest estimate to anchor the trajectory and compared the ensuing trajectory to the remaining estimates. We also compared the end points of the estimated NGA trajectories to the earliest (the 1960s decade) FAO data on crop productivities in the modern countries encompassing Seshat NGAs. We discuss the benefits of this methodology over previous efforts to estimate agricultural productivities in world history.

This article is a response to Slingerland et al. who criticize the quality of the data from Seshat: Global History Databank utilized in our Nature paper entitled “Complex Societies Precede Moralizing Gods throughout World History”. Their critique centres around the roles played by research assistants and experts in procuring and curating data, periodization structure, and so-called “data pasting” and “data filling”. We show that these criticisms are based on misunderstandings or misrepresentations of the methods used by Seshat researchers. Overall, Slingerland et al.’s critique (which is crosslinked online here) does not call into question any of our main findings, but it does highlight various shortcomings of Slingerland et al.’s database project. Our collective efforts to code and quantify features of global history hold out the promise of a new era in the study of global history but only if critique can be conducted constructively in good faith and both the benefits and the pitfalls of open science fully recognized.

This article introduces the Seshat: Global History Databank, its potential, and its methodology. Seshat is a databank containing vast amounts of quantitative data buttressed by qualitative nuance for a large sample of historical and archaeological polities. The sample is global in scope and covers the period from the Neolithic Revolution to the Industrial Revolution. Seshat allows scholars to capture dynamic processes and to test theories about the co-evolution (or not) of social scale and complexity, agriculture, warfare, religion, and any number of such Big Questions. Seshat is rapidly becoming a massive resource for innovative cross-cultural and cross-disciplinary research. Seshat is part of a growing trend to use comparative historical data on a large scale and contributes as such to a growing consilience between the humanities and social sciences. Seshat is underpinned by a robust and transparent workflow to ensure the ever growing dataset is of high quality.

This article revisits the prediction, made in 2010, that the 2010–2020 decade would likely be a period of growing instability in the United States and Western Europe Turchin P. 2018. This prediction was based on a computational model that quantified in the USA such structural-demographic forces for instability as popular immiseration, intraelite competition, and state weakness prior to 2010. Using these trends as inputs, the model calculated and projected forward in time the Political Stress Indicator, which in the past was strongly correlated with socio-political instability. Ortmans et al. Turchin P. 2010 conducted a similar structural-demographic study for the United Kingdom. Here we use the Cross-National Time-Series Data Archive for the US, UK, and several major Western European countries to assess these structural-demographic predictions. We find that such measures of socio-political instability as anti-government demonstrations and riots increased dramatically during the 2010–2020 decade in all of these countries.

Understanding why large, complex human societies have emerged and persisted more readily in certain regions of the world than others is an issue of long-standing debate. Here, we systematically test different hypotheses involving the social and ecological factors that may ultimately promote or inhibit the formation of large, complex human societies. We employ spatially explicit statistical analyses using data on the geographical and temporal distribution of the largest human groups over a 3000-year period of history. The results support the predictions of two complementary hypotheses, indicating that large-scale societies developed more commonly in regions where (i) agriculture has been practiced for longer (thus providing more time for the norms and institutions that facilitate large-scale organisation to emerge), and (ii) warfare was more intense (as proxied by distance from the Eurasian steppe), thus creating a stronger selection pressure for societies to scale up. We found no support for the influential idea that large-scale societies were more common in those regions naturally endowed with a higher potential for productive agriculture. Our study highlights how modern cultural evolutionary theory can be used to organise and synthesise alternative hypotheses and shed light on the ways ecological and social processes have interacted to shape the complex social world we live in today.