Existential risk pessimism and the time of perils (Part 5: An existential risk Kuznets curve?)

In a model of directed technical change … existential risk follows a Kuznets-style inverted U-shape. This suggests we could be living in a unique “time of perils,” having developed technologies advanced enough to threaten our permanent destruction, but not having grown wealthy enough yet to be willing to spend sufficiently on safety.

Leopold Aschenbrenner, “Existential risk and growth“

1. Recap

This is Part 5 in a series based on my paper “Existential risk pessimism and the time of perils“.

Part 1 and Part 2 set up a tension between two claims:

(Existential Risk Pessimism) Per-century existential risk is very high.

(Astronomical Value Thesis) Efforts to mitigate existential risk have astronomically high expected value.

Part 3 introduced a potential solution: the Time of Perils Hypothesis on which risk is high now, but will soon fall to a permanently low level.

Why should we believe the Time of Perils Hypothesis? Part 4 argued against one motivation for the Time of Perils Hypothesis, which appeals to space settlement. in this post, I look at a second justification which appeals to the economic concept of a Kuznets curve.

2. The environmental Kuznets curve

Consider the risk of climate catastrophe. Climate risk increases with growth in consumption, which emits greenhouse gases. Climate risk decreases with spending on climate safety, such as reforestation and other forms of carbon capture.

Economists have noted that two dynamics push towards reduced climate risk in sufficiently wealthy societies. First, the marginal utility of additional consumption decreases, reducing the benefits of fossil fuel use. Second, as society becomes wealthier we have more to lose by destroying our climate. These dynamics exert pressure towards an increase in safety spending relative to consumption.

Some economists think that this dynamic is sufficient to generate an environmental Kuznets curve (Figure 1): an inverse U-shaped relationship between per-capita income and environmental degradation. Societies initially become wealthy by polluting their environments. But past a high threshold of wealth, rational societies should be expected to improve the environment more quickly than they destroy it, due to the diminishing marginal utility of consumption and the increasing importance of climate safety.

Figure 1: The environmental Kuznets curve (Reprinted from Yandle et al. 2002)

Now everyone admits that this dynamic is not fast enough to stop the world from causing irresponsible levels of environmental harm. We’ve already done that. But it may well be enough to prevent the most catastrophic warming scenarios, where 10-20^∘C warming may lead to human extinction or permanent curtailment of human potential.

Could the same dynamic repeat for other risks beyond climate change?

3. An existential risk Kuznets curve

Leopold Aschenbrenner argues that the same dynamic repeats for other existential risks. In Aschenbrenner’s model, society is divided into separate consumption and safety sectors. At time t, the consumption sector produces consumption outputs C_t as a function of the current level of consumption technology A_t, and the labor force producing consumption goods L_ct:

Here is a constant determining the influence of technology on production. 

Similarly, the safety sector produces safety outputs H_t as a function of safety technology B_t and the labor force producing safety outputs L_ht.

As in the environmental case, Aschenbrenner takes existential risk to increase with consumption outputs and decrease with safety outputs. In particular, he assumes:

for constants .

Aschenbrenner proves that under a variety of conditions, optimal resource allocation should lead society to invest quickly enough in safety over consumption to drive existential risk towards zero. Leopold also shows that under a range of assumptions, his model grounds an existential risk Kuznets curve: a U-shaped relationship between time and existential risk (Figure 3). Although existential risk remains high today and may increase for several centuries, eventually the diminishing marginal utility of consumption and the increasing importance of safety should chase risk exponentially towards zero. Until that happens, humanity remains in a Time of Perils, but afterwards, we should expect low levels of post-peril risk continuing indefinitely into the future.

Figure 2: The existential risk Kuznets curve (reprinted from Aschenbrenner 2020)

4. Evaluating the argument

That’s … actually a pretty good argument. I think it is probably the best argument for the Time of Perils Hypothesis on offer. In general, I think that Aschenbrenner (2020) is a very good paper, and if you have the time I would recommend taking a look at it. (I am told by reliable sources that it has a good chance of being accepted into a world-leading economics journal).

However, I don’t think you should believe Aschenbrenner’s argument for the Time of Perils Hypothesis. Here’s why. Aschenbrenner’s model treats consumption as the source of existential risk. But most Pessimists do not think that consumption is even the primary determinant of existential risk. In the special case of climate risk, consumption does indeed drive risk by emitting greenhouse gases and causing other forms of environmental degradation. But Pessimists think most existential risk comes from things like rogue AI and engineered pandemics. These risks aren’t caused by consumption. They’re caused by technological growth. Risks from superintelligence grow with advances in technologies such as machine learning, and bioterrorism risks grow with advances in our capacity to synthesize, analyze and distribute biological materials. So a reduction in existential risk may be largely achieved through slowing growth of technology rather than slowing consumption.

We could revise (3) to let technologies A and B replace consumption outputs C as the main drivers of existential risk. But if we did this, we would lose all of the main theorems of the model, and hence we would not get a Time of Perils Hypothesis of the form we need.

As always, there’s a bit more to say here. Check out the full paper for a complete response to Aschenbrenner. But I think I have said enough for now.

5. Taking stock

As time goes on, the marginal utility of consumption decreases and the value of safety increases. This generates pressure to slow down consumption which generates existential risks. From this, we may well get an environmental Kuznets curve: risk increases initially, as societies grow rich from emissions-heavy consumption, but over time consumption decreases and brings a corresponding decrease in climate risk.

But most risks are not like this. The pessimist thinks that most existential risk is driven by technology, not by consumption. So even if we expect a Kuznets-like structure to risk-generating consumption, such as consumption driven by fossil fuel use, this does not give us any reason to expect a similar structure across other risks.

At least, so say I. But what do you think? Could there be a Kuznets-like structure to most existential risks? Why else might we expect an existential risk Kuznets curve? or are there other arguments against the existential risk Kuznets curve that I have overlooked? Let me know.