The problem is not so much an excess of technology as a lack of wisdom. Carl Sagan put this especially well: “Many of the dangers we face indeed arise from science and technology — but, more fundamentally because we have become powerful without becoming commensurately wise.”
Toby Ord, The precipice
1. Recap
This is Part 6 in a series based on my paper “Existential risk pessimism and the time of perils“.
Part 1 and Part 2 set up a tension between two claims:
(Existential Risk Pessimism) Per-century existential risk is very high.
(Astronomical Value Thesis) Efforts to mitigate existential risk have astronomically high expected value.
Part 3 introduced a potential solution: the Time of Perils Hypothesis on which risk is high now, but will soon fall to a permanently low level.
Why should we believe the Time of Perils Hypothesis? Part 4 argued against one motivation for the Time of Perils Hypothesis, which appeals to space settlement. Part 5 argued against another motivation, which appeals to the concept of an existential risk Kuznets curve.
Today, I want to look at a third argument for the Time of Perils Hypothesis, which appeals to wisdom.
2. The appeal to wisdom
Readers may recall from Part 3 that Carl Sagan took the problem to be that humanity’s technological capabilities are growing far more quickly than our wisdom. Until we gain the wisdom to handle new technologies, Sagan held, we will remain at peril. But once we grow in wisdom, we may become relatively safe.
This line has been taken up by other pessimists. Here is Ord, quoting Sagan:
The problem is not so much an excess of technology as a lack of wisdom. Carl Sagan put this especially well: “Many of the dangers we face indeed arise from science and technology — but, more fundamentally because we have become powerful without becoming commensurately wise.”
Toby Ord, The precipice
Sagan put a sharper edge on the point: “If we continue to accumulate only power and not wisdom, we will surely destroy ourselves”.
3. Fleshing out the argument: Bostrom
The trouble with this argument is that it is thin on details. Neither Sagan nor Ord tells us much about what it means to become wise; why we should expect dramatic future increases in wisdom; and how increased wisdom could lead to a short perilous period followed by a dramatic reduction in post-peril risk.
This will be a recurring theme in many of our discussions. Often it is difficult to evaluate arguments made by effective altruists because the arguments are not made in enough detail to be fully evaluable.
I will do my best to look at some ways we might fill out the appeal to wisdom. None of these ways of filling out the argument will ground a time of perils hypothesis of the right form.
One thing we might do is to point towards promising current trends in reasoning and related areas. In this vein, Nick Bostrom argues that:
An optimist could expect that the `sanity level’ of humanity will rise over the course of this century — that prejudices will (on balance) recede, that insights will accumulate, and that people will become more accustomed to thinking about abstract future probabilities and global risks. With luck, we could see a general uplift of epistemic standards in both individual and collective cognition.
It may not be unreasonable to hope for the uplift in epistemic standards that Bostrom describes. But the problem is that these and similar trends come nowhere close to grounding the manyfold reduction in post-peril risk that the pessimist needs. It is not so implausible to think that a reduction in prejudice or a rise in future-oriented thinking might lead humanity to take existential risks more seriously. But these are moderate and familiar trends, and on their own they are highly unlikely to be strong enough to take us out of the time of perils. Indeed, it is perhaps for this reason that Bostrom hedges his appeal to increased sanity by attributing this thought to an optimist, and does not saddle even the optimist with the claim that increased sanity alone will be powerful enough to take us out of the time of perils.
4. Fleshing out the argument: Ord
Ord strengthens Bostrom’s argument by appealing to civilizational virtues. Ord argues that we can treat humanity as a collective agent currently in its infancy. Humanity will grow in wisdom and reach adulthood by acquiring civilizational virtues such as prudence, patience, self-discipline, compassion, stewardship, gratitude, fairness, unity and solidarity. As humanity grows in virtue and hence in wisdom, humanity will act to substantially reduce existential risk, bringing an end to the time of perils. This view strengthens Bostrom’s argument by divorcing the concept of civilizational virtue from the virtues of individual humans. Because collective agents can have properties that their members lack, Ord holds, we may well hope that humanity as a whole will become substantially more patient or compassionate in the coming centuries, even if we doubt that the average human will grow in patience or compassion during this time.
At this point, the most helpful response would be to ask Ord for more details. We are not told much about why we should expect humanity to grow in virtue or how this growth could lead to a quick and substantial drop in existential risk. Without these details, it is hard to place much stock in the appeal to civilizational virtue, and it is also hard to know what to say in response. But we may get some handle on the prospects for Ord’s argument by thinking through some particular civilizational virtues.
Consider unity. Humanity becomes more unified as we build forms of international cooperation such as the United Nations, or international trade and climate agreements. Becoming unified ensures that humanity acts with a view to the interests of humanity as a whole, instead of each nation pursuing its own interest. This would increase pressure to address existential risks, since humanity would be concerned with the security of all humans and their descendants, instead of the security of a single nation and its descendants. But increased unity could only do so much to drive down existential risk. At the time of writing, many nations boast at least 5% of the world’s population within their own borders, and at least two contain over 15% of the world’s population. Unifying these nations into a single actor would increase their constituencies by a factor of no more than twenty, and hence in the best case it could not lead to more than a twentyfold increase in the importance of existential risk reduction. While that is nothing to sneeze at, it remains orders of magnitude lower than what the pessimist needs.
Next, consider patience. Humanity becomes more patient by adopting systems of government which better represent the interests of future people. Many political systems give inadequate weight to future generations, for example by instituting short election cycles which force politicians to deliver immediate results, or by giving no formal voice to unborn generations. These problems can, and have been partially addressed by mechanisms such as citizens’ assemblies elected to represent future generations, or government commissioners tasked with protecting future generations.
There is no doubt that institutional changes can increase the patience of political systems. For example, many of these changes have led to increased emphasis on mitigating climate risks to future generations. But the pessimist needs a mechanism by which a largely impatient group of humans could together become patient enough to make great sacrifices directed at reducing existential risks, based largely on the threat that those risks pose to far-future generations. It is hard to see how the features of current, or feasible near-term political changes could increase patience on such a scale. Indeed, one might reasonably expect constituents to reject any system of government that acted with substantially more patience than the average voter.
Now it could well be that there is a plausible story about how humanity might acquire some particular virtue that is strong enough to end the time of perils. Or perhaps the combination of many different virtues will be enough to tip the scales. But we have not seen a detailed argument for either of these conclusions, and we saw above that making the argument out is no easy task. So we cannot yet ground the time of perils hypothesis in the hope that humanity will increase in wisdom.
5. A race: Technology against wisdom
Even if the story about increasing wisdom were to work, it still might not be strong enough to do the trick. Existential risk pessimists also tell a story on which technological developments are inherently dangerous. As humans gain increasingly sophisticated technologies, we are at an ever-higher risk of destroying ourselves.
Suppose you accept that humans will grow in future wisdom, and that wisdom tends to drag down the risk of harmful misuse of technology. Then we have a race dynamic: technology increases risk, and wisdom decreases risk. Which is faster?
Naively, we would expect technological growth to be faster. After all, we are at a period of unprecedentedly high technological growth in the history of our species, and we are likely to continue in such a period for some time. Here is a chart of the history of major technological innovations over time:
This chart suggests that we are in a period of unprecedentedly fast technological change.
By contrast, the human brain changes slowly. Evolution is a long process, so we are unlikely to change much as a species over the next few centuries. This suggests that an appeal to individual growth in wisdom is doomed from the start: individual wisdom grows much slower than the rate of current technological growth.
With this in mind, we can see that Ord must have been right to appeal to collective wisdom growth, for there is no firm bound on the speed with which our societal institutions can change, binding us together to be collectively wiser. Perhaps Ord envisions a chart for the growth of recent or near-future collective wisdom that is as sharply peaked as the recent growth of technological capacities? (Actually, it would have to be much more sharply peaked to bring an end to the time of perils).
Perhaps. But this is a very high bar to clear, and it would take an awful lot of argument to convince most readers of such a conclusion.
At least, so say I. What say you?
Leave a Reply