One of the biggest changes in philanthropy and the international development community that has taken place … is the increased focus of independent organizations on measuring the impact of particular interventions to help people in extreme poverty, and in assessing the effectiveness of the organizations providing the most successful interventions. GiveWell … was the pioneer here, setting new standards for rigorous evaluation of the work of charities.Peter Singer, The life you can save
This is Part 5 in my series on epistemics: practices that shape knowledge, belief and opinion within a community. In this series, I focus on areas where community epistemics could be productively improved.
Part 1 introduced the series and briefly discussed the role of funding, publication practices, expertise and deference within the effective altruist ecosystem.
Part 2 discussed the role of examples within discourse by effective altruists, focusing on the cases of Aum Shinrikyo and the Biological Weapons Convention.
Part 3 looked at the role of peer review within the effective altruism movement.
Part 4 looked at the declining role of cost-effectiveness analysis within the effective altruism movement. Today’s post continues this discussion by highlighting four costs of turning away from cost-effectiveness analysis.
2. Practical cost
Right now, GiveWell estimates that about $5,000 could save a life today. The person saved would be a member of a significantly disadvantaged community, who otherwise would die because they had the misfortune to be born or to live in the wrong place at the wrong time. These claims are based on rigorous cost-effectiveness analyses. I believe them, and I hope you do too.
This puts a clear price on wasteful spending. Each transatlantic business class flight could instead have saved a person’s life. Each independent researcher publishing forum posts costs, in many cases, more than ten lives per year. Each regrantor given a million-dollar budget to spend with limited oversight had better save, in expectation, at least two hundred lives, or they will have killed more people than all but the worst murderers.
Consider, for example, a question posed by Ben Garfinkel in 2018: “How sure are we about this AI stuff?” Perhaps effective altruists are right. Perhaps transformative developments in artificial intelligence are just around the corner; perhaps these developments pose significant levels of near-term existential risk to humanity; perhaps humanity will not respond quickly enough to mitigate risks without being nudged to do so; and perhaps a team of young, motivated and intelligent altruists can put a nontrivial dent in those risks. If that is true, then effective altruists have done very well for themselves, and I myself may have much to apologize for.
But what if effective altruists aren’t right? Many scientists, artificial intelligence researchers, and policymakers think that these risks are negligible to non-existent, and that the AI safety research being funded by effective altruists is often of low quality, and unlikely to lower risk. They think it is impossible to put a rigorous value on AI safety investments by effective altruists, but that the value is probably near-zero, if not negative. If that is right, then longtermists may have a good deal to atone for.
How much would they need to atone? Here is one example. Over the years 2022 and 2023, Open Philanthropy reduced its planned giving to GiveWell by $350 million dollars [Edit: $400 million. Thanks, Evelyn!]. That money could have saved 70,000 members of disadvantaged communities from premature death, or helped to lift millions out of poverty. If longtermists are, in fact, wasting money on poorly justified projects, they will have a good number of people to answer to. Probably, they will have to answer to hundreds of thousands, if not millions, who died when they could have lived. At its peak valuation of over $40 billion dollars, the effective altruism movement could have saved about 8 million lives according to GiveWell estimates.
3. Symbolic cost
Even if free-spending effective altruist projects are in fact an effective way to promote value, it remains true that they look bad. Moments such as the Wytham Abbey purchase and the luxurious flights and living conditions of visitors to the Bahamas struck even many core effective altruists the wrong way, and many outside observers were appalled. It doesn’t help that many of the recipients of this money were privileged and successful members of wealthy societies, or that effective altruist money had previously been funneled almost exclusively towards some of the most disadvantaged groups on earth.
For example, George Rosenfeld wrote in mid-2022 that:
Over the past few months, I’ve heard critical comments about a range of spending decisions. Several people asked me whether it was really a good use of EA money to pay for my transatlantic flights for EAG. Others challenged whether EAs seriously claim that the most effective way to spend money is to send privileged university students to an AirBNB for the weekend. And that’s before they hear about the Bahamas visitor programme … I am not contesting here whether these programmes are worth the money. My own view is that most of them probably are and I try to lay this out to those who ask. But it is the perceptions which I find most concerning: many people see the current state of the movement and intuitively conclude that lots of EA spending is not only wasteful but also self-serving, straying far from what you’d expect the principles of an ‘effective altruism’ movement to be. Given the optics issues which have hindered the progress of EA in the past, we should be wary of this dynamic. Importantly, I’ve heard this claim not only from critics of EA, but also from committed group members and an aligned student who might otherwise be more involved.
I have discussed perceptions of wasteful spending on this blog in several places, including Part 6 of my series on billionaire philanthropy, and Part 4 of my series on epistemics, As expected, a number of commentators shared Rosenfeld’s reaction, and I have personally heard the same from many effective altruists as well as a growing number of others.
Here is one thing that personally bothers me about the optics of free-spending EA. As effective altruists became wealthier, the framing went from a rigorous emphasis on cost-effectiveness to the question of `How are we going to spend all of this money’? Six-figure grants were almost too small to bother with: megaprojects and massively scalable enterprises were the norm. Grantmakers continually bemoaned the fact that they couldn’t find projects able to absorb the available capital.
Only, there were of course a large number of known projects that could absorb the entirety of available funding. These are short-termist projects that had an excellent track record of reducing poverty and the global burden of disease, as well as improving animal welfare on a massive scale. Effective altruists had, for a long time, been funding these projects. The framing in terms of the question, `How are we going to spend all of this money?’ was only made possible because the question ruled out, by assumption, the idea of using the money to fund established, scalable, measurable and cost-effective assumptions that would benefit some of the most disadvantaged groups on earth. Rosenfeld continues:
One especially problematic framing concerns the apparent discrepancy between longtermist and neartermist funding. Many people find it understandably confusing to hear that ‘EA currently has more money than it can spend effectively’ whilst also noticing that problems like malaria and extreme poverty still exist, especially given how much EA focuses on how cheap it is to save a life and how important it is to practise what we preach.
When grantmakers flush with cash find themselves struggling to find a project, any project, which can absorb their money, then walk straight by some of the most disadvantaged groups on earth before handing out large sums of money in a freewheeling way to some of the most advantaged groups on earth, this behavior leaves a bad taste in many people’s mouth.
Maybe that’s because the behavior is wrong. I tend to think it is. But even if it is not wrong in itself, it certainly looks wrong. And looks are important.
For example, my colleague Andreas Mogensen confronted the following question in a recent paper: why is it wrong for societies to preferentially allocate healthcare resources to the most productive members of society, rather than distributing healthcare resources largely according to need? On a consequentialist approach, this may seem like a mistake, since some people do give back more to the community than others. Yet most of us recoil at the prospect of, for example, a technology mogul having a team of thirty personal doctors, devoted only to keeping him alive for a few extra years. Why is this?
Mogensen suggests that the problem is semiotic: it communicates something harmful by signaling that those who produce less are less deserving of a fundamental good, healthcare, which is bound up with the idea of their status as a citizen. By allocating healthcare according to productivity rather than need, we would communicate that less-productive members of society are second-class citizens. Mogensen writes:
The significance attached to medical care in societies where health is a socially recognized need interacts with the prevalence of meritocratic ideals. The result is that prioritizing treatment in this way represents some people as more deserving of a good that is bound up with each person’s standing as a citizen, thereby expressing something contrary to the ideal of equal citizenship.
In the case of healthcare allocation, semiotic objections have strength. It is, most people think, wrong to give much better healthcare to the most productive members of society. And the explanation for this wrongness is, plausibly, semiotic: it looks bad, and those looks cause lasting damage to people’s standing in the community.
In the same way, when foundations flush with cash walk straight by the needy, sinking money wildly into speculative far-future investments that most people do not want, and sinking an increasing fraction of those investments back into the wealthy communities that administer them, this behavior does not look good. It does not look good because it communicates symbolic disrespect for disadvantaged populations by denying their claims to goods such as healthcare, housing, and safe working conditions, goods which are fundamentally bound up with their status as citizens of a global community.
I think that whatever strength we accord to the semiotic objection to rationing healthcare according to productivity within a society, we should accord at least as much strength, and probably considerably more, to the semiotic objection to free-spending longtermist philanthropy. That’s not to say that semiotic considerations must take precedence over all others. But neither should they be ignored.
4. Foregone leadership
Effective altruists have changed many areas of the philanthropic landscape for the better. One of the best things they did for philanthropy was to pioneer the use and spread of up-to-date cost-effectiveness methods backed by high-quality data. Effective altruists did not merely use cost-effectiveness analysis themselves, but also through their example and spending power encouraged charities to produce better data, and philanthropists to act on the basis of better analyses.
We saw in the previous post in this series that Peter Singer expresses justifiable pride in this achievement in his book, The life you can save:
One of the biggest changes in philanthropy and the international development community that has taken place since I first wrote this book is the increased focus of independent organizations on measuring the impact of particular interventions to help people in extreme poverty, and in assessing the effectiveness of the organizations providing the most successful interventions. GiveWell … was the pioneer here, setting new standards for rigorous evaluation of the work of charities.
It is hard to overstate the importance of GiveWell’s leadership in spreading rigorous cost-effectiveness analysis.
When effective altruists abandon cost-effectiveness analysis, they lose their ability to lead. They can no longer in good faith, or with reliable effect ask others to adopt rigorous standards of cost-effectiveness analysis if they are no longer willing to adhere to these standards themselves.
Perhaps more damaging, effective altruists may lead others in the opposite direction. If even the movement behind GiveWell cannot adhere to such a rigorous regime, then others might rightly wonder whether cost-effectiveness analysis is all it is cracked up to be. Why shouldn’t they just fund other interventions on the basis of much less substantial analyses?
This would be a real shame. Effective altruists have led well in this area, and it would hurt many to see their leadership erode or backfire.
5. Philosophical cost
Early effective altruists were ruthless in their adherence to rigorous cost-benefit analysis. This spawned the institutional critique, which complained that a variety of systematic reforms were not being considered, let alone funded, because they could not be given a suitably rigorous cost-effectiveness analysis.
Effective altruists replied by doubling down on the requirement to conduct cost-effectiveness analyses. If systemic reforms could not be analyzed through standard forms of cost-effectiveness analysis, then they would not be considered.
Now effective altruists consider and fund a wide variety of speculative interventions that are much harder to evaluate than any of the institutional reforms which they previously refused to consider. We might not have airtight cost-effectiveness analyses for ongoing projects such as prison reform, the Movement for Black Lives, feminist activism, and other justice-focused movements. But we do know a good deal about how these movements have proceeded, what kinds of outcomes they might achieve, and what it might take to achieve those outcomes. We’re also, I hope, fairly sure that the outcomes are good.
By contrast, we know much less about the outcomes that our actions are likely to produce in the far future, and the likelihood of their producing these outcomes. Some have gone so far as to suggest that we are clueless about the long-term future. This contrast raises two philosophical implications.
First, longtermists need a new response to the institutional critique. They can’t say that interventions will be ruled out of consideration unless they can be supported by an especially rigorous cost-effectiveness analysis, because that insistence would also rule out many longtermist interventions that are currently being considered and funded.
Finding another response to the institutional critique may be possible, but it is not so easy. Is the problem that the social movements in question are ineffective? That’s a hard sell: many of these movements have been among the most successful liberatory forces in recent history. Is the problem that funding social movements in wealthy countries is very expensive? Fine. Many of these causes are even more dire in the developing world. Is the problem that effective altruism should remain apolitical? That ship sailed a long time ago. Effective altruism is now so political that they were bankrolling a congressional campaign last year (and executives at FTX were funding a good deal more besides).
Could the problem be that political action on one side of these issues would be cancelled out by other effective altruists batting for the opposite team? Well, yes, that could be a problem. And relatedly, the problem could just be that effective altruists don’t want to see the proposed reforms come to pass. But do effective altruists really want to say this? That would not be a good look.
Perhaps there are other ways for effective altruists to respond to the institutional critique. But their most historically successful response, an insistence on rigorous cost-effectiveness analysis, is now off the table.
A second problem is that effective altruists’ change of attitudes towards cost-effectiveness analysis may look unprincipled. Early effective altruists were quite insistent that cost-effectiveness analysis should be preserved, because methodological rigor is essential to identifying effective charities. Now, effective altruists have done an about-face and endorsed some of the most speculative and least rigorously evaluable interventions proposed by any group on earth. What explains this change of attitudes?
The change had better not be explicable in sociological terms. That is, it had better not turn out that effective altruists wanted to defend short-termist interventions in global health and development as well as animal welfare, of a type that tends to perform well on detailed cost-effectiveness analyses. Therefore, they settled on rigorous cost-effectiveness analysis because it supported their favored views. A few years later, effective altruists wanted to fund speculative longshots aimed at improving the far future. They realized that these longshots cannot possibly be assessed using traditional forms of cost-effectiveness analysis, so they decided that cost-effectiveness analysis was not such a good thing after all.
Is this what happened? I’m not sure I would go out and say that it is. But if this is not the explanation, what else could explain such a sharp methodological change? Did effective altruists discover some powerful new arguments against cost-effectiveness analysis that they were previously unaware of? Did they simply re-evaluate the strength of arguments against cost-effectiveness analysis that they had previously rejected. Perhaps. It would be good to hear more about what these arguments are, and where and when they became influential. Otherwise, critics may have some grounds to suspect the explanation for effective altruists’ changing attitudes towards cost-effectiveness analysis is sociological rather than philosophical. And that would not be good.
6. Taking stock
Part 4 of this series discussed the declining role of cost-effectiveness analysis within the effective altruism movement. We saw that the rise of longtermism and the increasing availability of funding led to a shift away from the movement’s initial focus on rigorous cost-effectiveness analysis towards a variety of less demanding forms of scrutiny.
Today’s post discussed some of the costs that effective altruists face when they turn away from cost-effectiveness analysis. The first cost is practical: without rigorous analysis, effective altruists run a heightened risk of funding ineffective causes. The opportunity cost of this mistake may run to the hundreds of thousands, or even millions of lives – a hefty price.
The second cost is symbolic: wasteful spending communicates symbolic disrespect for disadvantaged populations by denying their claims to goods such as healthcare, housing, and safe working conditions, and turning away from rigorous forms of analysis that would incorporate these claims.
The third cost involves foregone leadership: one of the most impactful things that effective altruists did for global philanthropy was to use their size and influence to pressure charities and philanthropists alike to invest in better data collection and analysis. When effective altruists turn away from cost-effectiveness analysis, they not only lose their ability to lead others, but may in fact actively discourage others by signaling that rigorous adherence to cost-effectiveness methods cannot be maintained.
A final cost is philosophical. Once effective altruists move beyond the initial insistence on rigorous cost-effectiveness analysis, they will need a new response to the institutional critique beyond the classic response that the effects of institutional change efforts are insufficiently measurable. And they will need to supply a principled, non-sociological explanation for their sudden abandonment of cost-effectiveness methods.
Might these costs be worth paying? Perhaps they are, and perhaps they are not. But they are undeniably costs, and they should be internalized, analyzed, and acknowledged as such.