Suppose that Alice and Bob have different beliefs about existential risk over the next 50 years. While Alice thinks it comes overwhelmingly from a single threat, Bob views it as a disjunction of four independent threats.
Alice:
Threat A: 20%
Bob:
Threat A: 6%
Threat B: 6%
Threat C: 5%
Threat D: 5%
Combined risk (given independence): ~20%
Setting aside your beliefs about actual threats such as AI misalignment or nuclear war, who looks more reasonable to you?
I think many people would say Bob. While Alice’s focus on a single big threat can come across as fanatical, Bob’s broader portfolio of lower credences has a more balanced and humble look.
But I take the opposite view. Serious existential threats seem to arise rarely: it’s debatable whether we even have one yet. This makes it decidedly unlikely that we have evidence of four near-term risks of 5 per cent or more. A single big risk is much more credible (if still unlikely), given the plausible hypothesis that the risk distribution is heavy-tailed.
Existential risk communication has often given comparable attention to several unrelated threats, creating the impression that they pose risks of roughly the same order. But if the near-term risk is indeed high, it’s more likely to be concentrated in one or two threats – AI and possibly synthetic biology. So it’s better to emphasise those threats than to use broader framings that make existential risk seem more disjunctive than it is.
I get your argument for why we should not expect independent multiple threats, but what if we have one underlying threat that can manifest in multiple ways say the underlying threat is technological progress resulting in ways to destroy us and that can manifest in multiple ways like nuclear weapons or AI. The point isn’t that I can refine words so that my argument fits within your terms, but rather that it’s changes in a single underlying variable which produce these threats. The argument being that while individual such changes are rare one such change can result in world destruction through multiple pathways.
Basically, I am arguing that just as it would not make sense to treat AI will create biological weapons to exterminate us, and AI will create nano-technology to exterminate us as two different threats for the purposes of your argument. Similarly, arguably it should not make sense to treat technological progress will result in AI wiping us out and technological progress will result in biological weapons, wiping us out as separate threats for the purposes of your argument because it is changes that cause human extinction which are rare and technological progress on the scale we have seen in recent history resulting in an unprecedented level of technological capability is a single such change.
It makes sense as a way of reasoning, but depends on two things:
- definition of existential threat. Are we talking about literally wiping out humanity, or would you count something like the plague (20-30% of world population killed)? Because in the second case I don't see how you cannot count nuclear war for example
- uncertainty. None of us is an expert in the many fields needed to independently assess existential risk from all plausible causes. When Bob is not an expert on X, and sees experts on both camps of "X is an existentialist risk", with somewhat reasonable (to him) arguments on both sides, it's perfectly rational to have an estimate of 5% that factors in his ignorance. Conversely, it's pretty safe to bet that your Alice is an expert in the field that she assigns high risk to. But then, how can she confidently say that risk from other causes is near zero?