The problem with disjunctive existential risk

Suppose that Alice and Bob have different beliefs about existential risk over the next 50 years. While Alice thinks it comes overwhelmingly from a single threat, Bob views it as a disjunction of four independent threats.

Alice:

• Threat A: 20%

Bob:

• Threat A: 6%

• Threat B: 6%

• Threat C: 5%

• Threat D: 5%

• Combined risk (given independence): ~20%

Setting aside your beliefs about actual threats such as AI misalignment or nuclear war, who looks more reasonable to you?

I think many people would say Bob. While Alice’s focus on a single big threat can come across as fanatical, Bob’s broader portfolio of lower credences has a more balanced and humble look.

But I take the opposite view. Serious existential threats seem to arise rarely: it’s debatable whether we even have one yet. This makes it decidedly unlikely that we have evidence of four near-term risks of 5 per cent or more. A single big risk is much more credible (if still unlikely), given the plausible hypothesis that the risk distribution is heavy-tailed.

Existential risk communication has often given comparable attention to several unrelated threats, creating the impression that they pose risks of roughly the same order. But if the near-term risk is indeed high, it’s more likely to be concentrated in one or two threats – AI and possibly synthetic biology. So it’s better to emphasise those threats than to use broader framings that make existential risk seem more disjunctive than it is.