I get your argument for why we should not expect independent multiple threats, but what if we have one underlying threat that can manifest in multiple ways say the underlying threat is technological progress resulting in ways to destroy us and that can manifest in multiple ways like nuclear weapons or AI. The point isn’t that I can refine words so that my argument fits within your terms, but rather that it’s changes in a single underlying variable which produce these threats. The argument being that while individual such changes are rare one such change can result in world destruction through multiple pathways.
Basically, I am arguing that just as it would not make sense to treat AI will create biological weapons to exterminate us, and AI will create nano-technology to exterminate us as two different threats for the purposes of your argument. Similarly, arguably it should not make sense to treat technological progress will result in AI wiping us out and technological progress will result in biological weapons, wiping us out as separate threats for the purposes of your argument because it is changes that cause human extinction which are rare and technological progress on the scale we have seen in recent history resulting in an unprecedented level of technological capability is a single such change.
It makes sense as a way of reasoning, but depends on two things:
- definition of existential threat. Are we talking about literally wiping out humanity, or would you count something like the plague (20-30% of world population killed)? Because in the second case I don't see how you cannot count nuclear war for example
- uncertainty. None of us is an expert in the many fields needed to independently assess existential risk from all plausible causes. When Bob is not an expert on X, and sees experts on both camps of "X is an existentialist risk", with somewhat reasonable (to him) arguments on both sides, it's perfectly rational to have an estimate of 5% that factors in his ignorance. Conversely, it's pretty safe to bet that your Alice is an expert in the field that she assigns high risk to. But then, how can she confidently say that risk from other causes is near zero?
Agree with the sentiment, though not which particular risks they are.
Therein lies the rub, I suppose.
You want "your" pet existential risk to be the real threat when in reality all of them are low probability, which drives up probability estimates amongst people who are most interested in that particular risk. In aggregrate it's terribly inflated.
I'm just saying it's plausible, that's enough to make a single big risk credible.
And it's plausible because we see heavy-tailed distributions across many domains, including existential risks for which we have decent data, such as supervolcanoes vs asteroids vs stellar explosions.
I get your argument for why we should not expect independent multiple threats, but what if we have one underlying threat that can manifest in multiple ways say the underlying threat is technological progress resulting in ways to destroy us and that can manifest in multiple ways like nuclear weapons or AI. The point isn’t that I can refine words so that my argument fits within your terms, but rather that it’s changes in a single underlying variable which produce these threats. The argument being that while individual such changes are rare one such change can result in world destruction through multiple pathways.
Basically, I am arguing that just as it would not make sense to treat AI will create biological weapons to exterminate us, and AI will create nano-technology to exterminate us as two different threats for the purposes of your argument. Similarly, arguably it should not make sense to treat technological progress will result in AI wiping us out and technological progress will result in biological weapons, wiping us out as separate threats for the purposes of your argument because it is changes that cause human extinction which are rare and technological progress on the scale we have seen in recent history resulting in an unprecedented level of technological capability is a single such change.
It makes sense as a way of reasoning, but depends on two things:
- definition of existential threat. Are we talking about literally wiping out humanity, or would you count something like the plague (20-30% of world population killed)? Because in the second case I don't see how you cannot count nuclear war for example
- uncertainty. None of us is an expert in the many fields needed to independently assess existential risk from all plausible causes. When Bob is not an expert on X, and sees experts on both camps of "X is an existentialist risk", with somewhat reasonable (to him) arguments on both sides, it's perfectly rational to have an estimate of 5% that factors in his ignorance. Conversely, it's pretty safe to bet that your Alice is an expert in the field that she assigns high risk to. But then, how can she confidently say that risk from other causes is near zero?
Agree with the sentiment, though not which particular risks they are.
Therein lies the rub, I suppose.
You want "your" pet existential risk to be the real threat when in reality all of them are low probability, which drives up probability estimates amongst people who are most interested in that particular risk. In aggregrate it's terribly inflated.
I’d love to understand this better. Why assume that distribution *between* tail risks would also be heavy tailed?
I'm just saying it's plausible, that's enough to make a single big risk credible.
And it's plausible because we see heavy-tailed distributions across many domains, including existential risks for which we have decent data, such as supervolcanoes vs asteroids vs stellar explosions.
https://forum.effectivealtruism.org/posts/Z5KZ2cui8WDjyF6gJ/some-thoughts-on-toby-ord-s-existential-risk-estimates