>>12114593You could say the same of nukes, including even today. They could spiral out of our control and kill most everyone or everyone. But we can still try to keep a handle on them to prevent that.
AGI might ultimately escape any kind of attempt to make it safe. But given it's probably going to be developed somewhere someday, be it decades or centuries away, and be it a very gradual ramp-up or a sudden one, we should do what we can to reduce the risks.
Just one hypothetical example: what if a US research lab (or union between the US and some other countries) creates the first superintelligent AGI, and what if we can manage to keep that safe and secure and somehow have it help us prevent other AGIs from posing risks, or at least mitigate those risks once they do present themselves? The future might be some weird dystopia of AGIs battling and outsmarting AGIs, with some representing competing human organizations or nations, and maybe some acting independently or totally aberrantly.
Maybe we'll have squads of human, robot, and AGI Bladerunners running around with the sole goal of destroying any potentially dangerous AGIs (and perhaps some of their human maintainers and security forces) before they become too risky, both with information warfare and kinetic force from guns and bombs.
Maybe it'll be like some bizarre new layer on top of "cyberwar". Maybe we won't be able to prevent unsafe AGIs from forming, but maybe we'll have at least some safe, good, stable, smart AGIs who can put the bad eggs down before it's too late.
Or maybe that won't even be the most serious risk for humanity and maybe we'll need to create a defensive AGI to help us deal with lone-wolf terrorists and Columbine and Aum Shinrikyo types who might be able to release newly discovered forms of biological and chemical attacks that can kill millions easily and cheaply. Maybe some death cults will try to seed the atmosphere with something that suffocates all life or something.
