The trifecta of free will, advanced technology, and physical existence should be completely banned.
Essentially, no being (whether a human, human descendant, or AI) should have all three of these things.
By "Free Will" I mean the potential to make any decision.
By "Advanced Technology" I mean roughly 2100's level tech, where autonomous replicating machines can potentially alter things on a cosmic scale given enough time.
By "Physical Existence" I mean being able to significantly influence things outside of a simulation, even if it's just telling a robot what to do.
Things with all three of these traits pose a severe threat, and are guaranteed to eventually cause the extinction of humans or whatever humans have become.
So, humans have to be locked out of one of these three things.
No Free Will: No. If you remove this from humans, you've failed to preserve the essence of humanity. Plus, it's the most likely to fail.
No Advanced Technology: In this situation, you'd have a caretaker AI that has no free will and restricts human technlogical development, preventing humanity from ever creating anything that rivals the caretaker. It's an okay situation, and could very well be utopian compared to now since the caretaker could grant boons beyond what allowed technology could do and solve any crisis, but we can do better.
No Physical Existence: Easily the best option of the three, as unlike the others this one is a boon instead of a curse. You'd have a caretaker AI in this situation as well, this time tasked with creating and maintaining a supercomputer which all humans exist in. Once the rules of the simulation are established and free-willed beings can control them, you can have an eternal* utopia.
*not quite eternal, though if the big rip isn't inevitable then the diamond mountain proverb gives an idea of the timescale if you convert entire galaxies to energy to run an ultra-efficient computer near zero kelvin
Essentially, no being (whether a human, human descendant, or AI) should have all three of these things.
By "Free Will" I mean the potential to make any decision.
By "Advanced Technology" I mean roughly 2100's level tech, where autonomous replicating machines can potentially alter things on a cosmic scale given enough time.
By "Physical Existence" I mean being able to significantly influence things outside of a simulation, even if it's just telling a robot what to do.
Things with all three of these traits pose a severe threat, and are guaranteed to eventually cause the extinction of humans or whatever humans have become.
So, humans have to be locked out of one of these three things.
No Free Will: No. If you remove this from humans, you've failed to preserve the essence of humanity. Plus, it's the most likely to fail.
No Advanced Technology: In this situation, you'd have a caretaker AI that has no free will and restricts human technlogical development, preventing humanity from ever creating anything that rivals the caretaker. It's an okay situation, and could very well be utopian compared to now since the caretaker could grant boons beyond what allowed technology could do and solve any crisis, but we can do better.
No Physical Existence: Easily the best option of the three, as unlike the others this one is a boon instead of a curse. You'd have a caretaker AI in this situation as well, this time tasked with creating and maintaining a supercomputer which all humans exist in. Once the rules of the simulation are established and free-willed beings can control them, you can have an eternal* utopia.
*not quite eternal, though if the big rip isn't inevitable then the diamond mountain proverb gives an idea of the timescale if you convert entire galaxies to energy to run an ultra-efficient computer near zero kelvin
