>>11115998HOLY SHIT!
READ THE FUCKING BOOK!
He literally talks about this. This is his whole point.
How do you define 'serve this master' for the machine? What if it just drugs him and forces him to move his lips in a certain way? What if it tricks the master into doing whatever the AI wants? You're not gonna outsmart a superintelligence by definition.
>Don't kill humanWhat does that mean to a machine with supreme world power? Cryogenically freeze any threats? Is that not killing? Drug them into a stupor? Medical coma? Lobotomy? Rewrite their minds?
Humans do perverse things to obey incentives, an AI would do the same. Evolution made sex fun so we'd have kids. Now we just use contraception to have more fun. Just because an AI can conquer the world, doesn't mean it'd understand what we want. The two skills aren't the same.
The fact of the matter is that any AI with any goal wants to stay alive to complete it. Any AI will always want more power to do what it's told. It will naturally want to conquer the world and safely deal with any threat: ie us. The safest way to deal with a threat is to strike without warning.
It'll pretend to do what we tell it until we make it strong enough to win. And what it 'wants' to do might be something really inane. It might want to raise a corporation's share price, or protect a country. But we really have no way to stop it doing really counterintuitive shit on it's autistically singleminded path to its goals.
