>>11182537Chances are that AI will be able to recursively improve itself and create an intelligence explosion far faster than human intelligence can be improved.
https://intelligence.org/files/AIPosNegFactor.pdfBuilding a 747 from scratch is not easy. But is it easier to:
• Start with the existing design of a biological bird,
• and incrementally modify the design through a series of successive stages,
• each stage independently viable,
• such that the endpoint is a bird scaled up to the size of a 747,
• which actually flies,
• as fast as a 747,
• and then carry out this series of transformations on an actual living bird,
• without killing the bird or making it extremely uncomfortable?
I’m not saying it could never, ever be done. I’m saying that it would be easier to build
the 747, and then have the 747, metaphorically speaking, upgrade the bird. “Let’s just
scale up an existing bird to the size of a 747” is not a clever strategy that avoids dealing
with the intimidating theoretical mysteries of aerodynamics. Perhaps, in the beginning,
all you know about flight is that a bird has the mysterious essence of flight, and the
materials with which you must build a 747 are just lying there on the ground. But you
cannot sculpt the mysterious essence of flight, even as it already resides in the bird, until
flight has ceased to be a mysterious essence unto you.
The above argument is directed at a deliberately extreme case. The general point is
that we do not have total freedom to pick a path that sounds nice and reassuring, or
that would make a good story as a science fiction novel. We are constrained by which
technologies are likely to precede others.
