Humans are physically diverse because we moved out all over the world. If we stayed in Africa, you'd have a similar situation as to the one today. Africans are so genetically diverse, the gap between the Khoisan and the ancestors of the Mandinka is anywhere between 260,000 and 350,000 years old. Meanwhile all Africans and Eurasians last shared a common ancestor roughly 60,000-80,000 years ago.
Now, keep this fact in mind: Homo sapiens was not the first species to leave Africa, we were possibly the final species to do so. Homo erectus evolved in Africa 1.9-2 million years ago, and migrated to the Middle East, West Asia/Europe (Georgia and possibly Italy and England), East Asia, and Southeast Asia. They ranged from around 137-185cm, and their skulls were so distinct, that many former species of Homo/human are now being reclassified as Homo erectus by some scientists.
https://en.wikipedia.org/wiki/Dmanisi_skullsFor example, the Dmanisi finds of what was once deemed to be Homo georgicus appears to be an extremely diverse population of Homo erectus. Correct me if I'm wrong, but not even Australian Aboriginals at their most robust look as weird as these finds.
