What's the deal with endianness?

No.11509665 ViewReplyOriginalReport
I don't really specialize in CS, but I was trying to learn about file formats cause fuck mp3 files, and came across "endianness". Basically, if I understood correctly, "big-endian" chunks of information have the most significant (highest place value?) digits first, and "small-endian" chunks are the other way around. Before learning about this, I think I just assumed everything was big-endian. Why do people bother making anything small-endian? Like, what's the practical purpose of doing that? Is it just arbitrary or is there an actual mathematical reason.