I have a hard time believing that if we never came up with -symbolic- language, and attributing meaning to various audible tones, that we would never have "come up with math", or would never "have any understanding of" "math".

There has to be some type of connection to, and understanding of something so universal beyond our symbolic representation