>>10195849linear algebra was a prereq at my undergrad university for physics majors. in any case, even if it weren't, it's essential for getting good at even intro-level quantum mechanics. also if you want to be able to tackle Jackson problems, you'll for sure want to know a bit of PDE's/complex analysis, and a bit of (real) analysis is very useful in physics.
comparing different universities, departments are seldom very consistent in the requirements for each degree program. i bet you i could find a good university that has very minimal math requirements for CS but very high math requirements for physics. it varies.
anyhow as an experimental physics grad student (graduating very soon) i do a lot of code-monkeying but also a lot of reading research papers (including on theoretical physics) and i find that my math double-major was crucial for helping me to understand all the crazy group theory involved in advanced physics, not to mention linear algebra and diffeq for QM, and complex analysis+PDE's for graduate-level E+M. in fact whenever i encounter a math problem i can't solve analytically, my first reflex is just to write up some easy numerical method in python or C++ that requires virtually 0 math knowledge