I suggest you take a look at Project Euler [0]. It's a bunch of math puzzles that usually require programming to solve. In order to solve most problems in a reasonable amount of time, you'll need to use results from number theory and other areas. Once you've used a result, you can try to prove it or to understand the proof.
Project Euler emphasizes number theory, which is the most approachable field of mathematics for total beginners because the background you need is just addition and multiplication. You should be able to make progress in number theory much faster than by taking the traditional route through calculus.
Another advantage of the Project Euler approach is that you'll learn how to put math into code, which is fun and tremendously valuable.
Another thing I recommend is learning geometry [1]. The way to do this is to use a ruler and compass to draw various shapes and then prove that those shapes have certain properties (e.g. prove that an angle really is a right angle). This approach also has more merit than the traditional approach, because you learn how to write proofs without driving yourself to exhaustion and frustration with calculus exercises.
I also suggest learning linear algebra before calculus, because it's more useful to programmers and more accessible. The way to learn linear algebra is to study OpenGL and OpenCV with an emphasis on graphics and machine vision theory. Making things work in OpenGL is more rewarding than just doing exercises out of a textbook.
At a certain point, you'll find that you can't progress any further in number theory or geometry without calculus and complex analysis, at which point calculus should be a fun challenge for you instead of a tough slog. You'll need multi-variable differential calculus and linear algebra to understand neural networks.
In summary: Have fun! Math is fun! Learn to write proofs! Watch Numberphile [2]!
[0]:
projecteuler.net/[1]:
wikipedia.org/wiki/Euclidean_geometry[2]:
numberphile.com/ 
