There is the notion of a "topos" which can be pictured as an alternate mathematical universe in which we can do mathematics. Most mathematicians spend all their professional life in just a single topos, the "standard topos".

In most toposes, mathematics plays out slightly differently than in the standard topos. For instance, there are toposes in which every function from the naturals to the naturals is computable or in which the intermediate value theorem from undergraduate calculus fails.

However, in all toposes, the laws of intuitionistic logic hold (ordinary classical logic is intuitionistic logic plus the law of excluded middle, and typically classically the axiom of choice is also added). This puts upper bounds on how strange toposes can be: For instance, in any topos, 2 + 2 equals 4, there are infinitely many prime numbers, the square root of two is not rational, the powerset of the naturals is uncountable, the fundamental theorem of Galois theory holds, and so on.

Incidentally, I'm writing an introductory paper on this topic, for an audience of philosophers of mathematics. It won't be finished before Tuesday, but I'm posting this now. You can find the current draft here: https://rawgit.com/iblech/internal-methods/master/paper-filmat.pdf Questions and comments are always welcome.