>>12603677It's a field of Mathematics that is worried about questions like the following:
> How can we measure the information content of the outcome of a random experiment?For example, in a system where you don't have information about what's going on, you only get to observe one final result that it produces. "How much information" are you gaining by observing that outcome? How many observations do you need until you have a good idea of what's going on?
> What's the best way to encode information?Let's say I want to send some message to someone (like a file), but there's a possibility that it'll get corrupted. What's the optimal way of adding redundancy so that, even if it does get corrupted, he can still decode the message with the intent that I had? "How many more bits" will the message need to have to be corruption-proof? Is there a way to make it, for example, just 1.05x longer but 2x less likely to corrupt?
> What do we mean by "redundancy"?> If I have some data with some redundancy in it, what's the best way to compress it?> How do I calculate the average information gained from an observation? (I think that's the definition of Shanon entropy)And honestly you could consider statistical inference to be a part of information theory (because it's kinda closely related to some of these questions), but most people don't see it that way.
So yeah, it's mostly about "How do we transfer data optimally" (sometimes called "coding theory"), but there's a bit of Statistics too, when we talk about random experiments.
