/sci/ - Science & Math » Thread #12618279

31KiB, 602x398, standard-deviation.png

View Same Google iqdb SauceNAO

Standard deviation

Anonymous Sat 23 Jan 16:04:20 2021 No.12618279 View Reply Original Report

Quoted By: >>12618368 >>12619201 >>12619276

Basic statistics question: why doesn't the standard deviation use simple arithmetic mean? What's the need in using the average of squared numbers and square rooting the whole thing? Mixing addition and exponentiation doesn't seems right, also the denominator wasn't even squared and it is also inside that radicand.

How this thing works and outputs a result that makes sense?

Anonymous

Anonymous Sat 23 Jan 2021 16:09:27 No.12618297 Report

Quoted By: >>12618333

The arithmetic mean would give you a standard deviation of 0. Clearly that's meaningless as the data has an obvious spread around the mean. So we do the next best thing and work with the root mean square. This is a common theme across all statistics and math.
>Try arithmetic mean
>If zero, when there is clear spread, shift to root mean square

Anonymous

Anonymous Sat 23 Jan 2021 16:19:52 No.12618333 Report

Quoted By: >>12618342 >>12618416 >>12618591 >>12618702 >>12620140

>>12618297
What if we use the absolute value to get rid of the negatives, just like the squaring does.

In the example it would be
(|-3| + |-1| + |-1| + |-1| + 0 + 0 + 2 + 4) / 8 = 1.5

Seems like a meaningful result. Looking this number I understand that the values deviate from the mean by +-1.5 in average.

Anonymous

Anonymous Sat 23 Jan 2021 16:23:46 No.12618342 Report

Quoted By:

>>12618333
Yeah so that's a good follow-up and was also what entered my mind. That one I honestly don't know. I suspect it has to do with how the absolute value isn't differentiable, and makes assessing properties of it difficult. But again, that's just a guess.

Anonymous

Anonymous Sat 23 Jan 2021 16:30:30 No.12618368 Report

Quoted By: >>12618376

>>12618279
First of all standard deviation is an invention of the eternal brainlet.
No such thing as a standard deviation. Variance, however is real.

The standard deviation was invented because if $X \sim \mathcal(\mu,\sigma^2)$ then
$z=\frac{X-\mu}{\sigma} \sim \mathcal{N}(0,1)$
and
$\mu + \sigma z \sim X$

So brainlets wanted a name for $\sigma$ for simple calculations on the normal distribution. Never reason on the standard deviation, it's futile for anything other than a Gaussian. Ask yourself what variance and central moments are.

Anonymous

Anonymous Sat 23 Jan 2021 16:32:07 No.12618376 Report

Quoted By:

>>12618368
* $X \sim \mathcal{N}(\mu,\sigma^2)$

Anonymous

Anonymous Sat 23 Jan 2021 16:39:25 No.12618416 Report

Quoted By: >>12618432

>>12618333
Thinking more, a practical reason may exist. Standard deviation quantifies error. The quadratic mean is always larger than or equal to the arithmetic mean (even with absolute values). By using quadratic mean, we potentially overestimate the error, which is always a good practice in statistics. Again, just a guess.

Anonymous

Anonymous Sat 23 Jan 2021 16:42:29 No.12618432 Report

Quoted By:

>>12618416
>The quadratic mean is always larger than or equal to the arithmetic mean (even with absolute values)
Not if $x_n \in (0,1) \textrm{ } n =1,2,...,N$

Anonymous

Anonymous Sat 23 Jan 2021 17:21:33 No.12618591 Report

Quoted By:

>>12618333
Pop ?{-3, -1, -1, -1, 0, 0, 2, 4} = 2 =! 1.5 when squaring. Anyways extremes are more prominent and it makes for easier work as we always get something differentiable. But there is no reason that I can tell why using absolute deviation is wrong or not useful. It is just harder to work with.

Anonymous

Anonymous Sat 23 Jan 2021 17:41:15 No.12618702 Report

Quoted By: >>12618810

>>12618333
Same reason we use minimum squares to solve fitting problems in linear algebra: squares are easier to work with than absolute values.

Anonymous

Anonymous Sat 23 Jan 2021 18:04:24 No.12618810 Report

Quoted By: >>12619276

>>12618702
I thought we used minimum squares because of the inner product which induces the euclidean distance?

Anonymous

Anonymous Sat 23 Jan 2021 19:52:01 No.12619201 Report

Quoted By:

>>12618279
serious answer, read the intro and first chapter in Risk and Asset Allocation by Meucci. if you think of any probability distribution (for instance, picking a random member from a population as in your example), then it should be obvious that we are interested in something like an "average" and something like a "width" of that distribution. it turns out that the mean and variance are very well behaved mathematically, so you can use them to do lots of nice things (means and variances sum when adding independent random variables, for instance). however, as others in the topic have noticed, sometimes squaring is an issue for outliers. one alternative would be using median/spread instead.

Anonymous

Anonymous Sat 23 Jan 2021 20:10:08 No.12619276 Report

Quoted By:

>>12618810
>>12618279
This is the correct answer, as you can think of it as the distance from the mean. You may notice that the standard deviation formula is very similar to the distance formula.

Anonymous

Anonymous Sat 23 Jan 2021 23:49:17 No.12620140 Report

Quoted By:

>>12618333
Yes we could use that. My probability theory book tells about it. But we won't becouse GAUSSIAN YAY BELL CURvE WE LOVE SCIENCE

Capcode	All Only User Posts Only Moderator Posts Only Admin Posts Only Developer Posts
Show Posts	All Only With Images Only Without Images
Deleted Posts	All Only Deleted Posts Only Non-Deleted Posts
Ghost Posts	All Only Ghost Posts Only Non-Ghost Posts
Post Type	All Only Sticky Threads Only Opening Posts Only Reply Posts
Results	All Grouped By Threads
Order	Latest Posts First Oldest Posts First

Your latest searches