Чи означає, що центрування зменшує коваріацію?

11

Якщо припустити, що у мене є дві незалежні випадкові величини, і я хочу максимально зменшити коваріацію між ними, не втрачаючи занадто багато "сигналу", чи означає центринг допомогу? Я десь читав, що середнє центрування зменшує кореляцію на значний фактор, тому я думаю, що це повинно зробити те ж саме для коваріації.

correlation covariance random-vector

— lvdp
джерело

30

Якщо $X$ і $Y$ - випадкові величини, а $a$ і $b$ - константи, то

\begin{aligned} Cov (X + a, Y + b) & = E [(X + a - E [X + a]) (Y + b - E [Y + b])] \\ = E [(X + a - E [X] - E [a]) (Y + b - E [Y] - E [b])] \\ = E [(X + a - E [X] - a) (Y + b - E [Y] - b)] \\ = E [(X - E [X]) (Y - E [Y])] \\ = Cov (X, Y) . \end{aligned}

$\begin{aligned} \operatorname{Cov}(X + a, Y + b) &= E[(X + a - E[X + a])(Y + b - E[Y + b])] \\ &= E[(X + a - E[X] - E[a])(Y + b - E[Y] - E[b])] \\ &= E[(X + a - E[X] - a)(Y + b - E[Y] - b)] \\ &= E[(X - E[X])(Y - E[Y])] \\ &= \operatorname{Cov}(X, Y). \end{aligned}$ Центрування - це особливий випадок

a = - E [X]

$a = -E[X]$ і

b = - E [Y]

$b = -E[Y]$ , тому центрування не впливає на коваріацію.

Крім того, оскільки кореляція визначається як

Corr (X, Y) = \frac{Cov (X, Y)}{\sqrt{Var (X) Var (Y)}},

$\operatorname{Corr}(X, Y) = \frac{\operatorname{Cov}(X, Y)}{\sqrt{\operatorname{Var}(X) \operatorname{Var}(Y)}},$ ми можемо бачити, що

\begin{aligned} Corr (X + a, Y + b) & = \frac{Cov (X + a, Y + b)}{\sqrt{Var (X + a) Var (Y + b)}} \\ = \frac{Cov (X, Y)}{\sqrt{Var (X) Var (Y)}}, \end{aligned}

$\begin{aligned} \operatorname{Corr}(X + a, Y + b) &= \frac{\operatorname{Cov}(X + a, Y + b)}{\sqrt{\operatorname{Var}(X + a) \operatorname{Var}(Y + b)}} \\ &= \frac{\operatorname{Cov}(X, Y)}{\sqrt{\operatorname{Var}(X) \operatorname{Var}(Y)}}, \end{aligned}$ тому, зокрема, на кореляцію не впливає і центрування.

Це була популяційна версія історії. Зразок версії такий же: Якщо ми використовуємо

\hat{Cov} (X, Y) = \frac{1}{n} \sum_{i = 1}^{n} (X_{i} - \frac{1}{n} \sum_{j = 1}^{n} X_{j}) (Y_{i} - \frac{1}{n} \sum_{j = 1}^{n} Y_{j})

$\widehat{\operatorname{Cov}}(X, Y) = \frac{1}{n} \sum_{i=1}^n \left(X_i - \frac{1}{n}\sum_{j=1}^n X_j\right)\left(Y_i - \frac{1}{n}\sum_{j=1}^n Y_j\right)$ як наша оцінка коваріації між

X

$X$ і

Y

$Y$ з парного зразка

(X_{1}, Y_{1}), \dots, (X_{n}, Y_{n})

$(X_1,Y_1), \ldots, (X_n,Y_n)$ , то

\begin{aligned} \hat{Cov} (X + a, Y + b) & = \frac{1}{n} \sum_{i = 1}^{n} (X_{i} + a - \frac{1}{n} \sum_{j = 1}^{n} (X_{j} + a)) (Y_{i} + b - \frac{1}{n} \sum_{j = 1}^{n} (Y_{j} + b)) \\ = \frac{1}{n} \sum_{i = 1}^{n} (X_{i} + a - \frac{1}{n} \sum_{j = 1}^{n} X_{j} - \frac{n}{n} a) (Y_{i} + b - \frac{1}{n} \sum_{j = 1}^{n} Y_{j} - \frac{n}{n} b) \\ = \frac{1}{n} \sum_{i = 1}^{n} (X_{i} - \frac{1}{n} \sum_{j = 1}^{n} X_{j}) (Y_{i} - \frac{1}{n} \sum_{j = 1}^{n} Y_{j}) \\ = \hat{Cov} (X, Y) \end{aligned}

$\begin{aligned} \widehat{\operatorname{Cov}}(X + a, Y + b) &= \frac{1}{n} \sum_{i=1}^n \left(X_i + a - \frac{1}{n}\sum_{j=1}^n (X_j + a)\right)\left(Y_i + b - \frac{1}{n}\sum_{j=1}^n (Y_j + b)\right) \\ &= \frac{1}{n} \sum_{i=1}^n \left(X_i + a - \frac{1}{n}\sum_{j=1}^n X_j - \frac{n}{n} a\right)\left(Y_i + b - \frac{1}{n}\sum_{j=1}^n Y_j - \frac{n}{n} b\right) \\ &= \frac{1}{n} \sum_{i=1}^n \left(X_i - \frac{1}{n}\sum_{j=1}^n X_j\right)\left(Y_i - \frac{1}{n}\sum_{j=1}^n Y_j\right) \\ &= \widehat{\operatorname{Cov}}(X, Y) \end{aligned}$ for any

a

$a$ and

b

$b$ .

— Artem Mavrin
джерело

thanks for the detailed answer. Does it mean that for sample covariance the sample size doesn't have any impact either? i.e. reducing the sample size does not reduce the sample covariance?

— lvdp

3

@lvdp That should probably be a separate question.

— Acccumulation

A reduced sample size can only come with a different sample. A different sample could show different covariance, therefore. But as sample covariance is defined as an average, sample size is scaled for in principle.

— Nick Cox

5

The definition of the covariance of $X$ and $Y$ is $E[(X-E[X])(Y-E[Y])]$ . The expression $X-E[X]$ in that formula is the centered version of $X$ . So we already center $X$ when we take the covariance, and centering is an idempotent operator; once a variable is centered, applying the centering process further times doesn't change it. If the formula didn't take the centered versions of the variables, then there would all sort of weird effects, such as the covariance between temperature and another variable being different depending on whether we measure temperature in Celsius or Kelvin.

— Acccumulation
джерело

3

"somewhere" tends to be a rather unreliable source...

Covariance/correlation are defined with explicit centering. If you don't center the data, then you are not computing covariance/correlation. (Precisely: Pearson correlation)

The main difference is whether you center based on a theoretical model (e.g., the expected value is supposed to be exactly 0) or based on the data (arithmetic mean). It is easy to see that the arithmetic mean will yield smaller Covariance than any different center.

However, smaller covariance does not imply smaller correlation, or the opposite. Assume that we have data X=(1,2) and Y=(2,1). It is easy to see that with arithmetic mean centering this will yield perfectly negative correlation, while if we know the generating process produces 0 on average, the data is actually positively correlated. So in this example, we are centering - but with the theoretical expected value of 0.

This can arise easily. Consider we have a sensor array, 11x11, with the cells numbered -5 to +5. Rather than taking the arithmetic mean, it does make sense to use the "physical" mean of our sensor array here when looking for the correlation of sensor events (if we enumerated the cells 0 to 10, we'd use 5 as fixed mean, and we would get the exact same results, so that indexing choice disappears from the analysis - nice).

— Has QUIT--Anony-Mousse
джерело

Thanks @Anony-Mousse, will the sample covariance depend on the sample size? I.e. smaller sample size will yield smaller covariance ( before centering).

— lvdp

1

Depends on the sample obviously. On average - I don't know. I'd expect smaller samples to have more variability mostly, so maybe more often more extreme values. But that is just an intuition.

— Has QUIT--Anony-Mousse