Intuition behind the formula for the variance of a sum of two variables

user1205901 05/15/2018. 3 answers, 482 views
variance covariance intuition

I know from previous studies that

$Var(A+B) = Var(A) + Var(B) + 2 Cov (A,B)$

However, I don't understand why that is. I can see that the effect will be to 'push up' the variance when A and B covary highly. It makes sense that when you create a composite from two highly correlated variables you will tend to be adding the high observations from A with the high observations from B, and the low observations from A with the low observations from B. This will tend tend to create extreme high and low values in the composite variable, increasing the variance of the composite.

But why does it work to multiply the covariance by exactly 2?

3 Answers

byouness 05/15/2018.

Simple answer:

The variance involves a square: $$Var(X) = E[(X - E[X])^2]$$

So, your question boils down to the factor 2 in the square identity:

$$(a+b)^2 = a^2 + b^2 + 2ab$$

Which can be understood visually as a decomposition of the area of a square of side $(a+b)$ into the area of the smaller squares of sides $a$ and $b$, in addition to two rectangles of sides $a$ and $b$:

enter image description here

More involved answer:

If you want a mathematically more involved answer, the covariance is a bilinear form, meaning that it is linear in both its first and second arguments, this leads to:

$$\begin{aligned} Var(A+B) &= Cov(A+B, A+B) \\ &= Cov(A, A+B) + Cov(B, A+B) \\ &= Cov(A,A) + Cov(A,B) + Cov(B,A) + Cov(B,B) \\ &= Var(A) + 2 Cov(A,B) + Var(B) \end{aligned}$$

In the last line, I used the fact that the covariance is symmetrical: $$Cov(A,B) = Cov(B,A)$$

To sum up:

It is two because you have to account for both $cov(A,B)$ and $cov(B,A)$.

Acccumulation 05/15/2018.

The set of random variables is a vector space, and many of the properties of Euclidean space can be analogized to them. The standard deviation acts much like a length, and the variance like length squared. Independence corresponds to being orthogonal, while perfect correlation corresponds with scalar multiplication. Thus, variance of independent variables follow the Pythagorean Theorem:
$var(A+B) = var(A)+var(B)$.

If they are perfectly correlated, then
$std(A+B) = std(A)+std(B)$

Note that this is equivalent to
$var(A+B) = var(A)+var(B)+2\sqrt{var(A)var(B)}$

If they are not independent, then they follow a law analogous to the law of cosines:
$var(A+B) = var(A)+var(B)+2cov(A,B)$

Note that the general case is one in between complete independence and perfect correlation. If $A$ and $B$ are independent, then $cov(A,B)$ is zero. So the general case is that $var(A,B)$ always has a $var(A)$ term and a $var(B)$ term, and then it has some variation on the $2\sqrt{var(A)var(B)}$ term; the more correlated the variables are, the larger this third term will be. And this is precisely what $2cov(A,B)$ is: it's $2\sqrt{var(A)var(B)}$ times the $r^2$ of $A$ and $B$.

$var(A+B) = var(A)+var(B)+MeasureOfCorrelation*PerfectCorrelationTerm$

where $MeasureOfCorrelation = r^2$ and $PerfectCorrelationTerm=2\sqrt{var(A)var(B)}$

Put in other terms, if $r = correl(A,B)$, then

$\sigma_{A+B} = \sigma_A^2+\sigma_B^2+ 2(r\sigma_A)(r\sigma_B)$

Thus, $r^2$ is analogous to the $cos$ in the Law of Cosines.

Bananin 05/16/2018.

I would add that what you cited is not the definition of $Var(A+B)$, but rather a consequence of the definitions of $Var$ and $Cov$. So the answer to why that equation holds is the calculation carried out by byouness. Your question may really be why that makes sense; informally:

How much $A+B$ will "vary" depends on four factors:

  1. How much $A$ would vary on its own.
  2. How much $B$ would vary on its own.
  3. How much $A$ will vary as $B$ moves around (or varies).
  4. How much $B$ will vary as $A$ moves around.

Which brings us to $$Var(A+B)=Var(A)+Var(B)+Cov(A,B)+Cov(B,A)$$ $$=Var(A)+Var(B)+2Cov(A,B)$$ because $Cov$ is a symmetric operator. - Download Hi-Res Songs

1 Martin Garrix

Yottabyte flac

Martin Garrix. 2018. Writer: Martin Garrix.
2 Alan Walker

Diamond Heart flac

Alan Walker. 2018. Writer: Alan Walker;Sophia Somajo;Mood Melodies;James Njie;Thomas Troelsen;Kristoffer Haugan;Edvard Normann;Anders Froen;Gunnar Greve;Yann Bargain;Victor Verpillat;Fredrik Borch Olsen.
3 Sia

I'm Still Here flac

Sia. 2018. Writer: Sia.
4 Blinders

Breach (Walk Alone) flac

Blinders. 2018. Writer: Dewain Whitmore;Ilsey Juber;Blinders;Martin Garrix.
5 Dyro

Latency flac

Dyro. 2018. Writer: Martin Garrix;Dyro.
6 Cardi B

Taki Taki flac

Cardi B. 2018. Writer: Bava;Juan Vasquez;Vicente Saavedra;Jordan Thorpe;DJ Snake;Ozuna;Cardi B;Selena Gomez.
7 Bradley Cooper

Shallow flac

Bradley Cooper. 2018. Writer: Andrew Wyatt;Anthony Rossomando;Mark Ronson;Lady Gaga.
8 Halsey

Without Me flac

Halsey. 2018. Writer: Halsey;Delacey;Louis Bell;Amy Allen;Justin Timberlake;Timbaland;Scott Storch.
9 Lady Gaga

I'll Never Love Again flac

Lady Gaga. 2018. Writer: Benjamin Rice;Lady Gaga.
10 Kelsea Ballerini

This Feeling flac

Kelsea Ballerini. 2018. Writer: Andrew Taggart;Alex Pall;Emily Warren.
11 Mako

Rise flac

Mako. 2018. Writer: Riot Music Team;Mako;Justin Tranter.
12 Dewain Whitmore

Burn Out flac

Dewain Whitmore. 2018. Writer: Dewain Whitmore;Ilsey Juber;Emilio Behr;Martijn Garritsen.
13 Bradley Cooper

Always Remember Us This Way flac

Bradley Cooper. 2018. Writer: Lady Gaga;Dave Cobb.
14 Little Mix

Woman Like Me flac

Little Mix. 2018. Writer: Nicki Minaj;Steve Mac;Ed Sheeran;Jess Glynne.
15 Charli XCX

1999 flac

Charli XCX. 2018. Writer: Charli XCX;Troye Sivan;Leland;Oscar Holter;Noonie Bao.
16 Rita Ora

Let You Love Me flac

Rita Ora. 2018. Writer: Rita Ora.
17 Diplo

Electricity flac

Diplo. 2018. Writer: Diplo;Mark Ronson;Picard Brothers;Wynter Gordon;Romy Madley Croft;Florence Welch.
18 Jonas Blue

Polaroid flac

Jonas Blue. 2018. Writer: Jonas Blue;Liam Payne;Lennon Stella.
19 Lady Gaga

Look What I Found flac

Lady Gaga. 2018. Writer: DJ White Shadow;Nick Monson;Mark Nilan Jr;Lady Gaga.
20 Avril Lavigne

Head Above Water flac

Avril Lavigne. 2018. Writer: Stephan Moccio;Travis Clark;Avril Lavigne.

Related questions

Hot questions


Popular Tags