6.5. Expectation¶

This section contains several results on expectation operator.

Any function $g (x)$ defines a new random variable $g (X)$ . If $g (X)$ has a finite expectation, then

E [g (X)] = \int_{- \infty}^{\infty} g (x) f_{X} (x) d x .

If several random variables $X_{1}, \dots, X_{n}$ are defined on the same sample space, then their sum $X_{1} + \dots + X_{n}$ is a new random variable. If all of them have finite expectations, then the expectation of their sum exists and is given by

E [X_{1} + \dots + X_{n}] = E [X_{1}] + \dots + E [X_{n}] .

If $X$ and $Y$ are mutually independent random variables with finite expectations, then their product is a random variable with finite expectation and

E (X Y) = E (X) E (Y) .

By induction, if $X_{1}, \dots, X_{n}$ are mutually independent random variables with finite expectations, then

E [\prod_{i = 1}^{n} X_{i}] = \prod_{i = 1}^{n} E [X_{i}] .

Let $X$ and $Y$ be two random variables with the joint density function $f_{X, Y} (x, y)$ . Let the marginal density function of $Y$ given $X$ be $f (y | x)$ . Then the conditional expectation is defined as follows:

E [Y | X] = \int_{- \infty}^{\infty} y f (y | x) d y .

$E [Y | X]$ is a new random variable.

\begin{aligned} E [E [Y | X]] & = \int_{- \infty}^{\infty} E [Y | X] f (x) d x \\ = \int_{- \infty}^{\infty} \int_{- \infty}^{\infty} y f (y | x) f (x) d y d x \\ = \int_{- \infty}^{\infty} y (\int_{- \infty}^{\infty} f (x, y) d x) d y \\ = \int_{- \infty}^{\infty} y f (y) d y = E [Y] . \end{aligned}

In short, we have

E [E [Y | X]] = E [Y] .

The covariance of $X$ and $Y$ is defined as

Cov (X, Y) = E [(X - E [X]) (Y - E [Y])] .

It is easy to see that

Cov (X, Y) = E [X Y] - E [X] E [Y] .

The correlation coefficient is defined as

ρ ≜ \frac{Cov (X, Y)}{\sqrt{V a r (X) V a r (Y)}} .

6.5.1. Independent Variables¶

If $X$ and $Y$ are independent, then

E [g_{1} (x) g_{2} (y)] = E [g_{1} (x)] E [g_{2} (y)] .

If $X$ and $Y$ are independent, then $Cov (X, Y) = 0$ .

6.5.2. Uncorrelated Variables¶

The two variables $X$ and $Y$ are called uncorrelated if $Cov (X, Y) = 0$ . Covariance doesn’t imply independence.

Topics in Signal Processing

Expectation

Contents

6.5. Expectation¶

6.5.1. Independent Variables¶

6.5.2. Uncorrelated Variables¶