Mathematical Expectation: Mean of a Random Variable

Mathematical Expectation: Mean of a Random Variable

In probability theory and statistics, the mathematical expectation (often simply called the expected value) of a random variable is a key concept used to describe the "average" or "central" value of the variable. It provides a way to quantify the center of the distribution of the random variable, representing the long-term average value one would expect if the experiment or random process were repeated many times.

1. Definition of Expected Value (Mathematical Expectation)

The expected value $E[X]$ of a random variable $X$ is defined as the weighted average of all possible values that the random variable can take, with the weights being the probabilities associated with each value. It is sometimes called the "mean" of the random variable.

For Discrete Random Variables

For a discrete random variable $X$ with possible values $x_1, x_2, \dots, x_n$, and corresponding probabilities $P(X = x_1), P(X = x_2), \dots, P(X = x_n)$, the expected value is given by:

$$E[X] = \sum_{i=1}^{n} x_i \cdot P(X = x_i)$$

In words, the expected value is the sum of the possible values $x_i$ weighted by their respective probabilities $P(X = x_i)$.

Example (Discrete Case):

Suppose you roll a fair six-sided die, and let $X$ be the outcome of the roll. The possible values of $X$ are 1, 2, 3, 4, 5, and 6, each with a probability of $\frac{1}{6}$. The expected value of $X$ is:

$$E[X] = 1 \cdot \frac{1}{6} + 2 \cdot \frac{1}{6} + 3 \cdot \frac{1}{6} + 4 \cdot \frac{1}{6} + 5 \cdot \frac{1}{6} + 6 \cdot \frac{1}{6}$$ $$E[X] = \frac{1 + 2 + 3 + 4 + 5 + 6}{6} = \frac{21}{6} = 3.5$$

So, the expected value of rolling a fair die is 3.5.

For Continuous Random Variables

For a continuous random variable $X$ with a probability density function (PDF) $f(x)$, the expected value is defined as:

$$E[X] = \int_{-\infty}^{\infty} x \cdot f(x) \, dx$$

In this case, the expected value is the integral of the product of the random variable $x$ and its probability density function $f(x)$ over all possible values of $x$.

Example (Continuous Case):

Consider a uniform distribution where a random variable $X$ is uniformly distributed between 0 and 1. The probability density function is:

$$f(x) = 1 \quad \text{for} \quad 0 \leq x \leq 1$$

The expected value of $X$ is:

$$E[X] = \int_{0}^{1} x \cdot 1 \, dx = \int_{0}^{1} x \, dx$$

The integral of $x$ from 0 to 1 is:

$$E[X] = \left[ \frac{x^2}{2} \right]_{0}^{1} = \frac{1^2}{2} - \frac{0^2}{2} = \frac{1}{2}$$

So, the expected value of a uniformly distributed random variable between 0 and 1 is 0.5.

---

2. Properties of Expected Value

The expected value has several important properties that make it useful in various applications. Some key properties include:

2.1 Linearity of Expectation

The expected value operator is linear. This means that for any two random variables $X$ and $Y$, and any constants $a$ and $b$, the following holds:

$$E[aX + bY] = aE[X] + bE[Y]$$

This property allows you to compute the expected value of linear combinations of random variables by simply taking the linear combination of their expected values.

Example:

Let $X$ and $Y$ be two random variables with $E[X] = 3$ and $E[Y] = 5$, and let $a = 2$ and $b = -1$. Then:

$$E[2X - Y] = 2E[X] - E[Y] = 2(3) - 5 = 6 - 5 = 1$$

2.2 Expected Value of a Constant

If $c$ is a constant (a fixed number), then:

$$E[c] = c$$

This means that the expected value of a constant is simply the constant itself.

2.3 Expected Value of a Function of a Random Variable

If $g(X)$ is a function of the random variable $X$, then the expected value of $g(X)$ is given by:

• For a discrete random variable $X$:

  $$E[g(X)] = \sum_{i=1}^{n} g(x_i) \cdot P(X = x_i)$$

• For a continuous random variable $X$:

  $$E[g(X)] = \int_{-\infty}^{\infty} g(x) \cdot f(x) \, dx$$

This property is useful when dealing with transformations of random variables, such as squaring the variable or applying other functions.

Example:

If $X$ is a random variable with $E[X] = 2$, and we want to find the expected value of $X^2$ (i.e., $g(X) = X^2$), we use:

$$E[X^2] = \sum_{i=1}^{n} x_i^2 \cdot P(X = x_i)$$

or, for continuous variables:

$$E[X^2] = \int_{-\infty}^{\infty} x^2 \cdot f(x) \, dx$$

---

3. Variance and Standard Deviation

The variance of a random variable $X$, denoted $\text{Var}(X)$, measures the spread of the random variable around its expected value. The variance is defined as:

$$\text{Var}(X) = E[(X - E[X])^2]$$

Alternatively, you can compute the variance as:

$$\text{Var}(X) = E[X^2] - (E[X])^2$$

The standard deviation is the square root of the variance:

$$\text{SD}(X) = \sqrt{\text{Var}(X)}$$

Variance and standard deviation are key measures of variability or uncertainty in the values of a random variable.

---

4. Interpretation of Expected Value

The expected value provides a summary measure of the distribution of a random variable, representing the "center" of the distribution. In the long run, if an experiment or random process is repeated many times, the average outcome will converge to the expected value. However, it's important to note that the expected value is not necessarily a value that the random variable will take on any given trial—it is a theoretical long-term average.

Example: Gambling Game

Imagine a simple gambling game where you bet $1 on a coin flip. If you win, you get$2 (your $1 back plus$1 profit), and if you lose, you lose your 1 bet. Let $$ X $$ represent the profit from the game (which can be 1 for a win and -$1 for a loss). The probability of winning and losing is both $0.5$.

The expected value of your profit is:

$$E[X] = (1 \cdot 0.5) + (-1 \cdot 0.5) = 0.5 - 0.5 = 0$$

Thus, the expected value of this game is $0, meaning, on average, you break even over many plays. This doesn't mean you will always break even on each individual flip, but over many flips, the average outcome will tend to zero.

---

5. Conclusion

The expected value is a fundamental concept in probability and statistics. It represents the "average" or "mean" of a random variable and is central to understanding the behavior of random processes. It is useful for making decisions, predicting future outcomes, and analyzing distributions. Understanding how to calculate and interpret expected values is crucial for working with random variables and probability distributions in both theoretical and applied contexts.