Sample variance

chella182 · #1 $Old$ November 3rd, 2009, 03:23 PM

This is a proof question that I just cannot get what's required for the life of me. This isn't part of my assessed homework, just fyi.

Suppose $X_1$ , $X_2$ , ..., $X_n$ is a random sample from a random variable $X$ which has expectation $\mu$ and variance $\sigma^2$ . Consider the sample variance

$S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$

Show that $E[S^2]=\sigma^2$

We're also given the hint that $X_i-\bar{X}$ may be written as $(X_i-\mu)-(\bar{X}-\mu)$ and that $Var(X)=E[(X-\mu)^2]$ . I've used both of these but I just can't seem to get $\sigma^2$ at all. Closest I've gotten is $\frac{n\sigma^2}{n-1}$

matheagle · #2 $Old$ November 3rd, 2009, 05:47 PM

why don't you write down what you've done.

obtain the expectation of...

$\sum_{i=1}^n(X_i-\bar X)^2 = \sum_{i=1}^n(X_i-\mu)^2-n(\bar X-\mu)^2$

$=E\biggl(\sum_{i=1}^n(X_i-\bar X)^2\biggr) = \sum_{i=1}^n V(X_i)-nV(\bar X)$

$=n\sigma^2-n{\sigma^2\over n} =n\sigma^2-\sigma^2=(n-1)\sigma^2$

Next divide by n-1 and it's over

theodds · #3 $Old$ November 3rd, 2009, 06:07 PM

Quote:

Originally Posted by chella182 $View Post$

This is a proof question that I just cannot get what's required for the life of me. This isn't part of my assessed homework, just fyi.

Suppose $X_1$ , $X_2$ , ..., $X_n$ is a random sample from a random variable $X$ which has expectation $\mu$ and variance $\sigma^2$ . Consider the sample variance

$S^2=\frac{1}{n-1}\sum_{i=1}^{n}(X_i-\bar{X})^2$

Show that $E[S^2]=\sigma^2$

We're also given the hint that $X_i-\bar{X}$ may be written as $(X_i-\mu)-(\bar{X}-\mu)$ and that $Var(X)=E[(X-\mu)^2]$ . I've used both of these but I just can't seem to get $\sigma^2$ at all. Closest I've gotten is $\frac{n\sigma^2}{n-1}$

Proving this fact is equivalent to proving $\mathbb{E}\sum_{i = 1} ^ n (Y_i - \bar{Y})^2 = (n - 1) \sigma^2$ . First, it's worth stating for clarity that, with respect to the summations, the term $\bar{Y}$ is a constant, so we can move it in and out of summations as we wish. Expand the quadratic and distribute the sum across the terms, which will give you $\mathbb{E}\left( \sum Y_i ^ 2 - 2 \bar{Y} \sum Y_i + \sum \bar{Y}^2 \right )$ . Now, note that $\sum Y_i = n \bar{Y}$ and $\sum \bar{Y}^2 = n\bar{Y}^2$ , which allows us to simplify our expression further to $\mathbb{E}\left( \sum Y_i ^ 2 - n \bar{Y}^2 \right )$ (the term n(ybar) is NOT in the sum). Now we use the linearity of the expectation operator to get $\sum \mathbb{E} Y_i ^ 2 - n\mathbb{E}\bar{Y}^2$ . The expectation of a square is the variance plus the square of the mean (you can get this by rearranging the formula for the variance), so use that to evaluate each of the expectations to get $\sum (\sigma^2 + \mu ^2) - n(\frac{\sigma^2}{n} + \mu^2)$ . This, of course, will simplify to $n \sigma^2 + n \mu ^ 2 - \sigma^2 - n \mu ^ 2 = n \sigma^2 - \sigma^2 = (n - 1)\sigma^2$ , completing the proof.

chella182 · #4 $Old$ November 4th, 2009, 03:21 AM

Thankyou! I think it was this key fact that I was missing...

Quote:

Originally Posted by theodds $View Post$

The expectation of a square is the variance plus the square of the mean (you can get this by rearranging the formula for the variance)

I was sat thinking "this must be something but couldn't fathom what. Cheers

Quote:

Originally Posted by matheagle $View Post$

why don't you write down what you've done.

I've gotten wrong for doing that in the past. Something about this site's not for checking work. + it would've taken me ages to type out what I'd done.