While studying from Larry Wasserman’s “All of Statistics”, I’ve found that the exposition of the Wald test was a little confusing to me, so that I struggled a bit in trying to derive a key result. Given that I didn’t find much on the internet to help me, and that I finally figured it out after a while, I thought of writing a small post for other confused students.
Given a scalar parameter of the distribution underlying the data, the Wald test uses its estimate to compute a statistic , which is then used to pit a null hypothesis against an alternative hypothesis . The distribution of the estimate is assumed to be asymptotically normal, centered on the true value of . The null hypothesis is , which states that the true value of the parameter is some scalar (so that, if we assume that is true and given that is asymptotically normal, would be centered on ). The alternative hypothesis is instead , which states the opposite.
Let be the estimated standard error of . If the null hypothesis is true, and thus the asymptotic distribution of is a normal centered on , then
where is the normal distribution with mean 0 and unit standard deviation.
The Wald statistic is defined as
and the size Wald test states: reject the null hypothesis when , where is equal to , being the inverse of the normal CDF. is asymptotically distributed as , so that the probability of rejecting the null hypothesis asymptotically converges to .
Now for the key result I wanted to prove:
Theorem: Let be the true value of , (i.e. the null hypothesis really is false). The power of correctly rejecting the null hypothesis is then approximately equal to
In proving this result I was initially confused by the fact that in Wasserman’s book is assumed to be asymptotically normal with center in (Theorem 10.3). I was then further led astray by this question on math.stackexchange, where the user asking the question incorrectly applies the definition of power of a test.
So here’s the proof:
Proof: The true value of is : given that is asymptotically normal and centered on , then . The power of the Wald test for is equal to
let , then , so that
concluding the proof