Deriving the power of Wald test for a single parameter

While studying from Larry Wasserman’s “All of Statistics”, I’ve found that the exposition of the Wald test was a little confusing to me, so that I struggled a bit in trying to derive a key result. Given that I didn’t find much on the internet to help me, and that I finally figured it out after a while, I thought of writing a small post for other confused students.

Given a scalar parameter $\theta$ of the distribution underlying the data, the Wald test uses its estimate $\hat{\theta}$ to compute a statistic ${W}$, which is then used to pit a null hypothesis $H_0$ against an alternative hypothesis $H_1$. The distribution of the estimate $\hat{\theta}$ is assumed to be asymptotically normal, centered on the true value of $\theta$. The null hypothesis is $H_0: \theta = \theta_0$, which states that the true value of the parameter $\theta$ is some scalar $\theta_0$ (so that, if we assume that $H_0$ is true and given that $\hat{\theta}$ is asymptotically normal, $\hat{\theta}$ would be centered on $\theta_0$). The alternative hypothesis is instead $H_1: \theta \ne \theta_0$, which states the opposite.

Let $\widehat{\text{se}}$ be the estimated standard error of $\hat{\theta}$. If the null hypothesis is true, and thus the asymptotic distribution of $\hat{\theta}$ is a normal centered on $\theta_0$, then

$$\begin{equation} \frac{\hat{\theta} - \theta_0}{\widehat{\text{se}}} \rightsquigarrow N(0,1), \end{equation}$$

where ${N}(0,1)$ is the normal distribution with mean 0 and unit standard deviation.

The Wald statistic is defined as

$$\begin{equation} W = \frac{\hat{\theta} - \theta_0}{\widehat{\text{se}}} \end{equation}$$

and the size $\alpha$ Wald test states: reject the null hypothesis when $\vert W \vert >z_\frac{\alpha}{2}$, where $z_\frac{\alpha}{2}$ is equal to $\Phi^{-1}(1-\frac{\alpha}{2})$, $\Phi^{-1}$ being the inverse of the normal CDF. ${W}$ is asymptotically distributed as ${N}(0,1)$, so that the probability of rejecting the null hypothesis asymptotically converges to $\alpha$.

Now for the key result I wanted to prove:

Theorem: Let $\theta_\ast$ be the true value of $\theta$, $\theta_\ast \ne \theta_0$ (i.e. the null hypothesis really is false). The power $\beta(\theta_\ast) = P_{\theta_\ast}(\vert W \vert >z_\frac{\alpha}{2})$ of correctly rejecting the null hypothesis is then approximately equal to

$$\begin{equation} 1 - \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \end{equation}$$

In proving this result I was initially confused by the fact that in Wasserman’s book $\hat{\theta}$ is assumed to be asymptotically normal with center in $\theta_0$ (Theorem 10.3). I was then further led astray by this question on math.stackexchange, where the user asking the question incorrectly applies the definition of power of a test.

So here’s the proof:

Proof: The true value of $\theta$ is $\theta_\ast$: given that $\hat{\theta}$ is asymptotically normal and centered on $\theta_\ast$, then $\frac{\hat{\theta} - \theta_\ast}{\widehat{\text{se}}} \rightsquigarrow N(0,1)$. The power of the Wald test for $\theta=\theta_\ast$ is equal to

$$\begin{align} \beta(\theta_\ast) & = P_{\theta_\ast}(\vert W \vert>z_\frac{\alpha}{2}) \\ & = P_{\theta_\ast}\left(\frac{\vert \hat\theta - \theta_0\vert}{\widehat{\text{se}}} > z_\frac{\alpha}{2}\right)\\ & = P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} > z_\frac{\alpha}{2} \right) + P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} < - z_\frac{\alpha}{2} \right) \\ & = P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} > \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2} \right) +\\ & \hphantom{{}={}} P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_*}{\widehat{\text{se}}} < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2} \right)\\ \end{align}$$

let $Z = \frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} = \frac{\hat{\theta} - \theta_\ast}{\widehat{\text{se}}}$, then $Z \rightsquigarrow N(0,1)$, so that

$$\begin{align} & \hphantom{{}={}}P_{\theta_\ast}\left(Z > \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \\\\ & = 1 - P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \\ & = 1 - \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \end{align}$$

concluding the proof $\blacksquare$