Deriving the power of Wald test for a single parameter

While studying from Larry Wasserman’s “All of Statistics”, I’ve found that the exposition of the Wald test was a little confusing to me, so that I struggled a bit in trying to derive a key result. Given that I didn’t find much on the internet to help me, and that I finally figured it out after a while, I thought of writing a small post for other confused students.

Given a scalar parameter \(\theta\) of the distribution underlying the data, the Wald test uses its estimate \(\hat{\theta}\) to compute a statistic \({W}\), which is then used to pit a null hypothesis \(H_0\) against an alternative hypothesis \(H_1\). The distribution of the estimate \(\hat{\theta}\) is assumed to be asymptotically normal, centered on the true value of \(\theta\). The null hypothesis is \(H_0: \theta = \theta_0\), which states that the true value of the parameter \(\theta\) is some scalar \(\theta_0\) (so that, if we assume that \(H_0\) is true and given that \(\hat{\theta}\) is asymptotically normal, \(\hat{\theta}\) would be centered on \(\theta_0\)). The alternative hypothesis is instead \(H_1: \theta \ne \theta_0\), which states the opposite.

Let \(\widehat{\text{se}}\) be the estimated standard error of \(\hat{\theta}\). If the null hypothesis is true, and thus the asymptotic distribution of \(\hat{\theta}\) is a normal centered on \(\theta_0\), then

\[\begin{equation} \frac{\hat{\theta} - \theta_0}{\widehat{\text{se}}} \rightsquigarrow N(0,1), \end{equation}\]

where \({N}(0,1)\) is the normal distribution with mean 0 and unit standard deviation.

The Wald statistic is defined as

\[\begin{equation} W = \frac{\hat{\theta} - \theta_0}{\widehat{\text{se}}} \end{equation}\]

and the size \(\alpha\) Wald test states: reject the null hypothesis when \(\vert W \vert >z_\frac{\alpha}{2}\), where \(z_\frac{\alpha}{2}\) is equal to \(\Phi^{-1}(1-\frac{\alpha}{2})\), \(\Phi^{-1}\) being the inverse of the normal CDF. \({W}\) is asymptotically distributed as \({N}(0,1)\), so that the probability of rejecting the null hypothesis asymptotically converges to \(\alpha\).

Now for the key result I wanted to prove:

Theorem: Let \(\theta_\ast\) be the true value of \(\theta\), \(\theta_\ast \ne \theta_0\) (i.e. the null hypothesis really is false). The power \(\beta(\theta_\ast) = P_{\theta_\ast}(\vert W \vert >z_\frac{\alpha}{2})\) of correctly rejecting the null hypothesis is then approximately equal to

\[\begin{equation} 1 - \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \end{equation}\]

In proving this result I was initially confused by the fact that in Wasserman’s book \(\hat{\theta}\) is assumed to be asymptotically normal with center in \(\theta_0\) (Theorem 10.3). I was then further led astray by this question on math.stackexchange, where the user asking the question incorrectly applies the definition of power of a test.

So here’s the proof:

Proof: The true value of \(\theta\) is \(\theta_\ast\): given that \(\hat{\theta}\) is asymptotically normal and centered on \(\theta_\ast\), then \(\frac{\hat{\theta} - \theta_\ast}{\widehat{\text{se}}} \rightsquigarrow N(0,1)\). The power of the Wald test for \(\theta=\theta_\ast\) is equal to

\[\begin{align} \beta(\theta_\ast) & = P_{\theta_\ast}(\vert W \vert>z_\frac{\alpha}{2}) \\ & = P_{\theta_\ast}\left(\frac{\vert \hat\theta - \theta_0\vert}{\widehat{\text{se}}} > z_\frac{\alpha}{2}\right)\\ & = P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} > z_\frac{\alpha}{2} \right) + P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} < - z_\frac{\alpha}{2} \right) \\ & = P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} > \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2} \right) +\\ & \hphantom{{}={}} P_{\theta_\ast}\left(\frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_*}{\widehat{\text{se}}} < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2} \right)\\ \end{align}\]

let \(Z = \frac{\hat\theta - \theta_0}{\widehat{\text{se}}} + \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} = \frac{\hat{\theta} - \theta_\ast}{\widehat{\text{se}}}\), then \(Z \rightsquigarrow N(0,1)\), so that

\[\begin{align} & \hphantom{{}={}}P_{\theta_\ast}\left(Z > \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \\\\ & = 1 - P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + P_{\theta_\ast}\left(Z < \frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \\ & = 1 - \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} + z_\frac{\alpha}{2}\right) + \Phi\left(\frac{\theta_0 - \theta_\ast}{\widehat{\text{se}}} - z_\frac{\alpha}{2}\right) \end{align}\]

concluding the proof \(\blacksquare\)