MIT 18.650 — Lecture 16

The Wald Test

Recall that a test $\Psi$ has level $\alpha$ if $\mathrm{size}(\Psi) \leq \alpha$ , where $\mathrm{size}(\Psi) := \max_{\theta \in \Theta_0} \mathbb{P}_{\theta}(\Psi = 1)$ is the maximum possible probability of a Type I error (when $H_0$ is true yet $\Psi = 1$ ). Also recall that the standard error of an estimator $\hat{\theta}$ is $\mathrm{se} := \sqrt{\mathbb{V}[\hat{\theta}]}$ .

Theorem. (The Wald Test) Suppose $\hat{\theta}$ is an asymptotically normal estimator of $\theta$ , and we seek a test $\Psi$ with level $\alpha$ . Note that this means $\frac{\hat{\theta} - \theta}{\widehat{\mathrm{se}}} \rightsquigarrow \mathcal{N}(0, 1)$ . Then $T_n := \frac{\hat{\theta} - \theta_0}{\widehat{\mathrm{se}}}$ is our test statistic, and our test $\Psi$ is defined by…

… if we seek the hypotheses $H_0: \theta = \theta_0$ and $H_1: \theta \neq \theta_0$ , then $\Psi := \mathds{1}\left(|T_n| \geq z_{\alpha / 2}\right)$ , where $\Phi(z_{\alpha / 2}) - \Phi(-z_{\alpha / 2}) = 1 - \alpha$ .
… if we seek the hypotheses $H_0: \theta \leq \theta_0$ and $H_1: \theta > \theta_0$ , then $\Psi := \mathds{1}\left(T_n \geq z_{\alpha}\right)$ , where $\Phi(z_{\alpha}) = 1 - \alpha$ .
… if we seek the hypotheses $H_0: \theta \geq \theta_0$ and $H_1: \theta < \theta_0$ , then $\Psi := \mathds{1}\left(T_n \leq -z_{\alpha}\right)$ , where $\Phi(-z_{\alpha}) = \alpha$ .

If you really wanted, you could also compute the power $\beta(\theta)$ of each of the above tests.

Remark. How do we actually compute an estimate $\widehat{\mathrm{se}}$ of the standard error?

If $\hat{\theta}$ is $\bar{X}_n$ , then we can use standard techniques: calculate the sample variance $s_n^2$ and estimate $\widehat{\mathrm{se}} = \frac{s_n}{\sqrt{n}}$ .
If $\hat{\theta}$ is the MLE, then compute $I(\hat{\theta})$ and declare that $\widehat{\mathrm{se}} = \frac{1}{\sqrt{n \cdot I(\hat{\theta})}}$ .
Otherwise, use bootstrapping: draw $B$ bootstrap samples, compute $\hat{\theta}_b^*$ for each, and set $\widehat{\mathrm{se}} = \sqrt{ \frac{1}{B - 1} \sum_{b = 1}^B \left( \hat{\theta}^*_b - \frac{1}{B} \sum_{c = 1}^B \hat{\theta}^*_c \right)^2 }$ .

The Z-test is the special case of the Wald test when the test statistic looks like $T_n = \frac{\bar{X}_n - \mu_0}{\sigma / \sqrt{n}}$ and when $\sigma$ is the true, known population standard deviation.