# Setup

Diaconis and Holmes (1994) compare the normal and $t$ approximations to the one-sample randomization distribution. The setup is as follows. We have $n$ pairs, $i=1,...,n$, each of which produces two measurements $(x*i, y_i)$. We denote the _absolute difference* of each pair by $d_i := |x_i - y_i|$.

Each time we generate a new randomization, we are equally likely to assign the higher member of the pair to the treatment as the lower pair. In other words, the *signed difference* is equally likely to be positive as negative. We can define random signs $\epsilon_1, ..., \epsilon_n$, which are equally likely to be $-1$ or $1$ and i.i.d. The mean difference between the treatment and the control can then be written as

$$ \bar D = \frac{1}{n} \sum_{i=1}^n \epsilon_i d_i. $$

Since $\text{Var}\bar D = \frac{\sum_{i=1}^n d_i^2}{n^2}$, we can standardize the mean difference

$$ Z = \frac{\bar D}{\sqrt{\sum_{i=1}^n d_i^2 / n^2}} $$

and compare to a $N(0, 1)$ distribution.

Alternatively, we can studentize with respect to the sample variance to obtain a different test statistic:

$$ T = \frac{\bar D}{\sqrt{\frac{\sum_{i=1}^n d_i^2 - n\bar D^2}{n-1} / n} } $$

and compare to a $t_{n-1}$ distribution.

Which approximation is better: $N(0,1)$ for the permutation distribution of $Z$ or $t_{n-1}$ for the permutation distribution of $T$?

# Reframing the Question

## (using a one-to-one transformation)

To answer this question, we first observe that there is a one-to-one transformation $h$, such that $T = h(Z)$. To see this, note that $\bar D = Z \sqrt{\sum_{i=1}^n d_i^2 / n^2}$. Substituting this into the expression for $T$, we obtain

$$ \displaystyle T = \frac{Z \sqrt{\sum d*i^2 / n^2}}{\sqrt{\frac{(1 - Z^2 / n) \sum*{i=1}^n d_i^2 }{n-1} / n}} = \sqrt{\frac{n-1}{n}} \frac{Z}{\sqrt{1 - Z^2 / n}} = h(Z). $$

Therefore, the question of whether $t*{n-1}$ is a good approximation to the distribution of $T$ is the same as the question of whether $h^{-1} \circ t*{n-1}$ is a good approximation to the distribution of $Z = h^{-1}(T)$.

Now we have two candidate approximations to the permutation distribution of $Z$.

- $N(0, 1)$
- $h^{-1} \circ t_{n-1}$

It is easy to evaluate the quality of approximation 1, since the Edgeworth expansion of the permutation distribution of $Z$ is (Albers *et al*, 1976)

$$ F_Z(z) \approx \Phi(z) + \frac{1}{n} \frac{\phi(z)}{12} \kappa (z^3 - 3z), $$

where $\kappa = n \sum d_i^4 / (\sum d_i^2)^2$ is the kurtosis. The leading term in the expansion is the normal c.d.f. $\Phi(z)$. The next term

$$ \frac{1}{n} C_1(z) = \frac{1}{n} \frac{ \phi(z)}{12}\kappa (z^3 - 3z) $$

describes the quality of the approximation.

To evaluate the quality of approximation 2, we need an approximation to the distribution of $Z$ of the form:

$ F*Z(z) \approx F*{h^{-1} \circ t_{n-1}}(z) + \frac{1}{n} C_2(z). $

To do this, we will use two Edgeworth expansions:

- $$ F_Z(z) \approx \Phi(z) + \frac{1}{n} C_1(z), $$ where $C_1(z)$ is defined above.
- $$ F
*{h^{-1} \circ t*{n-1}}(z) \approx \Phi(z) + \frac{1}{n} C_3(z), $$ where $C_3(z)$ will be calculated below.

To obtain the desired Edgeworth expansion, we subtract the second expression from the first. Therefore, $C_2(z) = C_1(z) - C_3(z)$.

Now, to determine which approximation is better, we only need to compare $|C_1(z)|$ with $|C_2(z)| = |C_1(z) - C_3(z)|$. However, first we need to derive an expression for $C_3(z)$.

# Edgeworth Expansion of $h^{-1} \circ t_{n-1}$

To obtain the Edgeworth expansion of $h^{-1} \circ t_{n-1}$, we first observe that the c.d.f. can be written:

$F*{h^{-1} \circ t*{n-1}}(z) = F*{t*{n-1}}(h(z)). $

There is a standard Edgeworth expansion for the $t_{n-1}$ distribution:

$$ F*{t*{n-1}}(y) \approx \Phi(y) - \frac{1}{n - 1} \frac{\phi(y)}{4} (y^3 + y) \approx \Phi(y) - \frac{1}{n} \frac{\phi(y)}{4} (y^3 + y).$$

Now, we substitute $h(z) \approx \sqrt{\frac{n-1}{n}} (z + \frac{z^3}{2n}) \approx z + \frac{1}{2n} (z^3 - z)$ for $y$:

$$ \displaystyle F*{h^{-1} \circ t*{n-1}}(z) \approx F*{t*{n-1}}( z + \frac{1}{2n} (z^3 - z) ) $$

$$ \displaystyle \approx \Phi(z + \frac{1}{2n} (z^3 - z)) - \frac{1}{n} \frac{\phi(z + \frac{1}{2n} (z^3 - z))}{4} ((z + \frac{1}{2n} (z^3 - z))^3 + (z + \frac{1}{2n} (z^3 - z))). $$

$$ \displaystyle \approx \Phi(z + \frac{1}{2n} (z^3 - z)) - \frac{1}{n} \frac{\phi(z)}{4} (z^3 + z). $$ (Throw away terms of order $1/n^2$ or lower.)

$$ \displaystyle \approx \Phi(z) + \frac{1}{n}\frac{\phi(z)}{2} (z^3 - z) - \frac{1}{n} \frac{\phi(z)}{4} (z^3 + z) $$

$$\displaystyle = \Phi(z) + \frac{1}{n} \frac{\phi(z)}{4}(z^3 - 3z). $$

So we see that $C_3(z) = \frac{\phi(z)}{4}(z^3 - 3z)$.

# Finishing It Off

Now we need to figure out when

$$C_1(z) = \frac{ \phi(z)}{12}\kappa (z^3 - 3z)$$

is less than

$$C_2(z) = C_1(z) - C_3(z) = \frac{ \phi(z)}{12}\kappa (z^3 - 3z) - \frac{\phi(z)}{4}(z^3 - 3z) = \frac{\phi(z)}{12} (\kappa - 3) (z^3 - 3z) $$

in absolute value.

This is easy:

$$|C_1(z)| \lt |C_2(z)|$$

$$|\kappa| \lt |\kappa - 3|$$ (since all the other terms cancel)

$$ \kappa \leq 3/2 $$