We have observational units, which we divide into 2 blocks (say, and ) of and experimental units respectively, in advance of an experiment. Thus, .
We conduct the experiment consisting in applying a different treatment to each block. We measure a non-categorical response , resulting in measurements .
Let and be the group means of the two groups.
Let
TeX Embedding failed!
be the mean difference.
We would like to test whether treatment has significantly larger mean than treatment . If there is no difference (the null hypothesis, or ), the assignment of the observational units to the blocks has no effect, and and are only labels. Thus, any label assignment should produce the same result as the selected one.
There are
TeX Embedding failed!
ways to assign labels and to observational units, and under null hypothesis, all of them are equally probable. Thus, we may consider a pure thought experiment in which we consider all possible label assignments. For every such assignment, we may compute the difference of the means .
The proportion of label assignments for which the diference is larger than that of the original assignment is the significance level of the test.
It can be shown that the normal theory can be applied to analyze the resulting test from the point of view of significance level and power. Below we outline the corresponding argument.
We see that
TeX Embedding failed!
where
TeX Embedding failed!
with probability each and is the value of the response of an observational unit.
Since all random variables and are jointly independent, the conditional distribution of conditioned on some particular set of values of variables is exactly the same as the marginal distribution with all discarded.
Thus, the distribution of is that of a difference of two random variables , where is an average of independent copies of and is an average of independent copies of . Thus, if and are approximately and thus large, by the Central Limit Theorem both these variables are approximately normally distributed with the same mean and variance as . Hence, we may assume that the distributions are approximately and .
We observe that we would come to exactly the same conclusion if the distribution of the response variable were .
