Bonferoni t-test

The problem can be formulated as follows: Given contrasts (between means) , , , , we would like to test the null hypothesis that all of them are . We have two significance levels:

  • For an individual contrast. We will call it .
  • Overall, for all contrasts simultaneously. We will call it

We must assume that the estimators of the contrasts are independent random variables. Perhaps we should recall that a contrast is a property of the joint distribution of the treatment groups:

An estimator of the contrast is a function of the sample, or a random variable. The mean of the -th treatment group is estimated as usual:

The estimators of contrasts are:

We also recall that the condition of independence, provided that the usual assumptions of normality and homoskedasticity (homogeneity of varances) is that the covariance between contrasts is . We already calculated the covariance to be:

where is the overall population variance.
For equal treatment groups, the condition for is equivalent to saying that the matrix has orthogonal columns. We recall that matrices of this sort are used to describe contrasts in the package software R. For uneven sizes of treatment groups, we modify the notion of orthogonality, so that the covariance condition still holds.
The Bonferoni t-test is based on a very simple observation that, if the contrasts are independent random variables, the probability of all of them simultanously being smaller than the  critical value is . Thus, the probability that at least one of them exceeds the critical value is . Thus the equation

. Hence, in order to test the simultaneous null hypothesis, we simply perform individual tests with the significance level which is determined from this formula, based on a given value of . Hence:

Another observation, which together with the prior observations, constitutes the Bonferoni t-test, is the following inequality:

which holds for all values . Thus, if then . Hence, if we set:

then we get a slightly weaker test, but asymptotically, as , we get essentially no difference.
Finally, we know that the null hypothesis for an individual contrast is based on the Student -distribution of the statistic:

where

is the estimator of the variance.