Virtual Laboratories > Hypothesis Testing > 1 2 3 [4] 5 6 7

4. Tests in the Bernoulli Model

Preliminaries

Suppose that I1, I2, ..., In is a random sample from the Bernoulli distribution with unknown parameter p in (0, 1). Thus, these are independent indicator variables taking the values 1 and 0 with probabilities p and 1 - p respectively. Usually, this model arises in one of the following contexts:

  1. There is an event of interest in a basic experiment, with unknown probability p. We replicate the experiment n times and define Ii = 1 if and only if the event occurred on the i'th run.
  2. We have a population of objects of several different types; p is the unknown proportion of objects of a particular type of interest. We select n objects at random from the population and let Ii = 1 if and only if the i'th object is of the type of interest. When the sampling is with replacement, these variables really do form a random sample from the Bernoulli distribution. When the sampling is without replacement, the variables are dependent, but the Bernoulli model may still be approximately valid. For more on these points, see the chapter on Finite Sampling Models.

In this section, we will construct hypothesis tests for the parameter p. This section parallels the section on Estimation in the Bernoulli Model in the Chapter on Interval Estimation.

Tests of p

The parameter space is {p: 0 < p < 1}, and all hypotheses define subsets of this space. Recall that

N = I1 + I2 + ··· + In

has the binomial distribution with parameters n and p and has mean and variance given by

E(N) = np, var(N) = np(1 - p).

Moreover, N is sufficient for p, so it is natural to construct a test statistic from N. For r in (0, 1), let br(n, p) denote the quantile of order r for the binomial distribution with parameters n and p. Since the binomial distribution is discrete, only certain (exact) quantiles are possible.

Mathematical Exercise 1. Show that the following tests have significance level r:

  1. Reject H0: p = p0 versus H1: p <> p0 if and only if N < br/2(n, p0) or N > b1 - r/2(n, p0).
  2. Reject H0: p <= p0 versus H1: p > p0 if and only if N > b1 - r(n, p0).
  3. Reject H0: p >= p0 versus H1: p < p0 if and only if N < br(n, p0).

When n is large, the distribution of N is approximately normal, by the central limit theorem. Thus, an approximate normal test can be constructed using the test statistic

Z0 = (N - np0) / [np0(1 - p0)]1/2.

Note that Z0 is the standard score of N, under the null hypothesis. As usual, for r in (0, 1), let zr denote the quantile of order r for the standard normal distribution.

Mathematical Exercise 2. Show that if n is large, the following tests have approximate significance level r:

  1. Reject H0: p = p0 versus H1: p <> p0 if and only if Z0 > z1 - r/2 or Z0 < -z1 - r/2.
  2. Reject H0: p <= p0 versus H1: p > p0 if and only if Z0 > z1 - r.
  3. Reject H0: p >= p0 versus H1: p < p0 if and only if Z0 < -z1 - r.

Simulation Exercise 3. In the proportion test experiment, set H0: p = p0, n = 10, significance level 0.1, and p0 = 0.5.

  1. For each p = 0.1, 0.2, ..., 0.9, run the experiment 1000 times, updating every 10 runs, and then note the relative frequency of rejecting H0 for each value of p.
  2. When p = 0.5, compare the relative frequency with the significance level.
  3. Based on these relative frequencies, sketch the empirical power curve.

Simulation Exercise 4. In the proportion test experiment, repeat the previous exercise with n = 20.

Simulation Exercise 5. In the proportion test experiment, set H0: p <= p0, n = 15, significance level 0.05, and p0 = 0.3.

  1. For each p = 0.1, 0.2, ..., 0.9, run the experiment 1000 times, updating every 10 runs, and then note the relative frequency of rejecting H0 for each value of p.
  2. When p = 0.3, compare the relative frequency with the significance level.
  3. Based on these relative frequencies, sketch the imperial power curve.

Simulation Exercise 6. In the proportion test experiment, repeat the previous exercise with n = 30.

Simulation Exercise 7. In the proportion test experiment, set H0: p >= p0, n = 20, significance level 0.01, and p0 = 0.6.

  1. For each p = 0.1, 0.2, ..., 0.9, run the experiment 1000 times, updating every 10 runs, and then note the relative frequency of rejecting H0 for each value of p.
  2. When p = 0.6, compare the relative frequency with the significance level.
  3. Based on these relative frequencies, sketch the imperial power curve.

Simulation Exercise 8. In the proportion test experiment, repeat the previous exercise with n = 50.

The Sign Test

Suppose now that we have a basic random experiment with a random variable X of interest. We assume that X has a continuous distribution. Let p0 be a specified number in (0, 1), and let m denote quantile of order p0 for the distribution of X. Thus, by definition,

p0 = P(X < m).

Suppose that m is unknown and that we want to construct hypothesis tests for m. For a given test value m0, let

p = P(X < m0).

Mathematical Exercise 9. Show that

  1. m = m0 if and only if p = p0.
  2. m < m0 if and only if p > p0.
  3. m > m0 if and only if p < p0.

As usual, we repeat the basic experiment n times to generate a random sample of size n from the distribution of X:

X1, X2, ..., Xn.

Let Ii be the indicator variable of the event {Xi < m0} for i = 1, 2, ..., n.

Mathematical Exercise 10. Show that I1, I2, ..., In is a random sample of size n from the Bernoulli distribution with parameter p.

From Exercises 9 and 10, tests of the unknown quantile m can be converted to tests of the Bernoulli parameter p, and thus the tests developed in the previous subsections apply. This procedure is known as the sign test, because essentially, only the sign of Xi - m0 is recorded for each i. This procedure is also an example of a nonparametric test, because no assumptions about the distribution of X are made (except for continuity). In particular, we do not need to assume that the distribution of X belongs to a particular parametric family.

The most important special case of the sign test is the case where p0 = 1/2; this is the sign test of the median. If the distribution of X is known to be symmetric, the median and the mean agree. In this case, sign tests of the median are also tests of the mean.

Simulation Exercise 11. In the sign test experiment, set the sampling distribution to normal with mean 0 and standard deviation 2. Set the sample size to 10 and the significance level to 0.1. For each of the 9 values of m0, run the simulation 1000 times, updating every 10 runs.

  1. When m0 = m, give the empirical estimate of the significance level of the test and compare with 0.1.
  2. In the other cases, give the empirical estimate of the power of the test.

Simulation Exercise 12. In the sign test experiment, set the sampling distribution to uniform on the interval [0, 5]. Set the sample size to 20 and the significance level to 0.05. For each of the 9 values of m0, run the simulation 1000 times, updating every 10 runs.

  1. When m0 = m, give the empirical estimate of the significance level of the test and compare with 0.05.
  2. In the other cases, give the empirical estimate of the power of the test.

Simulation Exercise 13. In the sign test experiment, set the sampling distribution to gamma with shape parameter a = 2 and scale parameter r = 1 . Set the sample size to 30 and the significance level to 0.025. For each of the 9 values of m0, run the simulation 1000 times, updating every 10 runs.

  1. When m0 = m, give the empirical estimate of the significance level of the test and compare with 0.025.
  2. In the other cases, give the empirical estimate of the power of the test.

Computational Exercises

Mathematical Exercise 14. In a pole of 1000 registered voters in a certain district, 427 prefer candidate X. At the 0.1 level, is the evidence sufficient to conclude that more that 40% of the registered voters prefer X?

Mathematical Exercise 15. A coin is tossed 500 times and results in 302 heads. At the 0.05 level, test to see if the coin is unfair.

Mathematical Exercise 16. A sample of 400 memory chips from a production line are tested, and 30 are defective. At the 0.05 level, test to see if the proportion of defective chips is less than 0.1.

Mathematical Exercise 17. A new drug is administered to 50 patients and the drug is effective in 42 cases. At the 0.1 level, test to see if the success rate for the new drug is greater that 0.8.

Data Analysis Exercise 18. Using the M&M data, test the following alternative hypotheses at the 0.1 significance level:

  1. The proportion of red M&Ms differs from 1/6.
  2. The proportion of green M&Ms is less than 1/6
  3. The proportion of yellow M&M is greater than 1/6

Data Analysis Exercise 19. Using the M&M data, test to see if the median weight exceeds 47.9 grams, at the 0.1 level.

Data Analysis Exercise 20. Using Fisher's iris data, perform the following tests, at the 0.1 level:

  1. The median petal length of Setosa irises differs from 15 mm.
  2. The median petal length of Verginica irises is greater than 52 mm.
  3. The median petal length of Versicolor irises is less than 42 mm.

Virtual Laboratories > Hypothesis Testing > 1 2 3 [4] 5 6 7
Contents | Applets | Data Sets | Biographies | Resources | Keywords | ©