Virtual Laboratories > Special Distributions > [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15

1. Introduction

In this chapter, we will study a number of parametric families of distributions that have special importance in probability and statistics. In some cases, a distribution is important because it occurs as the limit of many other distributions. In several cases, a parametric family is important because it can be used to model a wide variety of random phenomena. In turn, this is usually the case because the family has a rich collection of densities with a small number of parameters (usually 1 or 2). As a general philosophical principal, we try to model a random process with as few parameters as possible; this is sometimes referred to as the principal of parsimony of parameters. In turn, this is a special case of Ockham's razor, named in honor of William of Ockham, the principle that states that one should use the simplest model that adequately describes a given phenomenon.

There are several other parametric families of distributions that are studied elsewhere in this project, because the natural home for these distributions are various random processes. These include

• The binomial distribution
• The negative binomial distribution
• The multinomial distribution
• The hypergeometric distribution
• The multivariate hypergeometric distribution
• The Poisson distribution

Before we begin our study of special parametric families of distributions, we will study two general parametric families. Many of the special parametric families studied in this chapter belong to one or both of these general families.

Location-Scale Families

1. Suppose that a real-valued random variable Z has a continuous distribution with density function g and distribution function G. Let a, b be constants with b > 0. Show that X = a + bZ has density function f and distribution function F given by

1. F(x) = G[(x - a) / b]
2. f(x) = (1 / b) g[(x - a) / b]

This two-parameter family of distributions is called the location-scale family associated with the given distribution; a is called the location parameter and b the scale parameter. In the special case that b = 1, the one-parameter family is called the location family associated with the given distribution, and in the special case that a = 0, the one-parameter family is called the scale family associated with the given distribution.

2. Interpret the location and scale parameters graphically:

1. For the location family associated with g, show that the graph of f is obtained by shifting the graph of g, a units to the right if a > 0 and -a units to the left if a < 0.
2. For the scale family associated with g, show that if b > 1, the graph of f is obtained from the graph of g by stretching horizontally and compressing vertically, by a factor of b. If 0 < b < 1, the graph of f is obtained from the graph of g by compressing horizontally and stretching vertically, by a factor of b.

3. Show that if Z has a mode at z, then X has a mode at x = a + bz.

The following exercise relates the quantile functions.

4. Show that

1. F^-1(p) = a + bG^-1(p) for p in (0, 1).
2. If z is a quantile of order p for Z then x = a + bz is a quantile of order p for X.

5. Show that the uniform distribution on the interval (a, a + b), where a in R and b > 0 are parameters, is a location-scale family.

6. Let g(z) = exp(-z) for z > 0. This is the density function for the exponential distribution with parameter 1.

1. Find the location-scale family of densities.
2. Sketch the graphs.

The family of distributions in the previous exercise is known as the two-parameter exponential distribution.

7. Let g(z) = 1 / [(1 + z²)] for z in R. This is the density of the Cauchy distribution, named after Augustin Cauchy.

1. Find the location-scale family of densities.
2. Sketch the graphs.

The following exercise relates the mean and variance.

8. Show that

1. E(X) = a + bE(Z)
2. var(X) = b² var(Z)

The following exercise relates the moment generating functions:

9. Suppose that Z has moment generating function M. Show that X has moment generating function N given by

N(t) = exp(ta)M(tb).

Exponential Families

Suppose that X is random variable taking values in S, and that the distribution of X depends on an unspecified parameter a taking values in a parameter space A. In general, both X and a may be vector-valued. We will write f(x | a) for the density function of X at x in S, corresponding to a in A.

The distribution of X is a k-parameter exponential family if S does not depend on a and if the density function f can be written as

f(x | a) = c(a) r(x) exp[_{i = 1, ..., k} b_i(a) h_i(x)] for x S, a A.

where c and b₁, b₂, ..., b_k are functions on A, and where r and h₁, h₂, ..., h_k are functions on S. Moreover, k is assumed to be the smallest such integer. The parameters b₁(a), b₂(a), ..., b_k(a) are sometimes called natural parameters of the distribution, and the random variables h₁(X), h₂(X), ..., h_k(X) are sometimes called natural statistics of the distribution.

10. Suppose that X has the binomial distribution with parameters n and p, where n is fixed and p is in (0, 1). Show that the distribution is a one-parameter exponential family with natural parameter ln[(p / (1 - p)] and natural statistic X.

11. Suppose that X has the Poisson distribution with parameter a > 0. Show that the distribution is a one-parameter exponential family with natural parameter ln(a) and natural statistic X.

12. Suppose that X has the negative binomial distribution with parameters k and p, where k is fixed and p is in (0, 1). Show that the distribution is a one-parameter exponential family with natural parameter ln(1 - p) and natural statistic X.

In many cases, the distribution of a random variable X will fail to be an exponential family if the support set defined below depends on the parameter a.

{x: f(x | a) > 0}.

13. Suppose that X has the uniform distribution on (0, a) where a > 0. Show that the distribution of X is not an exponential family.

The next exercise shows that if we sample from the distribution of an exponential family, then the distribution of the random sample is itself an exponential family with the same natural statistics.

14. Suppose that the distribution of random variable X is a k-parameter exponential family with natural parameters b₁, b₂, ..., b_k, and natural statistics h₁(X), h₂(X), ..., h_k(X). Let X₁, X₂, ..., X_n be independent, each with the same distribution as X. Show that Y = (X₁, X₂, ..., X_n) is a k-parameter exponential family with natural parameters b₁, b₂, ..., b_k. and natural statistics

u_j(Y) = _{i = 1, ..., n} h_j(X_i) for j = 1, 2, ..., k.

Virtual Laboratories > Special Distributions > [1] 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Contents | Applets | Data Sets | Biographies | Resources | Keywords | ©