Blog

Contents 1. Preview 1 1.1 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Univariate Random Variables 5 2.1 Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 2.3 Discrete Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 2.4 Continuous Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.5 Location and Scale Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 27 2.6 Functions of a Random Variable . . . . . . . . . . . . . . . . . . . . . . . . 30 2.7 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 2.8 Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 2.9 Variance Stabilizing Transformation . . . . . . . . . . . . . . . . . . . . . . 44 2.10 Moment Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . . 45 2.11 Calculus Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 2.12 Chapter 2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 3. Multivariate Random Variables 61 3.1 Joint and Marginal Cumulative Distribution Functions . . . . . . . . . . . . 62 3.2 Bivariate Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 63 3.3 Bivariate Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . 66 3.4 Independent Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . 76 3.5 Conditional Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 3.6 Joint Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 3.7 Conditional Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 3.8 Joint Moment Generating Functions . . . . . . . . . . . . . . . . . . . . . . 102 3.9 Multinomial Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 3.10 Bivariate Normal Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 111 3.11 Calculus Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114 3.12 Chapter 3 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 iii iv CONTENTS 4. Functions of Two or More Random Variables 121 4.1 Cumulative Distribution Function Technique . . . . . . . . . . . . . . . . . 121 4.2 One-to-One Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . 125 4.3 Moment Generating Function Technique . . . . . . . . . . . . . . . . . . . . 137 4.4 Chapter 4 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147 5. Limiting or Asymptotic Distributions 151 5.1 Convergence in Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 152 5.2 Convergence in Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 5.3 Weak Law of Large Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 159 5.4 Moment Generating Function Technique for Limiting Distributions . . . . . 167 5.5 Additional Limit Theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 5.6 Chapter 5 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 6. Maximum Likelihood Estimation – One Parameter 183 6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 184 6.2 Maximum Likelihood Method . . . . . . . . . . . . . . . . . . . . . . . . . . 185 6.3 Score and Information Functions . . . . . . . . . . . . . . . . . . . . . . . . 191 6.4 Likelihood Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204 6.5 Limiting Distribution of Maximum Likelihood Estimator . . . . . . . . . . . 208 6.6 Con dence Intervals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 6.7 Approximate Con dence Intervals . . . . . . . . . . . . . . . . . . . . . . . 218 6.8 Chapter 6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 7. Maximum Likelihood Estimation – Multiparameter 227 7.1 Likelihood and Related Functions . . . . . . . . . . . . . . . . . . . . . . . . 228 7.2 Likelihood Regions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237 7.3 Limiting Distribution of Maximum Likelihood Estimator . . . . . . . . . . . 240 7.4 Approximate Con dence Regions . . . . . . . . . . . . . . . . . . . . . . . . 242 7.5 Chapter 7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 8. Hypothesis Testing 253 8.1 Test of Hypothesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 8.2 Likelihood Ratio Tests for Simple Hypotheses . . . . . . . . . . . . . . . . . 257 8.3 Likelihood Ratio Tests for Composite Hypotheses . . . . . . . . . . . . . . . 265 8.4 Chapter 8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 9. Solutions to Chapter Exercises 275 9.1 Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276 9.2 Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 9.3 Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305 9.4 Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 316 CONTENTS v 9.5 Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 318 9.6 Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325 9.7 Chapter 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 10. Solutions to Selected End of Chapter Problems 337 10.1 Chapter 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 338 10.2 Chapter 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 383 10.3 Chapter 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412 10.4 Chapter 5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 435 10.5 Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 448 10.6 Chapter 7 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460 10.7 Chapter 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465 11. Summary of Named Distributions 469 12. Distribution Tables 471 0 CONTENTS Preface In order to provide improved versions of these Course Notes for students in subsequent terms, please email corrections, sections that are confusing, or comments/suggestions to [email protected]. 1. Preview The following examples will illustrate the ideas and concepts discussed in these Course Notes. They also indicate how these ideas and concepts are connected to each other. 1.1 Example The number of service interruptions in a communications system over 200 separate days is summarized in the following frequency table: Number of interruptions 0 1 2 3 4 5 > 5 Total Observed frequency 64 71 42 18 4 1 0 200 It is believed that a Poisson model will t these data well. Why might this be a reasonable assumption (PROBABILITY MODELS ) If we let the random variable X = number of interruptions in a day and assume that the Poisson model is reasonable then the probability function of X is given by P (X = x) = xe x! for x = 0; 1; : : : where is a parameter of the model which represents the mean number of service inter- ruptions in a day. (RANDOM VARIABLES, PROBABILITY FUNCTIONS, EXPEC- TATION, MODEL PARAMETERS ) Since is unknown we might estimate it using the sample mean x = 64(0) + 71(1) + + 1(5) 200 = 230 200 = 1:15 (POINT ESTIMATION ) The estimate ^ = x is the maximum likelihood estimate of . It is the value of which maximizes the likelihood function. (MAXIMUM LIKELIHOOD ESTIMATION ) The likelihood function is the probability of the observed data as a function of the unknown parameter(s) in the model. The maximum likelihood estimate is thus the value of which maximizes the probability of observing the given data. 1 2 1. PREVIEW In this example the likelihood function is given by L() = P (observing 0 interruptions 64 times,. . . , > 5 interruptions 0 times;) = 200! 64!71! 1!0! 0e 0! 64 1e 1! 71 5e 5! 1 1P x=6 xe x! 0 = c64(0)+71(1)++1(5)e(64+71++1) = c230e200 for > 0 where c = 200! 64!71! 1!0! 1 0! 64 1 1! 71 : : : 1 5! 1 The maximum likelihood estimate of can be found by solving dLd = 0 or equivalently d logL d = 0 and verifying that it corresponds to a maximum. If we want an interval of values for which are reasonable given the data then we could construct a con dence interval for . (INTERVAL ESTIMATION ) To construct con dence intervals we need to nd the sampling distribution of the estimator. In this example we would need to nd the distribution of the estimator X = X1 +X2 + +Xn n where Xi = number of interruptions in a day i, i = 1; 2; : : : ; 200. (FUNCTIONS OF RANDOM VARIABLES: cumulative distribution function technique, one-to-one transfor- mations, moment generating function technique) Since Xi Poisson() with E(Xi) = and V ar(Xi) = the distribution of X for large n is approximately N(; =n) by the Central Limit Theorem. (LIMITING DISTRIBUTIONS ) Suppose the manufacturer of the communications system claimed that the mean number of interruptions was 1. Then we would like to test the hypothesis H : = 1. (TESTS OF HYPOTHESIS ) A test of hypothesis uses a test statistic to measure the evidence based on the observed data against the hypothesis. A test statistic with good properties for testing H : = 0 is the likelihood ratio statistic, 2 log [L (0) =L (^)]. (LIKELIHOOD RATIO STATISTIC ) For large n the distribution of the likelihood ratio statistic is approximately 2 (1) if the hypothesis H : = 0 is true. 1.2 Example The following are relief times in hours for 20 patients receiving a pain killer: 1:1 1:4 1:3 1:7 1:9 1:8 1:6 2:2 1:7 2:7 4:1 1:8 1:5 1:2 1:4 3:0 1:7 2:3 1:6 2:0 It is believed that the Weibull distribution with probability density function f(x) = x1e(x=) for x > 0; > 0; > 0 1.2. EXAMPLE 3 will provide a good t to the data. (CONTINUOUS MODELS, PROBABILITY DENSITY FUNCTIONS ) Assuming independent observations the (approximate) likelihood function is L(; ) = 20Q i=1 x1i e (xi=) for > 0; > 0 where xi is the observed relief time for the ith patient. (MULTIPARAMETER LIKELI- HOODS ) The maximum likelihood estimates ^ and ^ are found by simultaneously solving @ logL @ = 0 and @ logL @ = 0 Since an explicit solution to these equations cannot be obtained, a numerical solution must be found using an iterative method. (NEWTON S METHOD) Also, since the maximum likelihood estimators cannot be given explicitly, approximate con dence intervals and tests of hypothesis must be based on the asymptotic distributions of the maximum likelihood estimators. (LIMITING OR ASYMPTOTIC DISTRIBUTIONS OF MAXIMUM LIKE- LIHOOD ESTIMATORS ) 4 1. PREVIEW 2. Univariate Random Variables In this chapter we review concepts that were introduced in a previous probability course such as STAT 220/230/240 as well as introducing new concepts. In Section 2:1 the concepts of random experiments, sample spaces, probability models, and rules of probability are re- viewed. The concepts of a sigma algebra and probability set function are also introduced. In Section 2:2 we de ne a random variable and its cumulative distribution. It is important to note that the de nition of a cumulative distribution is the same for all types of ran- dom variables. In Section 2:3 we de ne a discrete random variable and review the named discrete distributions (Hypergeometric, Binomial, Geometric, Negative Binomial, Poisson). In Section 2:4 we de ne a continuous random variable and review the named continuous distributions (Uniform, Exponential, Normal, and Chi-squared). We also introduce new continuous distributions (Gamma, Two Parameter Exponential, Weibull, Cauchy, Pareto). A summary of the named discrete and continuous distributions that are used in these Course Notes can be found in Chapter 11. In Section 2:5 we review the cumulative distribution function technique for nding a function of a random variable and prove a theorem which can be used in the case of a monotone function. In Section 2:6 we review expectations of functions of random variables. In Sections 2:8 2:10 we introduce new material related to expectation such as inequalities, variance stabilizing transforma- tions and moment generating functions. Section 2.11 contains a number of useful calculus results which will be used throughout these Course Notes. 2.1 Probability To model real life phenomena for which we cannot predict exactly what will happen we assign numbers, called probabilities, to outcomes of interest which re ect the likelihood of such outcomes. To do this it is useful to introduce the concepts of an experiment and its associated sample space. Consider some phenomenon or process which is repeatable, at least in theory. We call the phenomenon or process a random experiment and refer to a single repetition of the experiment as a trial. For such an experiment we consider the set of all possible outcomes. 5 6 2. UNIVARIATE RANDOM VARIABLES 2.1.1 De nition – Sample Space A sample space S is a set of all the distinct outcomes for a random experiment, with the property that in a single trial, one and only one of these outcomes occurs. To assign probabilities to the events of interest for a given experiment we begin by de ning a collection of subsets of a sample space S which is rich enough to de ne all the events of interest for the experiment. We call such a collection of subsets a sigma algebra. 2.1.2 De nition – Sigma Algebra A collection of subsets of a set S is called a sigma algebra, denoted by B, if it satis es the following properties: (1) 2 B where is the empty set (2) If A 2 B then A 2 B (3) If A1; A2; : : : 2 B then 1S i=1 Ai 2 B Suppose A1; A2; : : : are subsets of the sample space S which correspond to events of interest for the experiment. To complete the probability model for the experiment we need to assign real numbers P (Ai) ; i = 1; 2; : : :, where P (Ai) is called the probability of Ai. To develop the theory of probability these probabilities must satisfy certain properties. The following Axioms of Probability are a set of axioms which allow a mathematical structure to be developed. 2.1.3 De nition – Probability Set Function Let B be a sigma algebra associated with the sample space S. A probability set function is a function P with domain B that satis es the following axioms: (A1) P (A) 0 for all A 2 B (A2) P (S) = 1 (A3) If A1; A2; : : : 2 B are pairwise mutually exclusive events, that is, Ai Aj = for all i 6= j, then P 1S i=1 Ai = 1P i=1 P (Ai) Note: The probabilities P (A1) ; P (A2) ; : : : can be assigned in any way as long they satisfy these three axioms. However, if we wish to model real life phenomena we would assign the probabilities such that they correspond to the relative frequencies of events in a repeatable experiment. 2.1. PROBABILITY 7 2.1.4 Example Let B be a sigma algebra associated with the sample space S and let P be a probability set function with domain B. If A;B 2 B then prove the following: (a) P ( ) = 0 (b) If A;B 2 B and A and B are mutually exclusive events then P (A [B) = P (A)+P (B). (c) P A = 1 P (A) (d) If A B then P (A) P (B) Note: A B means a 2 A implies a 2 B. Solution (a) Let A1 = S and Ai = for i = 2; 3; : : :. Since 1S i=1 Ai = S then by De nition 2.1.3 (A3) it follows that P (S) = P (S) + 1P i=2 P ( ) and by (A2) we have 1 = 1 + 1P i=2 P ( ) By (A1) the right side is a series of non-negative numbers which must converge to the left side which is 1 which is nite which results in a contradiction unless P ( ) = 0 as required. (b) Let A1 = A, A2 = B, and Ai = for i = 3; 4; : : :. Since 1S i=1 Ai = A [B then by (A3) P (A [B) = P (A) + P (B) + 1P i=3 P ( ) and since P ( ) = 0 by the result (a) it follows that P (A [B) = P (A) + P (B) (c) Since S = A [ A and A A = then by (A2) and the result proved in (b) it follows that 1 = P (S) = P A [ A = P (A) + P A or P A = 1 P (A) (d) Since B = (A B) [ A B = A [ A B and A A B = then by (b) P (B) = P (A) +P A B. But by (A1), P

A B 0 so it follows that P (B) P (A). 2.1.5 Exercise Let B be a sigma algebra associated with the sample space S and let P be a probability set function with domain B. If A;B 2 B then prove the following: (a) 0 P (A) 1 (b) P A B = P (A) P (A B) (c) P (A [B) = P (A) + P (B) P (A B) 8 2. UNIVARIATE RANDOM VARIABLES For a given experiment we are sometimes interested in the probability of an event given that we know that the event of interest has occurred in a certain subset of S. For example, the experiment might involve people of di¤erent ages and we may be interest in an event only for a given age group. This leads us to de ne conditional probability. 2.1.6 De nition – Conditional Probability Let B be a sigma algebra associated with the sample space S and suppose A;B 2 B. The conditional probability of event A given event B is P (AjB) = P (A B) P (B) provided P (B) > 0 2.1.7 Example The following table of probabilities are based on data from the 2011 Canadian census. The probabilities are for Canadians aged 25 34. Highest level of education attained Employed Unemployed No certi cate, diploma or degree 0:066 0:010 High school diploma or equivalent 0:185 0:016 Postsecondary certi cate, diploma or degree 0:683 0:040 If a person is selected at random what is the probability the person (a) is employed (b) has no certi cate, diploma or degree (c) is unemployed and has at least a high school diploma or equivalent (d) has at least a high school diploma or equivalent given that they are unemployed Solution (a) Let E be the event employed , A1 be the event no certi cate, diploma or degree , A2 be the event high school diploma or equivalent , and A3 be the event postsecondary certi cate, diploma or degree . P (E) = P (E A1) + P (E A2) + P (E A3) = 0:066 + 0:185 + 0:683 = 0:934 2.1. PROBABILITY 9 (b) P (A1) = P (E A1) + P E A1 = 0:066 + 0:010 = 0:076 (c) P (unemployed and have at least a high school diploma or equivalent) = P E (A2 [A3) = P E A2 + P E A3 = 0:016 + 0:040 = 0:056 (d) P A2 [A3j E = P E (A2 [A3) P E = 0:056 0:066 = 0:848 If the occurrence of event B does not a¤ect the probability of the event A, then the events are called independent events. 2.1.8 De nition – Independent Events Let B be a sigma algebra associated with the sample space S and suppose A;B 2 B. A and B are independent events if P (A B) = P (A)P (B) 2.1.9 Example In Example 2.1.7 are the events, unemployed and no certi cate, diploma or degree , independent events Solution The events unemployed and no certi cate, diploma or degree are not independent since 0:010 = P E A1 6= P EP (A1) = (0:066) (0:076) 10 2. UNIVARIATE RANDOM VARIABLES 2.2 Random Variables A probability model for a random experiment is often easier to construct if the outcomes of the experiment are real numbers. When the outcomes are not real numbers, the outcomes can be mapped to numbers using a function called a random variable. When the observed data are numerical values such as the number of interruptions in a day in a communications system or the length of time until relief after taking a pain killer, random variables are still used in constructing probability models. 2.2.1 De nition of a Random Variable A random variable X is a function from a sample space S to the real numbers <, that is, X : S ! < such that P (X x) is de ned for all x 2 <. Note: X x is an abbreviation for f! 2 S : X(!) xg where f! 2 S : X(!) xg 2 B and B is a sigma algebra associated with the sample space S. 2.2.2 Example Three friends Ali, Benita and Chen are enrolled in STAT 330. Suppose we are interested in whether these friends earn a grade of 70 or more. If we let A represent the event Ali earns a grade of 70 or more , B represent the event Benita earns a grade of 70 or more , and C represent the event Chen earns a grade of 70 or more then a suitable sample space is S = ABC; ABC;A BC;AB C; A BC;A B C; AB C; A B C Suppose we are mostly interested in how many of these friends earn a grade of 70 or more. We can de ne the random variable X = number of friends who earn a grade of 70 or more . The range of X is f0; 1; 2; 3g with associated mapping X (ABC) = 3 X ABC = X A BC = X AB C = 2 X A BC = X A B C = X AB C = 1 X A B C = 0 An important function associated with random variables is the cumulative distribution function. 2.2. RANDOM VARIABLES 11 2.2.3 De nition - Cumulative Distribution Function The cumulative distribution function (c.d.f.) of a random variable X is de ned by F (x) = P (X x) for x 2 < Note: The cumulative distribution function is de ned for all real numbers. 2.2.4 Properties - Cumulative Distribution Function (1) F is a non-decreasing function, that is, F (x1) F (x2) for all x1 < x2 (2) lim x!1F (x) = 0 and limx!1F (x) = 1 (3) F is a right-continuous function, that is, lim x!a+ F (x) = F (a) (4) For all a < b P (a < X b) = P (X b) P (X a) = F (b) F (a) (5) For all b P (X = b) = F (b) lim a!b F (a) 2.2.5 Example Suppose X is a random variable with cumulative distribution function F (x) = P (X x) = 8>>>>>>><>>>>>>>: 0 x < 0 0:1 0 x < 1 0:3 1 x < 2 0:6 2 x < 3 1 x 3 (a) Graph the function F (x). (b) Determine the probabilities (i) P (X 1) (ii) P (X 2) (iii) P (X 2:4) (iv) P (X = 2) (v) P (0 < X 2) 12 2. UNIVARIATE RANDOM VARIABLES -1 -0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1 x F(x) Figure 2.1: Graph of F (x) = P (X x) for Example 2.2.5 (vi) P (0 X 2). Solution (a) See Figure 2.1. (b) (i) P (X 1) = F (1) = 0:3 (ii) P (X 2) = F (2) = 0:6 (iii) P (X 2:4) = P (X 2) = F (2) = 0:6 (iv) P (X = 2) = F (2) lim x!2 F (x) = 0:6 0:3 = 0:3 or P (X = 2) = P (X 2) P (X 1) = F (2) F (1) = 0:6 0:3 = 0:3 (v) P (0 < X 2) = P (X 2) P (X 0) = F (2) F (0) = 0:6 0:1 = 0:5 (vi) P (0 X 2) = P (X 2) P (X < 0) = F (2) 0 = 0:6 2.2. RANDOM VARIABLES 13 2.2.6 Example Suppose X is a random variable with cumulative distribution function F (x) = P (X x) = 8>>>>><>>>>>: 0 x 0 2 5x 3 0 < x < 1 1 5 12x 3x2 7 1 x < 2 1 x 2 (a) Graph the function F (x). (b) Determine the probabilities (i) P (X 1) (ii) P (X 2) (iii) P (X 2:4) (iv) P (X = 0:5) (v) P (X = b), for b 2 < (vi) P (1 < X 2:4) (vii) P (1 X 2:4). Solution (a) See Figure 2.2 0 0.5 1 1.5 2 0 0.2 0.4 0.6 0.8 1 x F(x ) Figure 2.2: Graph of F (x) = P (X x) for Example 2.2.6 14 2. UNIVARIATE RANDOM VARIABLES (b) (i) P (X 1) = F (1) = 2 5 (1)3 = 2 5 = 0:4 (ii) P (X 2) = F (2) = 1 (iii) P (X 2:4) = F (2:4) = 1 (iv) P (X = 0:5) = F (0:5) lim x!0:5 F (x) = F (0:5) F (0:5) = 0 (v) P (X = b) = F (b) lim a!b F (a) = F (b) F (b) = 0 for all b 2 < (vi) P (1 < X 2:4) = F (2:4) F (1) = 1 0:4 = 0:6 (vii) P (1 X 2:4) = P (1 < X 2:4) since P (X = 1) = 0 = 0:6 2.3. DISCRETE RANDOM VARIABLES 15 2.3 Discrete Random Variables A set A is countable if the number of elements in the set is nite or the elements of the set can be put into a one-to-one correspondence with the positive integers. 2.3.1 De nition - Discrete Random Variable A random variable X de ned on a sample space S is a discrete random variable if there is a countable subset A < such that P (X 2 A) = 1. 2.3.2 De nition - Probability Function If X is a discrete random variable then the probability function (p.f.) of X is given by f(x) = P (X = x) = F (x) lim "!0+ F (x ") for x 2 < The set A = fx : f(x) > 0g is called the support set of X. 2.3.3 Properties – Probability Function (1) f(x) 0 for x 2 < (2) P x2A f(x) = 1 2.3.4 Example In Example 2.2.5 nd the support set A, show that X is a discrete random variable and determine its probability function. Solution The support set of X is A = f0; 1; 2; 3g which is a countable set. Its probability function is f(x) = P (X = x) = 8>>>>><>>>>>: 0:1 if x = 0 P (X 1) P (X 0) = 0:3 0:1 = 0:2 if x = 1 P (X 2) P (X 1) = 0:6 0:3 = 0:3 if x = 2 P (X 3) P (X 2) = 1 0:6 = 0:4 if x = 3 or x 0 1 2 3 Total f(x) = P (X = x) 0:1 0:2 0:3 0:4 1 16 2. UNIVARIATE RANDOM VARIABLES Since P (X 2 A) = 3P x=0 P (X = x) = 1, X is a discrete random variable. In the next example we review four of the named distributions which were introduced in a previous probability course. 2.3.5 Example Suppose a box containing a red balls and b black balls. For each of the following nd the probability function of the random variable X and show that P x2A f(x) = 1 where A = fx : f(x) > 0g is the support set of X. (a) X = number of red balls among n balls drawn at random without replacement. (b) X = number of red balls among n balls drawn at random with replacement. (c) X = number of black balls selected before obtaining the rst red ball if sampling is done at random with replacement. (d) X = number of black balls selected before obtaining the kth red ball if sampling is done at random with replacement. Solution (a) If n balls are selected at random without replacement from a box of a red balls and b black balls then the random variable X = number of red balls has a Hypergeometric distribution with probability function f (x) = P (X = x) = a x b n x a+ b n for x = max (0; n b) ; : : : ;min (a; n) By the Hypergeometric identity 2.11.6 P x2A f(x) = min(a;n)P x=max(0;nb) a x b n x a+ b n = 1 a+ b n 1P x=0 a x b n x = a+ b n a+ b n = 1 2.3. DISCRETE RANDOM VARIABLES 17 (b) If n balls are selected at random with replacement from a box of a red balls and b black balls then we have a sequence of Bernoulli trials and the random variable X = number of red balls has a Binomial distribution with probability function f (x) = P (X = x) = n x px (1 p)nx for x = 0; 1; : : : ; n where p = aa+b . By the Binomial series 2.11.3(1) P x2A f(x) = nP x20 n x px (1 p)nx = (p+ 1 p)n = 1 (c) If sampling is done with replacement then we have a sequence of Bernoulli trials and the random variable X = number of black balls selected before obtaining the rst red ball has a Geometric distribution with probability function f (x) = P (X = x) = p (1 p)x for x = 0; 1; : : : By the Geometric series 2.11.1 P x2A f(x) = 1P x20 p (1 p)x = p [1 (1 p)] = 1 (d) If sampling is done with replacement then we have a sequence of Bernoulli trials and the random variable X = number of black balls selected before obtaining the kth red ball has a Negative Binomial distribution with probability function f (x) = P (X = x) = x+ k 1 x pk (1 p)x for x = 0; 1; : : : Using the identity 2.11.4(1) x+ k 1 x = k x (1)x the probability function can be written as f (x) = P (X = x) = k x pk (p 1)x for x = 0; 1; : : : 18 2. UNIVARIATE RANDOM VARIABLES By the Binomial series 2.11.3(2)P x2A f(x) = 1P x20 k x pk (p 1)x = pk 1P x20 k x (p 1)x = pk (1 + p 1)k = 1 2.3.6 Example If X is a random variable with probability function f(x) = xe x! for x = 0; 1; : : : ; > 0 (2.1) show that 1P x=0 f(x) = 1 Solution By the Exponential series 2.11.7 1P x=0 f(x) = 1P x=0 xe x! = e 1P x=0 x x! = ee = 1 The probability function (2.1) is called the Poisson probability function. 2.3.7 Exercise If X is a random variable with probability function f(x) = (1 p)x x log p for x = 1; 2; : : : ; 0 < p < 1 show that 1P x=1 f(x) = 1 Hint: Use the Logarithmic series 2.11.8. Important Note: A summary of the named distributions used in these Course Notes can be found in Chapter 11. 2.4. CONTINUOUS RANDOM VARIABLES 19 2.4 Continuous Random Variables 2.4.1 De nition - Continuous Random Variable Suppose X is a random variable with cumulative distribution function F . If F is a continu- ous function for all x 2 < and F is di¤erentiable except possibly at countably many points then X is called a continuous random variable. Note: The de nition (2.2.3) and properties (2.2.4) of the cumulative distribution function hold for the random variable X regardless of whether X is discrete or continuous. 2.4.2 Example Suppose X is a random variable with cumulative distribution function F (x) = P (X x) = 8>>>>><>>>>>: 0 x 1 12 (x+ 1)2 + x+ 1 1 < x 0 1 2 (x 1)2 + x 0 < x < 1 1 x 1 Show that X is a continuous random variable. Solution The cumulative distribution function F is a continuous function for all x 2 < since it is a piecewise function composed of continuous functions and lim x!aF (x) = F (a) at the break points a = 1; 0; 1. The function F is di¤erentiable for all x 6= 1; 0; 1 since it is a piecewise function composed of di¤erentiable functions. Since lim h!0 F (1 + h) F (1) h = 0 6= lim h!0+ F (1 + h) F (1) h = 1 lim h!0 F (0 + h) F (0) h = 0 = lim h!0+ F (0 + h) F (0) h lim h!0 F (1 + h) F (1) h = 1 6= lim h!0+ F (1 + h) F (1) h = 0 F is di¤erentiable at x = 0 butnot di¤erentiable at x = 1; 1. The set f1; 1g is a countable set. Since F is a continuous function for all x 2 < and F is di¤erentiable except for countable many points, therefore X is a continuous random variable. 20 2. UNIVARIATE RANDOM VARIABLES 2.4.3 De nition - Probability Density Function If X is a continuous random variable with cumulative distribution function F (x) then the probability density function (p.d.f.) of X is f(x) = F 0 (x) if F is di¤erentiable at x. The set A = fx : f(x) > 0g is called the support set of X. Note: At the countably many points at which F 0 (a) does not exist, f (a) may be assigned any convenient value since the probabilities P (X x) will be una¤ected by the choice. We usually choose f(a) 0 and most often we choose f(a) = 0. 2.4.4 Properties – Probability Density Function (1) f(x) 0 for all x 2 < (2) 1R 1 f(x)dx = lim x!1F (x) limx!1F (x) = 1 (3) f(x) = lim h!0 F (x+h)F (x) h = limh!0 P (xXx+h) h if this limit exists (4) F (x) = xR 1 f(t)dt; x 2 < (5) P (a < X b) = P (X b) P (X a) = F (b) F (a) = bR a f(x)dx (6) P (X = b) = F (b) lim a!b F (a) = F (b) F (b) = 0 (since F is continuous). 2.4.5 Example Find and sketch the probability density function of the random variable X with the cumu- lative distribution function in Example 2.2.6. Solution By taking the derivative of F (x) we obtain F 0(x) = 8>>>>><>>>>>: 0 x < 0 d dx 2 5x 3 = 65x 2 0 < x < 1 d dx 12x3x27 5 = 15 (12 6x) 1 < x < 2 0 x > 2 We can assign any values to f (0), f (1), and f (2). For convenience we choose f (0) = f (2) = 0 and f (1) = 65 . 2.4. CONTINUOUS RANDOM VARIABLES 21 The probability density function is f(x) = F 0 (x) = 8>><>>: 6 5x 2 0 < x 1 1 5 (12 6x) 1 < x < 2 0 otherwise The graph of f (x) is given in Figure 2.3. -0.5 0 0.5 1 1.5 2 2.5 0 0.2 0.4 0.6 0.8 1 1.2 1.4 x f(x) Figure 2.3: Graph of probability density function for Example 2.4.5 Note: See Table 2.1 in Section 2.7 for a summary of the di¤erences between the properties of a discrete and continuous random variable. 2.4.6 Example Suppose X is a random variable with cumulative distribution function F (x) = 8>><>>: 0 x a xa ba a < x b 1 x > b where b > a. (a) Sketch F (x) the cumulative distribution function of X. (b) Find f (x) the probability density function of X and sketch it. (c) Is it possible for f (x) to take on values greater than one 22 2. UNIVARIATE RANDOM VARIABLES Solution (a) See Figure 2.4 for a sketch of the cumulative distribution function. a b 0 0.5 1.0 (a+b)/2 x F(x) Figure 2.4: Graph of cumulative distribution function for Example 2.4.6 (b) By taking the derivative of F (x) we obtain F 0(x) = 8>><>>: 0 x < a 1 ba a < x < b 0 x > b The derivative of F (x) does not exist for x = a or x = b. For convenience we de ne f (a) = f (b) = 1ba so that f(x) = ( 1 ba a x b 0 otherwise See Figure 2.5 for a sketch of the probability density function. Note that we could de ne f (a) and f (b) to be any values and the cumulative distribution function would remain the same since x = a or x = b are countably many points. The random variable X is said to have a Uniform(a; b) distribution. We write this as X Uniform(a; b). (c) If a = 0 and b = 0:5 then f (x) = 2 for 0 x 0:5. This example illustrates that the probability density function is not a probability and that the probability density function can take on values greater than one. The important restriction for continuous random variables is 1Z 1 f (x) dx = 1 2.4. CONTINUOUS RANDOM VARIABLES 23 a b 1/(b-a) (a+b)/2 x f(x) Figure 2.5: Graph of the probability density function of a Uniform(a; b) random variable 2.4.7 Example Consider the function f(x) = x+1 for x 1 and 0 otherwise. For what values of is this function a probability density function Solution Using the result (2.8) from Section 2.11 1Z 1 f (x) dx = 1Z 1 x+1 dx = lim b!1 bZ 1 x1dx = lim b!1 h xjb1