ECMT1020 Introduction to Econometrics Week 1, 2022S1
Lecture 1: Review: Random Variables
Instructor: Ye Lu
Please read Chapter R.1-R.4 of the textbook.
Contents
1 Random variables 2
1.1 probability mass functions of discrete random variables . . . . . . . . . . . . . 2
1.2 probability density functions of continuous random variables . . . . . . . . . . 3
2 Expectation, variance and covariance 5
2.1 Expectation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Covariance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 Variance and covariance rules . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3 Exercises 7
What is Econometrics See an explanation by an economist at International Monetary
Fund (IMF).
“Econometrics uses economic theory, mathematics, and statistical inference to quantify
economic phenomena.”
With the aid of econometrics, we may discriminate between competing economic the-
ories. We also try to convert qualitative statements from economic theory into quan-
titative statements.
In short, econometrics is taking economics to data! Some examples are
How much do we expect the consumption would increase if there is an increase in the
disposable income or wealth
How much impact on inflation rate will an increase in interest rate by 5 basis points
will make
In a course of econometrics, you are expected to learn
the techniques to analyze data, and more importantly,
why those techniques work or don’t work in certain circumstances.
To achieve these learning goals (especially for the second goal, as elaborated in R.1 of the
textbook), we need to start from building a solid foundation in probability and statistical
inference. Therefore, in the first two lectures of ECMT1020, we will go through the basic
concepts in probability and statistics that you have already learned in a pre-requisite course
such as ECMT1010. However, this time we will redo everything more in-depth, more for-
mally, and more rigorously. We will also introduce some further concepts that might look
new to you.
Without further ado, let’s get started!
1
1 Random variables
A random variable is a mathematical formalization of a quantity or object which depends on
random events. Loosely speaking, it is a variable whose value cannot be predicted exactly.
Depending on the nature of possible values that a random variable can take on, we have
1. Discrete random variable: the possible values are countable. Discrete random variables
are usually (but not necessarily) counts. Examples:
a dice roll: possible values are {1, 2, 3, 4, 5, 6}
number of cars sold by a car dealer next month: possible values are (potentially)
{0, 1, 2, . . .}1
total score when two dice are thrown: possible values are
2. Continuous random variable: the possible values are a continuous range of values, often
an interval, or a union of disjoint intervals, in the real number line (or the real line
itself). Continuous random variables are usually (but not necessarily) measurements.
Examples:
temperature (measured in Fahrenheit) in a room: possible values in [55, 75]
the S&P/ASX 200 index tomorrow: possible values in [0,∞)
measurement error (difference between a measured quantity and its true value):
possible values in ( ∞,∞)
In the following, we introduce two mathematical functions used to characterize the prob-
ability distributions of discrete random variables and continuous random variables, respec-
tively. They effectively quantify the uncertainty/randomness that a random variable repre-
sents.
1.1 probability mass functions of discrete random variables
The probability mass function (pmf) is a function which maps all possible values of a discrete
random variable to their probabilities. Therefore, it can be considered as a list of ‘discrete
probabilities’ (using the language of the textbook).
What’s the probability mass function, denoted by p(·), of a dice roll
p(1) =
1
6
, p(2) =
1
6
, p(3) =
1
6
, p(4) =
1
6
, p(5) =
1
6
, p(6) =
1
6
.
Let X denote the random variable representing the outcome of a dice roll, we may write
pi := p(xi) =
1
6
, for xi = i, i = 1, . . . , 6. (1)
Questions:
1. Can we plot the above probability mass function specified in (1) How does it look
like
1You might read something like ‘discrete random variables can take only a finite set of values’, but that’s
not true. The possible values can certainly be infinite as long as they are countable and the sum of the discrete
probabilities is one.
2
2. What is the probability mass function of a random variable Y which is defined as the
total score of a roll of two dice How does the plot look like
In general, if X can take values x1, x2, . . ., then the probability mass function of X is
pi := p(xi) = Pr(X = xi), i = 1, 2, . . . ,
where Pr denotes ‘probability of’. Note that we always have∑
all i
pi =
∑
all xi
p(xi) = 1. (2)
In the case (which we discuss mostly) when X can only take finite discrete values, that is, X
can take values x1, . . . , xn where n being a finite positive integer, then the probability mass
function of X is
pi := p(xi) = Pr(X = xi), i = 1, . . . , n,
and we have
n∑
i=1
pi =
n∑
i=1
p(xi) = 1.
A side note: You may want to watch Section B2.a video in Econometrics Video Handbook
(EVH, link via Canvas) to learn more about random variables and cumulative distribution
functions (cdf). The cdf is an important concept to characterize the probability distribution
of a random variable (either discrete or continuous) which is not covered in the textbook.
Also, Section B2.b and B2.e on the general concepts of discrete random variables (with pmf)
and continuous random variables (with pdf introduced below) are also excellent.
1.2 probability density functions of continuous random variables
Unlike the case of discrete random variable, for a continuous random variable X the proba-
bility that X takes any specific value is ZERO! For example, if X denotes the temperature
in a room with possible values in [55, 75] ( F ), then
Pr(X = x) = 0 for any x ∈ [55, 75].
Therefore, the probability mess function (or discrete probabilities) described above will be
useless here.
Instead, we need a different concept to describe the probability distribution of a con-
tinuous random variable, and that is the so-called probability density, or probability
density function (pdf) when it is regarded as a function.
Moreover, instead of describing the probability of the random variable taking on specific
values (like what pmf does), we focus on describing the probability of a continuous
random variable taking value over a specific range, say over [a, b], by finding the area
under the pdf over that specific range.
3
Example in the textbook about the ‘uniformly distributed’ temperature: let f(·) denote the
pdf of the room temperature which is uniformly distributed between 55 F and 75 F . Then
f is a function defined on the real number line, and
f(x) =
{
0.05 for 55 ≤ x ≤ 75
0 for x < 55 or x > 75
(3)
Note that here we use lower case x instead of upper case X (What textbook did in equation
(R.16) is not common or proper) to differentiate a fixed real number x and a random variable
X. The integration of f specified in (3) is∫ ∞
∞
f(x)dx =
∫ 75
55
f(x)dx = 0.05× (75 55) = 0.0
5× 20 = 1.
In fact, it is always true that ∫ ∞
∞
f(x)dx = 1 (4)
if f is the pdf of a continuous random variable. Note that the equation (4) is the analogue
of (2) in the discrete random variable case.
It is very important to understand that for f being the pdf of a continuous random
variable X, for any real number x,
Pr(X = x) = f(x)!!
The way we characterize the probability distribution of random variable X by its pdf is
through the relationship
Pr(a ≤ X ≤ b) =
∫ b
a
f(x)dx, for any real number a < b.
That is, by integrating (calculating an area under a curve) the pdf over [a, b], we obtain the
probability of the continuous random variable taking value between a and b. See textbook
examples in Figure 1.
Figure 1: Probability of uniformly distributed temperature random variable taking values in
certain ranges
4
2 Expectation, variance and covariance
2.1 Expectation
The expected value, or simply expectation, or mean of a discrete random variable which takes
finite number of values is a weighted average of all its possible values, taking probability of
each value as its weight.
Suppose X takes n particular values x1, . . . , xn and the pmf of X is
pi = p(xi) = Pr(X = xi), i = 1, . . . , n. (5)
Then the expectation of X is
μX := E(X) = x1p1 + · · ·+ xnpn =
n∑
i=1
xipi.
Questions:
1. If X is defined as the total score of a roll of two dice, what is the expectation of X
2. Define another random variable Y = X2. What is the expectation of Y
In general, for g(·) being a function and X being a random variable, g(X) is also a
random variable. If X is a discrete random variable with pmf in (5), then the expectation
of g(X) is simply
E[g(X)] = g(x1)p1 + · · ·+ g(xn)pn =
n∑
i=1
g(xi)pi. (6)
Exercises: Can you verify the expected value rules (R.3) and (R.4) in the textbook for discrete
random variables Namely, if X and Y are two random variables and b is a constant, can
you verify
E(X + Y ) = E(X) + E(Y ) and E(bX) = bE(X)
Moreover, the expected value rule (R.5) says that if b is a constant, then
E(b) = b.
Well, this should be straightforward if you consider a constant b as a ‘special’ random variable
which takes only one value b with probability 1.
Note: the expected value rules apply to both discrete and continuous random variables.
For the definition of expectation value of continuous random variable, see BOX R.1 in the
textbook (p. 18). In particular, we need to, again, replace the summation in the discrete
case with the integration in the continuous case. If X is a continuous random variable with
pdf f(x), then
μX := E(X) =
∫ ∞
∞
xf(x)dx.
5
2.2 Variance
The variance of a random variable X is a useful measure of the dispersion of its probability
distribution. It is defined as
σ2X := Var(X) = E[(X μX)2]. (7)
Note:
In the definition, μX = E(X) is a constant. What’s the interpretation of the random
variable u := X μX What’s the expectation of u
Can you see that, essentially, Var(X) = E(u2) Now, if X is a discrete random variable
with pmf in (5), then applying (6) with g(X) := u2 = (X μX)2, we have the variance
of X given by
Var(X) = E(u2) =
n∑
i=1
(xi μX)2pi.
Can you calculate the variance of the random variable X defined as the total score of a
roll of two dice
A good exercise of applying the expected value rules introduced above is to prove an
alternative useful formula of variance:
Var(X) = E(X2) E(X)2,
or put differently,
σ2X = E(X
2) μ2X .
A related concept here is standard deviation of X which is defined as
σX :=
√
Var(X) =
√
σ2X ,
which is also a measure of dispersion of the probability distribution of X.
2.3 Covariance
If we have two random variables, X and Y , with expectations μX and μY respectively, then
their covariance is defined as
σXY = Cov(X,Y ) = E[(X μX)(Y μY )]. (8)
It is clear that we have σ2X = Var(X) = Cov(X,X) = σXX and σ
2
Y = Var(Y ) = Cov(Y, Y ) =
σY Y by comparing the definitions in (7) and (8).
A related concept is correlation which, comparing to covariance, is invariant to the scale of
measurement. The correlation coefficient between two random variables X and Y is defined
6
as
ρXY =
σXY√
σ2Xσ
2
Y
=
σXY
σXσY
.
If ρXY = 0, then we say the random variables X and Y are uncorrelated.
Note that X and Y are said to be independent if
E[g(X)h(Y )] = E[g(X)]E[h(Y )]
for any functions g(·) and h(·). As a special case where g and h are taken as identity
functions2, X and Y are independent implies that
E(XY ) = E(X)E(Y ).
Can you prove the following fact
“If X and Y are independent random variables, then they are uncorrelated.”
Note that the reverse is NOT true.
2.4 Variance and covariance rules
All the variance and covariance rules stated in Section R.4 in the textbook can be derived
from the expectation rules (R.3)–(R.5), simply because, as seen from (7) and (8), the variance
and covariance are defined through expectation.
Covariance rules: In the following, X,Y, Z are random variables and b is a constant.
1. Cov(X,Y + Z) = Cov(X,Y ) + Cov(X,Z)
2. Cov(X, bY ) = bCov(X,Y )
3. Cov(X, b) = 0
Variance rules: In the following, X,Y are random variables and b is a constant.
1. Var(X + Y ) = Var(X) + Var(Y ) + 2Cov(X,Y )
2. Var(bX) = b2Var(X)
3. Var(b) = 0
4. Var(X + b) = Var(X).
3 Exercises
Below exercises are from the textbook Exercises R.1–14.
R.1 A random variable X is defined to be the difference between the higher value and the
lower value when two dice are thrown. If they have the same value, X is defined to be zero.
Find the probability distribution for X.
2An identify function is a function of the form f(x) = x.
7
R.2 A random variable X is defined to be the larger of the two values when two dice are
thrown, or the value if the values are the same. Find the probability distribution for X.
R.3 Find the expected value of X in Exercise R.1.
R.4 Find the expected value of X in Exercise R.2.
R.5 If X is a (discrete) random variable with mean μX , and λ is a constant, provide that
the expected value of λX is λμX .
R.6 Calculate E(X2) for X defined in Exercises R.1.
R.7 Calculate E(X2) for X defined in Exercise R.2.
R.8 Let X be the total when two dice are thrown. Calculate the possible values of Y , when
Y is given by Y = 2X +3 and hence calculate E(Y ). Show that this is equal to 2E(X)+3.
R.9 Calculate the population variance and the standard deviation of X as defined in Ex-
ercise R.1.
R.10 Calculate the population variance and the standard deviation of X as defined in
Exercise R.2.
R.11 Using the fact that σ2X = E(X
2) μ2X , find the variance of the random variable X
defined in Exercise R.1 and verify the answer is the same as that obtained in Exercise R.9.
(Note, you have already calculated μX in Exercise R.3 and E(X
2) in Exercise R.6.)
R.12 Using the fact that σ2X = E(X
2) μ2X , find the variance of the random variable X
defined in Exercise R.2 and verify the answer is the same as that obtained in Exercise R.10.
(Note, you have already calculated μX in Exercise R.4 and E(X
2) in Exercise R.7.)
R.13 Suppose a variable Y is an exact linear function of X:
Y = λ+ μX
where λ and μ are constants. Demonstrate that the correlation between X and Y is equal
to 1 or 1, according to the sign of μ.
R.14 Suppose a variable Y is an exact linear function of X:
Y = λ+ μX
where λ and μ are constants, and suppose that Z is a third variable. Show that ρXZ = ρY Z .
8