Homework Set Three
ECE 271A
Department of Computer and Electrical Engineering
University of California, San Diego
Nuno Vasconcelos
This HW set contains several problems. Only the problem labeled Quiz must be handed in and will
be graded. The remaining problems are for practice. You should not submit them for grade. By
submitting your Quiz solution, you agree to comply with the following.
1. The Quiz is treated as a take-home test and is an INDIVIDUAL effort. NO collaboration
is allowed. The submitted work must be yours and must be original.
2. The work that you turn-in is your own, using the resources that are available to all students in
the class.
3. You can use the help of GENERAL resources on programming, such as MATLAB tutorials, or
related activities.
4. You are not allowed to consult or use resources provided by tutors, previous students in the class,
or any websites that provide solutions or help in solving assignments
and exams.
5. You will not upload your solutions or any other course materials to any web-sites or in some
other way distribute them outside the class.
6. 0 points will be assigned if your work seems to violate these rules and, if recurrent, the incident(s)
will be reported to the Academic Integrity Office.
1. In this problem we will consider the issue of linear regression and the connections between maximum
likelihood and least squares solutions. Consider a problem where we have two random variables Z and
X, such that
z = f(x, θ) + , (1)
where f is a polynomial with parameter vector θ
f(x, θ) =
K∑
k=0
θkx
k (2)
and a Gaussian random variable of zero mean and variance σ2. Our goal is to estimate the best
estimate of the function given an iid sample D = {(Dx,Dz)} = {(x1, z1), . . . , (xn, zn)}.
a) Formulate the problem as one of least squares, i.e define z = (z1, . . . , zn)
T ,
Φ =
1 . . . x
K
1
…
…
1 . . . xKn