ELEC 9741: Assignment 1, 2022
Instructions
1 due in Moodle, Friday June 24, 4pm
2 Signed School Cover Sheet attached
3 TYPED PDF only – no microsoft word docs.
4 Follow the Homework Rules.
5 Computeroutput : no commentary no marks.
6 Analyticalresults : no working no marks.
7 means you can use Matlab; else not.
8 No Copyingexcept from lectures
No Discussion
No web searching
Q1 (14) Theory
(a) If θ is an estimator of a vector parameter θ show that
MSE = E(‖ θ θ ‖2) =‖ bias ‖2 +trace(var(θ ))
(a) For the regression problem y = Xn×pβ + where
～ N(0, σ2I)with true value βo, consider the biased
(’shrinkage’) estimator
β a = (1 a
T
)β
β = (X>X) 1X>y = LSE
(i) Find the bias and variance of β a.
(ii) Show β a has a smaller MSE than the LSE when
a <
2τ
vsnr + τT
where vsnr = ‖βo‖
2
σ2 and τ = T
∑p
1
1
λ2
k
and λk
are the singular values of X .
(b) Noise Model.
Consider the stationary process
Yt = a+ φYt 3 + t, t = 1, 2, · · ·
where t is a Gaussian white noise sequence of zero
mean and variance σ2.
(i) Explain what are the stability/stationarity con-
straints on φ
(ii) Derive closed form expressions for the mean and
acs of Yt.
Q2(14) (Impulse Response Estimation)
Consider a system with output sequence measured in noise
yt = st + nt, st = (h u)t, t = 1, · · · , T
with impulse response hr = Aγrsin(2pi rTo ), r = 1, 2, · · · ,m.
The input sequence is a (0, σ2u) white noise
1 independent
of the observation noise sequence which is a (0, σ2) white
noise sequence. The variance SNR is vsnr = σ
2
s
σ2u
.
(a) Ignoring start-up transients, and setting ωo = 2piTo ,
show that for large m
σ2s =
∑m
1 h
2
rσ
2
u ≈ σ2u
∑∞
1 h
2
r
= A2σ2u
γ2
2
[
1
(1 γ2) +
cos(2ωo) γ2
1 + γ4 2γ2cos(2ωo) ]
(b) Simulation.
Write an m-file to simulate the system for T = 100
with the values: [γ, To, σ, vsnr] = [.8, 7, 1, 1] and
m = mo = 20.
(c) Show four displays:
plots of st, yt on the same graph;
histograms of yt, st on the same graph;
What do these plots reveal about the signals
an impulse response plot;
a Bode plot i.e. system frequency response.
(c) Parameter Estimation.
Write an m-file to compute the penalized least squares
estimator, its standard errors2, the singular values of
the X matrix.
(d) To compute the standard errors
you need to derive the
following formula for the variance of the smoothness
penalized least squares estimator (SP-LSE).
var(β λ) = σ
2(X ′X+λD>D) 1X>X(X ′X+λD>D) 1
(e) Using the simulated data from (b) compute the SP-
LSE of β for m = mo and a grid of λ values as fol-
lows.
Plot the ’loss’ = ‖ β λ βo ‖2 versus λ to find the
minimizing value of λ. 3
Compute the corresponding ’minimizing’ SP-LSE, its
standard errors and plot the estimated IR overlaid with
the true IR and a 95% confidence interval. Also plot
the singular values of the X matrix and comment on
them.
Q3 (8). Statistical Graphics.
The graphics/plots you display in Q1, Q2, Q3 will earn up
to 8 marks.
1i.e. it has zero mean and variance σ2u
2se(β r) =
√
var(β r), r = 1, · · · ,m
3Such a plot is only possible in a simulation, not with real data.
Q4(14) (Noise Modeling)
Do not use any specialised matlab commands such as zp2tf,
arima, aic, bic etc.
(a) Write an mfile to simulate a stationary AR(3) time
series driven by a zero mean Gaussian white noise of
unit variance.
Your mfile should accept as input, three real roots or
one real root and a complex root; all non-zero.
It should produce the AR parameters & variance di-
rectly as well as the simulated values as output.
Show two simulations (T=200) (on a single page) one
for each of the above cases. List the two sets of pa-
rameters used. In each case ensure that γo ≥ 3.
(b) Using your mfile simulate an AR(3) with roots
(.9,.1,.6) for T=200. List the true parameter values.
Using least squares regression4 produce estimates for
the 3 parameters, the noise variance as well as stan-
dard errors for the parameters.
Are the estimates within 2 standard errors of the true
values
(c) Using your mfile simulate new data (T=100) from
the same model (ii) compute BIC5 and find its mini-
mizing order p . Show a single plot of BIC together
with its two components.
Give the parameter estimates corresponding to p and
their standard errors.
Also do a statistical model diagnosis using just the acs
of the residuals. What conclusions do you draw about
the quality of the estimated parameters and model or-
der
4write your own mfile; don’t use any matlab command for any regres-
sion related computations
5using your own mfile; not matlab’s BIC command