Time series analysis and digital signal processing are closely related. Unfortunately, the two fields use different terms to refer to the same things.
Suppose you have a sequence of inputs x[n] and a sequence of outputs y[n] for integers n.
Moving average / FIR
If each output depends on a linear combination of a finite number of previous inputs
y[n] = b0 x[n] + b1 x[n−1] + … + bq x[n − q]
then time series analysis would call this a moving average (MA) model of order q, provided b0 = 1. Note that this might not really be an average, i.e. the b‘s are not necessarily positive and don’t necessarily sum to 1.
Digital signal processing would call this a finite impulse response (FIR) filter of order q.
Autoregressive / IIR
If each output depends on a linear combination of a finite number of previous outputs
y[n] = a1 y[n −1] + … + ap y[n − p]
then time series analysis would call this an autoregressive (AR) model of order p.
Digital signal processing would call this an infinite impulse response (IIR) filter of order p.
Sometimes you’ll see the opposite sign convention on the a‘s.
ARMA / IIR
If each output depends on a linear combination of a finite number of previous inputs and outputs
y[n] = b0 x[n] + b1 x[n-1] + … + bq x[n − q] + a1 y[n − 1] + … + ap y[n − p]
then time series analysis would call this an autoregressive moving average (ARMA) model of order (p, q), i.e. p AR terms and q MA terms.
Digital signal processing would call this an infinite impulse response (IIR) filter with q feedforward coefficients and p feedback coefficients. Also, as above, you may see the opposite sign convention on the a‘s.
ARMA notation
Box and Jenkins use a‘s for input and z‘s for output. We’ll stick with x‘s and y’s to make the comparison to DSP easier.
Using the backward shift operator B that takes a sample at n to the sample at n-1, the ARMA system can be written
φ(B) y[n] = θ(B) x[n]
where φ and θ are polynomials
φ(B) = 1 − φ1B − φ2B² − … φpBp
and
θ(B) = 1 − θ1B − θ2B² − … θqBq.
System function notation
In DSP, filters are described by their system function, the z-transform of the impulse response. In this notation (as in Oppenheim and Shafer, for example) we have
The φk in Box and Jenkins correspond to the ak in Oppenheim and Schafer. The θk correspond to the (negative) bk.
The system function H(z) corresponds to θ(1/z) / φ(1/z).
Related
DSP and time series consulting
Hi John: Very nice. I’ve been teaching myself basic EE lately and there’s also a lot of overlap with econometrics. All the distributed lag models in econometreics have a linear system viewpoint. As you said, totally different terminology but it’s all quite similar and sometimes equivalent. All three communities ( time-series statistics, time-series econometrics and DSP ) contain so much overlap that I never knew about or understood until the last 3-4 years. Great article. Your have of order q for the IIR in the AR section when you meant p. All the best.
Cool article, I’m an EE learning econometrics. All this stuff is strangely similar.
Hi John, this is a great article merging terminology from two different fields together on an interesting topic. It’s neat to see the ARMA model formulated as a linear filter. Under the ARMA notation section, I think you may have meant to have θ(B) = 1 + θ1B + θ2B² + … θqBq instead of θ(B) = 1 – θ1B – θ2B² – … θqBq.