Selecting ARIMA Order (p, d, q)

After understanding the components of ARIMA models (AR, I, MA), the next step is selecting the appropriate orders for these components, represented by the parameters $(p, d, q)$. This selection process is fundamental to building an effective ARIMA model. It involves determining the level of differencing needed (the 'I' part) and then identifying the structure of the AR and MA parts based on the autocorrelation patterns of the stationary series.

Determining the Order of Differencing (d)

The 'I' in ARIMA stands for 'Integrated'. This component addresses non-stationarity in the time series, specifically non-stationarity related to trends or level shifts. The parameter $d$ represents the number of times the time series needs to be differenced to achieve stationarity.

1. Assess Initial Stationarity: First, examine your original time series. Use the visual inspection techniques (plotting the series, rolling mean/variance) and statistical tests (like the Augmented Dickey-Fuller test) discussed in Chapter 2.

   • If the series appears stationary (constant mean and variance, ADF test rejects the null hypothesis of non-stationarity), then no differencing is needed, and $d = 0$.
   • If the series is non-stationary, proceed to differencing.

2. Apply Differencing: Calculate the first difference of the series: $\Delta y_t = y_t - y_{t-1}$. Now, test this differenced series for stationarity.

   • If the first-differenced series is stationary, then $d = 1$. This is very common for series exhibiting a linear trend.

3. Consider Second Differencing (If Necessary): If the first-differenced series is still non-stationary (perhaps due to a quadratic trend or changing trend), calculate the second difference: $\Delta^2 y_t = \Delta y_t - \Delta y_{t-1} = (y_t - y_{t-1}) - (y_{t-1} - y_{t-2})$. Test this second-differenced series for stationarity.

   • If the second-differenced series is stationary, then $d = 2$.

Important Guideline: Use the minimum order of differencing required to make the series stationary. Over-differencing can introduce artificial patterns and dependencies into the data, complicating the model unnecessarily. It's rare to need $d > 2$. If stationarity isn't achieved after two rounds of differencing, you might need to consider other transformations (like logging) or alternative modeling approaches.

Once you have determined the value of $d$ and obtained a stationary time series (let's call it $y'_t$, which might be the original series if $d=0$, or a differenced version if $d>0$), you can proceed to identify the AR and MA orders ($p$ and $q$).

Determining AR (p) and MA (q) Orders using ACF and PACF

The Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots, discussed in Chapter 3, are the primary tools for inferring the orders $p$ and $q$ for the ARMA part of the model. Remember to generate these plots using the stationary (differenced) time series $y'_t$.

Here are the general patterns to look for:

• AR(p) Model Signature:

  • ACF: Tails off slowly or decays exponentially towards zero. It might show oscillations.
  • PACF: Cuts off sharply after lag $p$. This means the partial autocorrelations are statistically significant up to lag $p$ and then drop abruptly to near-zero (within the significance bounds).
  • Interpretation: If you see this pattern, it suggests an AR($p$) model might be appropriate, meaning $q=0$. The value of $p$ is determined by the lag where the PACF cuts off.

• MA(q) Model Signature:

  • ACF: Cuts off sharply after lag $q$. The autocorrelations are significant up to lag $q$ and then drop abruptly.
  • PACF: Tails off slowly or decays exponentially towards zero.
  • Interpretation: This pattern suggests an MA($q$) model, meaning $p=0$. The value of $q$ is determined by the lag where the ACF cuts off.

• ARMA(p, q) Model Signature:

  • ACF: Tails off slowly after lag $q$.
  • PACF: Tails off slowly after lag $p$.
  • Interpretation: If both plots show a tailing-off behavior, it suggests that both AR and MA terms are needed. Identifying the exact $p$ and $q$ can be more challenging. Often, you might observe exponential decay or damped sine waves in both plots. In this case, you might look for the lags where the decay effectively begins or try small values for $p$ and $q$ (e.g., $p=1, q=1$) as a starting point.

Visualizing ACF/PACF Interpretation:

Let's imagine two scenarios for a stationary series ($d$ already determined):

Scenario 1: Potential AR(2) Model

In Scenario 1, the ACF plot shows a slow, somewhat exponential decay. The PACF plot shows significant spikes at lags 1 and 2, then cuts off abruptly (spikes after lag 2 are within the significance bounds, represented by the dashed lines). This strongly suggests an AR(2) model, so $p=2$ and $q=0$. The full model would be ARIMA(2, d, 0).

Scenario 2: Potential MA(1) Model

In Scenario 2, the ACF plot shows a single significant spike at lag 1 and then cuts off. The PACF plot decays more slowly, potentially geometrically or oscillating. This pattern strongly suggests an MA(1) model, implying $p=0$ and $q=1$. The full model would be ARIMA(0, d, 1).

Iterative Selection and Information Criteria

Real-world ACF and PACF plots are rarely as clean as the idealized examples. Noise in the data can obscure the patterns. Therefore, selecting $(p, d, q)$ is often an iterative process:

1. Determine d: Find the minimum differencing needed for stationarity.
2. Analyze ACF/PACF: Examine the plots for the stationary series. Identify potential candidate orders for $p$ and $q$ based on the cut-off or tailing-off behavior. Start with simpler models (small $p, q$).
3. Fit Candidate Models: Fit the ARIMA($p, d, q$) model(s) using a library like statsmodels (covered next).
4. Check Diagnostics: Analyze the residuals of the fitted model(s). If the model is good, the residuals should resemble white noise (no significant autocorrelation left). This is covered in the "Model Diagnostics" section.
5. Compare Models: If multiple candidate models seem plausible (e.g., ARIMA(1, d, 0) vs. ARIMA(0, d, 1) vs. ARIMA(1, d, 1)), you can use Information Criteria like AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion) to help choose. These metrics balance model fit (how well it explains the data) against model complexity (number of parameters $p+q$). Models with lower AIC or BIC values are generally preferred. We will discuss evaluation metrics, including AIC/BIC, in more detail in Chapter 6.
6. Refine: Based on diagnostics and comparison metrics, refine your choice of $p$ and $q$. You might need to try slightly different orders and repeat steps 3-5.

This combination of analyzing ACF/PACF plots, fitting models, checking residuals, and potentially using information criteria provides a systematic way to arrive at a suitable ARIMA($p, d, q$) specification for your time series.