AIC and BIC

ARIMA Models in Python

James Fulton

Climate informatics researcher

AIC - Akaike information criterion

Lower AIC indicates a better model
AIC likes to choose simple models with lower order

BIC - Bayesian information criterion

Very similar to AIC
Lower BIC indicates a better model
BIC likes to choose simple models with lower order

AIC vs BIC

BIC favors simpler models than AIC
AIC is better at choosing predictive models
BIC is better at choosing good explanatory model

AIC and BIC in statsmodels

# Create model
model = ARIMA(df, order=(1,0,1))

# Fit model
results = model.fit()

# Print fit summary
print(results.summary())

                            Statespace Model Results                           
==============================================================================
Dep. Variable:                      y   No. Observations:                 1000
Model:               SARIMAX(2, 0, 0)   Log Likelihood               -1399.704
Date:                Fri, 10 May 2019   AIC                           2805.407
Time:                        01:06:11   BIC                           2820.131
Sample:                    01-01-2013   HQIC                          2811.003
                         - 09-27-2015                                         
Covariance Type:                  opg

AIC and BIC in statsmodels

# Create model
model = ARIMA(df, order=(1,0,1))

# Fit model
results = model.fit()

# Print AIC and BIC
print('AIC:', results.aic)
print('BIC:', results.bic)

AIC: 2806.36 
BIC: 2821.09

Searching over AIC and BIC

# Loop over AR order
for p in range(3):
    # Loop over MA order
    for q in range(3):

        # Fit model
        model = ARIMA(df, order=(p,0,q))
        results = model.fit()

        # print the model order and the AIC/BIC values
        print(p, q, results.aic, results.bic)

0 0 2900.13 2905.04
0 1 2828.70 2838.52
0 2 2806.69 2821.42
1 0 2810.25 2820.06
1 1 2806.37 2821.09
1 2 2807.52 2827.15
...

Searching over AIC and BIC

order_aic_bic =[]
# Loop over AR order
for p in range(3):
    # Loop over MA order
    for q in range(3):

        # Fit model
        model = ARIMA(df, order=(p,0,q))
        results = model.fit()

        # Add order and scores to list
        order_aic_bic.append((p, q, results.aic, results.bic))

# Make DataFrame of model order and AIC/BIC scores
order_df = pd.DataFrame(order_aic_bic, columns=['p','q', 'aic', 'bic'])

Searching over AIC and BIC

# Sort by AIC
print(order_df.sort_values('aic'))

   p  q      aic      bic
7  2  1  2804.54  2824.17
6  2  0  2805.41  2820.13
4  1  1  2806.37  2821.09
2  0  2  2806.69  2821.42
...

# Sort by BIC
print(order_df.sort_values('bic'))

   p  q      aic      bic
3  1  0  2810.25  2820.06
6  2  0  2805.41  2820.13
4  1  1  2806.37  2821.09
2  0  2  2806.69  2821.42
...

Non-stationary model orders

# Fit model
model = ARIMA(df, order=(2,0,1))
results = model.fit()

ValueError: Non-stationary starting autoregressive parameters
found with `enforce_stationarity` set to True.

When certain orders don't work

# Loop over AR order
for p in range(3):
    # Loop over MA order
    for q in range(3):


        # Fit model
        model = ARIMA(df, order=(p,0,q))
        results = model.fit()

        # Print the model order and the AIC/BIC values
        print(p, q, results.aic, results.bic)

When certain orders don't work

# Loop over AR order
for p in range(3):
    # Loop over MA order
    for q in range(3):

        try:
            # Fit model
            model = ARIMA(df, order=(p,0,q))
            results = model.fit()

            # Print the model order and the AIC/BIC values
            print(p, q, results.aic, results.bic)

        except:
            # Print AIC and BIC as None when fails
            print(p, q, None, None)

Let's practice!

ARIMA Models in Python