Reviewing the input data

Concevoir des pipelines de prévision pour la production

Rami Krispin

Senior Manager, Data Science and Engineering

The US hourly electricity demand

Time series plot showing seasonal trends in electricity demand year-to-year

Seasonality analysis

Time series plot showing seasonal trends in electricity demand at an hourly and daily level

Autocorrelation analysis

ACF plot showing daily and weekly lags

The EIA API

EIA Website

The EIA API

EIA API

No API key required to complete this course 🎉

¹ https://www.eia.gov/opendata/

The EIA API

import requests
import pandas as pd
import os

The EIA API

import requests
import pandas as pd
import os

api_url = "https://api.eia.gov/v2/"
api_path = "electricity/rto/region-data/"

The EIA API

import requests
import pandas as pd
import os

api_url = "https://api.eia.gov/v2/"
api_path = "electricity/rto/region-data/"
get_request = api_url + api_path + "data/&data[]=value"

print(get_request)

https://api.eia.gov/v2/electricity/rto/region-data/data/&data[]=value

The EIA API

eia_api_key = os.getenv('EIA_API_KEY')

get_request = get_request + "?api_key=" + eia_api_key

eia_api_key is personal and must be sourced from the EIA website

¹ https://www.eia.gov/opendata/

The EIA API

data = requests.get(get_request).json()
df = pd.DataFrame(data['response']['data'])

The EIA API

data = requests.get(get_request).json()
df = pd.DataFrame(data['response']['data'])

print(df.head())

     period           respondent    respondent-name                              value
0    2022-04-06T07    AVA           Avista Corporation                           462894.0    
1    2024-04-06T07    AZPS          Arizona Public Service Company               463663.0    
2    2024-04-07T07    BANC          Balancing Authority of Northern California   464916.0    
3    2024-04-07T07    BPAT          Bonneville Power Administration              459376.0    
4    2024-04-07T07    CAL           California                                   441989.0

The EIA API

EIA API

¹ EIA API website: https://www.eia.gov/opendata/ ² EIA API dashboard: https://www.eia.gov/opendata/browser/

Data preparation

Data input format for statsforecast and mlforecast packages:

unique_id - the series ID
ds - the series timestamp
y - the series values

df = pd.read_csv("data/data.csv")
ts = df[["period", "value"]]
ts["period"] = pd.to_datetime(ts["period"])
ts = ts.sort_values("period")

Data preparation

Data input format:

unique_id - the series ID
ds - the series timestamp
y - the series values

ts = ts[["period", "value"]]
ts["period"] = pd.to_datetime(ts["period"])
ts = ts.sort_values("period")

ts = ts.rename(columns = {"period": "ds", "value": "y"})
ts["unique_id"] = 1

Data preparation

print(ts.head())

     ds                     y           unique_id
0    2022-04-06 23:00:00    462894.0    1
1    2022-04-07 00:00:00    463663.0    1
2    2022-04-07 01:00:00    464916.0    1
3    2022-04-07 02:00:00    459376.0    1
4    2022-04-07 03:00:00    441989.0    1

Let's practice!

Concevoir des pipelines de prévision pour la production