Rolling window functions with pandas

Manipulating Time Series Data in Python

Stefan Jansen

Founder & Lead Data Scientist at Applied Artificial Intelligence

Window functions in pandas

Windows identify sub periods of your time series
Calculate metrics for sub periods inside the window
Create a new time series of metrics
Two types of windows:
- Rolling: same size, sliding (this video)
- Expanding: contain all prior values (next video)

Calculating a rolling average

data = pd.read_csv('google.csv', parse_dates=['date'], index_col='date')

DatetimeIndex: 1761 entries, 2010-01-04 to 2016-12-30
Data columns (total 1 columns):
price     1761 non-null float64
dtypes: float64(1)

ch3_1_v2 - Rolling Window Functions with Pandas.010.png

Calculating a rolling average

# Integer-based window size
data.rolling(window=30).mean() # fixed # observations

DatetimeIndex: 1761 entries, 2010-01-04 to 2017-05-24
Data columns (total 1 columns):
price    1732 non-null float64
dtypes: float64(1)

window=30: # business days
min_periods: choose value < 30 to get results for first days

Calculating a rolling average

# Offset-based window size
data.rolling(window='30D').mean() # fixed period length

DatetimeIndex: 1761 entries, 2010-01-04 to 2017-05-24
Data columns (total 1 columns):
price    1761 non-null float64
dtypes: float64(1)

30D: # calendar days

90 day rolling mean

r90 = data.rolling(window='90D').mean()

google.join(r90.add_suffix('_mean_90')).plot()

ch3_1_v2 - Rolling Window Functions with Pandas.017.png

90 & 360 day rolling means

data['mean90'] = r90

r360 = data['price'].rolling(window='360D'.mean()

data['mean360'] = r360; data.plot()

ch3_1_v2 - Rolling Window Functions with Pandas.020.png

Multiple rolling metrics (1)

r = data.price.rolling('90D').agg(['mean', 'std'])

r.plot(subplots = True)

ch3_1_v2 - Rolling Window Functions with Pandas.022.png

Multiple rolling metrics (2)

rolling = data.google.rolling('360D')

q10 = rolling.quantile(0.1).to_frame('q10')

median = rolling.median().to_frame('median')

q90 = rolling.quantile(0.9).to_frame('q90')

pd.concat([q10, median, q90], axis=1).plot()

ch3_1_v2 - Rolling Window Functions with Pandas.024.png

Let's practice!

Manipulating Time Series Data in Python