降採樣與彙總

Manipulating Time Series Data in Python

Stefan Jansen

Founder & Lead Data Scientist at Applied Artificial Intelligence

降採樣與彙總方法

  • 目前為止：升採樣、填補邏輯與插值
  • 接著：降採樣
    • 小時到天
    • 天到月，等等
  • 新時間點要如何表示既有數值？
    • 平均數、中位數、最後一筆？
Manipulating Time Series Data in Python

空氣品質：每日臭氧濃度

ozone = pd.read_csv('ozone.csv',
                     parse_dates=['date'],
                     index_col='date')

ozone.info()

DatetimeIndex: 6291 entries, 2000-01-01 to 2017-03-31
Data columns (total 1 columns):
Ozone    6167 non-null float64
dtypes: float64(1)

ozone = ozone.resample('D').asfreq()

ozone.info()

DatetimeIndex: 6300 entries, 1998-01-05 to 2017-03-31
Freq: D
Data columns (total 1 columns):
Ozone    6167 non-null float64
dtypes: float64(1)
Manipulating Time Series Data in Python

建立每月臭氧資料

ozone.resample('M').mean().head()

               Ozone
date
2000-01-31  0.010443
2000-02-29  0.011817
2000-03-31  0.016810
2000-04-30  0.019413
2000-05-31  0.026535

.resample().mean()：月平均，指派到該月月底

ozone.resample('M').median().head()

               Ozone
date
2000-01-31  0.009486
2000-02-29  0.010726
2000-03-31  0.017004
2000-04-30  0.019866
2000-05-31  0.026018
Manipulating Time Series Data in Python

建立每月臭氧資料

ozone.resample('M').agg(['mean', 'std']).head()

               Ozone
                mean       std
date
2000-01-31  0.010443  0.004755
2000-02-29  0.011817  0.004072
2000-03-31  0.016810  0.004977
2000-04-30  0.019413  0.006574
2000-05-31  0.026535  0.008409
  • .resample().agg()：可傳入彙總函式清單，類似 groupby
Manipulating Time Series Data in Python

繪製重採樣後的臭氧資料

ozone = ozone.loc['2016':]

ax = ozone.plot()

monthly = ozone.resample('M').mean()

monthly.add_suffix('_monthly').plot(ax=ax)

第 2 章第 4 張 v2 - 降採樣與彙總.018.png

Manipulating Time Series Data in Python

重採樣多個時間序列

data = pd.read_csv('ozone_pm25.csv',
                   parse_dates=['date'],
                   index_col='date')

data = data.resample('D').asfreq()

data.info()

DatetimeIndex: 6300 entries, 2000-01-01 to 2017-03-31
Freq: D
Data columns (total 2 columns):
Ozone    6167 non-null float64
PM25     6167 non-null float64
dtypes: float64(2)
Manipulating Time Series Data in Python

重採樣多個時間序列

data = data.resample('BM').mean()

data.info()

<class 'pandas.core.frame.DataFrame'>
DatetimeIndex: 207 entries, 2000-01-31 to 2017-03-31
Freq: BM
Data columns (total 2 columns):
ozone    207 non-null float64
pm25     207 non-null float64
dtypes: float64(2)
Manipulating Time Series Data in Python

重採樣多個時間序列

df.resample('M').first().head(4)

               Ozone       PM25
date
2000-01-31  0.005545  20.800000
2000-02-29  0.016139   6.500000
2000-03-31  0.017004   8.493333
2000-04-30  0.031354   6.889474

df.resample('MS').first().head()

               Ozone       PM25
date
2000-01-01  0.004032  37.320000
2000-02-01  0.010583  24.800000
2000-03-01  0.007418  11.106667
2000-04-01  0.017631  11.700000
2000-05-01  0.022628   9.700000
Manipulating Time Series Data in Python

一起來練習吧！

Manipulating Time Series Data in Python

Preparing Video For Download...