Clean your time series data

Visualizing Time Series Data in Python

Thomas Vincent

Head of Data Science, Getty Images

The CO2 level time series

A snippet of the weekly measurements of CO2 levels at the Mauna Loa Observatory, Hawaii.

datastamp    co2                   
1958-03-29  316.1
1958-04-05  317.3
1958-04-12  317.6
...
...
2001-12-15  371.2
2001-12-22  371.3
2001-12-29  371.5

Finding missing values in a DataFrame

print(df.isnull())

datestamp   co2
1958-03-29  False
1958-04-05  False
1958-04-12  False

print(df.notnull())

datestamp    co2
1958-03-29   True
1958-04-05   True
1958-04-12   True
...

Counting missing values in a DataFrame

print(df.isnull().sum())

datestamp     0
co2          59
dtype: int64

Replacing missing values in a DataFrame

print(df)

...
5  1958-05-03  316.9
6  1958-05-10    NaN
7  1958-05-17  317.5
...

df = df.fillna(method='bfill')
print(df)

...
5  1958-05-03  316.9
6  1958-05-10  317.5
7  1958-05-17  317.5
...

Let's practice!

Visualizing Time Series Data in Python