Parsing and manipulating dates

Data Transformation with Polars

Liam Brannigan

Data Scientist & Polars Contributor

Time series data

Time series for market prices

Time series data

Time series for sales

Time series data

Time series for streams

Time series data

Time series for count

Electricity prices dataset

shape: (24, 3)
| time             | price | solar |
| ---              | ---   | ---   |
| str              | f64   | f64   |
|------------------|-------|-------|
| 05/07/2025 00:00 | 34.2  | 0.0   |
| ...              | ...   | ...   |
| 05/07/2025 12:00 | 0.0   | 462.0 |

time - hourly values
price - £ per Megawatt hour
solar - W per square meter
solar > 400 = warm summer day

Price and solar relationship

Time series of electricity prices

Price and solar relationship

Time series of electricity prices

Datetime dtypes in Polars

Date - calendar date - 2000-01-01
Datetime - date and time - 2000-01-01 12:00:00
Time - time of day - 12:00:00
Duration - time span - 3 hours

Automatic datetime parsing

pl.read_csv(
  "electricity_prices",

)

Automatic datetime parsing

pl.read_csv(
  "electricity_prices",
  try_parse_dates=True
)

format: YYYY-MM-DD
e.g. 2025-07-05
e.g. 2025-07-05 12:00:00

Our data needs manual parsing

| time             | price | solar |
| ---              | ---   | ---   |
| str              | f64   | f64   |
|------------------|-------|-------|
| 05/07/2025 00:00 | 34.2  | 0.0   |
| 05/07/2025 01:00 | 28.7  | 0.0   |

format: DD/MM/YYYY HH:MM
not auto-detected by Polars

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(                             )
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime,                 )
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y      ")
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y %H:%M")
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y %H:%M")
)

| time                | price | solar |
| ---                 | ---   | ---   |
| datetime[us]        | f64   | f64   |
|---------------------|-------|-------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   |
| 2025-07-05 01:00:00 | 28.7  | 0.0   |
| 2025-07-05 02:00:00 | 27.5  | 0.0   |

Extracting datetime components

Table of time series data

Extracting datetime components

Table of time series data with the date highlighted on one row

Extracting datetime components

Table of time series data with the time highlighted on one row

Extracting datetime components

Table of time series data with the hour highlighted on one row

Extracting the date

prices.with_columns(
    pl.col("time").dt.date()
)

Extracting the date

prices.with_columns(
    pl.col("time").dt.date().alias("date")
)

Extracting the date

prices.with_columns(
    pl.col("time").dt.date().alias("date")
)

| time                | price | solar | date       |
| ---                 | ---   | ---   | ---        |
| datetime[µs]        | f64   | f64   | date       |
|---------------------|-------|-------|------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 2025-07-05 |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 2025-07-05 |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 2025-07-05 |

Extracting the time

prices.with_columns(
    pl.col("time").dt.time().alias("time_of_day")
)

Extracting the time

prices.with_columns(
    pl.col("time").dt.time().alias("time_of_day")
)

| time                | price | solar | time_of_day |
| ---                 | ---   | ---   | ---         |
| datetime[us]        | f64   | f64   | time        |
|---------------------|-------|-------|-------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 00:00:00    |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 01:00:00    |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 02:00:00    |

Extracting the hour

prices.with_columns(
    pl.col("time").dt.hour().alias("hour")
)

Extracting the hour

prices.with_columns(
    pl.col("time").dt.hour().alias("hour")
)

| time                | price | solar | hour |
| ---                 | ---   | ---   | ---  |
| datetime[µs]        | f64   | f64   | i8   |
|---------------------|-------|-------|------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 0    |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 1    |
| 2025-07-05 12:00:00 | 0.0   | 462.0 | 12   |

Other datetime components

.dt.year() - extract the year
.dt.month() - extract the month (1-12)
.dt.day() - extract the day of month
.dt.minute() - extract the minute
...

¹ https://docs.pola.rs/api/python/dev/reference/expressions/temporal.html

Adjusting datetimes

.dt.offset_by() - adjust datetimes
"1h" - forward 1 hour
"-2d" - backward 2 days

Adjusting datetimes

.dt.offset_by() - adjust datetimes
"1h" - forward 1 hour
"-2d" - backward 2 days

¹ https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.dt.offset_by.html

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h")
)

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h").alias("time_end")
)

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h").alias("time_end")
)

| time                | price | solar | time_end            |
| ---                 | ---   | ---   | ---                 |
| datetime[µs]        | f64   | f64   | datetime[µs]        |
|---------------------|-------|-------|---------------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 2025-07-05 01:00:00 |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 2025-07-05 02:00:00 |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 2025-07-05 03:00:00 |

Let's practice!

Data Transformation with Polars