Розбір і обробка дат

Перетворення даних у Polars

Liam Brannigan

Data Scientist & Polars Contributor

Дані часових рядів

Часовий ряд ринкових цін

Дані часових рядів

Часовий ряд продажів

Дані часових рядів

Часовий ряд стримів

Дані часових рядів

Часовий ряд підрахунків

Набір даних: ціни на електроенергію

shape: (24, 3)
| time             | price | solar |
| ---              | ---   | ---   |
| str              | f64   | f64   |
|------------------|-------|-------|
| 05/07/2025 00:00 | 34.2  | 0.0   |
| ...              | ...   | ...   |
| 05/07/2025 12:00 | 0.0   | 462.0 |

time — погодинні значення
price — £ за мегават-годину
solar — Вт на квадратний метр
solar > 400 = теплий літній день

Price and solar relationship

Time series of electricity prices

Price and solar relationship

Time series of electricity prices

Datetime dtypes in Polars

Date - calendar date - 2000-01-01
Datetime - date and time - 2000-01-01 12:00:00
Time - time of day - 12:00:00
Duration - time span - 3 hours

Automatic datetime parsing

pl.read_csv(
  "electricity_prices",

)

Automatic datetime parsing

pl.read_csv(
  "electricity_prices",
  try_parse_dates=True
)

format: YYYY-MM-DD
e.g. 2025-07-05
e.g. 2025-07-05 12:00:00

Our data needs manual parsing

| time             | price | solar |
| ---              | ---   | ---   |
| str              | f64   | f64   |
|------------------|-------|-------|
| 05/07/2025 00:00 | 34.2  | 0.0   |
| 05/07/2025 01:00 | 28.7  | 0.0   |

format: DD/MM/YYYY HH:MM
not auto-detected by Polars

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(                             )
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime,                 )
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y      ")
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y %H:%M")
)

Parsing with strptime

prices.with_columns(
    pl.col("time").str.strptime(pl.Datetime, "%d/%m/%Y %H:%M")
)

| time                | price | solar |
| ---                 | ---   | ---   |
| datetime[us]        | f64   | f64   |
|---------------------|-------|-------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   |
| 2025-07-05 01:00:00 | 28.7  | 0.0   |
| 2025-07-05 02:00:00 | 27.5  | 0.0   |

Extracting datetime components

Table of time series data

Extracting datetime components

Table of time series data with the date highlighted on one row

Extracting datetime components

Table of time series data with the time highlighted on one row

Extracting datetime components

Table of time series data with the hour highlighted on one row

Extracting the date

prices.with_columns(
    pl.col("time").dt.date()
)

Extracting the date

prices.with_columns(
    pl.col("time").dt.date().alias("date")
)

Extracting the date

prices.with_columns(
    pl.col("time").dt.date().alias("date")
)

| time                | price | solar | date       |
| ---                 | ---   | ---   | ---        |
| datetime[µs]        | f64   | f64   | date       |
|---------------------|-------|-------|------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 2025-07-05 |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 2025-07-05 |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 2025-07-05 |

Extracting the time

prices.with_columns(
    pl.col("time").dt.time().alias("time_of_day")
)

Extracting the time

prices.with_columns(
    pl.col("time").dt.time().alias("time_of_day")
)

| time                | price | solar | time_of_day |
| ---                 | ---   | ---   | ---         |
| datetime[us]        | f64   | f64   | time        |
|---------------------|-------|-------|-------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 00:00:00    |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 01:00:00    |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 02:00:00    |

Extracting the hour

prices.with_columns(
    pl.col("time").dt.hour().alias("hour")
)

Extracting the hour

prices.with_columns(
    pl.col("time").dt.hour().alias("hour")
)

| time                | price | solar | hour |
| ---                 | ---   | ---   | ---  |
| datetime[µs]        | f64   | f64   | i8   |
|---------------------|-------|-------|------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 0    |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 1    |
| 2025-07-05 12:00:00 | 0.0   | 462.0 | 12   |

Other datetime components

.dt.year() - extract the year
.dt.month() - extract the month (1-12)
.dt.day() - extract the day of month
.dt.minute() - extract the minute
...

¹ https://docs.pola.rs/api/python/dev/reference/expressions/temporal.html

Adjusting datetimes

.dt.offset_by() - adjust datetimes
"1h" - forward 1 hour
"-2d" - backward 2 days

Adjusting datetimes

.dt.offset_by() - adjust datetimes
"1h" - forward 1 hour
"-2d" - backward 2 days

¹ https://docs.pola.rs/api/python/stable/reference/expressions/api/polars.Expr.dt.offset_by.html

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h")
)

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h").alias("time_end")
)

Adding a time window end

prices.with_columns(
    pl.col("time").dt.offset_by("1h").alias("time_end")
)

| time                | price | solar | time_end            |
| ---                 | ---   | ---   | ---                 |
| datetime[µs]        | f64   | f64   | datetime[µs]        |
|---------------------|-------|-------|---------------------|
| 2025-07-05 00:00:00 | 34.2  | 0.0   | 2025-07-05 01:00:00 |
| 2025-07-05 01:00:00 | 28.7  | 0.0   | 2025-07-05 02:00:00 |
| 2025-07-05 02:00:00 | 27.5  | 0.0   | 2025-07-05 03:00:00 |

Let's practice!

Перетворення даних у Polars