Rijen filteren

Introductie tot Polars

Liam Brannigan

Data Scientist & Polars Contributor

Waarom een DataFrame filteren?

rentals

shape: (49, 8)
| name      | type    | price | bedrooms | doubles | singles | review | beach |
| ---       | ---     | ---   | ---      | ---     | ---     | ---    | ---   |
| str       | str     | i64   | i64      | i64     | i64     | f64    | bool  |
|-----------|---------|-------|----------|---------|---------|--------|-------|
| Waves     | Cottage | 540   | 4        | 1       | 2       | 8.9    | false |
| Seashells | Cottage | 540   | 4        | 2       | 2       | 8.7    | true  |
| ...       | ...     | ...   | ...      | ...     | ...     | ...    | ...   |

Introductie van filter

rentals.filter(
  # Predicaat
)

Predicaat: een voorwaarde evalueren tot True of False

Een groen vinkje voor een geslaagde test

Introductie van filter

rentals.filter(
  # Predicaat
)

Predicaat: een voorwaarde evalueren tot True of False

Een groen vinkje voor een geslaagde test

Een predicaat toevoegen

rentals.filter(
    pl.col("price") < 500
)

shape: (6, 8)
| name          | type    | price | bedrooms | doubles | singles | review | beach |
| ---           | ---     | ---   | ---      | ---     | ---     | ---    | ---   |
| str           | str     | i64   | i64      | i64     | i64     | f64    | bool  |
|---------------|---------|-------|----------|---------|---------|--------|-------|
| St Ives Bay   | Hotel   | 470   | 3        | 1       | 3       | 7.7    | true  |
| Perran View   | Hotel   | 341   | 3        | 1       | 4       | 8.2    | false |
| ...           | ...     | ...   | ...      | ...     | ...     | ...    | ...   |

Voorwaarden combineren met AND

rentals.filter(
   # Woningen onder 500 aan het strand
)

Voorwaarden combineren met AND

rentals.filter(
     pl.col("price") < 500
)

Voorwaarden combineren met AND

rentals.filter(
    (pl.col("price") < 500)
)

Voorwaarden combineren met AND

rentals.filter(
    (pl.col("price") < 500) &
)

Voorwaarden combineren met AND

rentals.filter(
    (pl.col("price") < 500) & (pl.col("beach") == True)
)

shape: (6, 8)
| name                | type    | price | ... | doubles | singles | review | beach |
| ---                 | ---     | ---   | ... | ---     | ---     | ---    | ---   |
| str                 | str     | i64   | ... | i64     | i64     | f64    | bool  |
|---------------------|---------|-------|---- |---------|---------|--------|-------|
| St Ives Bay         | Hotel   | 470   | ... | 1       | 3       | 7.7    | true  |
| Porth Retreat       | Hotel   | 249   | ... | 1       | 4       | null   | true  |
| Porth Caravan       | Caravan | 380   | ... | 1       | 4       | 7.2    | true  |
| ...                 | ...     | ...   | ... | ...     | ...     | ...    | ...   |

Voorwaarden combineren met OR

rentals.filter(

)

Voorwaarden combineren met OR

rentals.filter(
    (pl.col("price") < 500)
)

Voorwaarden combineren met OR

rentals.filter(
    (pl.col("price") < 500) | 
)

Voorwaarden combineren met OR

rentals.filter(
    (pl.col("price") < 500) | (pl.col("review") > 9.5)
)

shape: (18, 8)
| name              | type      | price | ... | doubles | singles | review | beach |
| ---               | ---       | ---   | ... | ---     | ---     | ---    | ---   |
| str               | str       | i64   | ... | i64     | i64     | f64    | bool  |
|-------------------|-----------|-------|-----|---------|---------|--------|-------|
| Bright House      | Cottage   | 956   | ... | 3       | null    | 9.9    | true  |
| Trewhiddle Villa  | Villa     | 1077  | ... | 3       | null    | 9.8    | false |
| ...               | ...       | ...   | ... | ...     | ...     | ...    | ...   |

Filteren op basis van een lijst

rentals.filter(
    pl.col("type").is_in(["Cottage", "Villa"])
)

shape: (34, 8)
| name          | type    | price | bedrooms | doubles | singles | review | beach |
| ---           | ---     | ---   | ---      | ---     | ---     | ---    | ---   |
| str           | str     | i64   | i64      | i64     | i64     | f64    | bool  |
|---------------|---------|-------|----------|---------|---------|--------|-------|
| Bright House  | Cottage | 956   | 4        | 3       | null    | 9.9    | true  |
| Bright House  | Cottage | 1050  | 4        | 3       | null    | 9.9    | true  |
| ...           | ...     | ...   | ...      | ...     | ...     | ...    | ...   |

Een predicaat negëren

rentals.filter(
    pl.col("type").is_in(["Cottage", "Villa"]).not_()
)

shape: (14, 8)
| name            | type      | price | ... | doubles | singles | review | beach |
| ---             | ---       | ---   | ... | ---     | ---     | ---    | ---   |
| str             | str       | i64   | ... | i64     | i64     | f64    | bool  |
|-----------------|-----------|-------| ... |---------|---------|--------|-------|
| Tregenna House  | Hotel     | 2411  | ... | 1       | 2       | 8.7    | true  |
| Tregenna House  | Hotel     | 2411  | ... | 1       | 2       | 8.7    | true  |
| ...             | ...       | ...   | ... | ...     | ...     | ...    | ...   |

Query-optimalisaties

(
  pl.scan_csv("vacation_rentals.csv")
  .filter(pl.col("type") == "Villa")

)

Query-optimalisaties

(
  pl.scan_csv("vacation_rentals.csv")
  .filter(pl.col("type") == "Villa")
  .explain()
)

Csv SCAN [vacation_rentals.csv]
PROJECT */8 COLUMNS
SELECTION: [(col("type")) == (String(Villa))]

Filtervoorwaarden

pl.col("price") < 500
pl.col("type").is_in(["Cottage", "Villa"])
pl.col("bedrooms").is_between(3,4)

¹ https://docs.pola.rs/api/python/stable/reference/expressions/boolean.html

Standaard Python-vergelijkingsoperatoren gebruiken

Operator	Betekenis
`==`	Gelijk aan
`!=`	Niet gelijk aan
`<`	Kleiner dan
`<=`	Kleiner of gelijk
`>`	Groter dan
`>=`	Groter of gelijk

Laten we oefenen!

Introductie tot Polars