Joining DataFrames

Data Transformation with Polars

Liam Brannigan

Data Scientist & Polars Contributor

Reviews data

shape: (5, 4)
| business         | location    | review | price |
| ---              | ---         | ---    | ---   |
| str              | str         | f64    | i64   |
|------------------|-------------|--------|-------|
| 7burgers         | Wakey Wakey | 4.2    | 15    |
| Bang Bang Burger | Forest Rd.  | 3.8    | 12    |
| Costa Coffee     | City Point  | 4.5    | 8     |
| Costa Coffee     | Waterloo    | 4.1    | 8     |
| The Queens Head  | Denman St.  | 4.7    | 25    |

The inspections dataset

inspections = pl.read_csv("restaurant_inspections.csv")

shape: (5, 4)
| business        | location    | type       | hygiene_rating |
| ---             | ---         | ---        | ---            |
| str             | str         | str        | i64            |
|-----------------|-------------|------------|----------------|
| 7burgers        | Wakey Wakey | restaurant | 4              |
| Costa Coffee    | City Point  | café       | 5              |
| Costa Coffee    | Waterloo    | café       | 3              |
| The Queens Head | Denman St.  | bar        | 5              |
| Wagamama        | Soho        | restaurant | 4              |

What is a join?

Diagram showing a single DataFrame.

What is a join?

Diagram showing two DataFrames.

What is a join?

Diagram showing two DataFrames with a matching key highlighted.

What is a join?

Diagram showing two DataFrames being combined by matching rows on shared columns.

What is a join?

Diagram showing two DataFrames being combined by matching rows on shared columns with a matching key highlighted.

Join on a single column

reviews.join(                                       )

Join on a single column

reviews.join(inspections,                           )

Join on a single column

reviews.join(inspections, on="business"             )

Join on a single column

reviews.join(inspections, on="business", how="inner")

shape: (6, 7)
| business      | location    | ... | location_right | hygiene_rating |
| ---           | ---         | ... | ---            | ---            |
| str           | str         | ... | str            | i64            |
|---------------|-------------|-----|----------------|----------------|
| 7burgers      | Wakey Wakey | ... | Wakey Wakey    | 4              |
| Costa Coffee  | City Point  | ... | City Point     | 5              |
| Costa Coffee  | Waterloo    | ... | City Point     | 5              |
| Costa Coffee  | City Point  | ... | Waterloo       | 3              |
| Costa Coffee  | Waterloo    | ... | Waterloo       | 3              |
| The Queens Hd | Denman St.  | ... | Denman St.     | 5              |

Join on multiple columns

reviews.join(inspections, on=["business", "location"])

Join on multiple columns

reviews.join(inspections, on=["business", "location"])

shape: (4, 6)
| business        | location    | review | price | type       | hygiene_rating |
| ---             | ---         | ---    | ---   | ---        | ---            |
| str             | str         | f64    | i64   | str        | i64            |
|-----------------|-------------|--------|-------|------------|----------------|
| 7burgers        | Wakey Wakey | 4.2    | 15    | restaurant | 4              |
| Costa Coffee    | City Point  | 4.5    | 8     | café       | 5              |
| Costa Coffee    | Waterloo    | 4.1    | 8     | café       | 3              |
| The Queens Head | Denman St.  | 4.7    | 25    | bar        | 5              |

Left join

reviews.join(inspections, on=["business", "location"],           )

Left join

reviews.join(inspections, on=["business", "location"], how="left")

shape: (5, 6)
| business         | location    | review | price | type       | hygiene_rating |
| ---              | ---         | ---    | ---   | ---        | ---            |
| str              | str         | f64    | i64   | str        | i64            |
|------------------|-------------|--------|-------|------------|----------------|
| 7burgers         | Wakey Wakey | 4.2    | 15    | restaurant | 4              |
| Bang Bang Burger | Forest Rd.  | 3.8    | 12    | null       | null           |
| Costa Coffee     | City Point  | 4.5    | 8     | café       | 5              |
| Costa Coffee     | Waterloo    | 4.1    | 8     | café       | 3              |
| The Queens Head  | Denman St.  | 4.7    | 25    | bar        | 5              |

Full join

reviews.join(inspections, on=["business", "location"],                          )

Full join

reviews.join(inspections, on=["business", "location"], how="full"               )

Full join

reviews.join(inspections, on=["business", "location"], how="full", coalesce=True)

shape: (6, 6)
| business         | location    | review | price | type       | hygiene_rating |
| ---              | ---         | ---    | ---   | ---        | ---            |
| str              | str         | f64    | i64   | str        | i64            |
|------------------|-------------|--------|-------|------------|----------------|
| 7burgers         | Wakey Wakey | 4.2    | 15    | restaurant | 4              |
| Costa Coffee     | City Point  | 4.5    | 8     | café       | 5              |
| Costa Coffee     | Waterloo    | 4.1    | 8     | café       | 3              |
| The Queens Head  | Denman St.  | 4.7    | 25    | bar        | 5              |
| Wagamama         | Soho        | null   | null  | restaurant | 4              |
| Bang Bang Burger | Forest Rd.  | 3.8    | 12    | null       | null           |

Choosing a join strategy

2 tables

Inner join

Inner join representation

Left join

Left join representation

Full join

Full join representation

Let's practice!

Data Transformation with Polars