Bekerja dengan Database

Menskalakan dan Mengoptimalkan Pipeline Data dengan Polars

Liam Brannigan

Data Scientist & Polars Contributor

Menquery database dari Polars

Diagram konseptual yang menunjukkan Polars mengirim kueri SQL dan uri ke database.

Menquery database dari Polars

Diagram konseptual yang menunjukkan Polars mengirim kueri SQL ke database dan menerima tabel hasil melalui engine database.

Database yang bisa di-query Polars

Database yang dapat digunakan Polars

Embedded → berjalan lokal, tanpa server
DuckDB → database analitis

Tabel izin bangunan

SELECT *
FROM building_permits
LIMIT 5

shape: (5, 5)
| issue_date | permit_type | review_type | street_name  | work_description             |
| ---        | ---         | ---         | ---          | ---                          |
| date       | str         | str         | str          | str                          |
|------------|-------------|-------------|--------------|------------------------------|
| 2026-04-30 | Signs       | SIGN PERMIT | CLARK ST     | Wall sign install            |
| 2026-04-28 | Renovation  | SELF CERT   | OAKLEY AVE   | Add dormer to 8-unit roof    |
| 2026-03-21 | Easy Permit | EASY PERMIT | SEMINARY AVE | Parapet masonry repair       |
| 2026-03-11 | Easy Permit | EASY PERMIT | RHODES AVE   | Rebuild south chimney side   |
| 2026-03-03 | Easy Permit | EASY PERMIT | GIDDINGS ST  | Porch lintel + tuckpointing  |

URI koneksi

import polars as pl

uri = "postgresql://analyst:[email protected]:5432/buildings"

Protokol postgresql://
analyst:secret nama pengguna dan sandi
data.chicago.org:5432 host dan port
Nama database buildings

Menquery database

query = """
    SELECT issue_date, permit_type
    FROM building_permits
    WHERE issue_date >= DATE '2026-01-01'
"""

Menquery database

query = """
    SELECT issue_date, permit_type
    FROM building_permits
    WHERE issue_date >= DATE '2026-01-01'
"""

permit_counts = (
    pl.read_database_uri(query=query, uri=uri)



)

Menquery database

query = """
    SELECT issue_date, permit_type
    FROM building_permits
    WHERE issue_date >= DATE '2026-01-01'
"""

permit_counts = (
    pl.read_database_uri(query=query, uri=uri)
    .group_by("permit_type")
    .len()
    .sort("len", descending=True)
)

Hitung izin per grup

permit_counts

shape: (3, 2)
| permit_type                     | len |
| ---                             | --- |
| str                             | u32 |
|---------------------------------|-----|
| Easy Permit                     | 58  |
| New Construction                | 14  |
| Renovation                      | 11  |

Pilih engine Anda

pl.read_database_uri(
    query=query,
    uri=uri,
    engine="connectorx",
)

Lebih banyak database didukung

pl.read_database_uri(
    query=query,
    uri=uri,
    engine="adbc",
)

Lebih cepat untuk database yang didukung

¹ https://arrow.apache.org/adbc/current/driver/status.html

Menulis kembali ke database

permit_counts.write_database(



)

Menulis kembali ke database

permit_counts.write_database(
    table_name="permit_type_summary",
    connection=uri,

)

Menulis kembali ke database

permit_counts.write_database(
    table_name="permit_type_summary",
    connection=uri,
    if_table_exists="replace",
)

append
fail

Integrasi DuckDB

import duckdb

db = duckdb.connect("permits.duckdb")

DuckDB → output Polars bawaan

Integrasi DuckDB

import duckdb

db = duckdb.connect("permits.duckdb")

result = db.sql("SELECT * FROM permit_type_summary").pl()

shape: (3, 2)
| permit_type                     | len |
| ---                             | --- |
| str                             | u32 |
|---------------------------------|-----|
| Easy Permit                     | 58  |
| New Construction                | 14  |
| Renovation                      | 11  |

Ayo berlatih!

Menskalakan dan Mengoptimalkan Pipeline Data dengan Polars