Calculando quantis e histogramas
Transformação de Dados com Polars
Liam Brannigan
Data Scientist & Polars Contributor
Quantis e histogramas
reviews = pl.read_csv("restaurants_quantiles.csv")
shape: (15, 4)
| business | location | review | price |
| --- | --- | --- | --- |
| str | str | f64 | i64 |
|------------------|---------------|--------|-------|
| 7burgers | Wakey Wakey | 4.2 | 15 |
| Bang Bang Burger | Forest Rd. | 3.8 | 12 |
| ... | ... | ... | ... |
| The Old Ivy | Angel | 4.1 | 28 |
| Franco Manca | Brixton Rd. | 4.0 | 22 |
Categorização por quantis
Categorização por quantis
Categorização por quantis
Conjunto de reviews
reviews = pl.read_csv("restaurants_quantiles.csv")
shape: (15, 4)
| business | location | review | price |
| --- | --- | --- | --- |
| str | str | f64 | i64 |
|------------------|---------------|--------|-------|
| 7burgers | Wakey Wakey | 4.2 | 15 |
| Bang Bang Burger | Forest Rd. | 3.8 | 12 |
| ... | ... | ... | ... |
| The Old Ivy | Angel | 4.1 | 28 |
| Franco Manca | Brixton Rd. | 4.0 | 22 |
Calcular quantis
reviews.select(
)
Calcular quantis
reviews.select(
pl.col("price")
)
Calcular quantis
reviews.select(
pl.col("price").quantile(0.33)
)
Calcular quantis
reviews.select(
pl.col("price").quantile(0.33).alias("budget_cutoff"),
)
Calcular quantis
reviews.select(
pl.col("price").quantile(0.33).alias("budget_cutoff"),
pl.col("price").quantile(0.67).alias("mid_cutoff")
)
shape: (1, 2)
| budget_cutoff | mid_cutoff |
| --- | --- |
| f64 | f64 |
|---------------|------------|
| 14.0 | 20.0 |
Usando quantis para criar categorias
reviews.with_columns(
pl.when(pl.col("price") <= pl.col("price").quantile(0.33)).then(pl.lit("budget"))
)
Usando quantis para criar categorias
reviews.with_columns(
pl.when(pl.col("price") <= pl.col("price").quantile(0.33)).then(pl.lit("budget"))
.when(pl.col("price") <= pl.col("price").quantile(0.67)).then(pl.lit("mid"))
)
Usando quantis para criar categorias
reviews.with_columns(
pl.when(pl.col("price") <= pl.col("price").quantile(0.33)).then(pl.lit("budget"))
.when(pl.col("price") <= pl.col("price").quantile(0.67)).then(pl.lit("mid"))
.otherwise(pl.lit("premium")).alias("price_band")
)
shape: (15, 5)
| business | location | review | price | price_band |
| --- | --- | --- | --- | --- |
| str | str | f64 | i64 | str |
|------------------|---------------|--------|-------|------------|
| 7burgers | Wakey Wakey | 4.2 | 15 | mid |
| Bang Bang Burger | Forest Rd. | 3.8 | 12 | budget |
| ... | ... | ... | ... | ... |
Quantis vs. quebras fixas
$$
Limiares por quantis
- Baseados nos dados atuais
- Tamanhos de grupos balanceados
- Adapta conforme os dados mudam
$$
Limiares fixos
- Ex.: 10 e 20 libras
- Tamanhos de grupos incertos
- Fáceis de entender
Categorias com limiares fixos
reviews.with_columns(
)
Categorias com limiares fixos
reviews.with_columns(
pl.col("price")
)
Categorias com limiares fixos
reviews.with_columns(
pl.col("price")
.cut(
)
Categorias com limiares fixos
reviews.with_columns(
pl.col("price")
.cut(breaks=[10, 20]
)
Categorias com limiares fixos
reviews.with_columns(
pl.col("price")
.cut(breaks=[10, 20], labels=["budget", "mid", "premium"])
)
Categorias com limiares fixos
reviews.with_columns(
pl.col("price")
.cut(breaks=[10, 20], labels=["budget", "mid", "premium"]).alias("price_band")
)
shape: (15, 5)
| business | location | review | price | price_band |
| --- | --- | --- | --- | --- |
| str | str | f64 | i64 | cat |
|------------------|---------------|--------|-------|------------|
| 7burgers | Wakey Wakey | 4.2 | 15 | mid |
| Costa Coffee | City Point | 4.5 | 8 | budget |
| The Queens Head | Denman St. | 4.7 | 25 | premium |
| ... | ... | ... | ... | ... |
Construir um histograma
reviews["price"]
Construir um histograma
reviews["price"].hist()
shape: (10, 3)
| breakpoint | category | count |
| --- | --- | --- |
| f64 | cat | u32 |
|------------|--------------|-------|
| 8.4 | [6.0, 8.4] | 3 |
| 10.8 | (8.4, 10.8] | 1 |
| ... | ... | ... |
| 27.6 | (25.2, 27.6] | 0 |
| 30.0 | (27.6, 30.0] | 2 |
Histograma com nº de bins definido
reviews["price"].hist(bin_count=5)
shape: (5, 3)
| breakpoint | category | count |
| --- | --- | --- |
| f64 | cat | u32 |
|------------|---------------|-------|
| 10.8 | [6.0, 10.8] | 4 |
| 15.6 | (10.8, 15.6] | 3 |
| 20.4 | (15.6, 20.4] | 3 |
| 25.2 | (20.4, 25.2] | 3 |
| 30.0 | (25.2, 30.0] | 2 |
Histograma com quebras definidas
reviews["price"].hist(bins=[0, 10, 20, 100])
shape: (3, 3)
| breakpoint | category | count |
| --- | --- | --- |
| f64 | cat | u32 |
|------------|----------------|-------|
| 10.0 | [0.0, 10.0] | 4 |
| 20.0 | (10.0, 20.0] | 6 |
| 100.0 | (20.0, 100.0] | 5 |
Plotar o histograma
import plotly.express as pxhist = reviews["price"].hist(bins=[0, 10, 20, 100])
Plotar o histograma
import plotly.express as px
hist = reviews["price"].hist(bins=[0, 10, 20, 100])
fig = px.bar(hist, x="category", y="count")
fig.show()
Plotar o histograma
Vamos praticar!
Transformação de Dados com Polars
