更多 Pandas
面向 R 用户的 Python
Daniel Chen
Instructor
缺失数据
- NaN 缺失值来自 numpy
np.NaN、np.NAN、np.nan等同于 R 的NA- 用
pd.isnull检查缺失- 用
pd.notnull检查非缺失 pd.isnull等同于pd.isna
- 用
处理缺失数据
df
name treatment_a treatment_b
0 John Smith NaN 2
1 Jane Doe 16.0 11
2 Mary Johnson 3.0 1
a_mean = df['treatment_a'].mean()
a_mean
9.5
Fillna
df['a_fill'] = df['treatment_a'].fillna(a_mean)
df
name treatment_a treatment_b a_fill
0 John Smith NaN 2 9.5
1 Jane Doe 16.0 11 16.0
2 Mary Johnson 3.0 1 3.0
更多 Pandas
- 应用自定义函数
- 分组操作
- 整理数据
应用自定义函数
- 内置函数
- 自定义函数
apply方法- 指定轴
R
df = data.frame('a' = c(1, 2, 3),
'b' = c(4, 5, 6))
apply(df, 2, mean)
a b
2 5
apply(df, 1, mean)
2.5 3.5 4.5
Python
import pandas as pd
df = pd.DataFrame({'A': [1, 2, 3],
'B':[4, 5, 6]})
df.apply(np.mean, axis=0)
A 2.0
B 5.0
dtype: float64
df.apply(np.mean, axis=1)
0 2.5
1 3.5
2 4.5
dtype: float64
整洁数据（Tidy）
- 重塑并整理数据
- Hadley Wickham，《Tidy Data》
- 每行代表一个观测
- 每列代表一个变量
- 每类观测单位组成一张表
Tidy Data 论文: http://vita.had.co.nz/papers/tidy-data.pdf
整洁：melt
df
name treatment_a treatment_b
0 John Smith NaN 2
1 Jane Doe 16.0 11
2 Mary Johnson 3.0 1
df_melt = pd.melt(df, id_vars='name')
df_melt
name variable value
0 John Smith treatment_a NaN
1 Jane Doe treatment_a 16.0
2 Mary Johnson treatment_a 3.0
3 John Smith treatment_b 2.0
...
整洁：pivot_table
df_melt_pivot = pd.pivot_table(df_melt,
index='name',
columns='variable',
values='value')
df_melt_pivot
variable treatment_a treatment_b
name
Jane Doe 16.0 11.0
John Smith NaN 2.0
Mary Johnson 3.0 1.0
重置索引
df_melt_pivot.reset_index()
variable name treatment_a treatment_b
0 Jane Doe 16.0 11.0
1 John Smith NaN 2.0
2 Mary Johnson 3.0 1.0
Groupby（分组）
groupby：分组-应用-合并- 将数据拆分为分区
- 对每个分区应用函数
- 合并结果
执行 groupby
name variable value
0 John Smith treatment_a NaN
1 Jane Doe treatment_a 16.0
2 Mary Johnson treatment_a 3.0
3 John Smith treatment_b 2.0
4 Jane Doe treatment_b 11.0
5 Mary Johnson treatment_b 1.0
df_melt.groupby('name')['value'].mean()
name
Jane Doe 13.5
John Smith 2.0
Mary Johnson 2.0
Name: value, dtype: float64
¡Vamos a practicar!
面向 R 用户的 Python
