安全公开发布数据集
Python 中的数据隐私与匿名化
Rebeca Gonzalez
Data engineer
探索数据集
为分析潜在的隐私风险，先获取相关领域与统计知识。
# 浏览数据集
cross_selling.head()
id Gender Age Driving_License Region_Code Previously_Insured Vehicle_Age Vintage Response
0 1 Male 44 1 28.0 0 > 2 Years 217 1
1 2 Male 76 1 3.0 0 1-2 Year 183 0
2 3 Male 47 1 28.0 0 > 2 Years 27 1
3 4 Male 21 1 11.0 1 < 1 Year 203 0
4 5 Female 29 1 41.0 1 < 1 Year 39 0
探索数据集
# 浏览数据集
cross_selling.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 381109 entries, 0 to 381108
Data columns (total 9 columns):
# Column Non-Null Count Dtype
------ -------------- -----
0 id 381109 non-null int64
1 Gender 381109 non-null object
2 Age 381109 non-null int64
3 Driving_License 381109 non-null int64
4 Region_Code 381109 non-null float64
5 Previously_Insured 381107 non-null int64
6 Vehicle_Age 381109 non-null object
7 Vintage 381109 non-null int64
8 Response 381109 non-null int64
dtypes: float64(1), int64(6), object(2)
memory usage: 26.2+ MB
探索数据集
# 计算 DataFrame 中唯一值的数量
cross_selling.nunique()
id 381109
Gender 2
Age 66
Driving_License 2
Region_Code 53
Previously_Insured 2
Vehicle_Age 3
Vintage 290
Response 2
dtype: int64
抑制唯一属性
# 对 id 列进行属性抑制 suppressed_df = cross_selling.drop('id', axis="columns")# 查看结果 DataFrame 的前几行 suppressed_df.head()
Gender Age Driving_License Region_Code Previously_Insured Vehicle_Age Vintage Response
0 Male 44 1 28.0 0 > 2 Years 217 1
1 Male 76 1 3.0 0 1-2 Year 183 0
2 Male 47 1 28.0 0 > 2 Years 27 1
3 Male 21 1 11.0 1 < 1 Year 203 0
4 Female 29 1 41.0 1 < 1 Year 39 0
清洗数据
# 删除 null 和 NaN 行
cleaned_df = suppressed_df.dropna(axis="index")
cleaned_df.head()
Gender Age Driving_License Region_Code Previously_Insured Vehicle_Age Vintage Response
0 Male 44 1 28.0 0 > 2 Years 217 1
1 Male 76 1 3.0 0 1-2 Year 183 0
2 Male 47 1 28.0 0 > 2 Years 27 1
3 Male 21 1 11.0 1 < 1 Year 203 0
4 Female 29 1 41.0 1 < 1 Year 39 0
从类别值采样
# 计算概率分布
cleaned_df['Gender'].value_counts(normalize=True)
Male 0.540957
Female 0.459043
Name: Gender, dtype: float64
从类别值采样
# 获取概率分布值 distributions = cleaned_df['Gender'].value_counts(normalize=True)# 按计算的概率分布进行采样 cleaned_df['Gender'] = np.random.choice(distributions.index, p=distributions, size=len(cleaned_df))
从类别值采样
# 查看结果数据集
cleaned_df
Gender Age Driving_License Region_Code Previously_Insured Vehicle_Age Vintage Response
0 Male 44 1 28.0 0 > 2 Years 217 1
1 Male 76 1 3.0 0 1-2 Year 183 0
2 Male 47 1 28.0 0 > 2 Years 27 1
3 Female 21 1 11.0 1 < 1 Year 203 0
4 Male 29 1 41.0 1 < 1 Year 39 0
... ... ... ... ... ... ... ... ...
381107 rows × 8 columns
从类别值采样
# 计算概率分布
cleaned_df['Gender'].value_counts(normalize=True)
Male 0.541973
Female 0.458027
Name: Gender, dtype: float64
移除列名
# 用数字替换列名
cleaned_df.columns = range(len(df.columns))
0 1 2 3 4 5 6 7
0 Male 44 1 28.0 0 > 2 Years 217 1
1 Male 76 1 3.0 0 1-2 Year 183 0
2 Male 47 1 28.0 0 > 2 Years 27 1
3 Female 21 1 11.0 1 < 1 Year 203 0
4 Male 29 1 41.0 1 < 1 Year 39 0
Vamos praticar!
Python 中的数据隐私与匿名化
