データ範囲の制約
Pythonで学ぶデータクリーニング
Adel Nehme
VP of AI Curriculum, DataCamp
動機づけ
movies.head()
movie_name avg_rating
0 The Godfather 5
1 Frozen 2 3
2 Shrek 4
...
動機づけ
import matplotlib.pyplot as plt
plt.hist(movies['avg_rating'])
plt.title('Average rating of movies (1-5)')
動機づけ
将来のサインアップはあり得ますか？
# Import date time
import datetime as dt
today_date = dt.date.today()
user_signups[user_signups['subscription_date'] > dt.date.today()]
subscription_date user_name ... Country
0 01/05/2021 Marah ... Nauru
1 09/08/2020 Joshua ... Austria
2 04/01/2020 Heidi ... Guinea
3 11/10/2020 Rina ... Turkmenistan
4 11/07/2020 Christine ... Marshall Islands
5 07/07/2020 Ayanna ... Gabon
範囲外データへの対処法
- データを削除
- 最小・最大をカスタム設定
- 欠損として扱い、補完
- 業務前提に応じて値を設定
映画の例
import pandas as pd
# Output Movies with rating > 5
movies[movies['avg_rating'] > 5]
movie_name avg_rating
23 A Beautiful Mind 6
65 La Vita e Bella 6
77 Amelie 6
# Drop values using filtering movies = movies[movies['avg_rating'] <= 5]# Drop values using .drop() movies.drop(movies[movies['avg_rating'] > 5].index, inplace = True)# Assert results assert movies['avg_rating'].max() <= 5
映画の例
# Convert avg_rating > 5 to 5
movies.loc[movies['avg_rating'] > 5, 'avg_rating'] = 5
# Assert statement
assert movies['avg_rating'].max() <= 5
出力がなければ成功です
日付範囲の例
import datetime as dt
import pandas as pd
# Output data types
user_signups.dtypes
subscription_date object
user_name object
Country object
dtype: object
# Convert to date
user_signups['subscription_date'] = pd.to_datetime(user_signups['subscription_date']).dt.date
日付範囲の例
today_date = dt.date.today()
データを削除する
# Drop values using filtering user_signups = user_signups[user_signups['subscription_date'] < today_date]# Drop values using .drop() user_signups.drop(user_signups[user_signups['subscription_date'] > today_date].index, inplace = True)
上限を固定して日付をハードコード
# Drop values using filtering
user_signups.loc[user_signups['subscription_date'] > today_date, 'subscription_date'] = today_date
# Assert is true
assert user_signups.subscription_date.max().date() <= today_date
Ayo berlatih!
Pythonで学ぶデータクリーニング
