Vincoli di intervallo dei dati
Pulizia dei dati in Python
Adel Nehme
VP of AI Curriculum, DataCamp
Motivazione
movies.head()
movie_name avg_rating
0 The Godfather 5
1 Frozen 2 3
2 Shrek 4
...
Motivazione
import matplotlib.pyplot as plt
plt.hist(movies['avg_rating'])
plt.title('Average rating of movies (1-5)')
Motivazione
Possono esistere iscrizioni future?
# Import date time
import datetime as dt
today_date = dt.date.today()
user_signups[user_signups['subscription_date'] > dt.date.today()]
subscription_date user_name ... Country
0 01/05/2021 Marah ... Nauru
1 09/08/2020 Joshua ... Austria
2 04/01/2020 Heidi ... Guinea
3 11/10/2020 Rina ... Turkmenistan
4 11/07/2020 Christine ... Marshall Islands
5 07/07/2020 Ayanna ... Gabon
Come gestire dati fuori intervallo?
- Eliminare i dati
- Impostare min e max personalizzati
- Trattare come mancanti e imputare
- Impostare valori in base ad assunzioni di business
Esempio sui film
import pandas as pd
# Output Movies with rating > 5
movies[movies['avg_rating'] > 5]
movie_name avg_rating
23 A Beautiful Mind 6
65 La Vita e Bella 6
77 Amelie 6
# Drop values using filtering movies = movies[movies['avg_rating'] <= 5]# Drop values using .drop() movies.drop(movies[movies['avg_rating'] > 5].index, inplace = True)# Assert results assert movies['avg_rating'].max() <= 5
Esempio sui film
# Convert avg_rating > 5 to 5
movies.loc[movies['avg_rating'] > 5, 'avg_rating'] = 5
# Assert statement
assert movies['avg_rating'].max() <= 5
Ricorda: nessun output = test superato
Esempio su intervalli di date
import datetime as dt
import pandas as pd
# Output data types
user_signups.dtypes
subscription_date object
user_name object
Country object
dtype: object
# Convert to date
user_signups['subscription_date'] = pd.to_datetime(user_signups['subscription_date']).dt.date
Esempio su intervalli di date
today_date = dt.date.today()
Elimina i dati
# Drop values using filtering user_signups = user_signups[user_signups['subscription_date'] < today_date]# Drop values using .drop() user_signups.drop(user_signups[user_signups['subscription_date'] > today_date].index, inplace = True)
Imposta un limite superiore fisso
# Drop values using filtering
user_signups.loc[user_signups['subscription_date'] > today_date, 'subscription_date'] = today_date
# Assert is true
assert user_signups.subscription_date.max().date() <= today_date
Ayo berlatih!
Pulizia dei dati in Python
