Costruire un classificatore Naive Bayes con BoW
Feature Engineering per NLP in Python
Rounak Banik
Data Scientist
Filtro antispam
| message | label |
|---|---|
| WINNER!! As a valued network customer you have been selected to receive a $900 prize reward! To claim call 09061701461 | spam |
| Ah, work. I vaguely remember that. What does it feel like? | ham |
Passi
- Preprocessamento del testo
- Creare un modello bag-of-words (rappresentazione)
- Machine Learning
Preprocessare il testo con CountVectorizer
Argomenti di CountVectorizer
lowercase:False,Truestrip_accents:'unciode','ascii',Nonestop_words:'english',list,Nonetoken_pattern:regextokenizer:function
Costruire il modello BoW
# Import CountVectorizer from sklearn.feature_extraction.text import CountVectorizer# Crea un oggetto CountVectorizer vectorizer = CountVectorizer(strip_accents='ascii', stop_words='english', lowercase=False)# Import train_test_split from sklearn.model_selection import train_test_split # Suddividi in training e test set X_train, X_test, y_train, y_test = train_test_split(df['message'], df['label'], test_size=0.25)
Costruire il modello BoW
... ... # Genera i vettori BoW di training X_train_bow = vectorizer.fit_transform(X_train)# Genera i vettori BoW di test X_test_bow = vectorizer.transform(X_test)
Addestrare il classificatore Naive Bayes
# Import MultinomialNB from sklearn.naive_bayes import MultinomialNB# Crea un oggetto MultinomialNB clf = MultinomialNB()# Allena clf clf.fit(X_train_bow, y_train)# Calcola l'accuratezza sul test set accuracy = clf.score(X_test_bow, y_test) print(accuracy)
0.760051
Esercitiamoci!
Feature Engineering per NLP in Python
