Let's predict the sentiment!

Sentiment Analysis in Python

Violeta Misheva

Data Scientist

Classification problems

  • Product and movie reviews: positive or negative sentiment (binary classification)
  • Tweets about airline companies: positive, neutral and negative (multi-class classification)
Sentiment Analysis in Python

Linear and logistic regressions

a graph showing a linear regression (left) fitted for a few values, and a logistic regression(right), fitted to similar values

Sentiment Analysis in Python

Logistic function

  • Linear regression: numeric outcome
  • Logistic regression: probability:

$$ Probability(sentiment=positive|review) $$

a graph showing a logistic(sigmoid) function, fitted to a few examples

Sentiment Analysis in Python

Logistic regression in Python

from sklearn.linear_model import LogisticRegression

log_reg = LogisticRegression().fit(X, y)
Sentiment Analysis in Python

Measuring model performance

Accuracy: Fraction of predictions our model got right.

  • The higher and closer the accuracy is to 1, the better
# Accuracy using score
score = log_reg.score(X, y)
print(score)

0.9009
Sentiment Analysis in Python

Using accuracy score

# Accuracy using accuracy_score
from sklearn.metrics import accuracy_score

y_predicted = log_reg.predict(X)
acurracy = accuracy_score(y, y_predicted)

0.9009
Sentiment Analysis in Python

Let's practice!

Sentiment Analysis in Python

Preparing Video For Download...