Intents and classification

Building Chatbots in Python

Alan Nichol

Co-founder and CTO, Rasa

Supervised learning

A classifier predicts the intent label given a sentence
"Fit" classifier by tuning it on training data
Evaluate performance on test data
Accuracy: the fraction of labels we predict correctly

ATIS dataset

Thousands of sentences with labeled intents and entities
Collected from a real flight booking service
Intents like
- atis_flight
- atis_airfare

ATIS dataset II

sentences_train[:2]                    
labels_train[:2]

[ "atis_flight",
  "atis_flight"]

["i want to fly from boston at 
  838 am and arrive in denver at 
  1110 in the morning",
  "what flights are available 
  from pittsburgh to baltimore 
  on thursday morning"]

import numpy as np
X_train_shape = (
    len(sentences_train),
    nlp.vocab.vectors_length)

X_train = np.zeros(X_train_shape)

for sentence in sentences_train:
    X_train[i,:] = nlp(sentence).vector

Nearest neighbor classification

Need training data
- Sentences which we've already labeled with their intents
Simplest solution:
- Look for the labeled example that's most similar
- Use its intent as a best guess
Nearest neighbor classification

Nearest neighbor classification in scikit-learn

from sklearn.metrics.pairwise import cosine_similarity
test_message = """
i would like to find a flight from charlotte
to las vegas that makes a stop in st. louis"""
test_x = nlp(test_message).vector

scores = [
    cosine_similarity(X[i,:], test_x) 
    for i in range(len(sentences_train)
]

labels_train[np.argmax(scores)]

'atis_flight'

Support vector machines

Nearest neighbors is very simple - we can do better
SVM / SVC: support vector machine / classifier

from sklearn.svm import SVC

clf = SVC()

clf.fit(X_train, y_train)

y_pred = clf.predict(X_test)

Let's practice!

Building Chatbots in Python