Training and updating models

Geavanceerde NLP met spaCy

Ines Montani

spaCy core developer

Why updating the model?

  • Better results on your specific domain
  • Learn classification schemes specifically for your problem
  • Essential for text classification
  • Very useful for named entity recognition
  • Less critical for part-of-speech tagging and dependency parsing
Geavanceerde NLP met spaCy

How training works (1)

  1. Initialize the model weights randomly with nlp.begin_training
  2. Predict a few examples with the current weights by calling nlp.update
  3. Compare prediction with true labels
  4. Calculate how to change weights to improve predictions
  5. Update weights slightly
  6. Go back to 2.
Geavanceerde NLP met spaCy

How training works (2)

Diagram of the training process

  • Training data: Examples and their annotations.
  • Text: The input text the model should predict a label for.
  • Label: The label the model should predict.
  • Gradient: How to change the weights.
Geavanceerde NLP met spaCy

Example: Training the entity recognizer

  • The entity recognizer tags words and phrases in context
  • Each token can only be part of one entity
  • Examples need to come with context
("iPhone X is coming", {'entities': [(0, 8, 'GADGET')]})
  • Texts with no entities are also important
("I need a new phone! Any tips?", {'entities': []})
  • Goal: teach the model to generalize
Geavanceerde NLP met spaCy

The training data

  • Examples of what we want the model to predict in context
  • Update an existing model: a few hundred to a few thousand examples
  • Train a new category: a few thousand to a million examples
    • spaCy's English models: 2 million words
  • Usually created manually by human annotators
  • Can be semi-automated – for example, using spaCy's Matcher!
Geavanceerde NLP met spaCy

Let's practice!

Geavanceerde NLP met spaCy

Preparing Video For Download...