ट्रांसक्राइब्ड टेक्स्ट पर named entity recognition

Python में Spoken Language Processing

Daniel Bourke

Machine Learning Engineer/YouTube Creator

spaCy इंस्टॉल करना

# Install spaCy
$ pip install spacy

# Download spaCy language model
$ python -m spacy download en_core_web_sm

spaCy का उपयोग

import spacy


# Load spaCy language model
nlp = spacy.load("en_core_web_sm")

# Create a spaCy doc
doc = nlp("I'd like to talk about a smartphone I ordered on July 31st from your 
Sydney store, my order number is 40939440. I spoke to Georgia about it last week.")

spaCy टोकन

# Show different tokens and positions
for token in doc:
  print(token.text, token.idx)

I 0
'd 1
like 4
to 9
talk 12
about 17
a 23
smartphone 25...

spaCy वाक्य

# Show sentences in doc
for sentences in doc.sents:
  print(sentence)

I'd like to talk about a smartphone I ordered on July 31st from your Sydney store, 
my order number is 4093829.
I spoke to one of your customer service team, Georgia, yesterday.

spaCy named entities

spaCy की कुछ बिल्ट-इन named entities:

PERSON लोग, काल्पनिक भी.
ORG कंपनियाँ, एजेंसियाँ, संस्थान आदि.
GPE देश, शहर, राज्य.
PRODUCT वस्तुएँ, वाहन, खाद्य आदि (सेवाएँ नहीं).
DATE पूर्ण या सापेक्ष तिथियाँ/अवधियाँ.
TIME दिन से छोटे समय.
MONEY धनराशि, यूनिट सहित.
CARDINAL अंक जो किसी अन्य प्रकार में नहीं आते.

spaCy named entities

# Find named entities in doc
for entity in doc.ents:
  print(entity.text, entity.label_)

July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

कस्टम named entities

# Import EntityRuler class
from spacy.pipeline import EntityRuler

# Check spaCy pipeline
print(nlp.pipeline)

[('tagger', <spacy.pipeline.pipes.Tagger at 0x1c3aa8a470>),
 ('parser', <spacy.pipeline.pipes.DependencyParser at 0x1c3bb60588>),
 ('ner', <spacy.pipeline.pipes.EntityRecognizer at 0x1c3bb605e8>)]

पाइपलाइन बदलना

# Create EntityRuler instance
ruler = EntityRuler(nlp)

# Add token pattern to ruler
ruler.add_patterns([{"label":"PRODUCT", "pattern": "smartphone"}])

# Add new rule to pipeline before ner
nlp.add_pipe(ruler, before="ner")

# Check updated pipeline
nlp.pipeline

पाइपलाइन बदलना

[('tagger', <spacy.pipeline.pipes.Tagger at 0x1c1f9c9b38>),
 ('parser', <spacy.pipeline.pipes.DependencyParser at 0x1c3c9cba08>),
 ('entity_ruler', <spacy.pipeline.entityruler.EntityRuler at 0x1c1d834b70>),
 ('ner', <spacy.pipeline.pipes.EntityRecognizer at 0x1c3c9cba68>)]

नई पाइपलाइन की जाँच

# Test new entity rule
for entity in doc.ents:
    print(entity.text, entity.label_)

smartphone PRODUCT
July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

चलो तेजी से spaCy का अभ्यास करें!

Python में Spoken Language Processing