Pengenalan entitas bernama pada teks transkripsi

Pemrosesan Bahasa Lisan dengan Python

Daniel Bourke

Machine Learning Engineer/YouTube Creator

Menginstal spaCy

# Install spaCy
$ pip install spacy

# Download spaCy language model
$ python -m spacy download en_core_web_sm

Menggunakan spaCy

import spacy


# Muat model bahasa spaCy
nlp = spacy.load("en_core_web_sm")

# Buat doc spaCy
text = ("I'd like to talk about a smartphone I ordered on July 31st "
        "from your Sydney store, my order number is 4093829. I spoke "
        "to one of your customer service team, Georgia, yesterday.")
doc = nlp(text)

Token spaCy

# Tampilkan token dan posisinya
for token in doc:
  print(token.text, token.idx)

I 0
'd 1
like 4
to 9
talk 12
about 17
a 23
smartphone 25...

Kalimat spaCy

# Tampilkan kalimat dalam doc
for sentence in doc.sents:
  print(sentence)

I'd like to talk about a smartphone I ordered on July 31st from your Sydney store, 
my order number is 4093829.
I spoke to one of your customer service team, Georgia, yesterday.

Entitas bernama spaCy

Beberapa entitas bernama bawaan spaCy:

PERSON Orang, termasuk fiksi.
ORG Perusahaan, lembaga, institusi, dll.
GPE Negara, kota, negara bagian.
PRODUCT Objek, kendaraan, makanan, dll. (Bukan layanan.)
DATE Tanggal absolut/relatif atau periode.
TIME Waktu kurang dari sehari.
MONEY Nilai uang, termasuk satuan.
CARDINAL Bilangan yang tidak masuk tipe lain.

Entitas bernama spaCy

# Temukan entitas bernama dalam doc
for entity in doc.ents:
  print(entity.text, entity.label_)

July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

Entitas bernama kustom

# Periksa pipeline spaCy
print(nlp.pipe_names)

['tok2vec', 'tagger', 'parser',
 'attribute_ruler', 'lemmatizer', 'ner']

Mengubah pipeline

# Tambahkan entity_ruler ke pipeline, sebelum ner
ruler = nlp.add_pipe("entity_ruler", before="ner")

# Tambahkan pola token ke ruler
ruler.add_patterns([{"label":"PRODUCT", "pattern": "smartphone"}])

# Periksa pipeline yang diperbarui
nlp.pipe_names

Mengubah pipeline

['tok2vec', 'tagger', 'parser', 'attribute_ruler',
 'lemmatizer', 'entity_ruler', 'ner']

Menguji pipeline baru

# Buat ulang doc dengan pipeline terbaru
doc = nlp(text)

# Uji aturan entitas baru
for entity in doc.ents:
  print(entity.text, entity.label_)

smartphone PRODUCT
July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

Ayo melesat dan berlatih spaCy!

Pemrosesan Bahasa Lisan dengan Python