转录文本的命名实体识别

Python 语音语言处理

Daniel Bourke

Machine Learning Engineer/YouTube Creator

安装 spaCy

# 安装 spaCy
$ pip install spacy

# 下载 spaCy 语言模型
$ python -m spacy download en_core_web_sm

使用 spaCy

import spacy


# 加载 spaCy 语言模型
nlp = spacy.load("en_core_web_sm")

# 创建一个 spaCy doc
doc = nlp("I'd like to talk about a smartphone I ordered on July 31st from your 
Sydney store, my order number is 40939440. I spoke to Georgia about it last week.")

spaCy 词元

# 显示不同词元及其位置
for token in doc:
  print(token.text, token.idx)

I 0
'd 1
like 4
to 9
talk 12
about 17
a 23
smartphone 25...

spaCy 句子

# 显示 doc 中的句子
for sentences in doc.sents:
  print(sentence)

I'd like to talk about a smartphone I ordered on July 31st from your Sydney store, 
my order number is 4093829.
I spoke to one of your customer service team, Georgia, yesterday.

spaCy 的命名实体

部分 spaCy 内置命名实体：

PERSON 人名（含虚构人物）。
ORG 组织：公司、机构等。
GPE 地缘政治实体：国家、城市、州。
PRODUCT 实物：物品、车辆、食品等（非服务）。
DATE 绝对或相对的日期或时期。
TIME 小于一天的时间。
MONEY 金额（含单位）。
CARDINAL 不属于其他类型的基数词。

spaCy 的命名实体

# 在 doc 中查找命名实体
for entity in doc.ents:
  print(entity.text, entity.label_)

July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

自定义命名实体

# 导入 EntityRuler 类
from spacy.pipeline import EntityRuler

# 查看 spaCy 流水线
print(nlp.pipeline)

[('tagger', <spacy.pipeline.pipes.Tagger at 0x1c3aa8a470>),
 ('parser', <spacy.pipeline.pipes.DependencyParser at 0x1c3bb60588>),
 ('ner', <spacy.pipeline.pipes.EntityRecognizer at 0x1c3bb605e8>)]

修改流水线

# 创建 EntityRuler 实例
ruler = EntityRuler(nlp)

# 向规则器添加词元模式
ruler.add_patterns([{"label":"PRODUCT", "pattern": "smartphone"}])

# 在 ner 之前将新规则加入流水线
nlp.add_pipe(ruler, before="ner")

# 查看更新后的流水线
nlp.pipeline

修改流水线

[('tagger', <spacy.pipeline.pipes.Tagger at 0x1c1f9c9b38>),
 ('parser', <spacy.pipeline.pipes.DependencyParser at 0x1c3c9cba08>),
 ('entity_ruler', <spacy.pipeline.entityruler.EntityRuler at 0x1c1d834b70>),
 ('ner', <spacy.pipeline.pipes.EntityRecognizer at 0x1c3c9cba68>)]

测试新流水线

# 测试新的实体规则
for entity in doc.ents:
    print(entity.text, entity.label_)

smartphone PRODUCT
July 31st DATE
Sydney GPE
4093829 CARDINAL
one CARDINAL
Georgia GPE
yesterday DATE

Let's rocket and practice spaCy!

Python 语音语言处理