问答

Python 中的自然语言处理（NLP）

Fouad Trad

Machine Learning Engineer

抽取式问答

答案直接从段落中复制

图片显示上下文"图书馆工作日于下午6点关闭"

图片显示抽取式答案："下午6点"

生成式问答

模型生成自然表述的答案

图片显示问题："图书馆工作日何时关闭？"

图片显示生成式答案："关闭时间为下午6点"

抽取式问答

适用于：
- 搜索引擎
- 文档检索系统
- 阅读理解应用
更准确，错误更少

生成式问答

适用于：
- 对话代理
- 虚拟助手
- 客服机器人
可能引入错误

抽取式问答

from transformers import pipeline

qa_pipeline = pipeline(task="question-answering", 
                       model="distilbert/distilbert-base-cased-distilled-squad")

context = """The Amazon rainforest is the largest tropical rainforest in the world, 
covering parts of Brazil, Peru, and Colombia."""

question = "Which countries does the Amazon rainforest cover?"

qa_answer = qa_pipeline(question=question, context=context)
print(qa_answer)

{'score': 0.9242347478866577,

 'start': 90, 'end': 116,

 'answer': 'Brazil, Peru, and Colombia'}

生成式问答

from transformers import pipeline
qa_pipeline = pipeline(task="text2text-generation",
                       model="fangyuan/hotpotqa_abstractive")

context = """The Amazon rainforest is the largest tropical rainforest in the world, 
covering parts of Brazil, Peru, and Colombia."""
question = "Which countries does the Amazon rainforest cover?"

input_text = f"question: {question}  context: {context}"

result = qa_pipeline(input_text)

print(result)

[{'generated_text': 'The Amazon rainforest covers parts of Brazil, Peru, and Colombia.'}]

Passons à la pratique !

Python 中的自然语言处理（NLP）