問答系統

Python 的 Natural Language Processing（NLP）

Fouad Trad

Machine Learning Engineer

擷取式 QA

答案直接從段落中擷取

Image showing a context "The library closes at 6 PM on weekdays"

Image showing an extractive answer: "6 PM"

生成式 QA

模型產生自然的答案

Image showing a question: "When does the library close on weekdays?"

Image showing an abstractive answer: "Closure is at 6 PM"

擷取式 QA

適用於：
- 搜尋引擎
- 文件檢索系統
- 閱讀理解 App
較精準，較不易出錯

生成式 QA

適用於：
- 對話型代理
- 虛擬助理
- 客服機器人
可能引入錯誤

擷取式 QA

from transformers import pipeline

qa_pipeline = pipeline(task="question-answering", 
                       model="distilbert/distilbert-base-cased-distilled-squad")

context = """The Amazon rainforest is the largest tropical rainforest in the world, 
covering parts of Brazil, Peru, and Colombia."""

question = "Which countries does the Amazon rainforest cover?"

qa_answer = qa_pipeline(question=question, context=context)
print(qa_answer)

{'score': 0.9242347478866577,

 'start': 90, 'end': 116,

 'answer': 'Brazil, Peru, and Colombia'}

生成式 QA

from transformers import pipeline
qa_pipeline = pipeline(task="text2text-generation",
                       model="fangyuan/hotpotqa_abstractive")

context = """The Amazon rainforest is the largest tropical rainforest in the world, 
covering parts of Brazil, Peru, and Colombia."""
question = "Which countries does the Amazon rainforest cover?"

input_text = f"question: {question}  context: {context}"

result = qa_pipeline(input_text)

print(result)

[{'generated_text': 'The Amazon rainforest covers parts of Brazil, Peru, and Colombia.'}]

一起來練習吧！

Python 的 Natural Language Processing（NLP）