文本摘要

使用 Hugging Face

Jacob H. Marquez

Lead Data Engineer

什么是摘要？

长文本

抽取式 vs. 抽象式

抽取式：

$$ ✅ 从文本中选取关键句

$$ ✅ 高效，资源占用低

$$ ❌ 灵活性差；可能不够连贯

抽象式：

$$ ✅ 生成重述后的新文本

$$ ✅ 更清晰、更易读

$$ ❌ 需更多资源与计算

抽取式摘要的用例

📑 法律文件：标出关键条款

💰 金融研究：提取洞见

法律文件

金融研究

抽象式摘要的用例

新闻文章

内容推荐.jpg

📰 新闻文章：生成精炼摘要

$$ $$

📍 内容推荐：生成吸引人的描述

抽取式摘要演示

from transformers import pipeline

# Load the extractive summarization pipeline
summarizer = pipeline("summarization", model="nyamuda/extractive-summarization")

text = "This is my really large text about Data Science..."
summary_text = summarizer(text)

print(summary_text[0]['summary_text'])

"data science is a field that combines mathematics, statistics...."

抽象式摘要演示

from transformers import pipeline

# Load the abstractive summarization pipeline
summarizer = pipeline("summarization", model="sshleifer/distilbart-cnn-12-6")


text = "This is my really large text about Data Science..."
summary_text = summarizer(text)
print(summary_text[0]['summary_text'])

"The global data science platform market is projected
 is projected to reach $140.9 billion by 2025..."

摘要参数

✅ min_new_tokens 与 max_new_tokens：控制摘要长度

summarizer = pipeline(task="summarization", min_new_tokens=10, max_new_tokens=150)

¡Vamos a practicar!

使用 Hugging Face