Natural Language Processing (NLP) in Python
Fouad Trad
Machine Learning Engineer
from transformers import pipeline
summarizer = pipeline(task="summarization", model="cnicu/t5-small-booksum")
text = """The Amazon rainforest, often referred to as the "lungs of the Earth," is one of the most biologically diverse regions in the world. Spanning over nine countries in South America, the majority of the forest lies in Brazil. It is home to an estimated 390 billion individual trees, divided into 16,000 different species. The rainforest plays a critical role in regulating the global climate by absorbing vast amounts of carbon dioxide and producing oxygen."""
result = summarizer(text)
print(result)
[{'summary_text': 'the Amazon rainforest is one of the most biologically diverse regions in the world.
The majority of the forest lies in Brazil. The rainforest plays a critical role in regulating the
global climate by absorbing vast amounts of carbon dioxide and producing oxygen.'}]
translator = pipeline(task="translation", model="Helsinki-NLP/opus-mt-en-fr")
sentence = "The rainforest helps regulate the Earth's climate."
result = translator(sentence)
print(result)
[{'translation_text': 'La forêt tropicale aide à réguler le climat de la Terre.'}]
generator = pipeline(task="text-generation", model="distilgpt2")
prompt = "Once upon a time,"
result = generator(prompt, max_length=30, num_return_sequences=3)
print(result)
[{'generated_text': "Once upon a time, my life wasn't so good, I kept my things tidy. The
more time I spend with my children the more ..."},
{'generated_text': 'Once upon a time, we began a process of finding the right answers to
some big questions," said Jim Pelterer, a lecturer at the
University...'},
{'generated_text': 'Once upon a time, a man came along and took in the city, and found out
that a strange woman had just walked in and was dancing about...'}]
Natural Language Processing (NLP) in Python