generator = pipeline(task="text-generation", model="distilgpt2")
prompt = "The Gion neighborhood in Kyoto is famous for"
output = generator(prompt, max_length=100, pad_token_id=generator.tokenizer.eos_token_id)
일관성
의미 있는 내용
사람과 유사한 텍스트
eos_token_id: 시퀀스 종료 토큰 ID
텍스트 생성
pad_token_id: max_length까지 빈 공간을 채움
패딩: 토큰 추가
generator.tokenizer.eos_token_id로 설정하면 학습을 통해 의미 있는 텍스트의 끝을 표시
모델은 max_length 또는 pad_token_id까지 생성
truncation = True
텍스트 생성
generator = pipeline(task="text-generation", model="distilgpt2")
prompt = "The Gion neighborhood in Kyoto is famous for"
output = generator(prompt, max_length=100, pad_token_id=generator.tokenizer.eos_token_id)
print(output[0]["generated_text"])
The Gion neighborhood in Kyoto is famous for its many colorful green forests, such as the
Red Hill, the Red River and the Red River. The Gion neighborhood is home to the world's
tallest trees.
프롬프트가 모호하면 출력이 불량할 수 있음
출력 유도
generator = pipeline(task="text-generation", model="distilgpt2")
review = "This book was great. I enjoyed the plot twist in Chapter 10."
response = "Dear reader, thank you for your review."
prompt = f"Book review:\n{review}\n\nBook shop response to the review:\n{response}"
output = generator(prompt, max_length=100, pad_token_id=generator.tokenizer.eos_token_id)
print(output[0]["generated_text"])
Dear reader, thank you for your review. We'd like to thank you for your reading!
언어 번역
Hugging Face에서 번역 태스크 및 모델 전체 목록 제공
translator = pipeline(task="translation_en_to_es", model="Helsinki-NLP/opus-mt-en-es")
text = "Walking amid Gion's Machiya wooden houses was a mesmerizing experience."
output = translator(text, clean_up_tokenization_spaces=True)
print(output[0]["translation_text"])
Caminar entre las casas de madera Machiya de Gion fue una experiencia fascinante.
Preparing Video For Download...
