Chain-of-thought och self-consistency-prompting

Prompt Engineering med OpenAI API

Fouad Trad

Machine Learning Engineer

Chain-of-thought-prompting

Kräver att LLM:er redovisar resonemangssteg innan de ger svar
Används för komplexa resonemangsuppgifter
Minskar modellfel

Bild som visar hur en chain-of-thought-prompt ber modellen lösa ett problem steg för steg, där utdata innehåller resonemanget för varje steg.

Chain-of-thought-prompting

Standardprompting för en resonemangsuppgift

prompt = """Q: You start with 15 books in your collection. At the bookstore, you 
purchase 8 new books. Then, you lend 3 to your friend and 2 to your cousin. Later, 
you visit another bookstore and buy 5 more books. How many books do you have now? 
A: The answer is"""

print(get_response(prompt))

25 books

Chain-of-thought-prompting

Chain-of-thought-prompting för en resonemangsuppgift

prompt = """Q: You start with 15 books in your collection. At the bookstore, you 
purchase 8 new books. Then, you lend 3 to your friend and 2 to your cousin. Later, 
you visit another bookstore and buy 5 more books. How many books do you have now? 
A: Let's think step by step""" 
print(get_response(prompt))

Step 1: Start with the number of books in your collection: 15 books
Step 2: Purchase 8 new books at the bookstore: 15 + 8 = 23 books
Step 3: Lend 3 books to your friend: 23 - 3 = 20 books
Step 4: Lend 2 books to your cousin: 20 - 2 = 18 books
Step 5: Visit another bookstore and buy 5 more books: 18 + 5 = 23 books
Therefore, you have 23 books now.

Chain-of-thought-prompting med few-shots

example = """
Q: The odd numbers in this group add up to an even number:  9, 10, 13, 4, 2.
A: Adding all the odd numbers (9, 13) gives 22. The answer is True.
"""

question = """
Q: The odd numbers in this group add up to an even number: 15, 13, 82, 7. 
A:
"""

prompt = example + question
print(get_response(prompt))

Adding all the odd numbers (15, 13, 7) gives 35. The answer is False.

Chain-of-thought kontra flerstegs-prompting

Flerstegs-prompts

Innehåller steg direkt i prompten

Diagram som visar att en flerstegs-prompt ger modellen en serie sekventiella steg att utföra.

Chain-of-thought kontra flerstegs-prompting

Flerstegs-prompts

Innehåller steg direkt i prompten

Diagram som visar att en flerstegs-prompt ger modellen en serie sekventiella steg att utföra.

Chain-of-thought-prompts

Ber modellen generera mellanliggande steg

Bild som visar att stegen i chain-of-thought-prompts genereras av modellen under utdataproduktionen.

Begränsning med chain-of-thought

Ett misslyckat resonemangssteg --> misslyckat resultat
Self-consistency-prompts introducerades

Diagram för en chain-of-thought-prompt som belyser att ett steg med felaktigt resonemang leder till ett misslyckat resultat.

Self-consistency-prompting

Genererar flera chain-of-thoughts genom att prompta modellen flera gånger
Majoritetsröstning för att få fram det slutliga svaret

Bild som visar att en self-consistency-prompt bygger på flera chain-of-thoughts, var och en med ett svar, där det slutliga resultatet bestäms genom majoritetsröstning.

Self-consistency-prompting

Kan göras med flera separata prompts eller en prompt som genererar flera svar.

self_consistency_instruction = "Imagine three completely independent experts who 
reason differently are answering this question. The final answer is obtained by 
majority vote. The question is: "


problem_to_solve = "If there are 10 cars in the parking lot and 3 more cars arrive. 
Half the original number of cars leave. Then, half of the current number of cars 
arrive. How many cars are there in the parking?"


prompt = self_consistency_instruction + problem_to_solve

print(get_response(prompt))

Self-consistency-prompt

Expert 1: Let's go step by step [...] Therefore, the total number of cars in the 
parking lot is 8 + 4 = 12.

Expert 2: First, let's calculate [...] Therefore, the total number of cars in the 
parking lot is now 5 + 2 = 7 cars.

Expert 3: Initially, there are 10 cars [...] Thus, the final answer is 8 + 4 = 12 
cars in the parking lot.

Based on the majority vote, the final answer is that there are 12 cars in the 
parking lot.

Nu kör vi en övning!

Prompt Engineering med OpenAI API