Prompting chain-of-thought e self-consistency

Prompt Engineering con l'API di OpenAI

Fouad Trad

Machine Learning Engineer

Chain-of-thought prompting

Richiede agli LLM di fornire i passi di ragionamento (pensieri) prima della risposta
Usato per compiti di ragionamento complessi
Aiuta a ridurre gli errori del modello

Immagine che mostra come un prompt chain-of-thought chieda al modello di risolvere un problema passo passo e l'output contenga il ragionamento per ogni passo.

Chain-of-thought prompting

Prompting standard per risolvere un compito di ragionamento

prompt = """Q: You start with 15 books in your collection. At the bookstore, you 
purchase 8 new books. Then, you lend 3 to your friend and 2 to your cousin. Later, 
you visit another bookstore and buy 5 more books. How many books do you have now? 
A: The answer is"""

print(get_response(prompt))

25 books

Chain-of-thought prompting

Chain-of-thought prompting per risolvere un compito di ragionamento

prompt = """Q: You start with 15 books in your collection. At the bookstore, you 
purchase 8 new books. Then, you lend 3 to your friend and 2 to your cousin. Later, 
you visit another bookstore and buy 5 more books. How many books do you have now? 
A: Let's think step by step""" 
print(get_response(prompt))

Step 1: Start with the number of books in your collection: 15 books
Step 2: Purchase 8 new books at the bookstore: 15 + 8 = 23 books
Step 3: Lend 3 books to your friend: 23 - 3 = 20 books
Step 4: Lend 2 books to your cousin: 20 - 2 = 18 books
Step 5: Visit another bookstore and buy 5 more books: 18 + 5 = 23 books
Therefore, you have 23 books now.

Chain-of-thought prompting con few-shot

example = """
Q: The odd numbers in this group add up to an even number:  9, 10, 13, 4, 2.
A: Adding all the odd numbers (9, 13) gives 22. The answer is True.
"""

question = """
Q: The odd numbers in this group add up to an even number: 15, 13, 82, 7. 
A:
"""

prompt = example + question
print(get_response(prompt))

Adding all the odd numbers (15, 13, 7) gives 35. The answer is False.

Chain-of-thought vs. multi-step prompting

Prompt multi-step

Includi passi nel prompt

Diagramma che mostra che un prompt multi-step fornisce al modello una serie di passi sequenziali da eseguire.

Chain-of-thought vs. multi-step prompting

Prompt multi-step

Includi passi nel prompt

Diagramma che mostra che un prompt multi-step fornisce al modello una serie di passi sequenziali da eseguire.

Prompt chain-of-thought

Chiedi al modello di generare i passi intermedi

Immagine che mostra che nei prompt chain-of-thought i passi sono prodotti dal modello durante la generazione dell'output.

Limite del chain-of-thought

Un pensiero errato --> risultato errato
Sono stati introdotti i prompt self-consistency

Diagramma per un prompt chain-of-thought che evidenzia che un singolo pensiero con ragionamento difettoso porta a un risultato errato.

Self-consistency prompting

Genera più chain-of-thought richiamando il modello più volte
Voto di maggioranza per l'output finale

Immagine che mostra che un prompt self-consistency genera più chain-of-thought, ognuno con un output, e il risultato finale si ottiene per voto di maggioranza.

Self-consistency prompting

Si può fare definendo più prompt o un prompt che genera più risposte.

self_consistency_instruction = "Imagine three completely independent experts who 
reason differently are answering this question. The final answer is obtained by 
majority vote. The question is: "


problem_to_solve = "If there are 10 cars in the parking lot and 3 more cars arrive. 
Half the original number of cars leave. Then, half of the current number of cars 
arrive. How many cars are there in the parking?"


prompt = self_consistency_instruction + problem_to_solve

print(get_response(prompt))

Self-consistency prompt

Expert 1: Let's go step by step [...] Therefore, the total number of cars in the 
parking lot is 8 + 4 = 12.

Expert 2: First, let's calculate [...] Therefore, the total number of cars in the 
parking lot is now 5 + 2 = 7 cars.

Expert 3: Initially, there are 10 cars [...] Thus, the final answer is 8 + 4 = 12 
cars in the parking lot.

Based on the majority vote, the final answer is that there are 12 cars in the 
parking lot.

Ayo berlatih!

Prompt Engineering con l'API di OpenAI