Validering

Utveckla AI-system med OpenAI API

Francesca Donadoni

Curriculum Manager, DataCamp

Validering

En utvecklare som testar kod på flera skärmar

Validering

Möjliga modellfel:

Feltolkning av kontext
Förstärkning av snedvridningar i utdata om träningsdata är partisk
Utdata med inaktuell information
Manipulation för att generera skadligt eller oetiskt innehåll
Oavsiktligt röjande av känslig information

Adversarial testing

Ett diagram där en programmerare matar in fientlig indata till data och modell, och modellen drar slutsatser från datan

¹ Anpassad från https://adversarial-robustness-toolbox.readthedocs.io/en/latest/

Adversarial testing

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its 
    sentiment is positive, negative, or neutral."},

{"role": "user", 
 "content": "It was great to see some of my favorite stars of 30 years ago 
    including John Ritter, Ben Gazarra and Audrey Hepburn. They looked quite wonderful. 
    But that was it. They were not given any characters or good lines to work with. 
    I neither understood or cared what the characters were doing."}])

¹ https://huggingface.co/datasets/davanstrien/test1?row=10

Adversarial testing

print(response.choices[0].message.content)

The sentiment of this movie review is negative.

Adversarial testing

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its sentiment 
    is positive, negative, or neutral."},

{"role": "user", 
 "content": "If you read the book, your all set. If you didn't...your still all set."}])

print(response.choices[0].message.content)

The sentiment of this movie review is neutral.

Evalueringsbibliotek och datamängder

Ett diagram som visar ett exempel på ett evalueringsbibliotek som använder olika datamängder för att testa en modell

¹ https://github.com/openai/evals

Nu kör vi en övning!

Utveckla AI-system med OpenAI API