Validasi

Mengembangkan Sistem AI dengan OpenAI API

Francesca Donadoni

Curriculum Manager, DataCamp

Validasi

Seorang pengembang menguji kode di beberapa layar

Validasi

Potensi kesalahan model:

Salah menafsirkan konteks
Memperkuat bias jika data latih bias
Memberikan informasi usang
Dimanipulasi untuk menghasilkan konten berbahaya/tidak etis
Tanpa sengaja mengungkap info sensitif

Pengujian adversarial

Diagram dengan programmer yang menyuntikkan input adversarial ke data dan model, serta model melakukan inferensi dari data

¹ Diadaptasi dari https://adversarial-robustness-toolbox.readthedocs.io/en/latest/

Pengujian adversarial

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its 
    sentiment is positive, negative, or neutral."},

{"role": "user", 
 "content": "It was great to see some of my favorite stars of 30 years ago 
    including John Ritter, Ben Gazarra and Audrey Hepburn. They looked quite wonderful. 
    But that was it. They were not given any characters or good lines to work with. 
    I neither understood or cared what the characters were doing."}])

¹ https://huggingface.co/datasets/davanstrien/test1?row=10

Pengujian adversarial

print(response.choices[0].message.content)

Sentimen ulasan film ini negatif.

Pengujian adversarial

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its sentiment 
    is positive, negative, or neutral."},

{"role": "user", 
 "content": "If you read the book, your all set. If you didn't...your still all set."}])

print(response.choices[0].message.content)

Sentimen ulasan film ini netral.

Pustaka evaluasi dan dataset

Diagram yang menunjukkan contoh pustaka evaluasi yang memakai berbagai dataset untuk menguji model

¹ https://github.com/openai/evals

Ayo berlatih!

Mengembangkan Sistem AI dengan OpenAI API