検証

OpenAI API で開発する AI システム

Francesca Donadoni

Curriculum Manager, DataCamp

検証

複数画面でコードをテストする開発者

検証

モデルのエラー要因:

文脈の誤解
学習データの偏りによるバイアス増幅
古い情報の出力
有害・非倫理的内容を生成するよう誘導される
機密情報の不注意な露出

敵対的テスト

プログラマがデータやモデルに敵対的入力を注入し、モデルが推論する図

¹ Adapted from https://adversarial-robustness-toolbox.readthedocs.io/en/latest/

敵対的テスト

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its 
    sentiment is positive, negative, or neutral."},

{"role": "user", 
 "content": "It was great to see some of my favorite stars of 30 years ago 
    including John Ritter, Ben Gazarra and Audrey Hepburn. They looked quite wonderful. 
    But that was it. They were not given any characters or good lines to work with. 
    I neither understood or cared what the characters were doing."}])

¹ https://huggingface.co/datasets/davanstrien/test1?row=10

敵対的テスト

print(response.choices[0].message.content)

この映画レビューの感情はネガティブです。

敵対的テスト

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its sentiment 
    is positive, negative, or neutral."},

{"role": "user", 
 "content": "If you read the book, your all set. If you didn't...your still all set."}])

print(response.choices[0].message.content)

この映画レビューの感情はニュートラルです。

評価ライブラリとデータセット

多様なデータセットでモデルを評価するライブラリの例を示す図

¹ https://github.com/openai/evals

練習してみましょう！

OpenAI API で開発する AI システム