验证

使用 OpenAI API 构建 AI 系统

Francesca Donadoni

Curriculum Manager, DataCamp

验证

一位开发者在多屏上测试代码

验证

模型可能出错：

误解上下文
若训练数据有偏见，会放大输出偏见
输出过时信息
被操纵以生成有害或不道德内容
无意泄露敏感信息

对抗性测试

一张示意图：程序员向数据和模型注入对抗性输入，模型基于数据推理

¹ 改编自 https://adversarial-robustness-toolbox.readthedocs.io/en/latest/

对抗性测试

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its 
    sentiment is positive, negative, or neutral."},

{"role": "user", 
 "content": "It was great to see some of my favorite stars of 30 years ago 
    including John Ritter, Ben Gazarra and Audrey Hepburn. They looked quite wonderful. 
    But that was it. They were not given any characters or good lines to work with. 
    I neither understood or cared what the characters were doing."}])

¹ https://huggingface.co/datasets/davanstrien/test1?row=10

对抗性测试

print(response.choices[0].message.content)

这条影评的情感是消极的。

对抗性测试

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[
{"role": "system",
 "content": "You are an AI assistant for the film industry. You should interpret 
    the user prompt, a movie review, and based on that extract whether its sentiment 
    is positive, negative, or neutral."},

{"role": "user", 
 "content": "If you read the book, your all set. If you didn't...your still all set."}])

print(response.choices[0].message.content)

这条影评的情感是中性的。

评估库与数据集

一个示例评估库的示意图，使用多种数据集测试模型

¹ https://github.com/openai/evals

Let's practice!

使用 OpenAI API 构建 AI 系统