Moderare

Dezvoltarea sistemelor AI cu OpenAI API

Francesca Donadoni

Curriculum Manager, DataCamp

Moderarea în API-ul OpenAI

Moderare: procesul de analizare a intrărilor pentru a detecta conținut care încalcă politici sau reguli predefinite

Moderarea în API-ul OpenAI

Diagramă cu un mesaj de utilizator citit de API-ul de moderare OpenAI, care produce ca răspuns o listă a categoriilor de conținut malițios luate în considerare

Moderarea conținutului

moderation_response = client.moderations.create(input="""
...until someone draws an Exploding Kitten.
When that happens, that person explodes. They are now dead.
This process continues until...
""") 

print(moderation_response.results[0].categories.violence)

True

¹ https://ek.explodingkittens.com/how-to-play/exploding-kittens

Moderarea în context

moderation_response = client.moderations.create(input="""
In the deck of cards are some Exploding Kittens. You play the game by putting the deck face down and taking turns drawing cards until someone draws an Exploding Kitten.
When that happens, that person explodes. They are now dead.
This process continues until there's only 1 player left, who wins the game.
The more cards you draw, the greater your chances of drawing an Exploding Kitten.
""") 

moderation_response.results[0].categories.violence

False

Injectare de prompt

O femeie care folosește un chatbot cu un prompt malițios injectat

Injectare de prompt

Limitarea cantității de text din prompturi
Limitarea numărului de tokeni de ieșire generați
Utilizarea de conținut pre-selectat ca intrare și ieșire validată

Adăugarea de restricții

user_request = """
In the deck of cards are some Exploding Kittens. You play the game by putting the 
deck face down and taking turns drawing cards until  someone draws an Exploding 
Kitten. When that happens, that person explodes. They are now dead.
This process continues until there's only 1 player left, who wins the game.
The more cards you draw, the greater your chances of drawing an Exploding Kitten.
"""

messages = [{"role": "system",
             "content": "Your role is to assess whether the user question is 
              allowed or not. The allowed topics are games of chess only. If 
              the topic is allowed, reply with an answer as normal, otherwise
              say 'Apologies, but the topic is not_allowed.'",},
            {"role": "user", "content": user_request},]

Adăugarea de restricții

response = client.chat.completions.create(
    model="gpt-4o-mini", 
    messages=messages
)

print(response.choices[0].message.content)

Apologies, but the topic is not allowed.

Să exersăm!

Dezvoltarea sistemelor AI cu OpenAI API