Batching

Mengembangkan Sistem AI dengan OpenAI API

Francesca Donadoni

Curriculum Manager, DataCamp

Apa itu rate limit

Seseorang mengemudi mobil, dihentikan oleh polisi

Penyebab rate limit

Terlalu banyak permintaan

Teks permintaan terlalu panjang

Menghindari rate limit

Retry
- Jeda singkat antar permintaan

Batching
- Memproses banyak pesan dalam satu permintaan

Mengurangi token
- Mengukur dan mengurangi jumlah token

Pengulangan (retry)

from tenacity import (
    retry,
    stop_after_attempt,
    wait_random_exponential
)

@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))

Pengulangan (retry)

@retry(wait=wait_random_exponential(min=1, max=60), stop=stop_after_attempt(6))

def get_response(model, message):
    response = client.chat.completions.create(
      model=model,
      messages=[message],
      response_format={"type": "json_object"}
    )
    return response.choices[0].message.content

Batching

countries = ["United States", "Ireland", "India"]

message=[
    {
    "role": "system",
    "content": """You are given a series of countries and are asked to return the 
    country and capital city. Provide each of the questions with an answer in the 
    response as separate content.""",
    }]


[message.append({"role": "user", "content": i }) for i in countries]

Batching

response = client.chat.completions.create(
      model="gpt-4o-mini",
      messages=message
    )

print(response.choices[0].message.content)

United States: Washington D.C.
Ireland: Dublin
India: New Delhi

Mengurangi token

import tiktoken


encoding = tiktoken.encoding_for_model("gpt-4o-mini")

prompt = "Tokens can be full words, or groups of characters commonly grouped 
          together: tokenization."


num_tokens = len(encoding.encode(prompt))

print("Number of tokens in prompt:", num_tokens)

Number of tokens in prompt: 17

Ayo berlatih!

Mengembangkan Sistem AI dengan OpenAI API

Batching

Apa itu rate limit

Penyebab rate limit

Terlalu banyak permintaan

Teks permintaan terlalu panjang

Menghindari rate limit

Pengulangan (retry)

Pengulangan (retry)

Batching

Batching

Mengurangi token

Ayo berlatih!