Zapytanie do modelu typu Chat Completion

Databricks z Python SDK

Avi Steinberg

Senior Software Engineer

Zapytanie do punktu końcowego

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import ChatMessage, ChatMessageRole
w = WorkspaceClient()
response = w.serving_endpoints.query(

    name="databricks-meta-llama-3-3-70b-instruct",

    messages=[
        ChatMessage(
            role=ChatMessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(role=ChatMessageRole.USER, content="<your-question>"),
    ],

    max_tokens=128)
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://databricks-sdk-py.readthedocs.io/en/stable/workspace/serving/serving_endpoints.html

ChatMessage

Parametry klasy ChatMessage():

ChatMessage:
  content: str
  role: ChatMessageRole

Przykładowa wiadomość czatu:

ChatMessage(
  role=ChatMessageRole.USER, 
  content="Can you summarize what happened in world war 1?"
)

ChatMessageRole

Wysyłanie ChatMessage z rolą SYSTEM:

ChatMessage(
  role=ChatMessageRole.SYSTEM,
  content="You are a helpful assistant."
)

Wysyłanie ChatMessage z rolą USER:

ChatMessage(
  role=ChatMessageRole.USER, 
  content="How do you use for loops in Python?"
)

Zapytanie do dużego modelu językowego (LLM) typu chat completion

w = WorkspaceClient()
response = w.serving_endpoints.query( # Zapytanie do modelu AI Meta Llama 3
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( # Wiadomość z rolą "SYSTEM" — wpływa na sposób odpowiadania agenta
            role=ChatMessageRole.SYSTEM, content=f"You are a helpful assistant. "
        ),

        ChatMessage( # Wiadomość z rolą "USER" — zadaje pytanie modelowi AI
            role=ChatMessageRole.USER, content="Can you explain what a fibonacci sequence is?"
        ),
    ],
  max_tokens=128, # Ograniczenie liczby słów w odpowiedzi modelu
)

# Wyodrębnienie i wyświetlenie odpowiedzi modelu AI
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models

Struktura odpowiedzi

Przetwarzanie odpowiedzi z punktu końcowego:

from databricks.sdk import WorkspaceClient
response = w.serving_endpoints.query(
    name="model-name", 
    messages=[...])

print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
The Fibonacci sequence is a series of 
numbers in which each number is the sum 
of the two preceding numbers, starting 
from 0 and 1. The sequence begins like 
this:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 
55, 89, 144...
To generate the next number in the 
sequence, you simply add the previous 
two numbers. For example:
* 0 + 1 = 1
* 1 + 1 = 2
* 1 + 2 = 3

Przykład: streszczanie tekstu

w = WorkspaceClient()
# Zapytanie do modelu AI Meta Llama 3
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( 
            # Prośba do agenta AI o streszczenie tekstu ze zmiennej some_long_text
            role=ChatMessageRole.USER, content=f"Summarize the text stored in {some_long_text}"
        ),
    ],

  max_tokens=100, # Maksymalna liczba słów w odpowiedzi: 100
)

# Wyświetlenie przetworzonej odpowiedzi modelu AI
print(f"RESPONSE:\n{response.choices[0].message.content}")

Przykład: generowanie treści

w = WorkspaceClient()
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
    ChatMessage(role=ChatMessageRole.SYSTEM, 
        content=f"You are a ghost writer for 
        famous country singers."
      ),

    ChatMessage(role=ChatMessageRole.USER, 
        content="Write the lyrics to a country 
        song that takes place in Mississippi")],
  max_tokens=100) 
print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
"Underneath the magnolia sky, where the
Mississippi River rolls by
I'm sittin' here, thinkin' 'bout you, with the
Delta blues in my soul tonight
The cypress trees are swayin' slow,
and the crickets are singin' our song
But without you, baby, this ol' 
Mississippi night just don't feel like home"

Czas na praktykę!

Databricks z Python SDK