การ Query โมเดล Chat Completion

Databricks กับ Python SDK

Avi Steinberg

Senior Software Engineer

Query Serving Endpoint

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import ChatMessage, ChatMessageRole
w = WorkspaceClient()
response = w.serving_endpoints.query(

    name="databricks-meta-llama-3-3-70b-instruct",

    messages=[
        ChatMessage(
            role=ChatMessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(role=ChatMessageRole.USER, content="<your-question>"),
    ],

    max_tokens=128)
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://databricks-sdk-py.readthedocs.io/en/stable/workspace/serving/serving_endpoints.html

ChatMessage

พารามิเตอร์ของคลาส ChatMessage():

ChatMessage:
  content: str
  role: ChatMessageRole

ตัวอย่าง Chat Message:

ChatMessage(
  role=ChatMessageRole.USER, 
  content="Can you summarize what happened in world war 1?"
)

ChatMessageRole

ส่ง ChatMessage โดยใช้ role SYSTEM:

ChatMessage(
  role=ChatMessageRole.SYSTEM,
  content="You are a helpful assistant."
)

ส่ง ChatMessage โดยใช้ role USER:

ChatMessage(
  role=ChatMessageRole.USER, 
  content="How do you use for loops in Python?"
)

การ Query Large Language Model (LLM) แบบ Chat Completion

w = WorkspaceClient()
response = w.serving_endpoints.query( # Query the Meta Llama 3 AI model
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( # Message with "SYSTEM" role to influence how the agent responds to future queries
            role=ChatMessageRole.SYSTEM, content=f"You are a helpful assistant. "
        ),

        ChatMessage( # Message with the "USER" role to ask the AI model a question
            role=ChatMessageRole.USER, content="Can you explain what a fibonacci sequence is?"
        ),
    ],
  max_tokens=128, # Limit the number of words that the AI model can use in the response
)

# Parse the AI model response and print the content
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models

โครงสร้างผลลัพธ์จากการ Query

การแยกวิเคราะห์ผลลัพธ์จาก Serving Endpoint:

from databricks.sdk import WorkspaceClient
response = w.serving_endpoints.query(
    name="model-name", 
    messages=[...])

print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
The Fibonacci sequence is a series of 
numbers in which each number is the sum 
of the two preceding numbers, starting 
from 0 and 1. The sequence begins like 
this:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 
55, 89, 144...
To generate the next number in the 
sequence, you simply add the previous 
two numbers. For example:
* 0 + 1 = 1
* 1 + 1 = 2
* 1 + 2 = 3

ตัวอย่าง: สรุปข้อความ

w = WorkspaceClient()
# Query the Meta Llama 3 AI model
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( 
            # Ask the AI agent to summarize the text stored in variable some_long_text
            role=ChatMessageRole.USER, content=f"Summarize the text stored in {some_long_text}"
        ),
    ],

  max_tokens=100, # Set the maximum number of words in the response to 100
)

# Print the parsed response from the AI model
print(f"RESPONSE:\n{response.choices[0].message.content}")

ตัวอย่าง: สร้างเนื้อหา

w = WorkspaceClient()
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
    ChatMessage(role=ChatMessageRole.SYSTEM, 
        content=f"You are a ghost writer for 
        famous country singers."
      ),

    ChatMessage(role=ChatMessageRole.USER, 
        content="Write the lyrics to a country 
        song that takes place in Mississippi")],
  max_tokens=100) 
print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
"Underneath the magnolia sky, where the
Mississippi River rolls by
I'm sittin' here, thinkin' 'bout you, with the
Delta blues in my soul tonight
The cypress trees are swayin' slow,
and the crickets are singin' our song
But without you, baby, this ol' 
Mississippi night just don't feel like home"

มาฝึกกันเถอะ!

Databricks กับ Python SDK