查詢聊天補全模型

使用 Python SDK 的 Databricks

Avi Steinberg

Senior Software Engineer

查詢服務端點

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import ChatMessage, ChatMessageRole
w = WorkspaceClient()
response = w.serving_endpoints.query(

    name="databricks-meta-llama-3-3-70b-instruct",

    messages=[
        ChatMessage(
            role=ChatMessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(role=ChatMessageRole.USER, content="<your-question>"),
    ],

    max_tokens=128)
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://databricks-sdk-py.readthedocs.io/en/stable/workspace/serving/serving_endpoints.html

ChatMessage

ChatMessage() 類別的參數：

ChatMessage:
  content: str
  role: ChatMessageRole

聊天訊息範例：

ChatMessage(
  role=ChatMessageRole.USER, 
  content="Can you summarize what happened in world war 1?"
)

ChatMessageRole

以 SYSTEM 角色送出 ChatMessage：

ChatMessage(
  role=ChatMessageRole.SYSTEM,
  content="You are a helpful assistant."
)

以 USER 角色送出 ChatMessage：

ChatMessage(
  role=ChatMessageRole.USER, 
  content="How do you use for loops in Python?"
)

查詢聊天補全大型語言模型（LLM）

w = WorkspaceClient()
response = w.serving_endpoints.query( # 查詢 Meta Llama 3 AI 模型
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( # 使用「SYSTEM」角色，影響代理後續回應方式
            role=ChatMessageRole.SYSTEM, content=f"You are a helpful assistant. "
        ),

        ChatMessage( # 使用「USER」角色向 AI 模型發問
            role=ChatMessageRole.USER, content="Can you explain what a fibonacci sequence is?"
        ),
    ],
  max_tokens=128, # 限制 AI 回應可使用的字數
)

# 解析 AI 模型回應並印出內容
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models

查詢回應結構

解析服務端點查詢回應：

from databricks.sdk import WorkspaceClient
response = w.serving_endpoints.query(
    name="model-name", 
    messages=[...])

print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
The Fibonacci sequence is a series of 
numbers in which each number is the sum 
of the two preceding numbers, starting 
from 0 and 1. The sequence begins like 
this:
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 
55, 89, 144...
To generate the next number in the 
sequence, you simply add the previous 
two numbers. For example:
* 0 + 1 = 1
* 1 + 1 = 2
* 1 + 2 = 3

範例：摘要文字

w = WorkspaceClient()
# 查詢 Meta Llama 3 AI 模型
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( 
            # 請 AI 代理摘要變數 some_long_text 中的文字
            role=ChatMessageRole.USER, content=f"Summarize the text stored in {some_long_text}"
        ),
    ],

  max_tokens=100, # 將回應的最大字數設為 100
)

# 印出 AI 模型解析後的回應
print(f"RESPONSE:\n{response.choices[0].message.content}")

範例：產生內容

w = WorkspaceClient()
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
    ChatMessage(role=ChatMessageRole.SYSTEM, 
        content=f"You are a ghost writer for 
        famous country singers."
      ),

    ChatMessage(role=ChatMessageRole.USER, 
        content="Write the lyrics to a country 
        song that takes place in Mississippi")],
  max_tokens=100) 
print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
"Underneath the magnolia sky, where the
Mississippi River rolls by
I'm sittin' here, thinkin' 'bout you, with the
Delta blues in my soul tonight
The cypress trees are swayin' slow,
and the crickets are singin' our song
But without you, baby, this ol' 
Mississippi night just don't feel like home"

一起來練習吧！

使用 Python SDK 的 Databricks