查询聊天补全模型

使用 Python SDK 的 Databricks

Avi Steinberg

Senior Software Engineer

查询服务端点

from databricks.sdk import WorkspaceClient
from databricks.sdk.service.serving import ChatMessage, ChatMessageRole
w = WorkspaceClient()
response = w.serving_endpoints.query(

    name="databricks-meta-llama-3-3-70b-instruct",

    messages=[
        ChatMessage(
            role=ChatMessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(role=ChatMessageRole.USER, content="<your-question>"),
    ],

    max_tokens=128)
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://databricks-sdk-py.readthedocs.io/en/stable/workspace/serving/serving_endpoints.html

ChatMessage

ChatMessage() 类的参数：

ChatMessage:
  content: str
  role: ChatMessageRole

示例 ChatMessage：

ChatMessage(
  role=ChatMessageRole.USER, 
  content="Can you summarize what happened in world war 1?"
)

ChatMessageRole

使用 SYSTEM 角色发送 ChatMessage：

ChatMessage(
  role=ChatMessageRole.SYSTEM,
  content="You are a helpful assistant."
)

使用 USER 角色发送 ChatMessage：

ChatMessage(
  role=ChatMessageRole.USER, 
  content="How do you use for loops in Python?"
)

查询聊天补全大型语言模型（LLM）

w = WorkspaceClient()
response = w.serving_endpoints.query( # 查询 Meta Llama 3 AI 模型
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( # 使用 "SYSTEM" 角色影响模型对后续查询的回应方式
            role=ChatMessageRole.SYSTEM, content=f"You are a helpful assistant. "
        ),

        ChatMessage( # 使用 "USER" 角色向模型提问
            role=ChatMessageRole.USER, content="Can you explain what a fibonacci sequence is?"
        ),
    ],
  max_tokens=128, # 限制模型回复的最大词数
)

# 解析模型响应并打印内容
print(f"RESPONSE:\n{response.choices[0].message.content}")

¹ https://docs.databricks.com/aws/en/machine-learning/model-serving/score-foundation-models

查询响应结构

解析服务端点查询响应：

from databricks.sdk import WorkspaceClient
response = w.serving_endpoints.query(
    name="model-name", 
    messages=[...])

print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
斐波那契数列是一串数字，
每一项等于前两项之和，
从 0 和 1 开始。序列开头为：
0, 1, 1, 2, 3, 5, 8, 13, 21, 34,
55, 89, 144...
要生成下一项，只需将前两项相加。例如：
* 0 + 1 = 1
* 1 + 1 = 2
* 1 + 2 = 3

示例：总结文本

w = WorkspaceClient()
# 查询 Meta Llama 3 AI 模型
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
        ChatMessage( 
            # 要求智能体总结变量 some_long_text 中的文本
            role=ChatMessageRole.USER, content=f"Summarize the text stored in {some_long_text}"
        ),
    ],

  max_tokens=100, # 将回复的最大词数设为 100
)

# 打印解析后的模型响应
print(f"RESPONSE:\n{response.choices[0].message.content}")

示例：生成内容

w = WorkspaceClient()
response = w.serving_endpoints.query(
  name="databricks-meta-llama-3-3-70b-instruct",

  messages=[
    ChatMessage(role=ChatMessageRole.SYSTEM, 
        content=f"You are a ghost writer for 
        famous country singers."
      ),

    ChatMessage(role=ChatMessageRole.USER, 
        content="Write the lyrics to a country 
        song that takes place in Mississippi")],
  max_tokens=100) 
print(
f"RESPONSE:\n{response.choices[0].message.content}"
)

RESPONSE:
"在玉兰花的天空下，
密西西比河缓缓流过
我独自坐着，想着你，
今夜三角洲蓝调在我心中回荡
柏树慢慢摇曳，
蟋蟀在唱我们的歌
没有你，宝贝，
这密西西比的夜晚不再像家"

让我们练习！

使用 Python SDK 的 Databricks