代理最终答案验证

使用 Hugging Face smolagents 的 AI Agents

Adel Nehme

VP of AI Curriculum, DataCamp

为何需要验证

代理的回答不够有用
客户体验受损

为避免此问题，smolagents 支持验证最终答案！

验证代理响应

def check_answer_length(final_answer, agent_memory):
    # Check if the answer is substantial enough
    if len(final_answer) < 200:
        raise Exception("Car recommendation is too brief")
    return True

若 final_answer 触发规则，则抛出异常；否则返回 True。

在代理中使用输出验证

car_advisor = CodeAgent(
    tools=[WebSearchTool()],
    model=InferenceClientModel(),
    final_answer_checks=[check_answer_length],
    verbosity_level=0
)

在回复前运行 check_answer_length 验证。
根据函数中定义的异常信息自动重试。

元评估：用 AI 验证 AI

validation_prompt = """
Reasoning process: {}

Agent's final answer: {}

Does the final answer logically follow
from the reasoning and solve the user's 
question? 

Respond only TRUE or FALSE. 
No other text.
"""

用元评估验证推理

def check_reasoning_accuracy(final_answer, agent_memory):
    evaluator_model = InferenceClientModel()
    reasoning_steps = agent_memory.get_succinct_steps()
    final_prompt = validation_prompt.format(reasoning_steps, final_answer)

    message = ChatMessage(role='user', content=final_prompt)
    evaluation = evaluator_model([message])

    if evaluation.content == "FALSE":
        raise Exception("The agent's reasoning process contains logical errors")
    else:
        return True

组合多重验证

car_advisor = CodeAgent(
    tools=[WebSearchTool()],
    model=InferenceClientModel(),
    final_answer_checks=[check_answer_length, check_reasoning_accuracy],
    verbosity_level=0
)

在用户看到前，更可能捕捉并纠正错误！

设计智能系统

Passons à la pratique !

使用 Hugging Face smolagents 的 AI Agents