写在前面：硬代码和LLM的取舍

LLM不是万能的。LLM这种基于概率的模型必然会有出错的可能，哪怕概率很小，哪怕使用了多复杂的工作流来抑制。需要完全准确的地方最好直接使用硬代码写死。LLM-based agent一般只用于和“复杂文本理解”沾边的地方，用于简化流程。
举例：

  1. 识别用户对话的特定格式，这种可以用LLM去做也可以用硬代码；
  2. 识别一个RSS并提取其内容，虽然现在有很多RSS阅读器（比如最近非常火的Follow），但是图快的话确实也可以用LLM-based agent解决，省去了很多和RSS打交道的麻烦。

简单的单次对话函数ask

本文从这里开始和知乎用户Tallisgo的Github仓库基本一致，感谢他的总结和分享

API建议使用兼容OpenAI格式的（DeepSeek很便宜，以下都用Deepseek的），没有API的话，用One-API搭一个API+本地部署一个LLM应该也行

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
from openai import OpenAI
from typing import List, Dict

YOUR_DEEPSEEK_KEY=""
DEEPSEEK_API_URL="https://api.deepseek.com"

def ask(messages: List[Dict]):
    client = OpenAI(api_key=YOUR_DEEPSEEK_KEY, base_url=DEEPSEEK_API_URL)
    response = client.chat.completions.create(
            model = 'deepseek-chat',
            temperature = 0,
            messages = messages
    )

    return response

其中参数Temperature是设置AI回答的灵活性和创造性，其取值为[0,1]，越高创造性越高，越低越严谨。

一次简单的对话

1
2
3
4
5
6
7
8
9
10
11
12
13
14
system_prompt = "You are a helpful assistant" 

question = """
Q: The cafeteria had 23 apples.
If they used 20 to make lunch and bought 6 more, how many apples do they have?
A:"""

messages = [
    {"role":"system", "content":system_prompt},
    {"role":"user", "content":question},
]

response = ask(messages)
print(response.choices[0].message.content)

结果如下：

1
2
3
4
5
6
7
8
To find out how many apples the cafeteria has now, we need to follow these steps:

1. Start with the initial number of apples: 23 apples.
2. Subtract the number of apples used for lunch: 23 - 20 = 3 apples remaining.
3. Add the number of apples bought: 3 + 6 = 9 apples.

So, the cafeteria now has 9 apples.
print(response.choices[0].message.content)

提供 Few-Shots 样本

1
2
3
4
5
6
question = """Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 tennis balls. How many tennis balls does he have now?
A: The answer is 11.
Q: The cafeteria had 23 apples.
If they used 20 to make lunch and bought 6 more, how many apples do they have?
A:"""

使用magic word（或称引导提示词）

比如下面的question就用了引导提示词Let's think step by step:

1
2
3
4
5
6
7
question = """Q: Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
Each can has 3 tennis balls. How many tennis balls does he have now?
A: The answer is 11.

Q: The cafeteria had 23 apples.
If they used 20 to make lunch and bought 6 more, how many apples do they have?
A: Let's think step by step."""

引导提示词也可以是特定的格式，比如给LLM一个只有键没有值的Json模板让他去做填空。接下来会经常见到这种写一半等填空的引导提示词。

多段 Prompt 修改

本段是原作者模仿Andrew Ng的translation-agent总结的多段 prompt递进修改，不全。
下一篇文章会详细描述吴恩达老师的这个reflection workflow.

任务拆分，第三方工具

需求：DuckDuckGo API分析问题“Compare the population gap between Toronto and New York city”
思路：任务拆分，单独解决，总结

任务拆分

1
2
3
4
5
6
plan_system_prompt = """
    Let's first understand the problem and devise a plan to solve the problem.
    Please output the plan starting with the header 'Plan:' and then followed by a numbered list of steps. 
    Please make the plan the minimum number of steps required to accurately complete the task. If the task is a question, 
    the final step should almost always be 'Given the above steps taken, please respond to the users original question'. 
"""

调用第三方工具：DuckDuckGo Search API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
from duckduckgo_search import DDGS

def internet_search(query: str):
    with DDGS() as ddgs:
        ddgs_gen = ddgs.text(
            query,
            max_results=5, 
            region="wt-wt", 
            safesearch="moderate", 
            timelimit="y",
            backend="api",
        )
        if ddgs_gen:
            return [r for r in ddgs_gen]
    return "No results found."

search_res = internet_search('the current population of Toronto')

这部分显然还没写完，dify跑通再说吧