The Art of Prompt Design
ReAct Design Pattern
Authors: @Sam1320, @slundberg
The ReAct prompting strategy (by Yao et al.) consists of prompting a language model to generate explicit reasoning traces that contain an “action” step. The action step selects actions to execute in order to gain more information. ReAct can be viewed as a specific form of chain of thought combined with tool use. To execute the ReAct pattern we just need to follow the standard tool use paradigm of executing the selected actions and injecting the result (aka. Observation) back into the prompt and repeating the process until a final answer is found. This
This notebook shows two different ways to leverage guidance
to implement this pattern:
Using the
tools
API of thegen
function.Using a ReAct-specific stateful guidance function.
Setup
First let’s import the necessary modules and load the LLM:
[1]:
import os
import math
import guidance
from guidance import models, gen, select
path_to_model = os.getenv('home/scottlundberg_google_com/models/mistral-7b-v0.1.Q8_0.gguf')
mistral7 = models.LlamaCpp(path_to_model, n_gpu_layers=-1)
We can use the same prompt for both approaches so let’s define it here. We will use two simple functions sqrt
and age
(returns age of a famous person) as tools to demonstrate the pattern.
[24]:
prompt = """Answer the following questions as best you can. You have access only to the following tools:
{tools}
Use the following format:
Question: the input question you must answer
Thought 1: you should always think about what to do
Action 1: the action to take, has to be one of {tool_names}
Observation 1: the result of the action
... (this Thought/Action/Action Input/Observation can repeat N times)
Thought N: I now know the final answer.
Final Answer: the final answer to the original input question.
Done.
Example:
Question: What is the square root of the age of Brad Pitt?
Thought 1: I should find out how old Brad Pitt is.
Action 1: age(Brad Pitt)
Observation 1: 56
Thought 2: I should find the square root of 56.
Action 2: sqrt(56)
Observation 2: 7.48
Thought 3: I now know the final answer.
Final Answer: 7.48
Done.
Question: {query}
"""
Implementing ReAct using the tools
API of the gen
function
We can define tools that can be used and then pass them as arguments to gen
. Guidance will identify when the model generates something that matches the grammar of a tool call, execute it and then resume generation.
Let’s first define our set of functions that can be used as tools. The output of these functions is inserted back into the program right after the call to the function. For this reason, in order to match the pattern of the prompt above, we’ll add “Observation N” before the output of the function.
[19]:
from guidance import regex
ages_db = {
"Leonardo DiCaprio": 49,
"Brad Pitt": 59
}
@guidance
def sqrt(lm, number):
lm += f'\nObservation {regex(r"[0-9]+")}: ' + f'{math.sqrt(float(number))}\n'
return lm
@guidance
def log(lm, number):
lm += f'\nObservation {regex(r"[0-9]+")}: {math.log(float(number)):.4f}\n'
return lm
@guidance
def age(lm, person):
lm += f'\nObservation {regex(r"[0-9]+")}: {ages_db.get(person)}\n'
return lm
tools = {
"sqrt": "Computes the square root of a number.",
"age": "Returns the age of a person.",
"log": "Computes the logarithm of a number."
}
tool_map = {
"sqrt": sqrt,
"age": age,
"log": log
}
Run the Program
Now we can start generation just by adding the model and the prompt to gen
and passing the tools as arguments.
[25]:
query = "What is the logarithm of Leonardo DiCaprio's age?"
prompt_with_query = prompt.format(tools=tools, tool_names=list(tools.keys()), query=query)
lm = mistral7 + prompt_with_query + gen(max_tokens=200, tools=[sqrt, age, log], stop="Done.")
Answer the following questions as best you can. You have access only to the following tools: {'sqrt': 'Computes the square root of a number.', 'age': 'Returns the age of a person.', 'log': 'Computes the logarithm of a number.'} Use the following format: Question: the input question you must answer Thought 1: you should always think about what to do Action 1: the action to take, has to be one of ['sqrt', 'age', 'log'] Observation 1: the result of the action ... (this Thought/Action/Action Input/Observation can repeat N times) Thought N: I now know the final answer. Final Answer: the final answer to the original input question. Done. Example: Question: What is the square root of the age of Brad Pitt? Thought 1: I should find out how old Brad Pitt is. Action 1: age(Brad Pitt) Observation 1: 56 Thought 2: I should find the square root of 56. Action 2: sqrt(56) Observation 2: 7.48 Thought 3: I now know the final answer. Final Answer: 7.48 Done. Question: What is the logarithm of Leonardo DiCaprio's age? Thought 1: I should find out how old Leonardo DiCaprio is. Action 1: age(Leonardo DiCaprio) Observation 1: 49 Thought 2: I should find the logarithm of 49. Action 2: log(49) Observation 2: 3.8918 Thought 3: I now know the final answer. Final Answer: 3.8918
Implementing ReAct Directly Using Stateful Control
Instead of passing the tools as arguments to gen
we can use stateful control to guide the execution of the program. This allows for more fine grained control.
In this case we’ll define a function that runs the ReAct loop. Note that now select
constrains the model to select one of the available tools. Also, we don’t need to add prefixes to the tools since all the context is finely controlled inside the loop.
[26]:
tool_map = {
"sqrt": lambda x: str(math.sqrt(float(x))),
"age": lambda x: str(ages_db.get(x)),
"log": lambda x: str(math.log(float(x)))
}
@guidance
def react_prompt_example(lm, question, tools, max_rounds=10):
tool_names = list(tools.keys())
lm += prompt.format(tools=tools, tool_names=tool_names, query=question)
i = 1
while True:
lm += f'Thought {i}: ' + gen(name='thought', suffix='\n')
if 'final answer' in lm['thought'] or i == max_rounds:
lm += 'Final Answer: ' + gen(name='answer', suffix='\n')
break
lm += f'Act {i}: ' + select(tool_names, name='act')
lm += '(' + gen(name='arg', suffix=')') + '\n'
if lm['act'] in tool_map:
lm += f'Observation {i}: ' + tool_map[lm['act']](lm['arg']) + '\n'
i += 1
return lm
[27]:
lm = mistral7
lm += react_prompt_example("What is the logarithm of Leonardo DiCaprio's age?", tools)
Answer the following questions as best you can. You have access only to the following tools: {'sqrt': 'Computes the square root of a number.', 'age': 'Returns the age of a person.', 'log': 'Computes the logarithm of a number.'} Use the following format: Question: the input question you must answer Thought 1: you should always think about what to do Action 1: the action to take, has to be one of ['sqrt', 'age', 'log'] Observation 1: the result of the action ... (this Thought/Action/Action Input/Observation can repeat N times) Thought N: I now know the final answer. Final Answer: the final answer to the original input question. Done. Example: Question: What is the square root of the age of Brad Pitt? Thought 1: I should find out how old Brad Pitt is. Action 1: age(Brad Pitt) Observation 1: 56 Thought 2: I should find the square root of 56. Action 2: sqrt(56) Observation 2: 7.48 Thought 3: I now know the final answer. Final Answer: 7.48 Done. Question: What is the logarithm of Leonardo DiCaprio's age? Thought 1: I should find out how old Leonardo DiCaprio is. Act 1: age(Leonardo DiCaprio) Observation 1: 49 Thought 2: I should find the logarithm of 49. Act 2: log(49) Observation 2: 3.8918202981106265 Thought 3: I now know the final answer. Final Answer: 3.8918202981106265
We can access the final answer which we stored in the program state.
[28]:
print(query)
print(f"Response: {lm['answer']}")
What is the logarithm of Leonardo DiCaprio's age?
Response: 3.8918202981106265
Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!