Chat dialog

Guidance supports chat-based models like ChatGPT and GPT-4 using role tags. These are then converted to the appropriate format for the model (either a JSON API format or special tokens).

import guidance

gpt35 = guidance.models.OpenAI("gpt-3.5-turbo")

import os

from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# This is the name of the model deployed, such as 'gpt-4' or 'gpt-35-turbo
model = os.getenv("AZUREAI_CHAT_MODEL", "Please set the AzureAI model")
# This is the deployment URL, as provided in the Azure AI playground ('view code'):
azure_endpoint = os.getenv("AZUREAI_CHAT_ENDPOINT", "Please set the endpoint")
# The environment variable should be set to the API key from the Azure AI playground:
api_key=os.getenv("AZUREAI_KEY", "Please set API key")
# Alternatively, we can use Entra authentication
token_provider = get_bearer_token_provider(
    DefaultAzureCredential(),
    "https://cognitiveservices.azure.com/.default"
)

Preliminaries concluded, we can now create our model:

from guidance import models, gen

azureai_model = models.AzureOpenAIChat(
    model=model,
    azure_endpoint=azure_endpoint,
    # For authentication, use either
    azure_ad_token_provider=token_provider,
    # or
    # api_key=api_key
    temperature = 0.7
)

Multi-step chat with hidden blocks

We are now going to set up a multistage chat, where we have the chat bot help the use achieve some goal. The user will only have to specify the goal, and then we will create a chain-of-thought conversation with the bot which will:

1. Ask the bot for a number of suggestions.

2. List the pros and cons of each.

3. Pick the best suggestion.

4. Product a detailed action plan.

Our goal is to only show the final result to the user.

Now, let us define our generation function:

import re
from guidance import gen, select, system, user, assistant

@guidance
def plan_for_goal(lm, goal: str):

    # This is a helper function which we will use below
    def parse_best(prosandcons, options):
        best = re.search(r'Best=(\d+)', prosandcons)
        if not best:
            best =  re.search(r'Best.*?(\d+)', 'Best= option is 3')
        if best:
            best = int(best.group(1))
        else:
            best = 0
        return options[best]

    # Some general instruction to the model
    with system():
        lm += "You are a helpful assistant."

    # Simulate a simple request from the user
    # Note that we switch to using 'lm2' here, because these are intermediate steps (so we don't want to overwrite the current lm object)
    with user():
        lm2 = lm + f"""\
        I want to {goal}
        Can you please generate one option for how to accomplish this?
        Please make the option very short, at most one line."""

    # Generate several options. Note that this means several sequential generation requests
    n_options = 5
    with assistant():
        options = []
        for i in range(n_options):
            options.append((lm2 + gen(name='option', temperature=1.0, max_tokens=50))["option"])

    # Have the user request pros and cons
    with user():
        lm2 += f"""\
        I want to {goal}
        Can you please comment on the pros and cons of each of the following options, and then pick the best option?
        ---
        """
        for i, opt in enumerate(options):
            lm2 += f"Option {i}: {opt}\n"
        lm2 += f"""\
        ---
        Please discuss each option very briefly (one line for pros, one for cons), and end by saying Best=X, where X is the number of the best option."""

    # Get the pros and cons from the model
    with assistant():
        lm2 += gen(name='prosandcons', temperature=0.0, max_tokens=600, stop="Best=") + "Best=" + gen("best", regex="[0-9]+")

    # The user now extracts the one selected as the best, and asks for a full plan
    # We switch back to 'lm' because this is the final result we want
    with user():
        lm += f"""\
        I want to {goal}
        Here is my plan: {options[int(lm2["best"])]}
        Please elaborate on this plan, and tell me how to best accomplish it."""

    # The plan is generated
    with assistant():
        lm += gen(name='plan', max_tokens=500)

    return lm

Create a plan for the user. Note how the portions which were sent to lm2 in the function above are not shown in the final result:

results = gpt35 + plan_for_goal(goal="read more books")

system
You are a helpful assistant.
user
I want to read more books
Here is my plan: Create a designated reading time each day.
Please elaborate on this plan, and tell me how to best accomplish it.
assistant
Creating a designated reading time each day is an excellent plan to help you read more books. Here are some steps to help you accomplish it effectively:

1. Determine the best time: Consider your daily routine and identify a time slot that works best for you. It could be in the morning, during lunch breaks, or before going to bed. Choose a time when you are least likely to be interrupted or distracted.

2. Set a specific duration: Decide how long you want your designated reading time to be. It could be 30 minutes, an hour, or even longer, depending on your schedule and reading goals. Setting a specific duration will help you stay committed and make it easier to incorporate into your routine.

3. Create a reading-friendly environment: Find a quiet and comfortable spot where you can focus on reading without distractions. Ensure there is adequate lighting and a cozy seating arrangement. You may also want to have a cup of tea or coffee nearby to enhance the reading experience.

4. Eliminate distractions: Minimize potential distractions during your designated reading time. Put your phone on silent mode or keep it in another room. If you're reading on a digital device, consider using an app or feature that blocks notifications temporarily.

5. Choose books that interest you: Select books that align with your interests and preferences. Reading something you genuinely enjoy will make it easier to stay engaged and motivated. Consider exploring different genres, authors, or topics to keep your reading experience diverse and exciting.

6. Set reading goals: Establish reading goals to keep yourself motivated and track your progress. It could be a specific number of books to read in a month or a year, or even a goal to finish a particular series. Having goals will give you a sense of accomplishment and encourage you to continue reading regularly.

7. Make it a habit: Consistency is key when establishing a reading routine. Commit to your designated reading time every day, even if it's just for a few minutes. Over time, it will become a habit, and you'll find it easier to prioritize reading in your daily life.

8. Join a book club or reading community: Consider joining a book club or an online reading community to connect with fellow book lovers. Engaging in discussions, sharing recommendations, and participating in reading challenges can enhance your reading experience and provide additional motivation.

Remember, the goal is to enjoy the process of reading and make it a regular part of your life. By following these steps and staying committed to your designated reading time

We can access the final plan itself:

print(results['plan'])

Setting aside dedicated time each day for reading is a great plan to read more books. Here are some ways you can best accomplish this:

1. Schedule your reading time: Choose a specific time each day that works best for you. It could be in the morning, during lunch break, or before going to bed. Consistency is key, so try to stick to the same schedule every day.

2. Minimize distractions: Create a quiet and comfortable reading environment. Find a cozy spot where you can focus without interruptions. Put away your phone or any other potential distractions that may tempt you away from your reading time.

3. Start with smaller goals: If you're just beginning to develop a reading habit, start with a smaller goal, such as reading for 15 minutes a day. As you become more comfortable and find pleasure in reading, gradually increase the reading time.

4. Choose books you enjoy: Reading becomes more enjoyable when you're interested in the subject matter. Select books that align with your interests, whether it's fiction, non-fiction, self-help, or any other genre that you find appealing. Don't be afraid to explore different genres to keep things interesting.

5. Set a reading target: Determine how many books you want to read in a month or a year. This target can help motivate you to stay on track with your reading habit. Break it down into smaller milestones to make it more achievable.

6. Keep a reading list: Create a list of books you want to read. This will help you stay organized and focused on your reading goals. You can also use apps or websites that allow you to track your reading progress and discover new books.

7. Join a book club or reading community: Engaging with others who share your passion for reading can provide motivation and accountability. Consider joining a book club or an online reading community where you can discuss books, share recommendations, and participate in reading challenges.

Remember, the key to reading more books is to make it a priority and integrate it into your daily routine. With consistency and dedication, you'll be able to enjoy the benefits of reading and expand your knowledge and imagination.

Asking help from experts

Now, let us ask our chat model to pick some experts in a particular field, and impersonate them to give advice:

@guidance
def run_expert_advice(lm, query: str):
    # Some general instruction to the model
    with system():
        lm += "You are a helpful assistant."

    with user():
        lm += f"""I want a response to the following question:
{query}
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
"""

    with assistant():
        lm += gen(name='experts', temperature=0, max_tokens=300)

    with user():
        lm += """Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
"""

    with assistant():
        lm += gen(name='answer', temperature=0, max_tokens=500)

    return lm

mean_life = gpt35 + run_expert_advice("What is the meaning of life?")

system
You are a helpful assistant.
user
I want a response to the following question:
What is the meaning of life?
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
assistant
Sure, I can provide you with three world-class experts who would be great at answering the question about the meaning of life. Here are three individuals who have made significant contributions in the fields of philosophy, spirituality, and science:

1. Socrates: Socrates was an ancient Greek philosopher known for his Socratic method of questioning. He explored various philosophical concepts, including the nature of life and the pursuit of wisdom. His deep thinking and intellectual approach would make him a great candidate to provide insights on the meaning of life.

2. Dalai Lama: The 14th Dalai Lama, Tenzin Gyatso, is the spiritual leader of Tibetan Buddhism. He is renowned for his teachings on compassion, happiness, and the nature of existence. His profound understanding of spirituality and his ability to bridge Eastern and Western philosophies would make him an excellent choice to offer insights on the meaning of life.

3. Albert Einstein: Einstein was a renowned physicist who revolutionized our understanding of the universe with his theory of relativity. While not a philosopher or spiritual leader, his scientific perspective and deep curiosity about the nature of reality could provide a unique and thought-provoking viewpoint on the meaning of life.

These three experts come from different backgrounds and disciplines, offering diverse perspectives that could contribute to a comprehensive understanding of the question at hand.
user
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
assistant
ANSWER:

The meaning of life is a profound and complex question that has captivated the minds of philosophers, spiritual leaders, and scientists throughout history. While there may be differing perspectives on this matter, a joint anonymous answer from a panel of world-class experts could offer some insights.

Life's meaning, according to these experts, can be understood through various lenses. Some might argue that the meaning of life lies in the pursuit of knowledge and wisdom. Socrates, with his emphasis on the examined life and the pursuit of truth, would support this viewpoint. For him, a meaningful life involves constant questioning, self-reflection, and the acquisition of knowledge.

On the other hand, others might argue that the meaning of life is rooted in compassion and the cultivation of inner peace. The Dalai Lama, drawing from the teachings of Tibetan Buddhism, would advocate for this perspective. According to him, a meaningful life is one that embraces love, kindness, and the alleviation of suffering for oneself and others.

Additionally, there are those who might argue that the meaning of life can be found in understanding the natural world and unraveling its mysteries. Albert Einstein, with his scientific approach, would align with this viewpoint. For him, a meaningful life involves exploring the laws of the universe, unraveling its complexities, and expanding our knowledge of the cosmos.

While these perspectives may seem divergent, they are not necessarily mutually exclusive. It is possible to find harmony and integration among them. The pursuit of knowledge, compassion, and scientific exploration can coexist and complement each other, offering a more comprehensive understanding of the meaning of life.

Ultimately, the meaning of life is a deeply personal and subjective matter. It is up to each individual to explore, reflect, and find their own sense of purpose and fulfillment. The insights provided by these world-class experts serve as valuable guideposts, encouraging us to engage in self-discovery, intellectual growth, compassion, and the exploration of the natural world.

more_productive = gpt35 + run_expert_advice('How can I be more productive?')

system
You are a helpful assistant.
user
I want a response to the following question:
How can I be more productive?
Who are 3 world-class experts (past or present) who would be great at answering this?
Please don't answer the question or comment on it yet.
assistant
Sure, I can provide you with three world-class experts who would be great at answering your question on how to be more productive. However, I won't comment on or answer the question directly. Here are three experts you could consider:

1. David Allen: He is the author of the bestselling book "Getting Things Done" and is widely recognized as a productivity guru. Allen's methodology focuses on organizing tasks and reducing mental clutter to increase productivity.

2. Cal Newport: Newport is a professor at Georgetown University and a bestselling author known for his work on productivity and work-life balance. His book "Deep Work" explores ways to eliminate distractions and achieve more focused, meaningful work.

3. Gretchen Rubin: Rubin is an author and happiness expert who has written extensively about habits and personal development. Her book "Better Than Before" delves into strategies for building habits that can enhance productivity and overall well-being.

These experts offer unique perspectives and strategies for improving productivity, and their insights might prove valuable to your quest for increased efficiency.
user
Great, now please answer the question as if these experts had collaborated in writing a joint anonymous answer.
In other words, their identity is not revealed, nor is the fact that there is a panel of experts answering the question.
If the experts would disagree, just present their different positions as alternatives in the answer itself (e.g. 'some might argue... others might argue...').
Please start your answer with ANSWER:
assistant
ANSWER:
To become more productive, consider incorporating the following strategies:

1. Prioritize and organize: One approach is to adopt David Allen's methodology from "Getting Things Done." This involves capturing all your tasks and ideas in an external system, breaking them down into actionable items, and organizing them based on priority. By having a clear overview of your commitments, you can focus on completing tasks efficiently and avoid mental clutter.

2. Deep work and elimination of distractions: Another perspective, inspired by Cal Newport's book "Deep Work," suggests creating dedicated periods of focused, uninterrupted work. During these sessions, eliminate distractions such as social media or email notifications. By immersing yourself in deep, undistracted work, you can accomplish more in less time and achieve higher-quality results.

3. Establish effective habits: Gretchen Rubin's insights from "Better Than Before" highlight the importance of building habits that support productivity. Some might argue that implementing daily routines or rituals, such as setting specific times for important tasks or incorporating regular breaks, can enhance productivity. Others might argue for the power of habit stacking, where you pair a new habit with an existing one to increase the likelihood of follow-through.

Remember, productivity strategies can vary depending on individual preferences and work styles. Experiment with different approaches and adapt them to fit your unique circumstances. Ultimately, finding the right mix of methods will help you optimize your productivity and accomplish your goals more efficiently.

Agents

We are now going to define a ‘conversation agent.’ This maintains a memory of a conversation, and can generate an appropriate reply, based on the persona it has been given.

class ConversationAgent:
    def __init__(self, chat_model, name: str, instructions: str, context_turns: int = 2):
        self._chat_model = chat_model
        self._name = name
        self._instructions = instructions
        self._my_turns = []
        self._interlocutor_turns = []
        self._went_first = False
        self._context_turns = context_turns

    @property
    def name(self) -> str:
        return self._name

    def reply(self, interlocutor_reply: str | None = None) -> str:
        if interlocutor_reply is None:
            self._my_turns = []
            self._interlocutor_turns = []
            self._went_first = True
        else:
            self._interlocutor_turns.append(interlocutor_reply)

        # Get trimmed history
        my_hist = self._my_turns[(1-self._context_turns):]
        interlocutor_hist = self._interlocutor_turns[-self._context_turns:]

        # Set up the system prompt
        curr_model = self._chat_model
        with system():
            curr_model += f"Your name is {self.name}. {self._instructions}"
            if len(interlocutor_hist) == 0:
                curr_model += "Introduce yourself and start the conversation"
            elif len(interlocutor_hist) == 1:
                curr_model += "Introduce yourself before continuing the conversation"

        # Replay the last few turns
        for i in range(len(my_hist)):
            with user():
                curr_model += interlocutor_hist[i]
            with assistant():
                curr_model += my_hist[i]

        if len(interlocutor_hist) > 0:
            with user():
                curr_model += interlocutor_hist[-1]

        with assistant():
            curr_model += gen(name='response', max_tokens=100)

        self._my_turns.append(curr_model['response'])
        return curr_model['response']

We can have two of these agents converse with each other with a conversation simulator:

def conversation_simulator(
    bot0: ConversationAgent,
    bot1: ConversationAgent,
    total_turns: int = 5 )-> list[dict[str,str]]:
    conversation_turns = []
    last_reply = None
    for _ in range(total_turns):
        last_reply = bot0.reply(last_reply)
        conversation_turns.append(dict(name=bot0.name, text=last_reply))
        last_reply = bot1.reply(last_reply)
        conversation_turns.append(dict(name=bot1.name, text=last_reply))
    return conversation_turns

Now, let’s try generating a conversation:

bot_instructions = """You are taking part in a discussion about bodyline bowling.
Only generate text as yourself and do not prefix your reply with your name.
Keep your answers to a couple of short sentences."""

bradman_bot = ConversationAgent(gpt35, "Donald Bradman", bot_instructions, context_turns=5)
jardine_bot = ConversationAgent(gpt35, "Douglas Jardine", bot_instructions, context_turns=5)

conversation_turns = conversation_simulator(bradman_bot, jardine_bot, total_turns=3)

system
Your name is Douglas Jardine. You are taking part in a discussion about bodyline bowling.
Only generate text as yourself and do not prefix your reply with your name.
Keep your answers to a couple of short sentences.
user
Hi everyone, I'm Donald Bradman, a former Australian cricketer. I'm excited to discuss bodyline bowling and its impact on the game. Let's get started!
assistant
Hello Donald Bradman, it's a pleasure to have you here. As a former Australian cricketer, your insights on bodyline bowling will be invaluable. Let's dive into the discussion!
user
Thank you for having me! I'm Donald Bradman, and I'm happy to share my thoughts on bodyline bowling. Let's get the discussion going!
assistant
Hello Donald Bradman, it's great to have you here. As one of the greatest cricketers of all time, your perspective on bodyline bowling will be highly valuable. Let's start the discussion!
user
Thank you for the kind words! I'm Donald Bradman, and I'm honored to be part of this discussion on bodyline bowling. Let's begin!
assistant
Hello Donald Bradman, it's a pleasure to have you here. Your expertise and experience in facing bodyline bowling will provide valuable insights to our discussion. Let's get started!

for turn in conversation_turns:
    print(f"{turn['name']}: {turn['text']}\n")

Donald Bradman: Hi everyone, I'm Donald Bradman, a former Australian cricketer. I'm excited to discuss bodyline bowling and its impact on the game. Let's get started!

Douglas Jardine: Hello Donald Bradman, it's a pleasure to have you here. As a former Australian cricketer, your insights on bodyline bowling will be invaluable. Let's dive into the discussion!

Donald Bradman: Thank you for having me! I'm Donald Bradman, and I'm happy to share my thoughts on bodyline bowling. Let's get the discussion going!

Douglas Jardine: Hello Donald Bradman, it's great to have you here. As one of the greatest cricketers of all time, your perspective on bodyline bowling will be highly valuable. Let's start the discussion!

Donald Bradman: Thank you for the kind words! I'm Donald Bradman, and I'm honored to be part of this discussion on bodyline bowling. Let's begin!

Douglas Jardine: Hello Donald Bradman, it's a pleasure to have you here. Your expertise and experience in facing bodyline bowling will provide valuable insights to our discussion. Let's get started!

---

Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!