Chat dialog

Guidance supports chat-based models like ChatGPT and GPT-4 using role tags. These are then converted to the appropriate format for the model (either a JSON API format or special tokens).

[1]:

call_delay_secs = 0

[2]:

import os

# Uncomment if using DefaultAzureCredential below
# from azure.identity import DefaultAzureCredential, get_bearer_token_provider

# This is the name of the model deployed, such as 'gpt-4' or 'gpt-3.5-turbo
model = os.getenv("AZUREAI_CHAT_MODEL", "Please set the model")

# This is the deployment URL, as provided in the Azure AI playground ('view code')
# It will end with 'openai.azure.com'
azure_endpoint = os.getenv("AZUREAI_CHAT_BASE_ENDPOINT", "Please set the endpoint")

# This is the name of the deployment specified in the Azure portal
azure_deployment = os.getenv("AZUREAI_CHAT_DEPLOYMENT", "Please set the deployment name")

# This is the deployed API version, such as 2024-02-15-preview
azure_api_version = os.getenv("AZUREAI_CHAT_API_VERSION", "Please set the API version")

# The environment variable should be set to the API key from the Azure AI playground:
api_key=os.getenv("AZUREAI_CHAT_KEY", "Please set API key")

# Alternatively, we can use Entra authentication
# token_provider = get_bearer_token_provider(
#     DefaultAzureCredential(),
#     "https://cognitiveservices.azure.com/.default"
#)

Preliminaries concluded, we can now create our model:

[3]:

from guidance import models, gen

azureai_model = models.AzureOpenAI(
    model=model,
    azure_endpoint=azure_endpoint,
    azure_deployment=azure_deployment,
    version=azure_api_version,
    # For authentication, use either
    # azure_ad_token_provider=token_provider,
    # or
    api_key=api_key,
)

Multi-step chat with hidden blocks

We are now going to set up a multistage chat, where we have the chat bot help the use achieve some goal. The user will only have to specify the goal, and then we will create a chain-of-thought conversation with the bot which will:

Ask the bot for a number of suggestions.
List the pros and cons of each.
Pick the best suggestion.
Product a detailed action plan.

Our goal is to only show the final result to the user.

Now, let us define our generation function:

[4]:

import re
import time

import guidance
from guidance import gen, select, system, user, assistant

@guidance
def plan_for_goal(lm, goal: str):

    # This is a helper function which we will use below
    def parse_best(prosandcons, options):
        best = re.search(r'Best=(\d+)', prosandcons)
        if not best:
            best =  re.search(r'Best.*?(\d+)', 'Best= option is 3')
        if best:
            best = int(best.group(1))
        else:
            best = 0
        return options[best]

    # Some general instruction to the model
    with system():
        lm += "You are a helpful assistant."

    # Simulate a simple request from the user
    # Note that we switch to using 'lm2' here, because these are intermediate steps (so we don't want to overwrite the current lm object)
    with user():
        lm2 = lm + f"""\
        I want to {goal}
        Can you please generate one option for how to accomplish this?
        Please make the option very short, at most one line."""

    # Generate several options. Note that this means several sequential generation requests
    n_options = 5
    with assistant():
        options = []
        for i in range(n_options):
            options.append((lm2 + gen(name='option', temperature=1.0, max_tokens=50))["option"])

    # Have the user request pros and cons
    with user():
        lm2 += f"""\
        I want to {goal}
        Can you please comment on the pros and cons of each of the following options, and then pick the best option?
        ---
        """
        for i, opt in enumerate(options):
            lm2 += f"Option {i}: {opt}\n"
        lm2 += f"""\
        ---
        Please discuss each option very briefly (one line for pros, one for cons), and end by saying Best=X, where X is the number of the best option."""

    # Get the pros and cons from the model
    with assistant():
        lm2 += gen(name='prosandcons', temperature=0.0, max_tokens=600, stop="Best=") + "Best=" + gen("best", regex="[0-9]+")
        time.sleep(call_delay_secs)

    # The user now extracts the one selected as the best, and asks for a full plan
    # We switch back to 'lm' because this is the final result we want
    with user():
        lm += f"""\
        I want to {goal}
        Here is my plan: {options[int(lm2["best"])]}
        Please elaborate on this plan, and tell me how to best accomplish it."""

    # The plan is generated
    with assistant():
        lm += gen(name='plan', max_tokens=500)
        time.sleep(call_delay_secs)

    return lm

Create a plan for the user. Note how the portions which were sent to lm2 in the function above are not shown in the final result:

[5]:

results = azureai_model + plan_for_goal(goal="read more books")

system
You are a helpful assistant.
user
I want to read more books
Here is my plan: Set a daily reading goal, like one chapter or 30 minutes per day.
Please elaborate on this plan, and tell me how to best accomplish it.
assistant
Absolutely, setting a daily reading goal is a great way to incorporate more reading into your routine. Here's a more detailed plan:

1. **Choose the Right Book**: Start with a book that interests you. It could be a novel, a self-help book, a biography, or any genre that you enjoy. The key is to choose a book that you're excited to read, as this will make it easier to stick to your reading plan.

2. **Set a Specific Goal**: You've mentioned reading one chapter or for 30 minutes each day. This is a great start. Make sure your goal is achievable and not too overwhelming. If a chapter is too long, you can set a goal to read a certain number of pages each day.

3. **Create a Reading Schedule**: Decide on a specific time to read each day. It could be in the morning, during lunch breaks, or before bed. Consistency is key, so try to stick to this schedule as closely as possible.

4. **Find a Quiet Place**: Choose a quiet and comfortable place to read where you won't be easily distracted. This could be a cozy corner in your home, a local library, or a park.

5. **Limit Distractions**: Turn off your phone or put it on silent mode to avoid interruptions. If you're reading on a digital device, try to close all unrelated apps and tabs.

6. **Take Notes**: If you come across an interesting idea or a word you don't understand, jot it down. This not only helps you remember what you've read but also makes the reading process more interactive and enjoyable.

7. **Review Your Progress**: Keep track of your reading progress. You can use a journal or a digital app for this. Celebrate your small victories, like finishing a chapter or a book.

8. **Join a Book Club or Reading Group**: This can provide motivation to read regularly. You'll also get to discuss the books and share your thoughts with others.

Remember, the goal is to make reading a habit and enjoy the process. Don't rush through books just to meet your goal. Happy reading!

We can access the final plan itself:

[6]:

print(results['plan'])

Absolutely, setting a daily reading goal is a great way to incorporate more reading into your routine. Here's a more detailed plan: