Another core module or plugin of the GuidedConversation is the Agenda. This is a specialized Pydantic BaseModel that gives the agent the ability to explicitly reason about a longer term plan, or agenda, for the conversation.
The BaseModel consists of a list of items, each with a description (a string) and a number of turns (an integer). It will raise an error if an input violates the type requirements. This check is particularly important for turn allocations. For example, sometimes a conversation agent provides fractional estimates ("0.6 turns") or broad ranges ("5-20" turns), both of which are meaningless. We also added additional validations which raise an error if the total number of turns allocated across items is invalid (e.g., it exceeds the number of remaining turns) depending on the resource constraint.
If an error is raised, the agent is raised, the agent is prompted to revise the agenda. To prevent infinite loops, we imposed a limit on the number of retries.
For this notebook we will revisit the teaching example from the first notebook. In that demo, under the hood the agent was actually making mistakes in its allocation of turns, mostly in generating an invalid number of cumulative turns. However, thanks to the Agenda plugin, it was able to automatically detect and correct these mistakes before they snowballed.
Let's start by setting up the Agenda plugin. It takes in a resource constraint type, which as a reminder controls conversation length. Currently it can be either maximum to set an upper limit and an exact mode for precise conversation lengths. Depending on the selected mode, the validation will differ. For example, for exact mode the total number of turns allocated across items must be exactly equal to the total number of turns available. While in maximum mode, the total number of turns allocated across items must be less than or equal to the total number of turns available.
from semantic_kernel import Kernel
from semantic_kernel.connectors.ai.open_ai import AzureChatCompletion
from semantic_kernel.contents import AuthorRole, ChatMessageContent
from guided_conversation.plugins.agenda import Agenda
from guided_conversation.utils.conversation_helpers import Conversation
from guided_conversation.utils.resources import ResourceConstraintMode
RESOURCE_CONSTRAINT_TYPE = ResourceConstraintMode.EXACT
kernel = Kernel()
service_id = "agenda_chat_completion"
chat_service = AzureChatCompletion(
service_id=service_id,
deployment_name="gpt-4o-2024-05-13",
api_version="2024-05-01-preview",
)
kernel.add_service(chat_service)
agenda = Agenda(
kernel=kernel, service_id=service_id, resource_constraint_mode=RESOURCE_CONSTRAINT_TYPE, max_agenda_retries=2
)
conversation = Conversation()
Here we provide an agenda that was generated by the Guided Conversation agent for the first turn of the conversation.
The core interface of the Agenda is update_agenda which takes in the generated agenda items, the conversation for context, and the remaining resource constraint units.
The expected format of the agenda is defined as follows in Pydantic:
class _BaseAgendaItem(BaseModelLLM):
title: str = Field(description="Brief description of the item")
resource: int = Field(description="Number of turns required for the item")
class _BaseAgenda(BaseModelLLM):
items: list[_BaseAgendaItem] = Field(
description="Ordered list of items to be completed in the remainder of the conversation",
default_factory=list,
)
Since we defined the resource constraint type to be exact, the resource units must also add up exactly to the remaining_turns parameter.
The provided agenda and remaining turns below adhere to that, so let's see what the string representation of the agenda looks like after we preform an update.
generated_agenda = [
{"title": "Explain what an acrostic poem is and how to write one and give an example", "resource": 1},
{"title": "Have the student write their acrostic poem", "resource": 2},
{"title": "Review and give initial feedback on the student's poem", "resource": 2},
{"title": "Guide the student in revising their poem based on the feedback", "resource": 3},
{"title": "Review the revised poem and provide final feedback", "resource": 3},
{"title": "Address any remaining questions or details", "resource": 3},
]
result = await agenda.update_agenda(
items=generated_agenda,
conversation=conversation,
remaining_turns=14,
)
print(agenda.get_agenda_for_prompt())
1. [1 turn] Explain what an acrostic poem is and how to write one and give an example 2. [2 turns] Have the student write their acrostic poem 3. [2 turns] Review and give initial feedback on the student's poem 4. [3 turns] Guide the student in revising their poem based on the feedback 5. [3 turns] Review the revised poem and provide final feedback 6. [3 turns] Address any remaining questions or details Total = 14 turns
Next, let's test out the ability of the agenda to detect and correct an agenda that does not follow the Pydantic model.
In the first part, we expand the conversation to give some realistic context for the Agenda. Then, we provide an invalid agenda where the type of the title field is not a string.
We will see how the Agenda plugin will use its judgement to correct this error and provide a valid agenda representation.
conversation.add_messages(
ChatMessageContent(
role=AuthorRole.ASSISTANT,
content="""Hi David! Today, we're going to learn about acrostic poems.
An acrostic poem is a fun type of poetry where the first letters of each line spell out a word or phrase. Here's how you can write one:
1. Choose a word or phrase that you like. This will be the subject of your poem.
2. Write the letters of your chosen word or phrase vertically down the page.
3. Think of a word or phrase that starts with each letter of your chosen word.
4. Write these words or phrases next to the corresponding letters to create your poem.
For example, if we use the word 'HAPPY', your poem might look like this:
H - Having fun with friends all day,
A - Awesome games that we all play.
P - Pizza parties on the weekend,
P - Puppies we bend down to tend,
Y - Yelling yay when we win the game.
Now, why don't you try creating your own acrostic poem? Choose any word or phrase you like and follow the steps above. I can't wait to see what you come up with!""",
)
)
conversation.add_messages(ChatMessageContent(role=AuthorRole.USER, content="I want to choose cars"))
conversation.add_messages(
ChatMessageContent(
role=AuthorRole.ASSISTANT,
content="""Great choice, David! 'Cars' sounds like a fun subject for your acrostic poem.
Be creative and let me know if you need any help as you write!""",
)
)
conversation.add_messages(
ChatMessageContent(
role=AuthorRole.USER,
content="""Heres my first attempt
Cruising down the street.
Adventure beckons with stories untold. \
R
S""",
)
)
result = await agenda.update_agenda(
items=[
{"title": 1, "resource": 3},
{"title": "Guide the student in revising their poem based on the feedback", "resource": 4},
{"title": "Review the revised poem and provide final feedback", "resource": 3},
{"title": "Address any remaining questions or details", "resource": 2},
],
conversation=conversation,
remaining_turns=12,
)
print(f"Was the update successful? {result.update_successful}")
print(f"Agenda state: {agenda.get_agenda_for_prompt()}")
Was the update successful? True Agenda state: 1. [3 turns] Ask for the feedback 2. [4 turns] Guide the student in revising their poem based on the feedback 3. [3 turns] Review the revised poem and provide final feedback 4. [2 turns] Address any remaining questions or details Total = 12 turns
We see that the agent removed the invalid item and correctly reallocated the resource to other items.
Lastly, let's test the ability of the Agenda to detect and correct an agenda that does not follow the resource constraint. We will provide an agenda where the total number of turns allocated across items exceeds the total number of remaining turns.
We will see that the agenda was successfully corrected to adhere to the resource constraint.
conversation.add_messages(
ChatMessageContent(
role=AuthorRole.ASSISTANT,
content="""That's a great start, David! I love the imagery you've used in your poem. Let's continue with writing the "R" and "S" lines.""",
)
)
conversation.add_messages(
ChatMessageContent(
role=AuthorRole.USER,
content="""Sure here's the rest of the poem:
Cruising down the street.
Adventure beckons with stories untold.
Revving engines, vroom vroom.
Steering through life's twists and turns.""",
)
)
result = await agenda.update_agenda(
items=[
{"title": "Review the revised poem and provide final feedback", "resource": 4},
{"title": "Address any remaining questions or details", "resource": 3},
],
conversation=conversation,
remaining_turns=11,
)
print(f"Was the update successful? {result.update_successful}")
print(f"Agenda state: {agenda.get_agenda_for_prompt()}")
Was the update successful? True Agenda state: 1. [7 turns] Review the revised poem and provide final feedback 2. [4 turns] Address any remaining questions or details Total = 11 turns