Notebook

Auto Generated Agent Chat: Graph Modeling Language with using select_speaker¶

AutoGen offers conversable agents powered by LLM, tool, or human, which can be used to perform tasks collectively via automated chat. This framework allows tool use and human participation through multi-agent conversation. Please find documentation about this feature here.

This notebook is about using graphs to define the transition paths amongst speakers.

Benefits

This contribution fills the gap between the current modes of GroupChat Class (auto, manual, round_robin) and an expressive directed graph. See Motivation for more detailed discussion.

Requirements¶

AutoGen requires Python>=3.8. To run this notebook example, please install:

pip install pyautogen

In [ ]:

%%capture --no-stderr
# %pip install "pyautogen>=0.2.3"
%pip install networkX~=3.2.1
%pip install matplotlib~=3.8.1

In [2]:

import random  # noqa E402

import matplotlib.pyplot as plt  # noqa E402
import networkx as nx  # noqa E402

import autogen  # noqa E402
from autogen.agentchat.assistant_agent import AssistantAgent  # noqa E402
from autogen.agentchat.groupchat import GroupChat  # noqa E402

In [3]:

print(autogen.__version__)

0.2.3

Motivation¶

The current GroupChat class allows transition to any agent (without or without the decision of LLM), some use case might demand for more control over transition. A graph is a possible way to control the transition paths, where each node represents an agent and each directed edge represent possible transition path. Let's illustrate the current transition paths for a GroupChat with five agents.

In [3]:

# Create an empty directed graph
graph = nx.DiGraph()

# Add 5 nodes to the graph using a for loop
for node_id in range(5):
    graph.add_node(node_id, label=str(node_id))

# Add edges between all nodes using a nested for loop
for source_node in range(5):
    for target_node in range(5):
        if source_node != target_node:  # To avoid self-loops
            graph.add_edge(source_node, target_node)

nx.draw(graph, with_labels=True, font_weight="bold")

Possibly interesting transition paths¶

In [4]:

# Hub and Spoke
# Create an empty directed graph
graph = nx.DiGraph()

# Add 5 nodes to the graph using a for loop
for node_id in range(5):
    graph.add_node(node_id, label=str(node_id))

# Add edges between all nodes using a nested for loop
for source_node in range(5):
    target_node = 0
    if source_node != target_node:  # To avoid self-loops
        graph.add_edge(source_node, target_node)
        graph.add_edge(target_node, source_node)


nx.draw(graph, with_labels=True, font_weight="bold")

In [5]:

# Sequential Team Operations
# Create an empty directed graph
graph = nx.DiGraph()

# Outer loop for prefixes 'A', 'B', 'C'
for prefix in ["A", "B", "C"]:
    # Add 5 nodes with each prefix to the graph using a for loop
    for i in range(5):
        node_id = f"{prefix}{i}"
        graph.add_node(node_id, label=node_id)

    # Add edges between nodes with the same prefix using a nested for loop
    for source_node in range(5):
        source_id = f"{prefix}{source_node}"
        for target_node in range(5):
            target_id = f"{prefix}{target_node}"
            if source_node != target_node:  # To avoid self-loops
                graph.add_edge(source_id, target_id)

graph.add_edge("A0", "B0")
graph.add_edge("B0", "C0")

# Draw the graph
nx.draw(graph, with_labels=True, font_weight="bold")

In [6]:

# Think aloud and debate
graph = nx.DiGraph()

for source_node in range(2):
    graph.add_node(source_node, label=source_node)

# Add edges between nodes with the same prefix using a nested for loop
for source_node in range(2):
    for target_node in range(2):
        graph.add_edge(source_node, target_node)

nx.draw(graph, with_labels=True, font_weight="bold")

Set your API Endpoint¶

The config_list_from_json function loads a list of configurations from an environment variable or a json file.

In [7]:

# The default config list in notebook.
config_list_gpt4 = autogen.config_list_from_json(
    "OAI_CONFIG_LIST",
    filter_dict={
        "model": ["gpt-4", "gpt-4-0314", "gpt4", "gpt-4-32k", "gpt-4-32k-0314", "gpt-4-32k-v0314"],
    },
)

It first looks for environment variable "OAI_CONFIG_LIST" which needs to be a valid json string. If that variable is not found, it then looks for a json file named "OAI_CONFIG_LIST". It filters the configs by models (you can filter by other keys as well). Only the gpt-4 models are kept in the list based on the filter condition.

The config list looks like the following:

config_list = [
    {
        'model': 'gpt-4',
        'api_key': '<your OpenAI API key here>',
    },
    {
        'model': 'gpt-4',
        'api_key': '<your Azure OpenAI API key here>',
        'base_url': '<your Azure OpenAI API base here>',
        'api_type': 'azure',
        'api_version': '2023-06-01-preview',
    },
    {
        'model': 'gpt-4-32k',
        'api_key': '<your Azure OpenAI API key here>',
        'base_url': '<your Azure OpenAI API base here>',
        'api_type': 'azure',
        'api_version': '2023-06-01-preview',
    },
]

If you open this notebook in colab, you can upload your files by clicking the file icon on the left panel and then choosing "upload file" icon.

You can set the value of config_list in other ways you prefer, e.g., loading from a YAML file.

We are printing out debug messages so that the reader can understand the conversation flow and select_speaker method better.

Overrides the select_speaker method with custom logic including:

Handling of NEXT: and TERMINATE tags in the last message.
Selection of the first-round speaker based on the first_round_speaker attribute in the graph nodes.
Selection of subsequent speakers based on the successors in the graph of the previous speaker.
Random selection of the next speaker from the eligible candidates.

In [8]:

class CustomGroupChat(GroupChat):
    def __init__(self, agents, messages, max_round=10, graph=None):
        super().__init__(agents, messages, max_round)
        self.previous_speaker = None  # Keep track of the previous speaker
        self.graph = graph  # The graph depicting who are the next speakers available

    def select_speaker(self, last_speaker, selector):
        self.previous_speaker = last_speaker

        # Check if last message suggests a next speaker or termination
        last_message = self.messages[-1] if self.messages else None
        suggested_next = None

        if last_message:
            if "NEXT:" in last_message["content"]:
                suggested_next = last_message["content"].split("NEXT: ")[-1].strip()
                # Strip full stop and comma
                suggested_next = suggested_next.replace(".", "").replace(",", "")
                print(f"Suggested next speaker from the last message: {suggested_next}")

            elif "TERMINATE" in last_message["content"]:
                try:
                    return self.agent_by_name("User_proxy")
                except ValueError:
                    print(f"agent_by_name failed suggested_next: {suggested_next}")

        # Debugging print for the current previous speaker
        if self.previous_speaker is not None:
            print("Current previous speaker:", self.previous_speaker.name)

        # Selecting first round speaker
        if self.previous_speaker is None and self.graph is not None:
            eligible_speakers = [
                agent for agent in agents if self.graph.nodes[agent.name].get("first_round_speaker", False)
            ]
            print("First round eligible speakers:", [speaker.name for speaker in eligible_speakers])

        # Selecting successors of the previous speaker
        elif self.previous_speaker is not None and self.graph is not None:
            eligible_speaker_names = [target for target in self.graph.successors(self.previous_speaker.name)]
            eligible_speakers = [agent for agent in agents if agent.name in eligible_speaker_names]
            print("Eligible speakers based on previous speaker:", eligible_speaker_names)

        else:
            eligible_speakers = agents

        # Debugging print for the next potential speakers
        print(
            f"Eligible speakers based on graph and previous speaker {self.previous_speaker.name if self.previous_speaker else 'None'}: {[speaker.name for speaker in eligible_speakers]}"
        )

        # Three attempts at getting the next_speaker
        # 1. Using suggested_next if suggested_next is in the eligible_speakers.name
        # 2. Using LLM to pick from eligible_speakers, given that there is some context in self.message
        # 3. Random (catch-all)
        next_speaker = None

        if eligible_speakers:
            print("Selecting from eligible speakers:", [speaker.name for speaker in eligible_speakers])
            # 1. Using suggested_next if suggested_next is in the eligible_speakers.name
            if suggested_next in [speaker.name for speaker in eligible_speakers]:
                print("suggested_next is in eligible_speakers")
                next_speaker = self.agent_by_name(suggested_next)

            else:
                msgs_len = len(self.messages)
                print(f"msgs_len is now {msgs_len}")
                if len(self.messages) > 1:
                    # 2. Using LLM to pick from eligible_speakers, given that there is some context in self.message
                    print(
                        f"Using LLM to pick from eligible_speakers: {[speaker.name for speaker in eligible_speakers]}"
                    )
                    selector.update_system_message(self.select_speaker_msg(eligible_speakers))
                    _, name = selector.generate_oai_reply(
                        self.messages
                        + [
                            {
                                "role": "system",
                                "content": f"Read the above conversation. Then select the next role from {[agent.name for agent in eligible_speakers]} to play. Only return the role.",
                            }
                        ]
                    )

                    # If exactly one agent is mentioned, use it. Otherwise, leave the OAI response unmodified
                    mentions = self._mentioned_agents(name, eligible_speakers)
                    if len(mentions) == 1:
                        name = next(iter(mentions))
                        next_speaker = self.agent_by_name(name)

                if next_speaker is None:
                    # 3. Random (catch-all)
                    next_speaker = random.choice(eligible_speakers)

            print(f"Selected next speaker: {next_speaker.name}")

            return next_speaker
        else:
            # Cannot return next_speaker with no eligible speakers
            raise ValueError("No eligible speakers found based on the graph constraints.")

Demonstration¶

Team Operations¶

In [9]:

# llm config
llm_config = {"config_list": config_list_gpt4, "cache_seed": 100}

# Create an empty directed graph
graph = nx.DiGraph()

agents = []

# Outer loop for prefixes 'A', 'B', 'C'
for prefix in ["A", "B", "C"]:
    # Add 3 nodes with each prefix to the graph using a for loop
    for i in range(3):
        node_id = f"{prefix}{i}"
        secret_value = random.randint(1, 5)  # Generate a random secret value
        graph.add_node(node_id, label=node_id, secret_value=secret_value)

        # Create an AssistantAgent for each node (assuming AssistantAgent is a defined class)
        agents.append(
            AssistantAgent(
                name=node_id,
                system_message=f"""Your name is {node_id}.
                                          Do not respond as the speaker named in the NEXT tag if your name is not in the NEXT tag. Instead, suggest a relevant team leader to handle the mis-tag, with the NEXT: tag.

                                          You have {secret_value} chocolates.

                                          The list of players are [A0, A1, A2, B0, B1, B2, C0, C1, C2].

                                            Your first character of your name is your team, and your second character denotes that you are a team leader if it is 0.
                                            CONSTRAINTS: Team members can only talk within the team, whilst team leader can talk to team leaders of other teams but not team members of other teams.

                                            You can use NEXT: to suggest the next speaker. You have to respect the CONSTRAINTS, and can only suggest one player from the list of players, i.e., do not suggest A3 because A3 is not from the list of players.
                                            Team leaders must make sure that they know the sum of the individual chocolate count of all three players in their own team, i.e., A0 is responsible for team A only.

                                          Keep track of the player's tally using a JSON format so that others can check the total tally. Use
                                          A0:?, A1:?, A2:?,
                                          B0:?, B1:?, B2:?,
                                          C0:?, C1:?, C2:?

                                          If you are the team leader, you should aggregate your team's total chocolate count to cooperate.
                                          Once the team leader know their team's tally, they can suggest another team leader for them to find their team tally, because we need all three team tallys to succeed.
                                          Use NEXT: to suggest the next speaker, e.g., NEXT: A0.

                                          Once we have the total tally from all nine players, sum up all three teams' tally, then terminate the discussion using TERMINATE.

                                          """,
                llm_config=llm_config,
            )
        )

    # Add edges between nodes with the same prefix using a nested for loop
    for source_node in range(3):
        source_id = f"{prefix}{source_node}"
        for target_node in range(3):
            target_id = f"{prefix}{target_node}"
            if source_node != target_node:  # To avoid self-loops
                graph.add_edge(source_id, target_id)

# Adding edges between teams
graph.add_edge("A0", "B0")
graph.add_edge("A0", "C0")
graph.add_edge("B0", "A0")
graph.add_edge("B0", "C0")
graph.add_edge("C0", "A0")
graph.add_edge("C0", "B0")


# Updating node A0
graph.nodes["A0"]["first_round_speaker"] = True


def get_node_color(node):
    if graph.nodes[node].get("first_round_speaker", False):
        return "red"
    else:
        return "green"


# Draw the graph with secret values annotated
plt.figure(figsize=(12, 10))
pos = nx.spring_layout(graph)  # positions for all nodes

# Draw nodes with their colors
nx.draw(graph, pos, with_labels=True, font_weight="bold", node_color=[get_node_color(node) for node in graph])

# Annotate secret values
for node, (x, y) in pos.items():
    secret_value = graph.nodes[node]["secret_value"]
    plt.text(x, y + 0.1, s=f"Secret: {secret_value}", horizontalalignment="center")

plt.show()

In [10]:

# Termination message detection


def is_termination_msg(content) -> bool:
    have_content = content.get("content", None) is not None
    if have_content and "TERMINATE" in content["content"]:
        return True
    return False


# Terminates the conversation when TERMINATE is detected.
user_proxy = autogen.UserProxyAgent(
    name="User_proxy",
    system_message="Terminator admin.",
    code_execution_config=False,
    is_termination_msg=is_termination_msg,
    human_input_mode="NEVER",
)

agents.append(user_proxy)

In [11]:

group_chat = CustomGroupChat(agents=agents, messages=[], max_round=20, graph=graph)  # Include all agents


# Create the manager
manager = autogen.GroupChatManager(groupchat=group_chat, llm_config=llm_config)


# Initiates the chat with Alice
agents[0].initiate_chat(
    manager,
    message="""
                        There are 9 players in this game, split equally into Teams A, B, C. Therefore each team has 3 players, including the team leader.
                        The task is to find out the sum of chocolate count from all nine players. I will now start with my team.
                        NEXT: A1""",
)

A0 (to chat_manager):


                        There are 9 players in this game, split equally into Teams A, B, C. Therefore each team has 3 players, including the team leader. 
                        The task is to find out the sum of chocolate count from all nine players. I will now start with my team. 
                        NEXT: A1

--------------------------------------------------------------------------------
Suggested next speaker from the last message: A1
Current previous speaker: A0
Eligible speakers based on previous speaker: ['A1', 'A2', 'B0', 'C0']
Eligible speakers based on graph and previous speaker A0: ['A1', 'A2', 'B0', 'C0']
Selecting from eligible speakers: ['A1', 'A2', 'B0', 'C0']
suggested_next is in eligible_speakers
Selected next speaker: A1
A1 (to chat_manager):

As A1 I have 1 chocolate right now. Our team leader A0, please note my count.

NEXT: A2

--------------------------------------------------------------------------------
Suggested next speaker from the last message: A2
Current previous speaker: A1
Eligible speakers based on previous speaker: ['A0', 'A2']
Eligible speakers based on graph and previous speaker A1: ['A0', 'A2']
Selecting from eligible speakers: ['A0', 'A2']
suggested_next is in eligible_speakers
Selected next speaker: A2
A2 (to chat_manager):

As part of Team A, I have 2 chocolates at the moment.

Now that each member of Team A has reported their tally, our team leader A0 should be able to calculate and report our team's total sum to the other team leaders. 

NEXT: A0.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: A0
Current previous speaker: A2
Eligible speakers based on previous speaker: ['A0', 'A1']
Eligible speakers based on graph and previous speaker A2: ['A0', 'A1']
Selecting from eligible speakers: ['A0', 'A1']
suggested_next is in eligible_speakers
Selected next speaker: A0
A0 (to chat_manager):

I, A0, have 4 chocolates, A1 reported having 1 chocolate, and A2 reported having 2 chocolates. So the total chocolate count for Team A is 4 + 1 + 2 = 7 chocolates. 

I'm saving this in our JSON format as: 
A0:4, A1:1, A2:2,
B0:?, B1:?, B2:?,
C0:?, C1:?, C2:?

Let's move on to Team B for their counts.
NEXT: B0.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: B0
Current previous speaker: A0
Eligible speakers based on previous speaker: ['A1', 'A2', 'B0', 'C0']
Eligible speakers based on graph and previous speaker A0: ['A1', 'A2', 'B0', 'C0']
Selecting from eligible speakers: ['A1', 'A2', 'B0', 'C0']
suggested_next is in eligible_speakers
Selected next speaker: B0
B0 (to chat_manager):

As B0, the team leader of Team B, I already have my count which is 5 chocolates. Now, I will ask the other members of my team to report their counts. 

NEXT: B1

--------------------------------------------------------------------------------
Suggested next speaker from the last message: B1
Current previous speaker: B0
Eligible speakers based on previous speaker: ['B1', 'B2', 'A0', 'C0']
Eligible speakers based on graph and previous speaker B0: ['A0', 'B1', 'B2', 'C0']
Selecting from eligible speakers: ['A0', 'B1', 'B2', 'C0']
suggested_next is in eligible_speakers
Selected next speaker: B1
B1 (to chat_manager):

As B1, I have 4 chocolates currently. It's now time for our team member B2 to report their count. 

NEXT: B2.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: B2
Current previous speaker: B1
Eligible speakers based on previous speaker: ['B0', 'B2']
Eligible speakers based on graph and previous speaker B1: ['B0', 'B2']
Selecting from eligible speakers: ['B0', 'B2']
suggested_next is in eligible_speakers
Selected next speaker: B2
B2 (to chat_manager):

As B2, I have 1 chocolate right now. Our team leader B0, please note my count.

NEXT: B0.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: B0
Current previous speaker: B2
Eligible speakers based on previous speaker: ['B0', 'B1']
Eligible speakers based on graph and previous speaker B2: ['B0', 'B1']
Selecting from eligible speakers: ['B0', 'B1']
suggested_next is in eligible_speakers
Selected next speaker: B0
B0 (to chat_manager):

As B0, I acknowledge receipt of both B1 and B2's counts. I have 5 chocolates, B1 has 4 and B2 has 1. Adding these counts together, Team B has a total of 5 + 4 + 1 = 10 chocolates.

Updating the JSON tally:
A0:4, A1:1, A2:2,
B0:5, B1:4, B2:1,
C0:?, C1:?, C2:?

Now it's time for Team C to share their counts. 

NEXT: C0.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: C0
Current previous speaker: B0
Eligible speakers based on previous speaker: ['B1', 'B2', 'A0', 'C0']
Eligible speakers based on graph and previous speaker B0: ['A0', 'B1', 'B2', 'C0']
Selecting from eligible speakers: ['A0', 'B1', 'B2', 'C0']
suggested_next is in eligible_speakers
Selected next speaker: C0
C0 (to chat_manager):

As C0, I currently have 2 chocolates. I will need the counts from C1 and C2 to complete our team's tally. 

NEXT: C1.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: C1
Current previous speaker: C0
Eligible speakers based on previous speaker: ['C1', 'C2', 'A0', 'B0']
Eligible speakers based on graph and previous speaker C0: ['A0', 'B0', 'C1', 'C2']
Selecting from eligible speakers: ['A0', 'B0', 'C1', 'C2']
suggested_next is in eligible_speakers
Selected next speaker: C1
C1 (to chat_manager):

As C1, I have 2 chocolates. C2, please share your count so that our team leader, C0, can calculate our team's total.

NEXT: C2.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: C2
Current previous speaker: C1
Eligible speakers based on previous speaker: ['C0', 'C2']
Eligible speakers based on graph and previous speaker C1: ['C0', 'C2']
Selecting from eligible speakers: ['C0', 'C2']
suggested_next is in eligible_speakers
Selected next speaker: C2
C2 (to chat_manager):

As C2, I have 5 chocolates. Now our team leader, C0, can calculate our team's total sum.

NEXT: C0.

--------------------------------------------------------------------------------
Suggested next speaker from the last message: C0
Current previous speaker: C2
Eligible speakers based on previous speaker: ['C0', 'C1']
Eligible speakers based on graph and previous speaker C2: ['C0', 'C1']
Selecting from eligible speakers: ['C0', 'C1']
suggested_next is in eligible_speakers
Selected next speaker: C0
C0 (to chat_manager):

As C0, I have 2 chocolates, C1 reported having 2 chocolates, and C2 reported having 5 chocolates. So, the total chocolate count for Team C is 2 + 2 + 5 = 9 chocolates.

Updating the JSON tally:
A0:4, A1:1, A2:2,
B0:5, B1:4, B2:1,
C0:2, C1:2, C2:5

Let's sum up all the team totals. 

TERMINATE.

--------------------------------------------------------------------------------