%matplotlib inline
import warnings
warnings.filterwarnings('ignore')
# setup disply parameters
from matplotlib import pylab as plt
from matplotlib.ticker import StrMethodFormatter
float_formatter = StrMethodFormatter('{x:0.03f}')
from IPython.core.display import display, HTML
display(HTML("<style>.container { width:95% !important; }</style>"))
SMALL_SIZE = 14
MEDIUM_SIZE = 16
BIGGER_SIZE = 20
plt.rc('font', size=SMALL_SIZE) # controls default text sizes
plt.rc('axes', titlesize=SMALL_SIZE) # fontsize of the axes title
plt.rc('axes', labelsize=MEDIUM_SIZE) # fontsize of the x and y labels
plt.rc('xtick', labelsize=SMALL_SIZE) # fontsize of the tick labels
plt.rc('ytick', labelsize=SMALL_SIZE) # fontsize of the tick labels
plt.rc('legend', fontsize=SMALL_SIZE) # legend fontsize
plt.rc('figure', titlesize=BIGGER_SIZE) # fontsize of the figure title
plt.rc('figure', figsize=(18, 6)) # set figure size
plt.rc("animation", html="html5")
from abc import abstractmethod, ABC
from collections import defaultdict
from random import shuffle, random, sample, randint
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from negmas import (
Action,
Agent,
NegotiatorMechanismInterface,
AgentWorldInterface,
Breach,
Contract,
Issue,
make_issue,
LinearUtilityFunction,
MechanismState,
Negotiator,
RandomNegotiator,
RenegotiationRequest,
SAONegotiator,
UtilityFunction,
World,
)
from negmas.serialization import to_flat_dict
from typing import Callable, List, Optional, Set, Dict, Any, Collection
from negmas import *
A simulation is an embedded domain in which agents behave. It is represented in NegMAS by a World
. This tutorial will take you through the process of developing a simple simulation (world).
A World
in NegMAS is not just a multi-agent world simulation. It was designed from the bottom up to simply the common tasks involved in constructing negotiation driven simulations.
The simulation is divided into simulation steps (not to be confused with negotiation rounds). At every step, agents are allowed to act proactively by executing actions in the world, reading their state from the world, or requesting/running negotiations with other agents.
The negotiation process follows the following steps:
request_negotiation
or run_negotiation
methods (the later does not return until the negotiation is complete. The World
can disable immediately running negotiations, or requesting negotiations that does not involve oneself, etc. The caller agent can also provide a Negotiator
to represent it in this negotiation.respond_to_negotiation_request
. The can accept by returning a Negotiator
object to represent them or reject.World
. The default rule is to start negotiations only if ALL partners accepted the request but other rules like starting negotiations that are accepted by at least two partners are possible.Mechanism
objects are created by the World
and negotiators supplied by all partners (that agreed to join the negotiation) are added to it. The World
designer can fix the type of the negotiation mechanism to be used (e.g. SAOMechanism
) or it can leave the choice to individual agents requesting negotiations.Contract
if an agreement is reached.Contract
s binding upon agreement by having them automatically signed or to give agents one final chance to sign the contracts. Only signed contracts are considered binding.World
can also run a simulation that is affected by signed contracts. A common case is for contracts to have some executing time or a similar attribute that defines when do they become due.Contract
becomes due, it is executed by the World
as part of the simulation it is controlling. In some cases, the contract cannot be executed and this is considered a Breach
.To create a new world, you need to do the following steps:
AgentWorldInterface
for your world (that inherits from AWI
) and provide easy to use methods that agents can use to access your world. A common functionality of the AWI is to confirm that negotiations make sense. We will see an example later in this tutorialWorld
class that inherits from the base World
and implement the required callback (your AWI from the first step will need to be registered in the world as will be shown later).Agent
class for your world that inherits from Agent
and provides the basic functionality shared by all agents in your world (if any)Let's consider a -- not so simple -- world. We have $m$ agents representing people agreeing about where to spend their vacation repeatedly. Every holiday season, an agent can request to negotiate with one or more other agents trying to arrange a trip. Each trip is described by the following attributes:
All agents involved in a negotiation must agree for the trip to be conducted. An agent can request at most $n$ negotiations in a single season. Each agent has some fixed but unknown probability of not honoring its agreement and failing to show up for the trip and only when all agents involved in a trip show up does the trip actually happen with each agent receiving its utility value as defined by its unchanging utility function.
Let's implement this world in NegMAS.
We will start by defining the interface between the world and the agent as an AWI class
class AWI(AgentWorldInterface):
@property
def n_negs(self):
"""Number of negotiations an agent can start in a step (holiday season)"""
return self._world.neg_quota_step
@property
def agents(self):
"""List of all other agent IDs"""
return list(_ for _ in self._world.agents.keys() if _ != self.agent.id)
def request_negotiation(
self, partners: List[str], negotiator: SAONegotiator
) -> bool:
"""A convenient way to request negotiations"""
if self.agent.id not in partners:
partners.append(self.agent.id)
req_id = self.agent.create_negotiation_request(
issues=self._world.ISSUES,
partners=partners,
negotiator=negotiator,
annotation=dict(),
extra=dict(negotiator_id=negotiator.id),
)
return self.request_negotiation_about(
issues=self._world.ISSUES, partners=partners, req_id=req_id
)
The minimum here is to define a way for agents to request negotiations form the world. The base AgentWorldInterface
has a request_negotiation_about
method that can be used for this purpose but it is too general and allows agents to set arbitrary issues and negotiation mechanisms. Usually you will want to restrict the types of negotiations allowed by defining a request_negotiation
method which decides as much as possible for the agent.
This is done here using the following method:
def request_negotiation(
self, partners: List[str], negotiator: SAONegotiator
) -> bool:
...
Here the agent is asked to provide only a list of partners
and a negotiator
to use.
if self.agent.id not in partners:
partners.append(self.agent.id)
create_negotiation_request
of the agent connected to the AWI which is used to keep track of which requests are out there and which are accepted/rejectedreq_id = self.agent.create_negotiation_request(
issues=self._world.ISSUES,
partners=partners,
negotiator=negotiator,
annotation=dict(),
extra=dict(negotiator_id=negotiator.id),
)
request_negotiation_about
method.return self.request_negotiation_about(
issues=self._world.ISSUES, partners=partners, req_id=req_id
)
Other than this commonly provided method, the AWI provides two properties that can be accessed by the agent, agents
which returns the IDs of all other agents in the world and n_negs
which gives the total number of negotiations that the agent can start in a single step (holiday season).
To implement the trips world, you will need to override the abstract methods of World
. You will usually need also to override __init__
to initialize your agent and join
to set up any agent specific information you need to keep. Here is the full implementation:
class TripsWorld(World):
ISSUES = [
make_issue((0.0, 1.0), "cost"),
make_issue(2, "active"),
make_issue((1, 7), "duration"),
]
def __init__(self, *args, **kwargs):
"""Initialize the world"""
kwargs["awi_type"] = AWI
kwargs["negotiation_quota_per_step"] = kwargs.get(
"negotiation_quota_per_step", 8
)
kwargs["force_signing"] = True
kwargs["default_signing_delay"] = 0
super().__init__(*args, **kwargs)
self._contracts: Dict[int, List[Contract]] = defaultdict(list)
self._total_utility: Dict[str, float] = defaultdict(float)
self._ufuns: Dict[str, UtilityFunction] = dict()
self._breach_prob: Dict[str, float] = dict()
def join(
self,
x: Agent,
preferences: Preferences | None = None,
breach_prob: float | None = None,
**kwargs,
):
"""Define the ufun and breach-probability for each agent"""
super().join(x, **kwargs)
weights = np.random.rand(len(self.ISSUES)) - 0.5
x.ufun = (
LinearUtilityFunction(weights, reserved_value=0.0) if ufun is None else ufun
)
self._ufuns[x.id] = x.ufun
self._breach_prob[x.id] = random() * 0.1 if breach_prob is None else breach_prob
def simulation_step(self, stage: int = 0):
"""What happens in this world? Nothing"""
pass
def get_private_state(self, agent: Agent) -> dict:
"""What is the information available to agents? total utility points"""
return dict(total_utility=self._total_utility[agent.id])
def execute_action(
self, action: Action, agent: Agent, callback: Callable | None = None
) -> bool:
"""Executing actions by agents? No actions available"""
pass
def on_contract_signed(self, contract: Contract) -> None:
"""Save the contract to be executed in the following hoiday season (step)"""
super().on_contract_signed(contract)
self._contracts[self.current_step + 1].append(contract)
def executable_contracts(self):
"""What contracts are to be executed in the current step?
Ones that were signed the previous step"""
return self._contracts[self.current_step]
def order_contracts_for_execution(
self, contracts: Collection[Contract]
) -> Collection[Contract]:
"""What should be the order of contract execution? Random"""
shuffle(contracts)
return contracts
def start_contract_execution(self, contract: Contract) -> Optional[Set[Breach]]:
"""What should happen when a contract comes due?
1. Find out if it will be breached
2. If not, add to each agent its utility from the trip
"""
breaches = []
for aid in contract.partners:
if random() < self._breach_prob[aid]:
breaches.append(
Breach(
contract,
aid,
"breach",
victims=[_ for _ in contract.partners if _ != aid],
)
)
if len(breaches) > 0:
return set(breaches)
for aid in contract.partners:
self._total_utility[aid] += self._ufuns[aid](contract.agreement)
return set()
def complete_contract_execution(
self, contract: Contract, breaches: List[Breach], resolution: Contract
) -> None:
"""What happens if a breach was resolved? Nothing. They cannot"""
pass
def delete_executed_contracts(self) -> None:
"""Removes all contracts for the current step"""
if self._current_step in self._contracts.keys():
del self._contracts[self.current_step]
def contract_record(self, contract: Contract) -> Dict[str, Any]:
"""Convert the contract into a dictionary for saving"""
return to_flat_dict(contract)
def breach_record(self, breach: Breach) -> Dict[str, Any]:
"""Convert the breach into a dictionary for saving"""
return to_flat_dict(breach)
def contract_size(self, contract: Contract) -> float:
"""How good is a contract? Welfare"""
if contract.agreement is None:
return 0.0
return sum(self._ufuns[aid](contract.agreement) for aid in contract.partners)
def post_step_stats(self):
for aid, agent in self.agents.items():
self._stats[f"total_utility_{agent.name}"].append(self._total_utility[aid])
We will now inspect each of these methods in turn.
The first thing to do when constructing the world in __init__
is to call the World
class constructor forcing some of the parameters. This is done here:
kwargs["awi_type"] = AWI
kwargs["negotiation_quota_per_step"] = kwargs.get(
"negotiation_quota_per_step", 8
)
kwargs["force_signing"] = True
kwargs["default_signing_delay"] = 0
super().__init__(*args, **kwargs)
Of note is setting the awi_type
to the AWI
class we have just created. This allows agents to access members of this class through their awi
property as we will see later.
Moreover, we force the negotiation_quota_per_step
to be no more than 8 (the default is $\inf$) and force signing of all contracts which will make contracts binding immediately once agreements are reached through negotiation.
We then define four data-members that we keep track of:
self._contracts: Dict[int, List[Contract]] = defaultdict(list)
self._total_utility: Dict[str, float] = defaultdict(float)
self._ufuns: Dict[str, UtilityFunction] = dict()
self._breach_prob: Dict[str, float] = dict()
Agents join the world by calls to the join
method.
join
method of the World
class. That is essential for the system to work properly. Whenever you override a method that is not marked abstract, you must call the base class version using super()
:def join(self, x: Agent, preferences: Preferences | None = None, breach_prob: float | None = None, **kwargs):
"""Define the ufun and breach-probability for each agent"""
super().join(x, **kwargs)
...
x.ufun = LinearUtilityFunction(
np.random.rand(len(self.ISSUES)) - 0.5
) if ufun is None else ufun
self._ufuns[x.id] = x.ufun
self._breach_prob[x.id] = random() * 0.1 if breach_prob is None else breach_prob
The TripsWorld
does not have a simulation. Nothing really happens in this world. This means we can just do nothing in the simulation_step
method
def simulation_step(self, stage: int = 0):
"""What happens in this world? Nothing"""
pass
Every world needs to define what is the private state of an agent (available to it through self.awi.state
). In our world, the private state of an agent is the total utility it collected so far.
def get_private_state(self, agent: Agent) -> dict:
"""What is the information available to agents? total utility points"""
return dict(total_utility=self._total_utility[agent.id])
As we have no actual simulation, there are not actions that the agent can execute in the world, so execute_action
does nothing.
def execute_action(
self, action: Action, agent: Agent, callback: Callable | None = None
) -> bool:
"""Executing actions by agents? No actions available"""
pass
The TripsWorld
is responsible of managing contracts. The base World
class will take care of most of the process but it needs the TripsWorld
to respond to some callbacks in order to manage contract execution, storage, and optionally renegotiations of breached contracts.
The callbacks related to this are:
def on_contract_signed(self, contract: Contract) -> None:
...
def executable_contracts(self) -> Collection[Contract]:
...
def order_contracts_for_execution( self, contracts: Collection[Contract]) -> Collection[Contract]:
...
def start_contract_execution(self, contract: Contract) -> Optional[Set[Breach]]:
...
def complete_contract_execution( self, contract: Contract, breaches: List[Breach]
, resolution: Contract) -> None:
...
def delete_executed_contracts(self) -> None:
...
def contract_record(self, contract: Contract) -> Dict[str, Any]:
...
def breach_record(self, breach: Breach) -> Dict[str, Any]:
...
def contract_size(self, contract: Contract) -> float:
...
The names are almost self-explanatory and we will go through them one by one:
self._contracts[self.current_step + 1].append(contract)
_contracts
mapping we updated in on_contract_signed
:return self._contracts[self.current_step]
executable_contracts
) are executed, here we just shuffle them randomly and return them:shuffle(contracts)
return contracts
breaches = []
for aid in contract.partners:
if random() < self._breach_prob[aid]:
breaches.append(
Breach(
contract, aid, "breach",
victims=[_ for _ in contract.partners if _ != aid],
)
)
if len(breaches) > 0:
return set(breaches)
If there are no breaches, the trip is assumed to execute successfully and every agent (of the partners) is assigned the utility value from that trip according to its utility function:
for aid in contract.partners:
self._total_utility[aid] = self._ufuns[aid](contract.agreement)
return set()
__init__
and updated in on_contract_signed
.if self._current_step in self._contracts.keys():
del self._contracts[self.current_step]
These six steps complete all processing of contracts. Nevertheless, we still need to override three other methods to define how contracts and breaches are stored and the value of a contract
to_flat_dict
:return to_flat_dict(contract)
return to_flat_dict(breach)
if contract.agreement is None:
return 0.0
return sum(self._ufuns[aid](contract.agreement) for aid in contract.partners)
This complete the world and agent-world-interface design. We can now develop our base agent class.
The base World
keeps track of negotiation related statistics (e.g. how many negotiations were requested very step, how many contracted were breached, etc). You can easily add to this set of statistics by overloading post_step_stats
(and the corresponding pre_step_stats
if needed). In our world, we just add one custom statistic: the total utility collected by the agent so far:
for aid, agent in self.agents.items():
self._stats[f"total_utility_{agent.name}"].append(self._total_utility[aid])
Note that we used the agent name not ID to differentiate these statistics. Because the system does not know or use our statistic, we can use the name which will usually be easier to read when inspecting these statistics as we will see in the following tutorial
Even though it is not strictly necessary (as with the case of agent-world-interface), it is useful to provide a base agent that hides unnecessary details from developers of agents targeting our TripsWorld
. This is the complete listing of our base agent:
class Person(Agent, ABC):
@abstractmethod
def step(self):
...
@abstractmethod
def init(self):
...
@abstractmethod
def respond_to_negotiation_request(
self,
initiator: str,
partners: List[str],
mechanism: NegotiatorMechanismInterface,
) -> Optional[Negotiator]:
...
def _respond_to_negotiation_request(
self,
initiator: str,
partners: List[str],
issues: List[Issue],
annotation: Dict[str, Any],
mechanism: NegotiatorMechanismInterface,
role: Optional[str],
req_id: Optional[str],
) -> Optional[Negotiator]:
return self.respond_to_negotiation_request(initiator, partners, mechanism)
def on_neg_request_rejected(self, req_id: str, by: Optional[List[str]]):
pass
def on_neg_request_accepted(
self, req_id: str, mechanism: NegotiatorMechanismInterface
):
pass
def on_negotiation_failure(
self,
partners: List[str],
annotation: Dict[str, Any],
mechanism: NegotiatorMechanismInterface,
state: MechanismState,
) -> None:
pass
def on_negotiation_success(
self, contract: Contract, mechanism: NegotiatorMechanismInterface
) -> None:
pass
def set_renegotiation_agenda(
self, contract: Contract, breaches: List[Breach]
) -> Optional[RenegotiationRequest]:
pass
def respond_to_renegotiation_request(
self, contract: Contract, breaches: List[Breach], agenda: RenegotiationRequest
) -> Optional[Negotiator]:
pass
def on_contract_executed(self, contract: Contract) -> None:
pass
def on_contract_breached(
self, contract: Contract, breaches: List[Breach], resolution: Optional[Contract]
) -> None:
pass
The first thing, our abstract-base-class (ABC) does is defining the abstract methods that must be implemented by any agent that is compatible with the TripsWorld
.
The first two abstract methods are init
and step
called by the world to initialize the agent (after its AWI is created) and at every simulation step. These methods are not abstract in the base Agent
class but we convert them to abstract methods to force all Person
based agents to provide some implementation for them
@abstractmethod
def step(self):
...
@abstractmethod
def init(self):
...
We then add a third method for responding to negotiation requests:
@abstractmethod
def respond_to_negotiation_request(
self, initiator: str, partners: List[str], mechanism: AgentMechanismInterface,
) -> Optional[Negotiator]:
...
World
and TripWorld
classes know nothing about this method, our base Person
class will call it when it receives a request to respond to a negotiation request from the world in _respond_to_negotiation_request
(notice the underscore which indicates that children should not modify this method):
return self.respond_to_negotiation_request(initiator, partners, mechanism)
This arrangement removes the need to pass several parameters of _respond_to_negotiation_request
that are not of value for our current simulation.
We provide a do-nothing implementation of all other callbacks expected during the simulation. These are:
def on_neg_request_rejected(self, req_id: str, by: Optional[List[str]]):
...
def on_neg_request_accepted(self, req_id: str, mechanism: AgentMechanismInterface):
...
def on_negotiation_failure( self, partners: List[str], annotation: Dict[str, Any],
mechanism: AgentMechanismInterface, state: MechanismState,):
...
def on_negotiation_success( self, contract: Contract, mechanism: AgentMechanismInterface):
...
def set_renegotiation_agenda(
self, contract: Contract, breaches: List[Breach]) -> Optional[RenegotiationRequest]:
...
def respond_to_renegotiation_request(
self, contract: Contract, breaches: List[Breach], agenda: RenegotiationRequest) -> Optional[Negotiator]:
...
def on_contract_executed(self, contract: Contract):
...
def on_contract_breached(self, contract: Contract, breaches: List[Breach]):
...
These callbacks are called by the world at key points of the process from a negotiation request to an exeucted/breached contract. The names are self-explanatory but we summarize them here:
request_negotiation
). Agents can access the current requests using their negotiation_requests
property to get more information about the request if needed.negotiations
property.We now have all the ingredients to create specific agents and start simulations. In the next tutorial we will develop an agent for this world and use it to test it.
World
¶This section is more of a reference. You need not go through it in details in your first read
You can control several options about how your world simulation runs by setting constructor parameters of the World
class (as we did earlier with force_signing
). Here we discuss briefly some of the most important options.
These are general parameters that do not directly affect how the world works. The most important of these are name
to set a name for the world, and awi_type
which controls the type of AWI used to connect agents to it.
AgentWorldInterface
)These options control how the simulation is run and the order of operations in each simulation step.
n_steps: Total simulation time in steps
time_limit: Real-time limit on the simulation
operations: A list of Operations
to run in order during every simulation step. You can use this parameter to set the order of events in your simulation. For example, you can choose when negotiations run relative to the simulation_step
of your world.
Available operations include:
_stats
by calling update_stats
of the World
class. Each time this operation is conducted a higher stage
is passed to the update_stats
method (the first such call will by default run pre_step_stats
and later calls will call post_step_stats
but you can change that.simulation_step
of the World
class. Each time this operation is conducted a higher stage
is passed to simulation_step
.step
methodControls all negotiations conducted during the simulation.
super().__init__(mechanisms={"negmas.sao.SAOMechanism": dict(offering_is_accepting=False), "negmas.st.STVetoMechanism": dict()},
...)
After negotiations are concluded with agreements, it is possible to have an extra signing step to confirm these agreements before they become binding contracts. This gives agents central control over the agreements reached by their negotiators. You can control whether or not this step is needed for any world simulation and how confirmation (i.e. signing) is done through these parameters.
force_signing
is False
default_singing_delay
is not effective and signature is immediateWhen contracts fail to execute, breaches occur. You can control what happens when breaches occur using this parameter.
BreachProcessing
values. Three options are available:NegMAS supports both general logs through the log*
methods of the World
class and agent specific logs through the agent_log*
methods of the AWI. These parameters control logging. The default logging location is ~/negmas/logs
.
draw
method are kept.These settings greatly affect the memory consumption of the simulation. It tells NegMAS what exactly do you need to save in-memory.
It is inevitable that exceptions will happen in agent code or the simulation. This set of parameters control how to handle these exceptions.
NegMAS can keep checkpoints of the world simulation that can be used to recover and continue the simulation later. These checkpoints are not stored by default but you can enable them and control their frequency and location using this set of parameters
We can now continue to the next tutorials in which we will develop agents for your newly created world.