Develop a factory manager (agent) for the SCM world¶
This tutorial describes how to develop an agent for the SCM world, test it, and submit it to the ANAC 2019 SCM league.
The first step is to install negmas
pip install negmas
Once you have this library installed, you can start developing your factory manager:
from negmas.apps.scml import FactoryManager
try:
class MyFactoryManager(FactoryManager):
"""My factory manager"""
f = MyFactoryManager()
except Exception as e:
print(e)
Can't instantiate abstract class MyFactoryManager with abstract methods confirm_contract_execution, confirm_loan, confirm_partial_execution, init, on_agent_bankrupt, on_cash_transfer, on_contract_breached, on_contract_cancelled, on_contract_executed, on_contract_nullified, on_contract_signed, on_inventory_change, on_neg_request_accepted, on_neg_request_rejected, on_negotiation_failure, on_negotiation_success, on_new_cfp, on_new_report, on_production_failure, on_production_success, on_remove_cfp, respond_to_negotiation_request, respond_to_renegotiation_request, set_renegotiation_agenda, sign_contract, step
You are told that you cannot instantiate your newly created class as did not implement the abstract (required) methods. These abstract methods are useful in giving you an idea of all the callback you should expect.
If you want some default behavior implemented for you, you can inherit
from one of the provided factory managers like
DoNothingFactoryManager
and GreedyFactoryManager
. In this case,
you only need to override the functions you modify
from negmas.apps.scml import DoNothingFactoryManager
class MyFactoryManager(DoNothingFactoryManager):
"""My factory manager"""
As the
documentation
states, this function is called whenever your factory manager receives a
request from another agent to negotiate. You can either return None
if you do not want to accept this negotiation or create a Negotiator
that represents your agent in it.
Your do-nothing agent is almost ready. Let’s try it:
The property stats
in SCML2020World
gives you several statistics
about the world for every time-step of the simulation.
Let’s check the contracts of this world:
Let’s try to run a tournament with this do-nothing agent against the built-in greedy agent (in the “collusion” track setting):
from negmas.apps.scml.utils import anac2019_collusion
from negmas.apps.scml import GreedyFactoryManager
results = anac2019_collusion(competitors=(MyFactoryManager, GreedyFactoryManager)
, agent_names_reveal_type=True
, n_configs=2 # create 10 different configs
, max_worlds_per_config=4 # create no more then 4 worlds per config
, n_runs_per_world=1 # number of runs of each configured world
, n_steps=50 # we are running each world for 50 steps only
)
You can see the scores that each individual factory manager got (just a random sample):
results.scores.tail()
agent_name | agent_type | log_file | score | stats_folder | world | |
---|---|---|---|---|---|---|
75 | greedy@3_0 | greedy_factory_manager | /Users/yasser/code/projects/negmas/notebooks/t... | -0.00735 | /Users/yasser/code/projects/negmas/notebooks/t... | 20190524-225118nW5E00003 |
76 | greedy@3_1 | greedy_factory_manager | /Users/yasser/code/projects/negmas/notebooks/t... | -0.00627 | /Users/yasser/code/projects/negmas/notebooks/t... | 20190524-225118nW5E00003 |
77 | my@3_2 | my_factory_manager | /Users/yasser/code/projects/negmas/notebooks/t... | 0.00000 | /Users/yasser/code/projects/negmas/notebooks/t... | 20190524-225118nW5E00003 |
78 | greedy@3_3 | greedy_factory_manager | /Users/yasser/code/projects/negmas/notebooks/t... | -0.00643 | /Users/yasser/code/projects/negmas/notebooks/t... | 20190524-225118nW5E00003 |
79 | my@4_0 | my_factory_manager | /Users/yasser/code/projects/negmas/notebooks/t... | 0.00000 | /Users/yasser/code/projects/negmas/notebooks/t... | 20190524-225118nW5E00003 |
You can also check the total scores for each factory manager type:
results.total_scores
agent_type | score | |
---|---|---|
0 | greedy_factory_manager | 0.010756 |
1 | my_factory_manager | 0.000000 |
If you want, you can check if these differences are statistically significant using a t-test:
results.ttest
a | b | p | t | |
---|---|---|---|---|
0 | greedy_factory_manager | my_factory_manager | 0.33257 | 0.975009 |
So the greedy factory manager is slightly better than the do-nothing factory manager for this short simulation getting an average gain of 1.1% compared with nothing (0%) for the do-nothing factory manager (as expected). Moreover, this difference is not statistically significant as the p-value is 0.333 > 0.05. If you try running this this tournament for less than 20, the greedy factory manager will most likely lose money. In the actual league, we will run each world simulation between 50 and 100 steps (more toward the later).
You can just check the winner(s) list
results.winners
['greedy_factory_manager']
and what was its/their score:
print(results.winners_scores)
[0.0107562]
To run a tournament in the “standard”/“sabotage” track settings, use “anac2019_std”/“anac2019_sabotage” instead of “anac2019_collusion”.
This information and much more is also stored in a log folder that gives
details of every world and total scores, etc. The default location of
this log folder is under negmas/logs/tournaments in your HOME directory
(this can be changed by passing a tournament_path
to the
anac2019_tournamet
function.
The information stored in this folder is:
File/Folder Name |
Format |
Content |
---|---|---|
base_configs |
FOLDER |
Contains one json file for each configuration tried during the tournament before assigning agents to factories. |
assigned_configs |
FOLDER |
Contains one json
file for each
configuration tried
during the tournament
after assigning
agents to factories.
You can re-run this
world using
|
params.json |
JSON |
The parameters used to create this tournament |
scores.csv |
CSV |
Scores of every agent in every world |
total_scores.csv |
CSV |
Scores of every agent type averaged over all runs |
winners.csv |
CSV |
Winner types and their average scores |
ttest.csv |
CSV |
Results of a factorial TTEST comparing the performance of all agent types |
Other than these files, a folder with the same number as the corresponding config file in the configs folder, keeps full statistics/log of every world with the following contents:
File Name |
Format |
Content |
---|---|---|
all_contracts.csv |
CSV |
A record of all contracts |
contracts_full_info.c sv |
CSV |
A record of all contracts with added information about the CFPs |
cancelled_contracts.c sv |
CSV |
Contracts that were cancelled because one partner refused to sign it |
signed_contracts.csv |
CSV |
Contracts that were actually signed |
negotiations.csv |
CSV |
A record of all negotiations |
breaches.csv |
CSV |
A record of all breaches |
stats.csv |
CSV |
Helpful statistics about the state of the world at every timestep (e.g. N. negotiations, N. Contracts Executed, etc) in CSV format |
stats.json |
JSON |
Helpful statistics about the state of the world at every timestep (e.g. N. negotiations, N. Contracts Executed, etc) in JSON format |
params.json |
JSON |
The arguments used to run the world |
logs.txt |
TXT |
A log file giving details of most important events during the simulation |
To develop a more useful agent, you will need to override one or more of
the available callbacks in FactroyManager
and use methods available
in the SCMLAWI
(SCML Agent SCML2020World Interface) to act in the
world in order to maximize your profit.
Most important callbacks:¶
The most important callbacks that your class is expected to override to be useful as a factory manager are the following:
init()
Called after the world is initialized, but before any simulation steps.step()
Called in the simulation loop. Simulates one step of the agent’s logic. You can use this call to be proactive.on_new_cfp()
Called whenever a new Call for Proposals (CFP) is published on the bulletin board. The agent can specify a condition (e.g., buy CFPs only) such that only those CFPs that satisfy this condition will trigger this callback. By default your agent will only receive CFPs about products that it can use for production or can produce. You can override that by changing theinsteresting_products
property of your agent (probably ininit()
). This callback can be used for implementing reactive behavior.on_cfp_removed()
Called whenever a CFP is removed from the bulletin board.on_negotiation_request_accepted()/on_negotiation_request_rejected()
Called when a negotiation request initiated by the agent is accepted/rejected.on_negotiation_success()
/on_negotiation_failure()
Called when a negotiation the agent is involved in terminates.sign_contract()
Called by the simulator when it is time to sign a contract. The agent can refuse to sign. By default, agents sign the contract.on_contract_signed()
/on_contract_canelled()
Called when a contract the agent is party to is signed/cancelled (contracts will be canceled if any of the partners party to it refused to sign it).on_production_failure()
Called whenever a production command scheduled by the agent cannot be executed (e.g. for lack of funds or need of input products).
More details¶
You can download a skeleton for developing your factory manager in either python or javahere.
For more details, refer to the detailed description of the SCM
world and the Agent
,
SCMLAgent
, and FactoryManager
documentation at NegMAS library
documentation
What can the agent do and know?¶
The agent can act by calling various methods of its awi
member
(Agent SCML2020World Interface). The most important of these are:
request_negotiation()
Requests a negotiation with another partnerregister_interest()
/unregister_interest
By default the agent will receiveon_*_cfp
callbacks only on products that its factory consumes or produces. To override this behavior, you can use these two methods of theawi
.register_cfp()
/remove_cfp()
Registers/removes a call for proposals indicating interest in buying/selling some product and giving the negotiation issues (e.g. deliver time, unit cost, quantity, penalty, signing delay).evaluate_insurance()
/buy_insurance()
Gets the insurance premium for some potential contract or buys oneexecute()
Executes an action in the world. The only supported actions are scheduling a production process to run at some future time-step, stopping (or canceling) a previously issued run command.
The agent can also access some useful information through its
awi
’s properties. Some of the most important such properties are:
state
The state of the factory giving its current storage, cash in wallet, and standing loans as well as all scheduled production commands.n_steps
SCML2020World simulation lengthcurrent_step
Current world simulation stepproducts
/processes
Information about products/processes defined in this world (these are also accessible through local properties of theFactoryManager
cfps
All calls for proposals currently published in the bulletin boardbreaches
All breaches currently published in the bulletin board
Participation in the ANAC 2019 SCM league¶
Now, you completed the development of your factory manager, tested it by running it in worlds and tournaments, what exactly should you do to participate in the SCM league @ ANAC 2019:
You need to submit the following items:
Names of all members of the team with their affiliations and email addresses
Either a single python file with the whole implementation of your agent with any supporting code or a zip file with a single folder containing your code. In the later case, you will need to indicate the class name of your factory manager. Any factory manager names are accepted except (Insurance, Bank, MFactoryManager, CFactoryManager).
A 2-pages academic report about your factory manager. Please check the submission website for details about this report.
That is it folks! You can now start developing your own factory manager. Have fun.
You can download a skeleton for developing your factory manager in either python or javahere.