Develop a factory manager (agent) for the SCM world

This tutorial describes how to develop an agent for the SCM world, test it, and submit it to the ANAC 2019 SCM league.

The first step is to install negmas

pip install negmas

Once you have this library installed, you can start developing your factory manager:

from negmas.apps.scml import FactoryManager
try:
    class MyFactoryManager(FactoryManager):
        """My factory manager"""
    f = MyFactoryManager()
except Exception as e:
    print(e)
Can't instantiate abstract class MyFactoryManager with abstract methods confirm_contract_execution, confirm_loan, confirm_partial_execution, init, on_agent_bankrupt, on_cash_transfer, on_contract_breached, on_contract_cancelled, on_contract_executed, on_contract_nullified, on_contract_signed, on_inventory_change, on_neg_request_accepted, on_neg_request_rejected, on_negotiation_failure, on_negotiation_success, on_new_cfp, on_new_report, on_production_failure, on_production_success, on_remove_cfp, respond_to_negotiation_request, respond_to_renegotiation_request, set_renegotiation_agenda, sign_contract, step

You are told that you cannot instantiate your newly created class as did not implement the abstract (required) methods. These abstract methods are useful in giving you an idea of all the callback you should expect.

If you want some default behavior implemented for you, you can inherit from one of the provided factory managers like DoNothingFactoryManager and GreedyFactoryManager. In this case, you only need to override the functions you modify

from negmas.apps.scml import DoNothingFactoryManager
class MyFactoryManager(DoNothingFactoryManager):
    """My factory manager"""

As the documentation states, this function is called whenever your factory manager receives a request from another agent to negotiate. You can either return None if you do not want to accept this negotiation or create a Negotiator that represents your agent in it.

Your do-nothing agent is almost ready. Let’s try it:

The property stats in SCML2020World gives you several statistics about the world for every time-step of the simulation.

Let’s check the contracts of this world:

Let’s try to run a tournament with this do-nothing agent against the built-in greedy agent (in the “collusion” track setting):

from negmas.apps.scml.utils import anac2019_collusion
from negmas.apps.scml import GreedyFactoryManager

results = anac2019_collusion(competitors=(MyFactoryManager, GreedyFactoryManager)
                              , agent_names_reveal_type=True
                              , n_configs=2        # create 10 different configs
                              , max_worlds_per_config=4 # create no more then 4 worlds per config
                              , n_runs_per_world=1 # number of runs of each configured world
                              , n_steps=50              # we are running each world for 50 steps only
                             )

You can see the scores that each individual factory manager got (just a random sample):

results.scores.tail()
agent_name agent_type log_file score stats_folder world
75 greedy@3_0 greedy_factory_manager /Users/yasser/code/projects/negmas/notebooks/t... -0.00735 /Users/yasser/code/projects/negmas/notebooks/t... 20190524-225118nW5E00003
76 greedy@3_1 greedy_factory_manager /Users/yasser/code/projects/negmas/notebooks/t... -0.00627 /Users/yasser/code/projects/negmas/notebooks/t... 20190524-225118nW5E00003
77 my@3_2 my_factory_manager /Users/yasser/code/projects/negmas/notebooks/t... 0.00000 /Users/yasser/code/projects/negmas/notebooks/t... 20190524-225118nW5E00003
78 greedy@3_3 greedy_factory_manager /Users/yasser/code/projects/negmas/notebooks/t... -0.00643 /Users/yasser/code/projects/negmas/notebooks/t... 20190524-225118nW5E00003
79 my@4_0 my_factory_manager /Users/yasser/code/projects/negmas/notebooks/t... 0.00000 /Users/yasser/code/projects/negmas/notebooks/t... 20190524-225118nW5E00003

You can also check the total scores for each factory manager type:

results.total_scores
agent_type score
0 greedy_factory_manager 0.010756
1 my_factory_manager 0.000000

If you want, you can check if these differences are statistically significant using a t-test:

results.ttest
a b p t
0 greedy_factory_manager my_factory_manager 0.33257 0.975009

So the greedy factory manager is slightly better than the do-nothing factory manager for this short simulation getting an average gain of 1.1% compared with nothing (0%) for the do-nothing factory manager (as expected). Moreover, this difference is not statistically significant as the p-value is 0.333 > 0.05. If you try running this this tournament for less than 20, the greedy factory manager will most likely lose money. In the actual league, we will run each world simulation between 50 and 100 steps (more toward the later).

You can just check the winner(s) list

results.winners
['greedy_factory_manager']

and what was its/their score:

print(results.winners_scores)
[0.0107562]

To run a tournament in the “standard”/“sabotage” track settings, use “anac2019_std”/“anac2019_sabotage” instead of “anac2019_collusion”.

This information and much more is also stored in a log folder that gives details of every world and total scores, etc. The default location of this log folder is under negmas/logs/tournaments in your HOME directory (this can be changed by passing a tournament_path to the anac2019_tournamet function.

The information stored in this folder is:

File/Folder Name

Format

Content

base_configs

FOLDER

Contains one json file for each configuration tried during the tournament before assigning agents to factories.

assigned_configs

FOLDER

Contains one json file for each configuration tried during the tournament after assigning agents to factories. You can re-run this world using run_world function in the tournament module.

params.json

JSON

The parameters used to create this tournament

scores.csv

CSV

Scores of every agent in every world

total_scores.csv

CSV

Scores of every agent type averaged over all runs

winners.csv

CSV

Winner types and their average scores

ttest.csv

CSV

Results of a factorial TTEST comparing the performance of all agent types

Other than these files, a folder with the same number as the corresponding config file in the configs folder, keeps full statistics/log of every world with the following contents:

File Name

Format

Content

all_contracts.csv

CSV

A record of all contracts

contracts_full_info.c sv

CSV

A record of all contracts with added information about the CFPs

cancelled_contracts.c sv

CSV

Contracts that were cancelled because one partner refused to sign it

signed_contracts.csv

CSV

Contracts that were actually signed

negotiations.csv

CSV

A record of all negotiations

breaches.csv

CSV

A record of all breaches

stats.csv

CSV

Helpful statistics about the state of the world at every timestep (e.g. N. negotiations, N. Contracts Executed, etc) in CSV format

stats.json

JSON

Helpful statistics about the state of the world at every timestep (e.g. N. negotiations, N. Contracts Executed, etc) in JSON format

params.json

JSON

The arguments used to run the world

logs.txt

TXT

A log file giving details of most important events during the simulation

To develop a more useful agent, you will need to override one or more of the available callbacks in FactroyManager and use methods available in the SCMLAWI (SCML Agent SCML2020World Interface) to act in the world in order to maximize your profit.

Most important callbacks:

The most important callbacks that your class is expected to override to be useful as a factory manager are the following:

  • init() Called after the world is initialized, but before any simulation steps.

  • step() Called in the simulation loop. Simulates one step of the agent’s logic. You can use this call to be proactive.

  • on_new_cfp() Called whenever a new Call for Proposals (CFP) is published on the bulletin board. The agent can specify a condition (e.g., buy CFPs only) such that only those CFPs that satisfy this condition will trigger this callback. By default your agent will only receive CFPs about products that it can use for production or can produce. You can override that by changing the insteresting_products property of your agent (probably in init()). This callback can be used for implementing reactive behavior.

  • on_cfp_removed() Called whenever a CFP is removed from the bulletin board.

  • on_negotiation_request_accepted()/on_negotiation_request_rejected() Called when a negotiation request initiated by the agent is accepted/rejected.

  • on_negotiation_success()/on_negotiation_failure() Called when a negotiation the agent is involved in terminates.

  • sign_contract() Called by the simulator when it is time to sign a contract. The agent can refuse to sign. By default, agents sign the contract.

  • on_contract_signed()/on_contract_canelled() Called when a contract the agent is party to is signed/cancelled (contracts will be canceled if any of the partners party to it refused to sign it).

  • on_production_failure() Called whenever a production command scheduled by the agent cannot be executed (e.g. for lack of funds or need of input products).

More details

You can download a skeleton for developing your factory manager in either python or javahere.

For more details, refer to the detailed description of the SCM world and the Agent, SCMLAgent, and FactoryManager documentation at NegMAS library documentation

What can the agent do and know?

The agent can act by calling various methods of its awi member (Agent SCML2020World Interface). The most important of these are:

  • request_negotiation() Requests a negotiation with another partner

  • register_interest() / unregister_interest By default the agent will receive on_*_cfp callbacks only on products that its factory consumes or produces. To override this behavior, you can use these two methods of the awi.

  • register_cfp() / remove_cfp() Registers/removes a call for proposals indicating interest in buying/selling some product and giving the negotiation issues (e.g. deliver time, unit cost, quantity, penalty, signing delay).

  • evaluate_insurance() / buy_insurance() Gets the insurance premium for some potential contract or buys one

  • execute() Executes an action in the world. The only supported actions are scheduling a production process to run at some future time-step, stopping (or canceling) a previously issued run command.

The agent can also access some useful information through its awi’s properties. Some of the most important such properties are:

  • state The state of the factory giving its current storage, cash in wallet, and standing loans as well as all scheduled production commands.

  • n_steps SCML2020World simulation length

  • current_step Current world simulation step

  • products/processes Information about products/processes defined in this world (these are also accessible through local properties of the FactoryManager

  • cfps All calls for proposals currently published in the bulletin board

  • breaches All breaches currently published in the bulletin board

Participation in the ANAC 2019 SCM league

Now, you completed the development of your factory manager, tested it by running it in worlds and tournaments, what exactly should you do to participate in the SCM league @ ANAC 2019:

You need to submit the following items:

  • Names of all members of the team with their affiliations and email addresses

  • Either a single python file with the whole implementation of your agent with any supporting code or a zip file with a single folder containing your code. In the later case, you will need to indicate the class name of your factory manager. Any factory manager names are accepted except (Insurance, Bank, MFactoryManager, CFactoryManager).

  • A 2-pages academic report about your factory manager. Please check the submission website for details about this report.

That is it folks! You can now start developing your own factory manager. Have fun.

You can download a skeleton for developing your factory manager in either python or javahere.

More Information

For more information, please refer to the links in the CFP

Download Notebook.