AAMAS Experiments

AAMAS Experiments

Use this before submission when the empirical or simulation story is not yet locked. At AAMAS the experiment exists to test the interaction claim, not to top a benchmark.

Experiment audit

• Map each empirical claim to a game, a self-play run, a population sweep, an ablation, or a deviation test.
• Choose opponents deliberately: self-play alone rarely suffices; include held-out opponents, population sets, or classical strategies as the claim requires.
• Separate simulations that validate a solution concept (where the equilibrium is known) from real or applied studies that show practical multiagent behavior.
• Report uncertainty for stochastic results over both seeds and opponents: standard errors, confidence intervals, or paired tests.
• Report the environment, number of agents, training regime, evaluation protocol, metrics, hyperparameter ranges, chosen settings, seeds, hardware, software versions, and runtime.
• Add ablations for the interaction mechanism (communication, reward sharing, the payment rule), not just cosmetic variants.
• Audit for the mismatch between the strategic claim and the setup: an equilibrium claim tested against only one fixed opponent, or a cooperation claim that hides a reward-shaping constant.

What experiments are for at this venue

• The strongest design shows the interaction under stress: agents that can deviate, opponents the method did not train against, and populations that vary in size or composition.
• One experiment that lets agents try to exploit the mechanism and fails to profit is worth more than five extra environments where nothing strategic is tested.
• Reviewers, often game theorists, check whether the metric matches the claim: convergence to a named solution concept, exploitability, social welfare, or regret - not just episodic return.

Interaction-validation design table