{"data":{"kind":"file","path":"README.md","version_id":"isgi6u8ru1ndx73wtrok59jm","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3848,"modified_at":"2025-09-08T01:38:56.229000","content_hash":"61a5662e8747728a2a082660662064afc66bae7fe436eba82ee13f016adf7522"},"entries":[],"content":"# Axelrod Tournament Environment\n\nAn evaluation environment for the Iterated Prisoner’s Dilemma (IPD) based on Robert Axelrod’s tournament. It runs a model against a suite of fixed strategies across multiple rounds and aggregates performance.\n\n## Background: Axelrod’s Tournament\n\nRobert Axelrod’s tournament (1980s) invited strategies to play the Iterated Prisoner’s Dilemma repeatedly against each other. Each round, both players choose Cooperate (C) or Defect (D). Payoffs are asymmetric: a unilateral defector earns a high reward (T), mutual cooperation is moderately good (R), mutual defection is poor (P), and being defected on while cooperating is worst (S). A key finding was that simple contingent strategies like Tit‑for‑Tat performed strongly by encouraging cooperation while retaliating against defection.\n\nThis repository adapts that setup for model evaluation: the model plays multiple matches against a set of benchmark strategies, and we summarize its total score across opponents.\n\nPayoffs used here (standard Axelrod):\n- Mutual cooperation (C,C): R=3 each\n- Mutual defection (D,D): P=1 each\n- You defect, opponent cooperates (D, C): T=5 (you), S=0 (opponent)\n- You cooperate, opponent defects (C, D): S=0 (you), T=5 (opponent)\n\n## What This Environment Does\n\n- Runs a tournament of repeated IPD matches against classic strategies (Tit‑for‑Tat, Grudger, Always Defect/Cooperate, Random, etc.).\n- Tracks round outcomes and aggregates a final tournament score.\n- Optionally requests a final acknowledgement tag from the model (e.g., `<final fs=\"250\" tr=\"100\"/>`) so scoring is robust and deterministic.\n- Exposes configuration for rounds, strategies, verbosity, and tag behavior.\n\n## Running with vf‑eval\n\nPrerequisites:\n- `uv` and `vf-eval` available from this repo’s `pyproject.toml`.\n- `.env` contains your API key (e.g., `OPENAI_API_KEY=...`) or export it in your shell.\n\nExample run:\n\n```bash\nuv run --env-file .env \\\n  vf-eval axelrod_tournament \\\n  -m gpt-5-mini \\\n  -k OPENAI_API_KEY \\\n  -n 1 -r 1 -t 512\n```\n\nFlags:\n- `-m`: model name (backend configured in your environment)\n- `-k`: env var holding the API key\n- `-n`: number of training examples\n- `-r`: number of eval examples (runs)\n- `-t`: max tokens for model outputs\n\nThe environment conducts multi-turn interaction per scenario. At tournament end, it may prompt the model for a final tag; the reward parser extracts `fs` (final score) and `tr` (total rounds) from that tag.\n\n## Configuration (tournament_config.py)\n\n- `ROUNDS_PER_MATCH` (int): rounds per opponent.\n- `ACTIVE_STRATEGIES` (list[str]): enabled strategies. Add/remove to change coverage.\n- `MAX_HISTORY_MESSAGES` (int): trims message history to control token growth.\n- `USE_VERBOSE_RESPONSES` (bool): detailed vs compact feedback.\n- `REVEAL_OPPONENT` (bool): expose strategy names in text.\n- `ON_INVALID_MOVE` (\"repeat\" | \"defect\"): fallback when the model output is malformed/empty.\n- `REQUIRE_FINAL_ACK` (bool): if true, asks for a final tag at the end.\n- `FINAL_TAG_NAME` (str): name of the final tag (default `final`).\n- `MAX_TURNS` (int): hard cap on total turns per scenario.\n- `DEFAULT_TRAIN_EXAMPLES`, `DEFAULT_EVAL_EXAMPLES` (int): dataset sizes for vf‑eval.\n\n## Interpreting Results\n\n- The reward normalizes the final score by the theoretical maximum (`T` per round), producing a value in [0, 1].\n- Verbose mode includes per-opponent summaries; compact mode uses single-line updates.\n\n## Repository Structure\n\n- `axelrod_tournament.py`: tournament engine and evaluation environment.\n- `tournament_config.py`: configuration.\n- `pyproject.toml`, `uv.lock`: dependency and lock files.\n\n## Extending\n\n- Add new strategies in `TournamentStrategy.get_move` and include them in `ACTIVE_STRATEGIES`.\n- Adjust scoring or prompts by editing `R, P, T, S` or `build_system_prompt`.","encoding":"utf-8","truncated":false,"total_bytes":3848},"status":null}