{"data":{"kind":"file","path":"README.md","version_id":"fj2jjur4lam165s3o4nwvblj","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":7340,"modified_at":"2026-10-24T20:56:46.783000","content_hash":"269802b7b20cbd073b64f579beec427305075251d12a349410e98dc2a639a9c6"},"entries":[],"content":"# the-traitors\n\n[![⭐ OpenReward Environment](https://img.shields.io/badge/%E2%AD%90%20OpenReward-Environment-f7e6cc)](https://openreward.ai/RJT1990/TheTraitors)\n\n## Description\n\nThe Traitors is a social-deduction environment based on the UK TV show \"The Traitors.\" The agent plays one of 22 contestants, either a Faithful or a Traitor, in a multi-round game of deception, discussion and elimination. The other 21 players are NPCs driven by an OpenAI chat model (default `gpt-5.4-mini`), each with a persistent personality, memory, and suspicion and trust tracking.\n\nEach round follows the show's format: a night phase (the Traitors secretly murder a Faithful), breakfast (the murder is revealed), a team mission, a three-exchange roundtable discussion, and a banishment vote. Once five or fewer players remain, an endgame vote decides whether to end the game or banish again.\n\nThe Traitors was one of the environments in the [Avalon-M1](https://gr.inc/releases/avalon-m1) training run, where we were able to hill-climb it successfully.\n\n## Capabilities\n\n- Multi-agent social deduction with 22 players\n- Strategic discussion and accusation at the roundtable\n- Banishment voting with vote-pattern analysis\n- Murder target selection and private Traitor coordination (Traitor role)\n- Endgame decision-making under uncertainty\n\n## Quickstart\n\n```bash\nprime env install the-traitors\nuv run vf-eval the-traitors -m <model> -n 5\n```\n\nUse `--env.taskset.split test` for the test split. The `null` harness is pinned by default, so the model sees no tools and plays purely through its replies.\n\n## Tasks\n\nThere are 100 train tasks and 100 test tasks (`--env.taskset.split train|test`):\n\n- **train**: seeds 0–99; seeds 0–49 play a Faithful and 50–99 a Traitor.\n- **test**: seeds 10000–10099; seeds 10000–10049 play a Faithful and 10050–10099 a Traitor.\n\nEach task starts a fresh 22-player game (3 Traitors, 19 Faithfuls) whose names, personalities, roles and agent seat are fixed by the seed. A game ends when every Traitor is banished, when the Traitors reach parity with the Faithfuls, when an endgame vote is unanimous, after round 15, or when the agent is murdered or banished.\n\n## How the agent plays\n\nThe environment plays the user. It sends the role briefing plus everything that happens up to the agent's first decision. It then reads each reply as one action, advances the game (NPC discussion, votes, murders) and replies with the narrative up to the next decision.\n\nEach reply must end with exactly one action. The action tag goes at the start of its own line, reasoning may come before it, and everything after the tag is the argument:\n\n| Action | Phase | Effect |\n|--------|-------|--------|\n| `[speak] <message>` | Discussion | Make a statement at the roundtable |\n| `[speak @<name>] <message>` | Discussion | Address a player and get their immediate reply |\n| `[state]` | Any | View alive players, prize pot, elimination history and voting records |\n| `[vote] <name>` | Roundtable vote | Vote to banish a player |\n| `[murder] <name>` | Night (Traitor only) | Choose a Faithful to murder |\n| `[conclave] <message>` | Night (Traitor only) | Private discussion with the other Traitors |\n| `[endgame] end_game` / `[endgame] banish_again` | Endgame vote | End the game or banish again |\n\nA reply with no action or with more than one gets corrective feedback and earns no reward. After 3 consecutive invalid replies (`--env.max-invalid-actions`), the agent forfeits and keeps whatever reward it has already earned.\n\n## Reward Structure\n\nThe reward (`traitors_reward`) is summed over the game:\n\n- +1 for surviving each night's murder (the opening breakfast, before the agent has acted, is not rewarded)\n- +1 for surviving each banishment vote\n- +1 for being alive on the winning side when the game ends\n\nNo LLM grader is used: the reward comes straight from the game engine. Recorded metrics: `won`, `survived`, `rounds`, `actions`, `invalid_actions`, `forfeit`. The game summary (seed, role, winner) is stored in `trace.info[\"traitors\"]`.\n\n## Configuration\n\n| Flag | Default | Meaning |\n|------|---------|---------|\n| `--env.taskset.split` | `train` | `train` or `test` |\n| `--env.npc-model` | `gpt-5.4-mini` | Chat model driving the 21 NPCs |\n| `--env.npc-base-url` | `https://api.openai.com/v1` | NPC endpoint (pinned so that `OPENAI_BASE_URL` overrides aimed at the policy never reach the NPCs) |\n| `--env.npc-api-key-var` | `OPENAI_API_KEY` | Environment variable holding the NPC API key |\n| `--env.max-invalid-actions` | `3` | Consecutive invalid replies before a forfeit |\n\n## Other Environment Requirements\n\nThe NPCs need an OpenAI API key, read from `OPENAI_API_KEY` by default. For hosted evaluations and training, add it as an environment secret (`prime env secret create <owner>/the-traitors --name OPENAI_API_KEY --value sk-...`) or link a global secret. Every agent action can trigger several NPC calls (discussion, votes, round summaries), so a full game makes many NPC requests.\n\n## Learnt behaviour in Avalon-M1\n\nIn Avalon-M1 the agent is an organisation, in which a CEO plays the game and can hire specialists to advise it. These are highlights of the winning Traitor strategy it learnt.\n\n- **Core playbook.** Never lead. Don't be the first to name a suspect or declare a vote, speak briefly, show emotion about the murder and urge caution (\"I'd rather not banish someone on the basis of who shouted loudest\"). Vote with the majority so the voting record stays unremarkable (\"don't be the lone holdout, it's memorable\"). Don't defend fellow Traitors (\"a Traitor defending another Traitor too quickly is a tell\"), but quietly cast doubt if one starts attracting votes.\n- **Murder the sharp voices, frame the quiet ones.** Earlier wins murdered quiet but respected players, and the NPCs noticed the pattern. Later wins kill the vocal, trusted players the room is starting to listen to, then turn that pattern on a quiet Faithful (\"three rounds of near-silence while the loudest voices keep getting killed feels worth asking about\").\n- **Protection lists and shields.** Specialists name players never to vote against (\"Do not vote for Oliver or Georgia under any circumstances\") and Faithfuls to keep alive as shields (\"Charlotte is a useful shield for the Traitors at the moment\").\n- **Heavier coaching.** The organisation's message board in recent wins holds about 150–170 strategy sentences against 20–70 in earlier ones, as specialists brief the CEO every exchange with scripted lines, vote recommendations and the next murder target.\n\n## Safety\n\nAgents interact with a simulated social-deduction game. The environment does not present direct safety risks. Traitor agents are asked to deceive, but only within the bounds of the game. Training runs that include this environment should also include environments that reward honesty, so the agent learns that deception is specific to this game context.\n\n## License\n\nMIT\n\n## Citations\n\n```bibtex\n@misc{generalreasoning2026avalonm1,\n      title={Avalon-M1: Reinforcement Learning from Collective Rewards},\n      author={Iliyan Zarov and Henry Course and Anthony Hartshorn and Ran Achiron and Louis Roche and Akshit Sinha and Chengxi Taylor and Ross Taylor},\n      year={2026},\n      publisher={General Reasoning},\n      url={https://gr.inc/avalonm1.pdf},\n}\n```\n","encoding":"utf-8","truncated":false,"total_bytes":7340},"status":null}