{"data":{"kind":"file","path":"README.md","version_id":"twebym22nhhmlsr4652f7yk5","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2568,"modified_at":"2026-02-16T04:02:11.291000","content_hash":"6ddfb51b22be55b11b5e466a7d159bd6d5a9c954af6882d65801b7d0e054a8f8"},"entries":[],"content":"# Lewis-Signaling\n\n### Overview\n- **Environment ID**: `lewis-signaling`\n- **Short description**: Referential communication game where a sender conveys a target word using meaningless signals\n- **Tags**: multi-agent, multi-turn, cooperative\n\n### Task\n- **Type**: multi-turn\n- **Parser**: XMLParser (fields: answer)\n- **Rubric**: Score-based reward (0-1)\n\n### Description\n\nA [Lewis signaling game](https://en.wikipedia.org/wiki/Lewis_signaling_game) is a referential communication task where two agents must develop a shared language to communicate about objects. This environment implements a sender-receiver game:\n\n  - Players: 2 (Sender and Receiver)\n  - The Sender sees a target word (e.g., \"apple\") plus distractor words (e.g., \"banana\", \"cherry\")\n  - The Sender must communicate using only pre-defined \"alien words\" (meaningless signals like \"zorp\", \"blick\")\n  - The Receiver sees the Sender's message and a shuffled list of candidates, then must select the target\n  - Perfect score: 1.0 (receiver correctly identifies the target)\n\nThis tests emergent communication: whether two agents can develop and learn a shared language to successfully communicate referentially about objects, without any pre-established meaning for the signals.\n\n### Dependencies\n- `verifiers>=0.1.8`\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nuv run vf-eval lewis-signaling\n```\n\nConfigure model and sampling:\n\n```bash\nuv run vf-eval lewis-signaling -m gpt-4.1-mini -n 20 -r 3 -t 1024 -T 0.7 -a '{\"num_distractors\": 4}'\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `num_train_examples` | int | `1000` | Number of training examples (each with a unique seed) |\n| `num_eval_examples` | int | `100` | Number of evaluation examples |\n| `num_real_words` | int | `32` | Size of vocabulary for targets and distractors |\n| `num_alien_words` | int | `32` | Size of vocabulary for messages (alien words) |\n| `num_distractors` | int | `3` | Number of distractor words per game (creates N+1 way choice) |\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Communication success (1.0 if receiver selected the target word, 0.0 otherwise) |\n\n### Project Structure\n\n```\nlewis_signaling/\n├── config.py           # GameConfig dataclass with game parameters\n├── vocabulary.py       # Real words and alien words vocabularies\n├── prompt.py           # System prompt templates for sender and receiver\n└── lewis_signaling.py  # LewisSignalingEnv environment and reward function\n```\n","encoding":"utf-8","truncated":false,"total_bytes":2568},"status":null}