{"data":{"kind":"file","path":"README.md","version_id":"uuz9frv8l704r2hzzj584jjf","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1577,"modified_at":"2026-01-23T10:03:44.658000","content_hash":"0922ff88973eff5ebd38d51ea5738904b74a4f6a53bb4e02e054860da9f419ee"},"entries":[],"content":"# countdown\n\n### Overview\n- **Environment ID**: `sami/countdown`\n- **Short description**: Countdown Numbers Game - reach a target using arithmetic operations\n- **Tags**: single-turn, math, reasoning, train, eval\n\n### Datasets\n- **Primary dataset(s)**: Procedurally generated arithmetic puzzles\n- **Source links**: Generated at runtime\n- **Split sizes**: Configurable (default: 200 train, 30 eval)\n\n### Task\n- **Type**: single-turn\n- **Parser**: XMLParser with fields `[\"reasoning\", \"solution\"]`\n- **Rubric overview**: Rewards exact solutions, closeness to target, and proper XML formatting\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nprime eval run sami/countdown\n```\n\nConfigure model and sampling:\n\n```bash\nprime eval run sami/countdown \\\n  -m qwen/qwen3-8b \\\n  -n 20 -r 3 -t 1024 -T 0.7 \\\n  -a '{\"num_examples\": 100, \"ensure_solvable\": false}'\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `num_examples` | int | `200` | Number of training puzzles to generate |\n| `eval_examples` | int | `30` | Number of evaluation puzzles |\n| `ensure_solvable` | bool | `True` | Only generate puzzles with known solutions |\n| `seed` | int | `42` | Random seed for reproducibility |\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Weighted sum of all reward functions |\n| `exact_match_reward` | 1.0 if solution reaches target exactly |\n| `closeness_reward` | Exponential decay based on distance from target |\n| `format_reward` | Proper XML formatting with reasoning/solution tags |\n","encoding":"utf-8","truncated":false,"total_bytes":1577},"status":null}