{"data":{"kind":"file","path":"README.md","version_id":"n1znq10hih3itq5rxj08vgns","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1956,"modified_at":"2025-09-08T02:11:25.064000","content_hash":"4e71e40ca5daf1bd37bee1881ec91938182ecdceeea251251fbfbc300c0e91f7"},"entries":[],"content":"# hangman\r\n\r\n\r\n### Overview\r\n- **Environment ID**: `hangman`\r\n  - **Short description**: A simple hangman game with reward for completion and correct guesses, as well as a penalty for wrong guesses\r\n- **Tags**: games, multi-turn, wordle, xml, feedback\r\n\r\n### Datasets\r\n- **Primary dataset(s)**: TextArena `Hangman-v0` (environment provides episodes)\r\n- **Source links**: (Hangman By TextArena)[https://www.textarena.ai/environments/hangman]\r\n- **Split sizes**: Number of episodes controlled via args\r\n\r\n### Task\r\n- **Type**: Multi-Turn\r\n- **Parser**: XMLParser\r\n- **Rubric overview**: Exact guess, Correct letter guesses, Penalty for wrong guess, Reward for guessing quickly\r\n\r\n### Quickstart\r\nRun an evaluation with default settings:\r\n\r\n```bash\r\nuv run vf-eval hangman\r\n```\r\n\r\nConfigure model and sampling:\r\n\r\n```bash\r\nuv run vf-eval hangman   -m gpt-4.1-mini   -n 20 -r 3 -t 1024 -T 0.7   -a '{\"key\": \"value\"}'  # env-specific args as JSON\r\n```\r\n\r\nNotes:\r\n- Use `-a` / `--env-args` to pass environment-specific configuration as a JSON object.\r\n\r\n### Environment Arguments\r\n\r\nThis documents the supported arguments for the `Hangman-v0` environment.\r\n\r\n| Arg | Type | Default | Description |\r\n| --- | ---- | ------- | ----------- |\r\n| `hardcore` | bool | `False` | If `True`, the game uses a larger, more difficult vocabulary. If `False`, it uses a smaller, more common word list. |\r\n\r\n---\r\n\r\n### Metrics\r\n\r\nThis summarizes the key metrics our rubric emits for the Hangman game and how to interpret them.\r\n\r\n| Metric | Meaning |\r\n| ------ | ------- |\r\n| `final_outcome_reward_func` | A binary score indicating the final game outcome. **1.0** for a win, **0.0** for a loss. |\r\n| `progress_reward_func` | A dense, per-turn score that rewards revealing letters (**+0.2** per letter) and penalizes wrong guesses (**-0.1**). |\r\n| `efficiency_reward_func` | A score that rewards solving the puzzle faster. Calculated as `1 / num_turns` only if the game is won. |","encoding":"utf-8","truncated":false,"total_bytes":1956},"status":null}