{"data":{"kind":"file","path":"README.md","version_id":"c870yhz4gq4g86zyx321c81u","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5239,"modified_at":"2026-08-07T07:00:35.460000","content_hash":"80d461d084b6aa3d987c59d54c2146dd2667b3abaf65167a102d4f242321d0a7"},"entries":[],"content":"# textarena-env\n\nRL environment in which the model **plays a single-player text game to completion** and the\ngame's own verdict is the reward. Built on\n[TextArena](https://github.com/TextArena/TextArena).\n\n- **Tasks:** 35 seeded games across 7 titles (deduction, word, planning, constraint)\n- **Dataset:** [`textarena-tasks-v1`](https://huggingface.co/datasets/eltociear/textarena-tasks-v1)\n- **Reward:** the game's verdict — 1.0 for a win, 0.0 otherwise. No LLM judge.\n- **Dependencies in the sandbox:** `textarena` only. **No Docker sandbox is used at all.**\n\n```bash\nprime env install eltociear/textarena-env\n```\n\n## The one environment here whose full loop is actually executed\n\nThe nineteen sibling environments run the model's code in a Docker sandbox, and that transport\ncannot be exercised without a Docker runtime. This one has no sandbox: TextArena is pure Python\nand the game runs in process, so a rollout is `setup_state` → `env_response` ×N →\n`is_completed` → rubric, and a scripted policy can drive the whole thing with no model and no\ncontainer.\n\n`test_play.py` does exactly that, with three policies, because one is not enough to trust a\nreward:\n\n| Policy | What it does | Must score |\n|---|---|---|\n| **perfect** | reads the game's hidden state and plays the winning move | **1.0** |\n| **hopeless** | a legal-looking but wrong action every turn until turns run out | **0.0** |\n| **garbage** | text the game cannot parse at all | **0.0**, *and* set `made_invalid_move` |\n\nThe third exists because a reward that cannot separate \"played badly\" from \"never played\" is\nnot measuring the game. **3/3 games pass.**\n\n## Seeds, not answers\n\nThere is no answer key, and there must not be one. A fixed key for Wordle or Mastermind *is*\nthe answer to the puzzle — grading against it would reward reciting a secret rather than\ndeducing it. What is fixed is the **seed**, which fixes the hidden word, the secret code and\nthe board. `env.reset(num_players=1, seed=N)` fully determines the game state, and\n`build_tasks.py --verify` asserts that for every row: same seed → same state, twice over.\n\nTower of Hanoi is reported as seed-invariant, which is correct rather than a bug — its starting\nposition is fixed by the puzzle.\n\n## The reward is clamped, on purpose\n\n`env.close()` returns the game's rewards, and some TextArena games return **-1** for a loss. A\nnegative loss reward would make *\"run out of turns without ever submitting a valid move\"* score\nhigher than *\"play and lose\"* — exactly the behaviour an RL environment must not teach. So the\nreward is clamped to 1.0 for a win and 0.0 otherwise.\n\n`made_invalid_move` and `turns_used` are recorded at **weight 0**. A malformed action is already\npunished by the turn it wastes; weighting it again would grade formatting twice.\n\n## Two things recorded rather than hidden\n\n- **`Wordle-v0` is deliberately excluded.** It raises `UnicodeDecodeError` here because\n  TextArena opens its word list without specifying an encoding and Windows falls back to the\n  system codepage. It would almost certainly work on Linux — but \"almost certainly\" is not\n  verified, and this project does not ship what it has not run.\n- **`import textarena` pulls in NLTK**, so every game needs the `words` corpus — a network fetch\n  on first use. It took three wrong measurements to establish that (watching stdout for the\n  download reports the truth exactly once, on an uncached machine; checking `sys.modules`\n  in-process poisons every game after the first, and the boolean written to compensate\n  collapsed into a tautology). `setup_state` pre-fetches the corpus so a rollout never stalls\n  mid-game.\n\n## Environment arguments\n\n| Argument | Default | Meaning |\n|---|---|---|\n| `split` | `'train'` | Dataset split to load. |\n| `dataset_name` | `'eltociear/textarena-tasks-v1'` | Hugging Face dataset of seeded games. |\n| `max_turns` | `30` | Turns before the rollout is cut off. Higher than the siblings because a game needs many. |\n| `**kwargs` | — | Passed through to `MultiTurnEnv`. |\n\n## Reward rubric\n\n| Reward function | Weight | What it returns |\n|---|---|---|\n| `game_reward` | 1.0 | 1.0 if the game says the player won, else 0.0. |\n| `made_invalid_move` | 0.0 | Informational: did the game ever reject an action as malformed? |\n| `turns_used` | 0.0 | Informational: how many turns the game took. |\n\n## Dependencies\n\n`datasets>=4.1.0`, `textarena>=0.7.0`, `verifiers>=0.1.8`\n\n## Verify it yourself\n\n```bash\npython environments/textarena_env/build_tasks.py --verify   # 35/35 seeded games reproduce\npython environments/textarena_env/test_play.py              # 3/3 games played end-to-end\n```\n\n## Sample `vf-eval` usage\n\n```bash\nuv run vf-install textarena-env\nuv run vf-eval -s textarena-env -m gpt-4.1 -n 5 -r 3\nuv run vf-tui                      # inspect the outputs/ folder it writes\n```\n\n## Known limitation\n\nUnlike its siblings there is **no unexercised sandbox transport** — this environment does not\nuse one. What has not been run is a rollout driven by an actual model rather than a scripted\npolicy, which needs an inference provider. The game loop, the reward, and the separation\nbetween a win, a loss and a malformed action are all verified by `test_play.py`.\n","encoding":"utf-8","truncated":false,"total_bytes":5239},"status":null}