{"data":{"kind":"file","path":"README.md","version_id":"fsqvto1hi2owpuli9fqe6ir8","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2301,"modified_at":"2026-03-03T23:23:17.746000","content_hash":"4c335c157015f5bb7ad31149022d35bedb53199ea9158ac3a3c674cff074018e"},"entries":[],"content":"# Word Game — Wordle (MultiTurnEnv + partial rewards)\n\nA Wordle-style 5-letter word guessing game using `vf.MultiTurnEnv`.  \nThis is the primary recipe for learning the multi-turn environment pattern.\n\n## How it works\n\n1. The model receives the system prompt explaining Wordle rules.\n2. Each turn: model guesses a 5-letter word.\n3. Environment responds with letter feedback: `G B Y G B`\n4. Model has 6 attempts to guess correctly.\n5. Reward is higher for fewer guesses needed:\n\n| Guesses needed | Reward |\n|----------------|--------|\n| 1 | 1.00 |\n| 2 | 0.90 |\n| 3 | 0.75 |\n| 4 | 0.60 |\n| 5 | 0.45 |\n| 6 | 0.30 |\n| Failed | 0.00 |\n\n## Setup\n\n```bash\npip install verifiers>=0.1.10\n```\n\n## Quick eval\n\n```python\nfrom lab_cookbook.recipes.word_game.word_game import load_environment\nimport verifiers as vf\n\nenv = load_environment(num_examples=20)\nvf.evaluate(env, model=\"gpt-4.1-mini\", rollouts_per_example=4)\n```\n\n## Training run\n\n```bash\nprime rl run config.toml\n```\n\n## Key patterns demonstrated\n\n### Custom MultiTurnEnv subclass\n```python\nclass WordleEnv(vf.MultiTurnEnv):\n    def setup_state(self, state):    # per-rollout init\n        state[\"secret\"] = ...\n\n    def env_response(self, messages, state):  # game logic\n        return feedback_string, state\n\n    @vf.stop\n    def stop_on_solved(self, messages, state):  # custom termination\n        return state[\"solved\"]\n\n    async def reward(self, completion, answer, state, **kwargs):\n        return partial_score\n```\n\n### Feedback logic\nThe `_compute_feedback(guess, secret)` function correctly handles:\n- Duplicate letters (counts available letters before awarding Y)\n- All-correct (GGGGG) \n- No matches (BBBBB)\n\n## Expected training behaviour\n\n- Starting reward: ~0.15 (random baseline — occasionally guesses correctly)\n- After 50 steps: ~0.35 (model learns to use feedback)\n- After 150 steps: ~0.50-0.60 (model develops Wordle strategy)\n- The model should learn to: use process of elimination, avoid already-ruled-out letters, prioritise common letters early\n\n## Adapting this pattern\n\nThe `MultiTurnEnv` + `setup_state` + `env_response` + `@vf.stop` pattern works for any  \nturn-based game or multi-step interactive task:\n- 20 Questions (yes/no narrowing)\n- Text adventures\n- Negotiation dialogues\n- Multi-step form filling with validation\n","encoding":"utf-8","truncated":false,"total_bytes":2301},"status":null}