{"data":{"kind":"file","path":"README.md","version_id":"wjft4dpqeyy82vm2pg0fwyyu","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2614,"modified_at":"2026-11-05T09:25:40.058000","content_hash":"26348aa992a63f9921aa73d71eb3a732bd7cd951a282b0c1af18d6b100be2f41"},"entries":[],"content":"# chess-rl\n\nMVP RL environment: an LLM plays one side of a chess position against a weakened\nStockfish, for up to 20 moves, and is graded by a stronger Stockfish.\n\n| Phase | Status |\n|---|---|\n| 1. Framework-free core (`chess_rl/game.py`, Gym-style `reset`/`step`) | done |\n| 2. `verifiers` v1 wrapper + LLM evals via Prime Inference | done |\n| 3. Publish privately to the Environments Hub | next |\n| 4. Small GRPO training run | optional |\n\n## Layout\n\n```\nchess_rl/game.py      ChessGame: rules, Stockfish opponent + grader, rubric (no LLM code)\nchess_rl/tasks.py     seeded task generator -> chess_rl/tasks.jsonl (50 balanced middlegames)\nchess_rl/taskset.py   verifiers wrapper: ChessTaskset + ChessGameEnv (env plays the \"user\")\ncheck_env.py          self-check for the core (no model calls)\nsummarize.py          per-model reward breakdown from outputs/*/traces.jsonl\n```\n\n## Setup\n\n```bash\nbrew install stockfish\nuv sync\nuv run python check_env.py                    # core self-check, free\nuv run python -m chess_rl.tasks 50 0          # regenerate tasks (deterministic)\n```\n\n## Running evals\n\n`.env` holds `PRIME_API_KEY`. Credits live on the team wallet, so requests must carry the team id:\n\n```bash\nset -a && source .env && set +a\nexport PRIME_TEAM_ID=<your-team-id>                      # from `prime teams list`\nuv run vf-run chess-rl -m openai/gpt-oss-20b -n 6 -r 1   # 6 positions, 1 game each\nuv run python summarize.py\nprime wallet                                              # check spend\n```\n\nKnobs: `--env.max-agent-moves`, `--env.opponent-depth`, `--env.opponent-random-move-prob`,\n`--env.grader-depth`, `--env.player.sampling.max-tokens`.\n\n## Rubric\n\n`reward = outcome + move_quality + illegal_move_penalty`\n\n- `outcome`: 1 / 0.5 / 0 if the game ended, else Stockfish's expected score for the agent\n- `move_quality`: harmonic mean of Lichess per-move accuracy (one blunder drags it down)\n- `illegal_move_penalty`: -0.1 per illegal / unparseable / cut-off reply (episode ends at 5)\n- Metrics (weight 0): `start_win_probability`, `agent_moves`, `illegal_moves`\n\n## Known gaps\n\n- `vf-run`'s per-rollout log line prints `reward=0.000`: rewards are recorded after the\n  conversation closes. The saved traces and the run summary are correct.\n- Not yet guarded: forcing repetition draws (0.5) from lost positions; `outcome` is\n  absolute, not improvement over `start_win_probability`.\n- `move_quality` is noisy at 6 episodes and does not rank models monotonically yet.\n- Claude models are billed at full input price on Prime (no cache discount listed);\n  one 6-game Sonnet 5.5 run cost about $2.78.\n","encoding":"utf-8","truncated":false,"total_bytes":2614},"status":null}