{"data":{"kind":"file","path":"README.md","version_id":"bt25w2ulq2ic3n61y4f028am","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2563,"modified_at":"2026-03-03T23:23:17.745000","content_hash":"da3cf77918312496bb5cbfddd12bd19afc6ab7332de93ba953bc50fba4b7a30a"},"entries":[],"content":"# Multi-Environment (EnvGroup — math + tools + word game)\n\nCombines three distinct environments into a single training run using `vf.EnvGroup`.  \nThis is the recipe to use when you want a model to generalise across multiple capability dimensions.\n\n## Sub-environments\n\n| Environment | Type | Task | Starting Reward |\n|-------------|------|------|----------------|\n| `math_rl` | SingleTurnEnv | Arithmetic in `\\boxed{}` | ~0.70 |\n| `tool_use` | ToolEnv | 4 stateless tools | ~0.30 |\n| `word_game` | MultiTurnEnv | Wordle (6 guesses) | ~0.15 |\n\nCombined starting reward: **~0.38** (weighted average)\n\n## Setup\n\n```bash\npip install verifiers>=0.1.10\n```\n\n## Quick eval\n\n```python\nfrom lab_cookbook.recipes.multi_env.multi_env import load_environment\nimport verifiers as vf\n\nenv = load_environment(math_examples=50, tools_examples=50, word_examples=50)\nvf.evaluate(env, model=\"gpt-4.1-mini\", rollouts_per_example=4)\n```\n\n## Training run\n\n```bash\nprime rl run config.toml\n```\n\n## How vf.EnvGroup works\n\n```python\nvf.EnvGroup(\n    envs=[math_env, tools_env, word_env],\n    env_names=[\"math\", \"tools\", \"word\"],\n)\n```\n\n- Each batch interleaves examples from all environments\n- Each environment uses its own reward function\n- The combined reward signal is normalized per-environment before advantage computation\n- Sub-environment metrics are logged separately (e.g., `math/mean_reward`, `tools/mean_reward`)\n\n## Why multi-environment training?\n\n**Generalization**: A model trained on only math tends to lose general tool-use ability.  \nEnvGroup prevents reward collapse to any single task.\n\n**Capability balance**: Harder environments (word game) benefit from the momentum of  \neasier ones (math) during early training.\n\n**Efficiency**: One training run produces a model capable of multiple behaviors,  \nrather than requiring separate fine-tuning runs.\n\n## Scaling up\n\nTo add a fourth environment:\n\n```python\nfrom lab_cookbook.recipes.document_search import document_search_l1\n\ndoc_env = document_search_l1.load_environment(num_examples=200)\n\nvf.EnvGroup(\n    envs=[math_env, tools_env, word_env, doc_env],\n    env_names=[\"math\", \"tools\", \"word\", \"docs\"],\n)\n```\n\n## Tips\n\n- **Balance example counts**: Equal counts per environment (~200 each) work well for similar-sized environments\n- **Match difficulty**: Avoid one environment dominating with very high or very low reward\n- **Monitor per-env metrics**: If one environment's reward drops, investigate that sub-environment\n- **Start reward target**: Aim for 0.25-0.45 across the combined group for good training signal\n","encoding":"utf-8","truncated":false,"total_bytes":2563},"status":null}