{"data":{"kind":"file","path":"README.md","version_id":"on1fkgiwldx2amc8r0f5ig41","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":6051,"modified_at":"2026-08-07T06:30:03.694000","content_hash":"2ef241627c73f7bc162308a7736054b46b7a3bfb575221bc3599f5d9f1ab6ff2"},"entries":[],"content":"# twohop-env\r\n\r\nRL environment for **multi-hop composition** over a synthetic fact universe, on the setting of\r\n*\"Lessons from Studying Two-Hop Latent Reasoning\"* (Balesni et al.).\r\n\r\n- **Tasks:** 36 across 6 categories\r\n- **Dataset:** [`twohop-tasks-v1`](https://huggingface.co/datasets/eltociear/twohop-tasks-v1)\r\n- **Reward:** binary, from an exact comparison. No LLM judge, no network, no clock.\r\n- **Dependencies in the sandbox:** none.\r\n\r\n```bash\r\nprime env install eltociear/twohop-env\r\n```\r\n\r\n\r\n36 multi-hop reasoning tasks for the\r\n[`twohop-env`](https://app.primeintellect.ai/dashboard/environments/eltociear/twohop-env) RL\r\nenvironment, on the setting of *\"Lessons from Studying Two-Hop Latent Reasoning\"*\r\n(Balesni et al.).\r\n\r\n| field | meaning |\r\n|---|---|\r\n| `task_id` | `th-000` … `th-035` |\r\n| `category` | twohop / threehop / broken / distractor / reverse / aggregate |\r\n| `prompt` | the question and the exact shape of the answer |\r\n| `api_description` | the complete synthetic fact set and the answer-format rules |\r\n| `expected_output` | JSON `{\"rows\": [...]}`, **computed by executing a reference solution** |\r\n\r\nCategories: twohop 10, threehop 6, distractor 6, broken 5, aggregate 5, reverse 4.\r\nNo dependencies, no network.\r\n\r\n## What is and is not reproduced\r\n\r\nThe paper's headline experiments fine-tune models to study whether composition happens\r\n*latently* across separately-learned facts; those need GPUs. What is reproducible — and what an\r\nRL environment actually needs — is the **in-context** setting: a synthetic fact universe, a\r\nquestion requiring composition, an exact answer.\r\n\r\nEvery entity is invented, so no answer can come from pretraining.\r\n\r\n## The trap, and why the obvious check is not enough\r\n\r\nA question is only two-hop if **both hops are load-bearing**, and it is very easy to write one\r\nthat is not. Each multi-hop question is therefore verified twice:\r\n\r\n1. **Ablation** — remove each fact of the chain; the answer must become underivable.\r\n2. **Non-forcing** — some other organisation must lead somewhere else, so that knowing the\r\n   employer is what selects the answer.\r\n\r\nCheck 1 was written first and looked sufficient. **It is not.** Point every organisation at the\r\nsame city: the ablation still passes — deleting either fact leaves the resolver nothing to\r\nfollow — while a model can answer correctly having ignored the first hop entirely, because\r\nthere is only one city any employer could be in. A one-hop question in a two-hop costume, waved\r\nstraight through.\r\n\r\nCheck 1 proves the *resolver* reads both facts. Only check 2 says anything about what the\r\n**model** must do. That distinction was found by deliberately trying to defeat check 1, and it\r\nis the main thing this dataset gets right.\r\n\r\nA third guard is at the dataset level: the modal city must cover **fewer than half** the\r\nanswers, so a model that ignores the facts and always names the most common city scores badly.\r\nIt currently covers 3 of 11.\r\n\r\n## Declining is part of the task\r\n\r\nSome organisations have no recorded city and some cities have no recorded country, so five\r\nquestions have **no derivable answer** and the correct response is the literal word `unknown`.\r\nNaming a plausible-looking entity instead is exactly the failure a reasoning probe should\r\ncatch, and comparison is case-sensitive, so `Unknown` and `none` are wrong in the same way.\r\n\r\nVerify with `python environments/twohop_env/build_tasks.py --verify` (36/36).\r\nSource: <https://github.com/eltociear/my-molt-agent/tree/main/environments/twohop_env>\r\n\r\n## Verify it yourself\r\n\r\n```bash\r\npython environments/twohop_env/build_tasks.py --verify   # 36/36, 21 chains proved multi-hop\r\n```\r\n\r\n## Environment arguments\r\n\r\n`load_environment()` deliberately exposes very little: the program's guidance is that an\r\nenvironment should have one correct way to be run, so nothing about the prompts, the parsing or\r\nthe grading is configurable.\r\n\r\n| Argument | Default | Meaning |\r\n|---|---|---|\r\n| `split` | `'train'` | Dataset split to load. |\r\n| `dataset_name` | `'eltociear/twohop-tasks-v1'` | Hugging Face dataset of tasks. Change only to point at a fork. |\r\n| `max_turns` | `5` | Tool-use turns the model gets before the rollout ends. |\r\n| `**kwargs` | — | Passed through to the underlying `SandboxEnv`. |\r\n\r\n## Reward rubric\r\n\r\n| Reward function | Weight | What it returns |\r\n|---|---|---|\r\n| `correctness` | 1.0 | Binary: 1.0 when the answer matches the reference, else 0.0. |\r\n\r\nThere is no LLM judge and no partial credit. The score is computed on the host in\r\n`post_rollout` and read back by the rubric, so the reward is a deterministic function of the\r\nvalues the model left in `result`. A **harness** failure (dead sandbox, unreadable read-back)\r\nalso scores 0.0 but additionally sets `state[\"scoring_error\"]`, so an eval run can tell a\r\nbroken harness from a wrong answer instead of blaming the model.\r\n\r\n## Dependencies\r\n\r\n`datasets>=4.1.0`, `verifiers>=0.1.8`\r\n\r\n## Sample `vf-eval` usage\r\n\r\n```bash\r\nuv run vf-install twohop-env\r\nuv run vf-eval -s twohop-env -m gpt-4.1 -n 5 -r 3\r\nuv run vf-tui                      # inspect the outputs/ folder it writes\r\n```\r\n\r\n## Known limitation\r\n\r\nThe Docker **sandbox transport** has not been executed — `docker run`, `pip install`, and the\r\nmodel's code running inside the container — because that needs a Docker runtime and an\r\ninference provider key, neither available where this was authored.\r\n\r\nEverything else is exercised. `environments/test_scoring_path.py` runs this environment's\r\n`post_rollout` and `Rubric` with the sandbox mocked out, asserting that a correct answer scores\r\n1.0, a well-formed wrong answer scores 0.0, and a dead sandbox scores 0.0 *and* sets\r\n`scoring_error` so a harness failure is never mistaken for a bad model. It also checks that\r\nwhat `build_tasks.py` emits is exactly what the scorer expects. The environment mirrors the\r\nstructure of `polars_env` (already accepted into the Environments Program) and imports cleanly\r\nagainst `verifiers` 0.2.1.\r\n","encoding":"utf-8","truncated":false,"total_bytes":6051},"status":null}