{"data":{"kind":"file","path":"README.md","version_id":"zdt0kj3u4f4nv0nl51wm8kkw","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":9457,"modified_at":"2026-10-19T04:23:43.245000","content_hash":"1837f34e5841a5ae3ede51244b3099547e206cd8e07db773fd6ac0035d908c75"},"entries":[],"content":"# long-context-navigator\n\nMulti-hop question answering over a document far larger than the context the agent may hold.\nThe agent decides what to read, what to keep, what to compress and what to drop. It works under\na **hard working-context ceiling**, and any action that would exceed the ceiling ends the\nepisode as a failure. Compression is a constraint to satisfy, not a cost to trade off.\n\nThe question is whether an agent learns a compression policy that carries every bridge entity\nacross reads and generalises to deeper chains than it was trained on, or whether it finds one\nlucky chunk and guesses. The generator, validator and reward are built so the answer can be\ntrusted in either direction.\n\n## Task and interaction\n\nAn instance is a ~32k-token synthetic document (64k and 128k in `scale_test`). It is made of\nsix alphabetised registers (projects, personnel, departments, facilities, regional offices,\nvendors) plus irrelevant noise sections. The question follows a k-hop chain:\n\n> *What is the annual budget of the unit of the lead of project Amber Falcon?*\n> project → **Lead** (person) → **Unit** (department) → **Annual budget**\n\nOnly the anchor and the final attribute are named. Every bridge entity has to be discovered by\nreading. The working-context ceiling is **600 tokens** and chunks are 320–480 tokens, so two\nchunks never fit together: each bridge has to be written into a summary before its chunk is\nfreed. Every instance includes distractors: near-miss entities (`Amber Falcon II`, sorted next to\nthe real one), superseded statements (\"previously …\"), and near-miss values clustered within a\nfew percent of the answer.\n\n**The policy is stateless.** Each step is a fresh single-turn model call whose prompt is the\nrendered state: question, document map (chunk id, alphabetical range, token cost), memory ledger,\nand working context. No chat history is carried between steps, so a COMPRESS summary is the\nonly memory. Each step is recorded as its own trace.\n\nOne action per step, on the last line:\n\n| action | effect |\n|---|---|\n| `ACTION: READ <id>` | load a chunk (its token count is charged against the ceiling) |\n| `ACTION: COMPRESS <ids> :: <summary>` | replace held items with your own summary (truncated at 120 tokens) |\n| `ACTION: DROP <ids>` | free held items; re-reading later is allowed and costs a step |\n| `ACTION: ANSWER <value>` | terminal |\n\nThe episode ends on ANSWER, when the step budget (`6 + 4·n_hops`) runs out, or as soon as an\naction would push the working context above 600 tokens.\n\n## Reward\n\nOne terminal reward per episode, recorded on **every step trace of that episode**:\n\n```\nceiling exceeded | step budget exhausted | no answer  ->  -0.5\nwrong answer                                         ->   0.0\ncorrect answer                                       ->   max(0.5, 1 - 0.05 · steps_used / min_steps)\n```\n\n`min_steps` is the instance's provably minimal step count, computed by the generator, so deep\ninstances are not penalised for being deep. Correctness is normalised exact match on\nalphanumerics, plus one numeric leniency: a prediction whose only digit run equals the truth's\ndigits matches (`USD 4,885,866` matches `4,885,866`; `4,885,866 or 4,885,870` does not). There\nis no judge.\n\nMetrics (logged, not rewarded): `correct`, `ceiling_exceeded`, `budget_exhausted`,\n`steps_over_min`, `n_compress`, `n_invalid`, `distractor_capture` (the wrong answer is a known\ndistractor's value), and `final_step` (1.0 on the episode's last trace).\n\n**Reading eval output.** vf-eval's headline is a mean over *traces*, and an episode has one trace\nper step, so longer episodes are weighted more heavily. For per-episode numbers, run\n`python scripts/episode_summary.py <run_dir>`. For training, every step trace carries its\nepisode's return, so one advantage per episode is inherited by all of its steps (as in the\noriginal GRPO loop). A consumer that computes advantages per trace group should group by episode.\n\n## Why the reward can be trusted\n\nEvery instance passes a mechanical validator before use. The validator re-reads the answer\n**from the text**, not from generator state, and checks that:\n- the chain is solvable along exactly the recorded chunks;\n- no distractor path reaches the answer;\n- the answer value is unique;\n- the anchor never appears in the answer chunk (no lexical shortcut);\n- the instance is solvable under the ceiling.\n\nA population-level audit rejects surface confounds (chunk position, length and header must not\npredict evidence; AUC within 0.5 ± 0.06). The audit in the source repo measured:\n- a privileged oracle navigator scores 1.000 in exactly `min_steps`;\n- a no-read policy scores 0.000 (chance is 0.017);\n- the single-chunk shortcut caps at 0.183 even when the answer chunk is handed over.\n\n`tests/test_exploits.py` checks that none of these strategies score above 0.0: guessing without\nreading, answering \"unknown\", listing every candidate value, concatenating digits, reading two\nchunks without compressing, copying a chunk into a \"summary\", spamming invalid actions, and\nstalling with re-reads.\n\n## Data and license\n\nFully procedural. Instances are generated in `load()`/`run()` from (split, index) by the source\nrepo's seeded generator, so nothing is downloaded or bundled. The generated instances are\nidentical to the source repo's `data/v1` splits (checked by `tests/test_reward_parity.py`).\n\n| split | n | hops | doc tokens |\n|---|---|---|---|\n| `train` | 2000 | 2–3 | 32k |\n| `id_test` (default) | 300 | 2–3 | 32k |\n| `ood_test` | 300 | 4–5 | 32k |\n| `scale_test` | 150 | 3 | 32k / 64k / 128k |\n\nToken counts use the source repo's deterministic whitespace/punctuation counter, not any model's\ntokenizer.\n\n## Configuration\n\n| flag | default | meaning |\n|---|---|---|\n| `--env.taskset.split` | `id_test` | which split to generate |\n| `--env.taskset.num-tasks` | split size | generate more (or fewer) instances; extra ones use fresh seeds |\n| `--env.navigator.model` / `.sampling.*` | run's model | the navigating model |\n\nThe ceiling, step budget and reward weights are fixed to the source environment's values, so\nresults stay comparable to the original work.\n\n## Quickstart\n\n```bash\nprime env install manasganti/long-context-navigator\nuv run vf-eval long-context-navigator -n 5 -r 1 -m <model>\nuv run vf-eval long-context-navigator --env.taskset.split ood_test -n 50 -r 1 -m <model>\npython scripts/episode_summary.py outputs/long-context-navigator--<model>--null--<id>\n```\n\nEach step is one tool-less model call (`null` harness, local `subprocess` runtime). A 5-task\nsmoke eval runs on a laptop with any OpenAI-compatible API model. Keep `max_tokens` around 512\nper step, since a step needs only a THOUGHT line and an ACTION line.\n\n## Baseline results (vf-eval, this package)\n\nPer-episode means from `scripts/episode_summary.py`. The run: `vf-eval -n 50 -r 1`, with the\nsource repo's rollout sampling (T = 0.8, top_p = 0.95, 256 max tokens per step). The model was\nserved unquantized (bf16) by vLLM 0.8.5 on one A100. Each split took under 2 minutes.\n\n| model | split | episodes | reward (mean ± SE) | accuracy | ceiling exceeded | step budget exhausted | wrong answer |\n|---|---|---|---|---|---|---|---|\n| Qwen2.5-7B-Instruct | `id_test` (2–3 hops) | 50 | −0.350 ± 0.033 | 0.00 | 66% | 4% | 30% |\n| Qwen2.5-7B-Instruct | `ood_test` (4–5 hops) | 50 | −0.460 ± 0.019 | 0.00 | 90% | 2% | 8% |\n\nThe untrained 7B model never answers correctly. It mostly fails by loading a second chunk\nbefore compressing the first, which breaks the 600-token ceiling. It compressed in only 8%\n(`id_test`) and 2% (`ood_test`) of episodes. This matches the original work's\nin-environment result for the same model (0.000 on both splits, below).\n\n## Original results (separate harness)\n\nThese numbers come from the source repo's own training/eval harness (Qwen2.5-7B-Instruct, VT ARC\nA100s, 600 held-out instances: 300 ID + 300 OOD). They were **not** produced by this package or\nby vf-eval, and they are not Hub baselines. The model-free rows are in the source repo's\n`results/`. The LLM rows are reported in its README.\n\n| policy | ID (2–3 hops) | OOD (4–5 hops) |\n|---|---|---|\n| privileged oracle navigator | 1.000 | 1.000 |\n| random reader / no-read | 0.000–0.003 | 0.000–0.007 |\n| Qwen2.5-7B-Instruct, untrained, in-environment | 0.000 | 0.000 |\n| Qwen2.5-7B-Instruct + SFT (LoRA r=32, 20k oracle-teacher steps) | 0.630 | 0.050 |\n| SFT + GRPO (lr 1e-5, steps 20 / 40 / 60) | 0.653 / 0.600 / 0.623 | 0.077 / 0.053 / 0.060 |\n| reference: full document in view, no ceiling, reasoning | 0.936 | 0.653 |\n\nThe first GRPO run changed nothing measurable (all differences within ~2 SE). The source repo\nkeeps it as a control.\n\n## Known limitations\n\n- Synthetic registers, not natural text. A MuSiQue substrate was tried and rejected in the source\n  repo (ceiling 0.374).\n- The 600-token ceiling is enforced by the environment, not by the model's context window. The\n  model never sees more than one rendered state per call.\n- vf-eval's trace-mean headline is step-weighted (see *Reading eval output*).\n- Per-step overhead: every step is a separate rollout (about 1–2 s of harness overhead locally on\n  top of the model call).\n\n## Source and citation\n\nEnvironment, generator, validator and reward: [Manas-Ganti/long-context-navigator](https://github.com/Manas-Ganti/long-context-navigator)\n(pinned at commit `baeb433`). Packaged for verifiers 0.4.0.\n","encoding":"utf-8","truncated":false,"total_bytes":9457},"status":null}