{"data":{"kind":"file","path":"README.md","version_id":"flvcunp66zqrctkxegcqg6ta","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3100,"modified_at":"2026-07-02T18:24:28.642000","content_hash":"e3a80085eb5c552b179f3856987448c26628f756a0caaf57958e030f1af77c65"},"entries":[],"content":"# doc-edit\n\nAgentic document-editing RL environment (verifiers / prime-rl compatible).\n\nThe agent operates on a **paginated plain-text purchase order it cannot see\ndirectly**, through four tools:\n\n| tool | behavior |\n|---|---|\n| `search_doc(query)` | case-insensitive substring search across all pages (≤12 hits, with page/line numbers) |\n| `read_page(page)` | full text of one page |\n| `edit_text(page, old_text, new_text)` | exact-string replacement; `old_text` must be unique on the page |\n| `submit()` | end the episode |\n\n## Task families\n\n- **`reference_and_update`** — one authoritative fact changes (an item quantity,\n  a unit price, the freight quote, the order date, the PO number, a contact\n  email). The change ripples through the document's dependency graph: line total\n  → subtotal → sales tax → order total, order date → due date, plus prose\n  *echoes* of those values in cover letters, payment terms, appendices, and page\n  headers — often in a different format (`$4,530.00` vs `USD 4,530.00`, `March\n  14, 2026` vs `2026-03-14`, `twelve (12)`). The agent must compute the new\n  derived values itself (no calculator tool) and find every stale mention.\n- **`audit_and_correct`** — 1–3 rendered instances of derived values were\n  corrupted. The instruction names the authoritative inputs; the agent must find\n  what disagrees and fix it, editing nothing else.\n\n## Reward (fully programmatic, no LLM judge)\n\nDocuments are generated from a schema (values + formulas) and rendered\ndeterministically from typed-placeholder templates. The expected document is a\nre-render of the same layout with corrected values; ground truth is the line\ndiff between the two renders. Sites are therefore *computed, never annotated*.\n\n- `consistency_reward` (weight 1.0) = fraction of required change-sites whose\n  corrected line is present, minus 0.1 per *collateral* line (a line matching\n  neither the original nor the expected document), clamped to [0, 1].\n- Zero-weight metrics logged per rollout: `site_recall`,\n  `exact_document_match`, `collateral_lines`, `stale_sites`, `num_edits`,\n  `submitted`, per-tool call counts, `num_turns` — watch strategy emerge during\n  training, not just reward.\n\n## Synthetic data pipeline\n\nValues are sampled procedurally; prose section templates are authored offline\nby a frontier LLM (`scripts/author_templates.py` at the repo root) and\n**machine-validated** (placeholder syntax, field/format legality, required\nechoes, forbidden fields) before entering the pool — no LLM output is ever\nconsumed at reward time. Decoy placeholders fill boilerplate with plausible\nnear-miss numbers and dates.\n\n## Knobs (`load_environment` kwargs)\n\n`num_examples`, `seed`, `families`, `n_fillers` (document length / search\npressure), `n_appendices` (echo surface), `max_turns` (budget pressure),\n`templates` (`\"seed\"` or `\"all\"`).\n\n## Quick start\n\n```bash\nuv run vf-eval doc-edit -p openrouter -m poolside/laguna-xs.2:free \\\n  -n 18 -r 2 --max-tokens 4000 --save-results\n\n# eyeball generated tasks (writes previews/)\nuv run python scripts/preview_task.py 6\n```\n","encoding":"utf-8","truncated":false,"total_bytes":3100},"status":null}