{"data":{"kind":"file","path":"README.md","version_id":"p0sflouwo8bvanosld3a9tk8","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5379,"modified_at":"2026-07-15T14:32:38.933000","content_hash":"372e1d4095026b28dc6ff7d2e461aefe05326bd574cb3ad979f370c9c2bed82b"},"entries":[],"content":"# tmax-opsd-v1\n\nA [verifiers](https://github.com/PrimeIntellect-ai/verifiers)-v1 environment that packages the\n[TMax](https://arxiv.org/abs/2606.23321) terminal-agent dataset (14,601 Docker-image tasks) with a\nTMax-faithful **vanillux** harness, plus a set of per-task **hints** for on-policy self-distillation\n(OPSD) training.\n\nIt supports two training signals over the same tasks:\n- **GRPO** — the binary solved/not-solved reward from each task's hidden verifier.\n- **OPSD** — dense per-token distillation from the policy conditioned on a method-level **hint**\n  (the `demo` field). This repo ships the generated hints.\n\n## What's in here\n\n| Path | What |\n|------|------|\n| `tmax_opsd_v1/` | the environment package (taskset, harness, reward, dataset builder) |\n| `data/demos/<task_id>.md` | **9,959** generated OPSD hints, one file per task |\n| `scripts/hintgen/` | the hint-generation pipeline + `RESUME.md` (how to generate the rest) |\n| `scripts/prepare_data.py` | build `tasks.parquet` ahead of time |\n| `docs/superpowers/specs/` | the hint-generation design spec |\n\n`data/tasks.parquet` and `data/raw/` are **not committed** (regenerable — see [Data](#data)).\n\n## The environment\n\n- **Taskset** `TMaxTaskset` — joins Ai2's two TMax releases (`allenai/TMax-15K` and\n  `allenai/tmax-15k-open-instruct`); each task runs in its own Docker image. Built lazily on first\n  `load()` if `tasks.parquet` is absent.\n- **Harness** `VanilluxHarness` — a single persistent-shell `bash` tool, the\n  `COMPLETE_TASK_AND_SUBMIT_FINAL_OUTPUT` submit marker, 10k head/tail observation truncation, and a\n  format-error nudge — matching TMax's mini-SWE-agent-derived harness.\n- **Reward** `solved` — binary 0/1 from the task's hidden pytest verifier\n  (`/logs/verifier/reward.txt`), run once at scoring time. Plus `@metric pass_fraction` and\n  `@metric hacked` (an oracle-copy detector), which are logged but never summed into the reward.\n- **`validate()`** — a pre-flight fixture check that flags dead/broken images (a known ~15% of the\n  corpus); pair with `blocklist_path` to skip them.\n- Public API: `from tmax_opsd_v1 import TMaxTaskset, VanilluxHarness`.\n\nEach task row (`TMaxData`) carries `task_id`, taxonomy fields (`domain`, `skill_type`, …), the\n`verifier_kind`, and a `demo` slot for the OPSD hint. `truth` (the answer key) and `test_script`\n(the verifier) are excluded from every serialization — they are never shown to the policy.\n\n## The OPSD hints\n\nOPSD (on-policy self-distillation) teaches the policy from a *hint-conditioned* version of itself:\nthe hint is prepended, the policy's own rollout is prefill-scored under it, and the resulting\nper-token log-probs become the distillation target. The hint must teach the **method** without\nleaking the answer.\n\nThe hints in `data/demos/<task_id>.md` are exactly that — short, method-level demonstrations\ngenerated with **Claude Sonnet 5** from each task's `truth`, with the literal answer and any\noracle/shortcut path redacted (the discover-vs-given rule; see the design spec). Each is a single\nparagraph of imperative, terminal-oriented guidance.\n\n**Coverage: 9,959 / 14,601 tasks.** The remainder are either not yet generated or a small set of\noffensive-security-framed tasks that Claude's safety filter declines — those stay GRPO-only.\n\n### Wiring hints into a training run\n\nHint support is built in and **ships inside the package** (`tmax_opsd_v1/demos.parquet`), so a\npulled/installed env carries the hints with no extra data step. `build.py` left-joins them into a\n`demo` column, `TMaxData` carries the `demo` field, and the taskset config exposes `require_demo`\nto select only hinted tasks. OPSD reads the hint through its `demo_key`.\n\nRegenerate the sidecar only if you add/change hints:\n\n```bash\nuv run python scripts/hintgen/assemble_demos.py   # data/demos/*.md -> tmax_opsd_v1/demos.parquet\n```\n\n**Pure OPSD** — a ready config is at [`configs/tmax_opsd_pure.toml`](configs/tmax_opsd_pure.toml):\n\n```toml\n[orchestrator.algo]\ntype = \"opsd\"\ndemo_key = \"demo\"\n\n[[orchestrator.train.env]]\n# the id goes INSIDE taskset (v1 dispatch); require_demo selects hinted tasks\ntaskset = { id = \"tmax_opsd_v1\", require_demo = true }\n```\n\n**RL + OPSD** — two env entries over the same env: a GRPO entry (all tasks) and an OPSD entry\n(`require_demo = true`), packed into one batch by the trainer.\n\n### Generating / resuming the hints\n\nThe full set is generated with Claude subagents (no API key needed — runs on a Claude plan), in\nresumable chunks with a mapping-integrity gate. See **`scripts/hintgen/RESUME.md`** for the exact\nloop (slice → generate → gate → repair → repeat).\n\n## Data\n\n`tasks.parquet` is built from the two public allenai releases and cached under\n`$TMAX_OPSD_DATA_DIR` (default `~/.cache/tmax-opsd-v1/`). Build it ahead of time:\n\n```bash\nuv run python scripts/prepare_data.py\n```\n\n…or let `TMaxTaskset.load()` build it lazily on first use. It is not committed because it contains\nsynthetic secret-shaped strings (fake keys/tokens in the security-themed task content) that trip\nGitHub's secret scanning.\n\n## Install & test\n\n```bash\nuv sync --all-extras\nuv run pytest            # docker-marked tests are skipped by default\n```\n\nRuntime: use the `prime` runtime (Prime Sandboxes) or a Linux `docker` host. Set\n`TMAX_OPSD_DATA_DIR` to control where the built parquet is cached.\n","encoding":"utf-8","truncated":false,"total_bytes":5379},"status":null}