{"data":{"kind":"file","path":"README.md","version_id":"fb5khymn4wufbidkv2acovgr","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4007,"modified_at":"2026-08-30T10:32:55.306000","content_hash":"37180c34da335034eb2cb8fd10f9f391a72d207015ecc2b8acc535b46148b278"},"entries":[],"content":"# pact\n\nA coding agent drives a LIBERO robot arm by **writing Python programs** over eight deterministic\nprimitives, scored on the simulator's own goal predicate. It never emits joint torques and never\nsees an image: it writes a program, runs it, reads structured failures, and rewrites.\n\n## Status\n\n**The taskset is ready.** `Taskset` / `Task` / reward / metrics are wired to verifiers v1 and the\nwhole chain runs end to end — setup → daemon → agent program → postconditions → attested verdict →\nreward — with no infrastructure failures.\n\n**The simulator is a sandbox image in `scripts/pact/`**, not part of this package. `make build`\nproduces `pact-sim`, and the build fails if its own selftest fails.\n\n## What happens in a rollout\n\nEach rollout gets **its own container and its own simulator daemon**. `Task.setup` launches\n`python -m pact_sim` in the background; it compiles the LIBERO scene (~8 s of MJCF parsing, meshes\nand collision structures) and then listens on a unix socket. Setup polls the socket — not the log,\nsince a process can print \"ready\" before it calls `listen()` — until it answers.\n\nThe agent then runs `robot-python solution.py` as many times as it likes. Each run is a **separate\nprocess**, but they all connect to the **same daemon**, so the world persists across them: a bowl\nknocked over in one program is still knocked over in the next. That is the property the whole\ndesign exists for, and it is asserted per episode by hashing `qpos` at each connection boundary.\n\nThe daemon is strictly serial — MuJoCo's GL context is not thread-safe — and a second connection is\nrefused in-band rather than left hanging.\n\nAt scoring, `Task.finalize` asks the daemon for a verdict over a fresh stdlib-only script. **Reward\nis read from a process the agent never had a handle on.** The daemon's dispatch table has ten verbs\nand none of them writes state, so `qpos[bowl] = plate_position` — which would score 1.0 having\ngrasped nothing, far cheaper than solving the task — is not in the search space at all.\n\n## Harness independence\n\nThe primitives ship as an **installed Python package on the agent's `PYTHONPATH`**, not as an MCP\ntoolset and not as skills. Any harness that can run a shell command can drive this environment,\nregardless of its capability flags — `bash` supports MCP but not `SKILL.md`, `terminus_2` supports\nneither, and none of that matters here. The agent writes a file and runs it.\n\n## What has been tested\n\nThe complete **`libero_goal` suite — all ten tasks**: one scene, seven objects, ten goals, so a\ndifference between tasks is attributable to the goal rather than to the scene.\n\n| | |\n|---|---|\n| episodes | 143 |\n| harness | `bash` |\n| model | `claude-sonnet-5` |\n| tier | T1 — simulator segmentation + `locate()`; names are ground truth, geometry is measured from depth (0.1–4 cm real error) |\n| frozen baseline | 4 tasks verified 2/2, `validate` 20/20 green at **zero token cost** |\n\n![baseline](baseline.png)\n\n**Read any success rate with the noise floor attached.** Two runs of an identical configuration —\nsame seeds, same image, zero errors — scored **0/12 and 3/12**. At n=12 the floor is 0–25%, wider\nthan any between-condition difference measured so far.\n\n## Next\n\n- **Cross-harness / cross-model baseline** — `bash` and `claude_code` × sonnet and opus, with a\n  replicate arm so the plot carries an error band rather than implying precision we do not have.\n- **Tiers.** T0 (privileged pose) and T2 (`locate()` removed, agent writes the unprojection) are\n  built and unrun.\n- **Legibility ablation.** One daemon flag degrades every `Result` to a bare bool, holding\n  primitives, tasks and model fixed. ASPIRE's largest reported lever, never isolated.\n- **Statistical power.** Separating 10% from 30% needs n≈60–80 per arm, not 12.\n\n```bash\ncd scripts/pact && make build\nuv run validate --taskset.id pact --runtime.type docker --taskset.seeds 2 -n 20 -c 4   # no model\nuv run eval @ configs/pact/eval.toml\n```\n","encoding":"utf-8","truncated":false,"total_bytes":4007},"status":null}