{"data":{"kind":"file","path":"README.md","version_id":"ddq1938d3ca7kt5qaevd0jhp","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":9595,"modified_at":"2026-11-04T01:43:03.475000","content_hash":"1be8f5e7150a7a12c387caf9adccf81895544735974b5555eff3a98696c0d20d"},"entries":[],"content":"# Weight Space Breakout: hosted repair pipeline\n\n**Version 0.4.5 adds [automatic submission of the first qualifying edit](QUALIFIED_REPAIR.md).**\nThe LLM proposes weight assignments using tools. The first proposal that passes the\npredefined 32-game practice check ends editing automatically; a separate 512-game\ngate still decides promotion. There is no editor-turn cap. This is a new protocol,\nnot a change to any already-running trial.\nThe software checks include a real tool-server lifecycle with a scripted provider:\none nonqualifying proposal stays open, the next qualifying proposal ends editing,\nand the collector verifies the committed receipt. This is a software test, not a\nhosted LLM success claim.\n\n**Version 0.4.4 adds a separately specified [basic tool-assisted repair trial](BASIC_REPAIR.md).**\nIt tests one broad improvement and the complete deployment path. The example miss is\ndiagnostic only in this new protocol; prior failure-driven trials keep their stricter\nrules. The new workflow passes 44 tests, including both protocols through the real\nagent, tool transport, final submission, and verifier with a scripted test provider.\nA passing software test is not a real hosted LLM result.\nSee the [independent design review](reviews/2026-11-03-independent-review.md).\n\nThe subsequent 0.4.3 attempt completed 141 valid practice tests but never submitted\nan edit in 144 turns. It stopped on a provider HTTP 429 concurrency-limit error.\nNo independent promotion check ran and no weights were promoted. This separates\nworking practice tools from the still-unproven hosted-editor submission milestone.\n\n**Version 0.4.3:** [prepared practice tools and a final submission action](PREPARED_EDITOR.md),\nwith no editor-turn limit. Startup prepares the numerical tools, and the last action\ncommits the proposed weight edit for independent verification. Tool calling is enabled.\n\nThe 0.4.1 hosted attempt made 123 model turns but never submitted an edit. It\nrepeated an oversized patch and eventually ended with a provider connection error.\nNo weights were promoted. Version 0.4.2 improves tool constraints and repeated-patch\nfeedback and preserves failed-attempt audits; its effect on editor behavior still\nrequires a new hosted trial. See [the full recorded outcome](PREPARED_EDITOR.md).\nVersion 0.4.3 updates the verifier dependency for the current hosted runtime;\nthe 39 tests also pass against that dependency.\n\n**Previous 0.3.0 trial:** [failure-driven diagnosis and replay](FAILURE_DRIVEN.md). The next\neditor receives a specific failed episode without a supplied diagnosis, can use tools,\nand must propose an explanation plus weight edits. Promotion requires preventing that\nmiss under controlled replay and improving broader play. Its first three-attempt trial\nis now complete: all attempts exhausted a budget before submitting valid edits, so\nthe faulty weights remained unchanged (final audit **1.3809 → 1.3809**).\nSee [the failure-driven results](FAILURE_DRIVEN.md) and\n[its separate demo](https://huggingface.co/spaces/HarleyCooper/weight-space-breakout-failure-driven-demo-01).\nThe successful earlier-protocol run described below remains a separate experiment.\n\nA working experiment that starts with known-bad policy weights, asks a hosted LLM for\nan edit, scores the candidate in Prime's environment, publishes accepted weights to\nHugging Face, and runs the next simulation from that exact checkpoint.\n\n**The first bounded run completed: one repair accepted, two regressions rejected.**\nOn a separate 1,024-game final audit, average reward improved from **1.3018 to 2.9180**.\nThe accepted edit changed two of 17,253 parameters and exactly restored the intact\nhandwritten reference policy. The other two candidates made play worse.\n\n[Live repair-loop simulation](https://huggingface.co/spaces/HarleyCooper/weight-space-breakout-repair-loop) ·\n[Versioned weights and full audit](https://huggingface.co/HarleyCooper/weight-space-breakout-iterative) ·\n[Prime environment](https://app.primeintellect.ai/dashboard/environments/harleycooper/weight-space-breakout) ·\n[Pipeline source](https://github.com/HarleyCoops/weight-space-breakout-pipeline)\n\n## What runs where\n\n| Piece | Job | Location |\n| --- | --- | --- |\n| Editor: `qwen/qwen3.6-35b-a3b` | Reads current weights, practice trajectories, and prior-round feedback; proposes scalar edits | Prime-hosted inference |\n| Game and verifier | Runs practice games, compares candidate to incumbent, decides acceptance, runs final audit | Prime-hosted evaluation |\n| Controller | Checks traces and hashes, serializes weights, publishes versions, verifies deployment | Lightweight Python orchestration; no local model or evaluation games |\n| Game policy | Maps a 10x10 observation to five action probabilities | 17,253-parameter NumPy network on Hugging Face CPU Basic |\n\nThe controller was run from a workstation for this demonstration. It can also run on\nordinary hosted CPU compute. Prime and Hugging Face handled the experiment's model\ninference and gameplay. Local development unit tests exercised the small game separately.\nThe three-round run has finished; no editor job or ongoing repair schedule remains active.\n\n## First iterative run: November 3, 2026\n\n| Round | Incumbent return | Candidate return | Mean gain | Decision |\n| --- | ---: | ---: | ---: | --- |\n| 1 | 1.4512 | 2.7773 | +1.3262 | Accepted: reverse the two output signs |\n| 2 | 2.8535 | 2.4395 | -0.4141 | Rejected: stronger controls and fewer no-ops |\n| 3 | 2.8535 | 2.3789 | -0.4746 | Rejected: extreme control strengths |\n\nEach row uses 512 fresh matched seeds. Row means are not directly comparable across\nrounds. The next round uses the last **accepted** policy and the previous attempt's\nfeedback. A rejected candidate remains in the audit but never replaces live weights.\n\nAcceptance requires a real parameter change, mean gain above 0.10, and\n`gain - 2 * paired_standard_error > 0`. This is an engineering screen, not a general\nstatistical guarantee. The final audit used 1,024 additional seeds, never fed to the\neditor or used for promotion: baseline **1.3017578125**, selected **2.91796875**, paired\ngain **1.6162109375**, standard error **0.1107816995**. Neither policy cleared a board\non that audit. Individual games still often fail quickly.\n\nFor every round, the deployed Hugging Face episode matched the Prime verifier's\npolicy hash, seed, complete action sequence, reward, and termination. Immutable\nHub revisions preserve the initial bad policy and every published state.\n\nThis establishes the end-to-end feedback, rejection, publication, and inference\nmechanism. It does not establish reliable recursive improvement or language-model\nrepair. This is a handwritten teaching controller with a deliberately simple fault,\nnot a trained Qwen policy. Qwen is the editor, not the exported game network.\n\n## Use and reproduce\n\n- **Play:** open the live Space and choose **Play latest weights** for a fresh seed.\n  **Replay pinned version** repeats the displayed seed and exact model revision.\n  The animation loops one computed episode; playing does not trigger weight editing.\n- **Run the pipeline:** see [setup and commands](pipeline/README.md).\n- **Inspect the protocol:** [PIPELINE.md](PIPELINE.md) specifies seeds, gates, compute,\n  and the disclosed tool-harness limitation.\n- **Inspect all attempts:** [public experiment records](https://huggingface.co/HarleyCooper/weight-space-breakout-iterative/tree/main/experiment).\n- **Extend to languages:** [Cree and Dakota assessment](LANGUAGE_EXTENSION.md).\n- **Compare the earlier one-shot experiment:** [ONE_SHOT.md](ONE_SHOT.md). That earlier\n  experiment used a different protocol and local game evaluation; its results remain separate.\n\nThe new failure-driven protocol allows tools and multiple editor turns under an explicit budget.\nThis initial run used exactly one response and zero tool calls per round. Prime's\nharness advertised tools despite the prompt saying they were unavailable; actual\nnon-use is verified from each trace. See the protocol disclosure for details.\n\n## Code and verification\n\n`weight_space_breakout/` contains the game, policy, data-only edit parser, Prime taskset,\nand verifier. `pipeline/control.py` implements the resumable bounded loop.\n`pipeline/space_app.py` serves immutable policy revisions. `tests/` covers game rules,\ninvalid edits, the acceptance gate, trace tampering, tool-use checks, and interrupted\nstate recovery. The original build passed 18 tests; the current suite passes 39 on Linux, including\nthe complete agent/tool/submission lifecycle with a scripted provider, failure replay,\npractice-tool execution, and rejecting a targeted rescue that harms\nbroader play. All three original real hosted deployment\ncomparisons passed. The run used environment **0.2.1**; subsequent documentation or\npackaging releases do not change those recorded results.\n\n## Credit and scope\n\nIndependent conceptual replication attempt and teaching adaptation inspired by\n[Sparks of Weight-Space Programming](https://rikiyat.github.io/blog/weight-space-programming/)\nby Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab (MIT CSAIL, November 2, 2026).\n[Original X post](https://x.com/rikiyatakehi/status/2117313742891327994) ·\n[Experiment 1 logs](https://github.com/RikiyaT/lm_understand-logs/tree/main/exp1_which_game).\n\nThe original experiment identifies a game from its weights. We reconstructed our own\ngame and tested repairs to intentionally faulty handwritten weights. This is not an\nexact reproduction of their simulator, PPO training, or reported scores.\r\n","encoding":"utf-8","truncated":false,"total_bytes":9595},"status":null}