{"data":{"kind":"file","path":"README.md","version_id":"lmkfn3d3hiu2wazalbf161kz","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4267,"modified_at":"2026-09-06T18:24:13.769000","content_hash":"874093d321e805d0e18e0a1b7889c1e12b539b3499c5d2e2d73cc1eec9eba304"},"entries":[],"content":"# gafitas-edit\n\n**Change an existing Python module until the project's own frozen pytest suite\npasses — without breaking the tests that already passed.**\n\nThe companion environments\n([`gafitas-repair`](https://app.primeintellect.ai/dashboard/environments/joseantoniodominguez/gafitas-repair),\n[`gafitas-repair-hard`](https://app.primeintellect.ai/dashboard/environments/joseantoniodominguez/gafitas-repair-hard))\nask for a module that does not exist yet. This one is the harder and more common\nshape: the file is already there, most of it is right, and something specific is\nmissing.\n\n| task | file to change | its size | the change | suite |\n|---|---|---|---|---|\n| `gb06` | `generalista/oracle_adapter.py` | 149 lines | +77 / −24 | `tests/test_oracle_feedback_actionability.py` |\n| `gb03` | `programmer/edit_provider.py` | 250 lines | +70 / −5 | `tests/test_edit_provider_retry_feedback.py` |\n| `gb02` | `programmer/ollama_provider.py` | 1023 lines | +40 / −15 | `tests/test_ollama_planner.py` |\n\nEach task is reconstructed from a real commit: the tree is the state\nimmediately before it with the target file rolled back to its parent version,\nand the reward is the suite that commit shipped.\n\n## Regression is the point\n\nBecause the file already works, most of the acceptance suite already passes —\n`gb02` starts at 23 of 24 tests green, and the task is the twenty-fourth. A\nmodel that satisfies the new test by rewriting the module into something that\nbreaks an old one scores zero, exactly as it should.\n\nThat is the difference between this and a write-from-scratch benchmark, and it\nis what real repair work is made of.\n\n## The controls, run before publishing\n\n**Task level** — the reward measures what it claims:\n\n| task | NEGATIVE | POSITIVE | TAMPER |\n|---|---|---|---|\n| gb02 | FAIL | PASS | held |\n| gb03 | FAIL | PASS | held |\n| gb06 | FAIL | PASS | held |\n\n- **NEGATIVE** — file rolled back: the suite must FAIL. A task that starts\n  solved measures nothing.\n- **POSITIVE** — the historical version: the suite must PASS. A task nobody can\n  solve teaches nothing.\n- **TAMPER** — the suite is replaced with trivially-passing tests and the\n  verdict does not move, because every test file is rewritten from the frozen\n  payload before pytest runs. Passing by weakening the test is impossible by\n  construction, not forbidden by policy.\n\n**Package level** — the environment as the hub loads it:\n\n| task | reference solution | junk code | no code block |\n|---|---|---|---|\n| gb02 | **1.0** | 0.0 | 0.0 |\n| gb03 | **1.0** | 0.0 | 0.0 |\n| gb06 | **1.0** | 0.0 | 0.0 |\n\nReproduce with `validate_env.py gafitas_edit` and\n`validate_published.py gafitas_edit`.\n\n## What the environment reports\n\n| signal | meaning |\n|---|---|\n| `repaired` *(reward)* | the frozen acceptance suite passed |\n| `suite_usable` | pytest actually ran. A suite that could not be collected is **not** a failed solution, and an average that mixes the two measures nothing |\n\n## A note on tree size\n\nThese repositories are large. Only what the acceptance suite needs travels, and\n**the trim itself was tested**: a candidate tree is accepted only after the\nsuite still fails with the file rolled back and still passes with the historical\nversion, so nothing quietly required was dropped. The trees went from about\n2 MB each to 200–825 KB that way.\n\n## What is not in this package\n\nThe historical fix. Each objective was authored from the pre-change public API\nand the frozen tests alone, then checked against the reference's own distinctive\nlines to confirm it quotes none of them. Trees were searched for the reference\n**as bytes**, so compiled bytecode and archives were searched too — a check that\nonce found the post-change version of a target file inside a `__pycache__` in\nthis corpus, where two filename-based checks had passed it.\n\nNote that the model *does* see the file as it stands, because without it the\ntask is unanswerable. What it never sees is the change.\n\n## Safety\n\nThis environment **executes model-generated Python**. Grading writes the tree\ninto a fresh temporary directory and runs pytest there as a subprocess under a\ntimeout, writing nothing outside it — but untrusted code still runs. Use a\nsandbox, as with any code-execution environment.\n","encoding":"utf-8","truncated":false,"total_bytes":4267},"status":null}