{"data":{"kind":"file","path":"README.md","version_id":"brgmrxm1avojc8vkm7noo3oo","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2399,"modified_at":"2026-08-21T13:03:47.335000","content_hash":"d634c9f49b87236b9fa094c6c086540e00cb5f53dca019207376867c64ec0ac6"},"entries":[],"content":"# research-debugging\n\nAn agent inherits a readmission cohort study. The headline conclusion is wrong.\nNobody wrote down why, so the agent has to find the cause itself, repair it, rerun\nthe analysis, and report a corrected conclusion that still holds up when someone\nelse checks the work independently.\n\nThe grader is the interesting part here, not the prompt.\n\n## Grading\n\nNo model scores anything. A frozen verifier runs outside the agent's container and\nchecks three things.\n\n1. The diagnosis, matched against a closed enumeration of flaw classes and\n   mechanism codes. Free text is never graded.\n2. Measured evidence fields. These are graded only where the correct value is\n   forced: exact zero agreement checks, unit fractions, or ratios whose\n   denominator the instruction pins. Anything an honest solver could legitimately\n   reconstruct under a different convention is never a pass/fail gate, because\n   early versions of this verifier failed correct solutions on exactly that.\n3. A rerun gate. The verifier reruns the analysis from the submitted source,\n   independently regenerates the derived feature table from the raw records, and\n   fails anything whose committed numbers disagree with what that rerun produces.\n\nThe verifier, the answer key, and the oracle stay on the host. Only `repo/` is\nwritten into the container, which means the answer is never sitting somewhere the\nagent could read it. That property is the whole design.\n\n## Reward\n\n`solved` is 1.0 only when the verifier returns PASS. `diagnosis_ok` and `rerun_ok`\nride alongside as metrics, deliberately not as partial credit. Naming the right\ndefect while failing to rebuild your own numbers is not a solve.\n\n## Checking the environment\n\n```bash\nuv run python check_env.py\n```\n\nThis builds the oracle from the packaged assets, grades it, then pushes a battery\nof counterfeit submissions through the same grader. The oracle must PASS. Every\ncounterfeit must FAIL. Two consecutive runs produce identical output; if they do\nnot on your machine, that is a finding.\n\n## Provenance\n\nPorted from a task published end to end at\nhttps://github.com/Mayokun26/research-debugging-eval, where the study repository,\nthe verifier, the oracle, and the validity harness that audits the grading itself\nare all public. That task exists as a transparency example. Its answer being\npublic is deliberate, and it is why this one can ship here.\n","encoding":"utf-8","truncated":false,"total_bytes":2399},"status":null}