{"data":{"kind":"file","path":"README.md","version_id":"fenovwhtmxj8zqnciitjp56k","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4127,"modified_at":"2026-09-06T21:24:08.172000","content_hash":"f69373f87a92e186279f20bf8671f9a7bf8b04b0aa391ef89316afdacea1988c"},"entries":[],"content":"# gafitas-django\n\nWrite Django code. Django decides whether you got it right.\n\nFour single-turn tasks of rising difficulty. The model is given a spec and one\nempty module, and writes it. Grading exercises the result through Django's real\nmachinery — a live ORM against an in-memory database, real form validation and\nerror placement, the real URL resolver, the real test client. Nothing is\nstring-matched, and no reference solution ships with the package.\n\n| task | | what it measures |\n|---|---|---|\n| `dj01` | easy | custom queryset, chainable in either order, conditional aggregate that keeps zero-count rows |\n| `dj02` | medium | field-level vs cross-field validation, and *where* each error must land |\n| `dj03` | medium | path converters, status codes, 404 semantics, the real resolver |\n| `dj04` | hard | **query budget** — `prefetch_related`, enforced by `assertNumQueries` |\n\n## Why dj04 exists\n\nEvery other task asks whether the answer is right. `dj04` asks whether it was\nwritten the way someone who knows Django writes it.\n\nA correct implementation that loops over authors and touches `.books` inside\nthe loop returns exactly the right report and still fails. That is not an\nopinion about style — it was measured before publishing. A naive solution run\nthrough this grader:\n\n```\n4 of 5 tests passed          — every correctness test\n1 failed: \"13 queries executed, 2 expected\"\n```\n\nSo the environment carries a `correct_but_naive` signal (weight 0) that\nseparates *wrong* from *right but N+1*. Those are different failures. A reward\nthat collapses them teaches a model the difference does not matter, and in\nproduction it is frequently the only thing that does.\n\n## What makes the reward mean something\n\n**The grader cannot be edited.** Every acceptance file is rewritten from the\nfrozen payload immediately before pytest runs. Verified per task: an empty\nmodule that *also* replaces the suite with trivially-passing tests still scores\n0.\n\n**No answer key ships.** References live only in the build script that proved\nthe tasks. The published payload carries the given tree, the frozen tests and\nthe objective.\n\n**A broken harness is not a failed solution.** `suite_usable` separates \"pytest\nran and tests failed\" from \"pytest could not collect at all\". Conflating them\nrewards a model for emitting something that merely imports.\n\n**The tasks start unsolved and are solvable.** Each was verified to FAIL with\nthe module empty and PASS with a reference, each control run twice — a task\nwhose verdict is not reproducible was not shipped.\n\n## Verified as published\n\nThe controls above are checked against the *installed package*, loaded the way\nthe hub loads it, not against the build script. That distinction is not\npedantry — it is how the two defects below were found, and neither was visible\nby reading the code:\n\n- an earlier sibling shipped without declaring `pytest`, so in a clean venv\n  every task scored 0.0 **including the reference** — a total silent failure\n  that reads exactly like a model being bad at the task;\n- `datasets` unions the schema across rows, so each task's file map arrives\n  carrying `None` for every path that exists only in a *sibling* task. Writing\n  that `None` raised, and would have zeroed every score in the same\n  indistinguishable way.\n\n```\ndj01 reference implemented=1.0   junk=0.0   empty=0.0\ndj02 reference implemented=1.0   junk=0.0   empty=0.0\ndj03 reference implemented=1.0   junk=0.0   empty=0.0\ndj04 reference implemented=1.0   junk=0.0   empty=0.0\ndj04 naive     implemented=0.0   correct_but_naive=1.0\n```\n\n## Safety\n\nThis environment **executes model-generated Python**. Grading writes into a\nfresh temporary directory and runs pytest there as a subprocess under a\ntimeout, writing nothing outside it — but untrusted code still runs. Sandbox\nit, as you would any code-execution environment.\n\n## Usage\n\n```python\nfrom gafitas_django import load_environment\n\nenv = load_environment()                      # all four\nenv = load_environment(tasks=(\"dj04\",))       # just the query-budget task\n```\n\nRequires Django ≥ 5.0 and pytest, both declared.\n\nMIT.\n","encoding":"utf-8","truncated":false,"total_bytes":4127},"status":null}