{"data":{"kind":"file","path":"README.md","version_id":"lzyv7i4s3oikjct0k0yrghn2","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":6867,"modified_at":"2026-08-07T06:30:03.689000","content_hash":"025a5e8be7f76911f7d1d2d1c01211013b321bcd5c66d3fe0f46dfb146dfa3ad"},"entries":[],"content":"# deeplearning-env\r\n\r\nRL environment for **the mathematics of deep learning** on the topics of *Deep Learning*\r\n(Goodfellow, Bengio & Courville): information theory, backpropagation, optimisation, ML linear\r\nalgebra and numerical stability.\r\n\r\n- **Tasks:** 36 across 5 categories\r\n- **Dataset:** [`deeplearning-tasks-v1`](https://huggingface.co/datasets/eltociear/deeplearning-tasks-v1)\r\n- **Reward:** binary, from an exact comparison. No LLM judge, no network, no clock.\r\n- **Dependencies in the sandbox:** none, and numpy is deliberately absent.\r\n\r\n```bash\r\nprime env install eltociear/deeplearning-env\r\n```\r\n\r\n\r\n36 exact tasks for the\r\n[`deeplearning-env`](https://app.primeintellect.ai/dashboard/environments/eltociear/deeplearning-env)\r\nRL environment, on the topics of *Deep Learning* (Goodfellow, Bengio & Courville): information\r\ntheory, backpropagation, optimisation, the linear algebra used in ML, and numerical stability.\r\n\r\n| field | meaning |\r\n|---|---|\r\n| `task_id` | `dl-000` … `dl-035` |\r\n| `category` | information / backprop / optimisation / linalg / numerical |\r\n| `prompt` | the question, the units, and the exact shape of the answer |\r\n| `api_description` | the fixed network and distributions, plus the exact-arithmetic rules |\r\n| `expected_output` | JSON `{\"rows\": [...]}`, **computed by executing a reference solution** |\r\n\r\nCategories: information 9, backprop 8, optimisation 7, linalg 6, numerical 6. No dependencies.\r\n\r\n## How an inexact subject was made exactly gradeable\r\n\r\nEntropy, KL and cross-entropy involve logarithms; a gradient through a sigmoid involves *e*.\r\nThe usual answer is to grade floats with a tolerance — which cannot then distinguish a wrong\r\nformula from rounding, at exactly the level where these quantities get confused. **A nat and a\r\nbit differ by a factor of ln 2**, and no tolerance wide enough for float noise is tight enough\r\nto catch that.\r\n\r\nSo the exact case is made reachable instead:\r\n\r\n- every distribution is **dyadic** — every probability is a power of 1/2 — so `log2(p)` is an\r\n  **integer** and entropy, cross-entropy, KL and mutual information are exact rationals.\r\n  `H([1/2, 1/4, 1/8, 1/8])` is exactly `7/4` bits, not `1.7499999999999998`;\r\n- every network is **piecewise linear** (affine, ReLU, squared error), so forward values and\r\n  backpropagated gradients are exact rationals — no sigmoid, no softmax value, no *e*;\r\n- the stability tasks ask **which failure occurs** (`inf`, `nan`, `zero`, `finite`), not what a\r\n  float came out as.\r\n\r\nEvery task states its units. There is not one float in the dataset.\r\n\r\n## Two defects the constraint itself caught\r\n\r\n- **A dyadic joint does not have dyadic marginals.** The first joint distribution had all four\r\n  cells dyadic and a marginal of `5/8`, at which point `log2` was no longer an integer and the\r\n  exactness argument collapsed. It is not a matter of picking better numbers either: on a\r\n  binary axis the only dyadic distribution is `[1/2, 1/2]`, and the only dyadic split of a\r\n  dyadic row sum into two parts is into equal halves — which forces **independence**, making\r\n  the mutual information identically zero and the task empty. The fix is the one construction\r\n  that keeps both: let `b` be a **deterministic function** of `a`. Two tasks now use it, and\r\n  they show `I(a;b) = H(a)` when the function is invertible and `I(a;b) = H(b)` when it is not.\r\n- **A KL-asymmetry task returned two equal numbers.** `D(P‖Q)` and `D(Q‖P)` were both `1/4`,\r\n  and `D(P‖R)` and `D(R‖P)` were both `7/8` because `R` is `P` reversed — the symmetry was\r\n  structural, not luck. A distribution `S` was added for which the two divergences are `1/2`\r\n  and `5/8`, and the builder asserts they differ.\r\n\r\n## Gradients are cross-checked against an exact finite difference\r\n\r\nNot a float finite difference — an **exact** one, with a rational step. Because the network is\r\npiecewise linear and the step does not cross a ReLU kink, the symmetric difference *equals* the\r\nderivative, so it is a genuinely independent computation rather than an approximation that has\r\nto be trusted to a tolerance. Every gradient in the key is checked this way before any task is\r\nbuilt.\r\n\r\nVerify with `python environments/deeplearning_env/build_tasks.py --verify` (36/36).\r\nSource: <https://github.com/eltociear/my-molt-agent/tree/main/environments/deeplearning_env>\r\n\r\n## Verify it yourself\r\n\r\n```bash\r\npython environments/deeplearning_env/build_tasks.py --verify   # 36/36\r\n```\r\n\r\n## Environment arguments\r\n\r\n`load_environment()` deliberately exposes very little: the program's guidance is that an\r\nenvironment should have one correct way to be run, so nothing about the prompts, the parsing or\r\nthe grading is configurable.\r\n\r\n| Argument | Default | Meaning |\r\n|---|---|---|\r\n| `split` | `'train'` | Dataset split to load. |\r\n| `dataset_name` | `'eltociear/deeplearning-tasks-v1'` | Hugging Face dataset of tasks. Change only to point at a fork. |\r\n| `max_turns` | `5` | Tool-use turns the model gets before the rollout ends. |\r\n| `**kwargs` | — | Passed through to the underlying `SandboxEnv`. |\r\n\r\n## Reward rubric\r\n\r\n| Reward function | Weight | What it returns |\r\n|---|---|---|\r\n| `correctness` | 1.0 | Binary: 1.0 when the answer matches the reference, else 0.0. |\r\n\r\nThere is no LLM judge and no partial credit. The score is computed on the host in\r\n`post_rollout` and read back by the rubric, so the reward is a deterministic function of the\r\nvalues the model left in `result`. A **harness** failure (dead sandbox, unreadable read-back)\r\nalso scores 0.0 but additionally sets `state[\"scoring_error\"]`, so an eval run can tell a\r\nbroken harness from a wrong answer instead of blaming the model.\r\n\r\n## Dependencies\r\n\r\n`datasets>=4.1.0`, `verifiers>=0.1.8`\r\n\r\n## Sample `vf-eval` usage\r\n\r\n```bash\r\nuv run vf-install deeplearning-env\r\nuv run vf-eval -s deeplearning-env -m gpt-4.1 -n 5 -r 3\r\nuv run vf-tui                      # inspect the outputs/ folder it writes\r\n```\r\n\r\n## Known limitation\r\n\r\nThe Docker **sandbox transport** has not been executed — `docker run`, `pip install`, and the\r\nmodel's code running inside the container — because that needs a Docker runtime and an\r\ninference provider key, neither available where this was authored.\r\n\r\nEverything else is exercised. `environments/test_scoring_path.py` runs this environment's\r\n`post_rollout` and `Rubric` with the sandbox mocked out, asserting that a correct answer scores\r\n1.0, a well-formed wrong answer scores 0.0, and a dead sandbox scores 0.0 *and* sets\r\n`scoring_error` so a harness failure is never mistaken for a bad model. It also checks that\r\nwhat `build_tasks.py` emits is exactly what the scorer expects. The environment mirrors the\r\nstructure of `polars_env` (already accepted into the Environments Program) and imports cleanly\r\nagainst `verifiers` 0.2.1.\r\n","encoding":"utf-8","truncated":false,"total_bytes":6867},"status":null}