{"data":{"kind":"file","path":"README.md","version_id":"pydisvmauhkqbdal96s7q4pg","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":6385,"modified_at":"2026-08-07T06:30:03.683000","content_hash":"df2996304d28fc221d4fea42271ff5059ff4f8b7df8e97ae4ced8853caef7e6e"},"entries":[],"content":"# uv-env\r\n\r\nRL environment for the semantics that decide what **`uv` actually installs**: PEP 440 version\r\nordering and specifier matching, PEP 508 environment markers, extras expansion, and dependency\r\nresolution with backtracking over a fixed index.\r\n\r\n- **Tasks:** 35 across 5 categories\r\n- **Dataset:** [`uv-tasks-v1`](https://huggingface.co/datasets/eltociear/uv-tasks-v1)\r\n- **Reward:** binary, from an exact comparison. No LLM judge, no network, no clock.\r\n- **Dependencies in the sandbox:** `packaging>=24.0` only.\r\n\r\n```bash\r\nprime env install eltociear/uv-env\r\n```\r\n\r\n\r\n35 Python-packaging tasks for the\r\n[`uv-env`](https://app.primeintellect.ai/dashboard/environments/eltociear/uv-env) RL\r\nenvironment: PEP 440 version ordering, PEP 440 specifier matching, PEP 508 environment markers,\r\nextras expansion, and dependency resolution with backtracking over a fixed index.\r\n\r\n| field | meaning |\r\n|---|---|\r\n| `task_id` | `uv-000` … `uv-034` |\r\n| `category` | versions / specifiers / markers / resolution / extras |\r\n| `prompt` | the question and the exact shape of the answer |\r\n| `api_description` | the fixed package index with every version and requirement, the resolution rules, and the three marker environments |\r\n| `expected_output` | JSON `{\"rows\": [...]}`, **computed by executing a reference solution** |\r\n\r\nCategories: versions 7, specifiers 8, markers 6, resolution 9, extras 5.\r\nRequires `packaging>=24.0` in the sandbox; nothing else, and no network.\r\n\r\n## Why this dataset exists\r\n\r\n`uv` sat unclaimed on the reasoning that *\"resolution needs the network, so it cannot be made\r\ndeterministic.\"* That is only true of resolution against a **live** index. Everything that makes\r\nuv's behaviour hard to predict is **offline** semantics — version ordering, specifier matching,\r\nmarker evaluation, extras, and the resolver's own backtracking. State the index in the prompt\r\nand the whole thing is a pure function.\r\n\r\n## The trap, measured rather than assumed\r\n\r\n```python\r\nVersion(\"1.1.0rc1\") in SpecifierSet(\">=1.0\")        # -> True\r\nSpecifierSet(\">=1.0\").filter([..., \"1.1.0rc1\"])     # -> 1.1.0rc1 is EXCLUDED\r\n```\r\n\r\n`__contains__` admits pre-releases when `prereleases` is `None`; `filter` excludes them. A\r\nresolver written with `in` silently installs release candidates and one written with `filter`\r\ndoes not — and `filter` is what uv and pip do.\r\n\r\nThat single asymmetry is the difference between a correct resolution answer key and a\r\nconfidently wrong one. It was found by running both, not by reasoning about the spec, and two\r\ntasks ask about it directly. The builder asserts the two still disagree, so an upstream change\r\nto `packaging` cannot quietly turn those tasks into tautologies.\r\n\r\n## Other things the tasks pin down\r\n\r\n- `python_version` is compared as a **version, not lexically**: `\"3.10\" >= \"3.9\"` is `False` as\r\n  strings and `True` as versions, and one task shows both side by side.\r\n- `~=1.4.2` matches `1.4.9` but not `1.5.0`; `~=1.4` matches `1.9.0` but not `2.0.0`.\r\n- `1.0.0+local` sorts **after** `1.0.0` and **before** `1.0.0.post1`.\r\n- `1.0.0.RELEASE` is not a valid PEP 440 version at all, while `1.0.0-alpha.1` normalises to\r\n  `1.0.0a1`.\r\n- One resolution is deliberately **unsatisfiable**, and one forces a **backtrack** off\r\n  `delta 2.0.0` because it requires `gamma<0.3` while `beta 1.1.0` requires `gamma>=0.3`.\r\n\r\n## The resolver's own answers are re-validated\r\n\r\nA backtracking resolver that is subtly wrong will happily return a subtly wrong answer, so\r\nevery resolution in the answer key is re-checked from scratch by a **separate function that\r\nshares no code with the search**: every root constraint and every requirement of every chosen\r\ndistribution must hold against the chosen versions.\r\n\r\nVerify with `python environments/uv_env/build_tasks.py --verify` (35/35).\r\nSource: <https://github.com/eltociear/my-molt-agent/tree/main/environments/uv_env>\r\n\r\n## Verify it yourself\r\n\r\n```bash\r\npython environments/uv_env/build_tasks.py --verify   # 35/35\r\n```\r\n\r\n## Environment arguments\r\n\r\n`load_environment()` deliberately exposes very little: the program's guidance is that an\r\nenvironment should have one correct way to be run, so nothing about the prompts, the parsing or\r\nthe grading is configurable.\r\n\r\n| Argument | Default | Meaning |\r\n|---|---|---|\r\n| `split` | `'train'` | Dataset split to load. |\r\n| `dataset_name` | `'eltociear/uv-tasks-v1'` | Hugging Face dataset of tasks. Change only to point at a fork. |\r\n| `max_turns` | `5` | Tool-use turns the model gets before the rollout ends. |\r\n| `**kwargs` | — | Passed through to the underlying `SandboxEnv`. |\r\n\r\n## Reward rubric\r\n\r\n| Reward function | Weight | What it returns |\r\n|---|---|---|\r\n| `correctness` | 1.0 | Binary: 1.0 when the answer matches the reference, else 0.0. |\r\n\r\nThere is no LLM judge and no partial credit. The score is computed on the host in\r\n`post_rollout` and read back by the rubric, so the reward is a deterministic function of the\r\nvalues the model left in `result`. A **harness** failure (dead sandbox, unreadable read-back)\r\nalso scores 0.0 but additionally sets `state[\"scoring_error\"]`, so an eval run can tell a\r\nbroken harness from a wrong answer instead of blaming the model.\r\n\r\n## Dependencies\r\n\r\n`datasets>=4.1.0`, `packaging>=24.0`, `verifiers>=0.1.8`\r\n\r\n## Sample `vf-eval` usage\r\n\r\n```bash\r\nuv run vf-install uv-env\r\nuv run vf-eval -s uv-env -m gpt-4.1 -n 5 -r 3\r\nuv run vf-tui                      # inspect the outputs/ folder it writes\r\n```\r\n\r\n## Known limitation\r\n\r\nThe Docker **sandbox transport** has not been executed — `docker run`, `pip install`, and the\r\nmodel's code running inside the container — because that needs a Docker runtime and an\r\ninference provider key, neither available where this was authored.\r\n\r\nEverything else is exercised. `environments/test_scoring_path.py` runs this environment's\r\n`post_rollout` and `Rubric` with the sandbox mocked out, asserting that a correct answer scores\r\n1.0, a well-formed wrong answer scores 0.0, and a dead sandbox scores 0.0 *and* sets\r\n`scoring_error` so a harness failure is never mistaken for a bad model. It also checks that\r\nwhat `build_tasks.py` emits is exactly what the scorer expects. The environment mirrors the\r\nstructure of `polars_env` (already accepted into the Environments Program) and imports cleanly\r\nagainst `verifiers` 0.2.1.\r\n","encoding":"utf-8","truncated":false,"total_bytes":6385},"status":null}