{"data":{"kind":"file","path":"README.md","version_id":"lv4pbm6cw43fyg1cyo7vivoy","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4683,"modified_at":"2026-08-17T08:40:17.267000","content_hash":"5c76a8b3f91918be7c7d5f1b8e37d58d4364a9eee24ff34eb83d668665ab549a"},"entries":[],"content":"# MCMCEnv — Prime Intellect environment\n\nMCMCEnv is a standalone Verifiers v1 package for reinforcement learning over\none-shot MCMC sampler design. One task is one immutable posterior instance; the\npolicy selects a sampler, the environment\nruns multiple chains, and negative Wasserstein distance scores the result.\n\n```text\ncategorical action -> canonical typed action -> MCMC chains -> -Wasserstein reward\n```\n\nThe trainable `wasserstein_reward` is the negative standardized sliced\nWasserstein-1 distance from fresh evaluator-private reference draws. The raw\nreference draws and posterior moments never appear in policy state or prompts;\nonly the configured scalar/full/binary reward projection can be returned.\nCODA diagnostics remain trace telemetry and have zero weight in reward. The\naction cannot change the model, data, monitored nodes, chain budget, reference\nevaluator, or reward function.\n\n## Runtime boundary\n\nThe Python wheel is self-contained. It includes the benchmark generator,\nvalidator, reward implementation, prompt, and versioned R verifier sources; it\ndoes not import the parent repository at runtime. R is never launched as a host\nsubprocess. The environment provisions the task's `vf.Runtime`, uploads a\ncontent-hashed R bundle to it, and invokes `Rscript` through\n`vf.Runtime.run`. The same runtime is reused for every pilot in the episode.\n\nThe task-provided image supplies the locked execution toolchain: R 4.6.1,\nNIMBLE 1.4.2, coda 0.19-4.1, and jsonlite 2.0.0. Build it from the environment\ndirectory:\n\n```bash\ndocker build \\\n  -f environments/mcmcbench_prime/Dockerfile.runtime \\\n  -t mcmcbench-nimble:0.2.0 \\\n  environments/mcmcbench_prime\n```\n\nPush that image to a registry accessible by the selected Prime runtime. Set its\npullable reference with `taskset.task.runtime_image`; the task copies it to\n`TaskData.image` and declares `NEEDS_CONTAINER = True`. The local default is\n`mcmcbench-nimble:0.2.0`.\n\n## Install and validate\n\nUse Python 3.12 or 3.13 and install the package directly:\n\n```bash\nuv pip install -e environments/mcmcbench_prime\nprime env install punnettsquare9331/mcmcenv@0.7.2\nuv run validate mcmcenv --only-gold --num-tasks 1\n```\n\nFor a focused transport test that does not require Docker or R:\n\n```bash\nPYTHONPATH=environments/mcmcbench_prime \\\n  python -m pytest environments/mcmcbench_prime/tests/test_runtime_execution.py\n```\n\nThe native package exports exactly one `Taskset` and one `Env`. Procedural mode\nprovides a deterministic, unbounded training stream; fixed mode provides the\nscientific evaluation suite. Task configuration controls the feedback condition\n(`full`, `scalar`, `binary`, or `none`), pilot limit, invalid-action limit,\nchain budget, seed, NIMBLE compilation, and runtime image. A terminal\n`{\"mode\":\"stop\"}` deploys the latest valid sampler configuration, or exact\nNIMBLE defaults when selected immediately.\n\nThe executable action language covers the installed NIMBLE sampler registry,\nordered/overlapping/repeated kernels, initialization, proposal covariance, and\nan environment-owned Metropolis engine. That engine supports Gaussian and\nheavy-tailed Student-t random walks, explicit or generated replica-temperature\nladders, and partial or full symbolic coordinate maps. SymPy verifies both\ninverse compositions, the signed Jacobian, and that the determinant is globally\nnonzero before execution. Maps requiring domain assumptions stay gated on a\ntrusted external Lean artifact. Structured transition telemetry records the\nexecuted proposal, swaps, and transformed nodes without changing the\nWasserstein-only reward. Arbitrary R is never executed.\n\nFor curriculum learning, the same validator also accepts eight compact sampler\npresets (default, scalar/block NIMBLE random walk, scalar NIMBLE slice,\nGaussian or Student-t Metropolis blocks, scalar Metropolis, and replica-tempered\nMetropolis). A preset contains a strategy name and a few\nnumeric knobs and deterministically expands into the ordinary typed action.\nThis gives small base models executable actions early in training; advanced\ntraining can then switch to the unrestricted low-level schema without changing\nthe target, transition implementation, or reward.\n\nThe recommended RL bootstrap uses a 50-way categorical action adapter. The\npolicy replies with `A00`-`A49`; the adapter expands it to canonical typed JSON.\nEmbedded tokens are recovered and any other text is deterministically projected\nonto the catalog, so every rollout reaches the sampler and receives a real\nposterior-discrepancy reward. `action_codec_exact` records clean deployment\nsyntax separately. The unrestricted JSON language remains available for later\nadvanced experiments.\n","encoding":"utf-8","truncated":false,"total_bytes":4683},"status":null}