{"data":{"kind":"file","path":"README.md","version_id":"uk9n8e8qqcakfn41b2dguzm5","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5377,"modified_at":"2026-08-18T19:47:16.351000","content_hash":"e8eb53764aa2de9ebf9ab46d40dc2d49f5af9da05c23dc4e6b1711beefffb399"},"entries":[],"content":"# omp-coding\n\n`omp-coding` version `1.1.7` is a Prime Verifiers v1 environment. Each rollout\nruns one real OMP episode in a new task container. Prime records the policy\nturns. A second container grades the declared output files with sealed cases.\n\n## Public classes\n\nThe package exports three Verifiers v1 classes and one hosted loader:\n\n- `OmpTaskset` loads one fixed task split.\n- `OmpHarness` runs pinned OMP 17.2.15 through the Prime policy endpoint.\n- `OmpEnv` grades the collected files and records the reward.\n- `load_environment` connects the Prime hosted evaluator to the v1 environment.\n\nThe wheel contains all task data, OMP RPC support, candidate workers, and the\ntraining data exporter. The hosted loader converts evaluator data only. It\nkeeps `OmpEnv` as the execution path.\n\n## Runtime\n\nEach solver uses this image:\n\n```text\nnode@sha256:934240a162082fd8b8a2f90cd5114446443f1eba1c5378f6687167ca405e6584\n```\nThe task runtime has one CPU, 2 GiB of memory, 64 processes, and no network.\nThe harness downloads the pinned OMP file for the runtime architecture. It\nsupports Linux arm64 and x64. It checks the SHA-256 value and version before\nuse.\n\nOMP has five host tools:\n\n- `sandbox_read`\n- `sandbox_write`\n- `sandbox_edit`\n- `sandbox_exec`\n- `run_tests`\n\nThe file tools reject undeclared paths. `sandbox_exec` runs as the unprivileged\nuser and has a 30 second command limit. `run_tests` sends only aggregate case\ncounts to the model. The sealed case file stays outside the solver container.\n\n## Install\n\nFrom the repository root, install it in the Prime tool environment:\n\n```sh\nenv -u VIRTUAL_ENV -u UV_PROJECT_ENVIRONMENT \\\n  prime --plain env install omp-coding --path environments\n```\n\n## Evaluate\n\n```sh\nset -a\n. .env\nset +a\n\neval omp-coding \\\n  --model openai/gpt-4.1-mini \\\n  --num-tasks 1 \\\n  --no-push \\\n  --no-rich \\\n  --sampling.max-tokens 32768 \\\n  --env.taskset.split validation \\\n  --client.base-url https://openrouter.ai/api/v1 \\\n  --client.api-key-var OPENROUTER_API_KEY \\\n  --max-concurrent 1\n```\n\nFor a Prime hosted run, link the model API key to the environment. Then run:\n\n```sh\nprime --plain eval run dylantirandaz/omp-coding \\\n  --hosted \\\n  --follow \\\n  --model openai/gpt-4.1-mini \\\n  --api-client-type openai_chat_completions \\\n  --api-base-url https://openrouter.ai/api/v1 \\\n  --api-key-var OPENROUTER_API_KEY \\\n  --env-args '{\"split\":\"validation\"}' \\\n  --num-examples 1 \\\n  --rollouts-per-example 1 \\\n  --max-tokens 32768 \\\n  --max-concurrent 1 \\\n  --allow-sandbox-access \\\n  --allow-tunnel-access\n```\n\nThe result is a Prime v1 `traces.jsonl` file. The `tests` reward is the number\nof passed sealed cases divided by the total case count. The token limit stops\nmore model turns. It does not override sealed grading. The\n`token_budget_exceeded` metric records final use above the task limit. An\ninfrastructure failure that prevents grading gets reward `0.0`.\n\nThe `challenge` split has four larger tasks for model comparison. Do not use\nchallenge traces as training data.\n\nUse a 32,768-token response limit for challenge runs. Each challenge task has\na separate 262,144-token cumulative limit and a 900-second agent limit. A\nlarger task limit does not increase the response limit.\n\n## Train\n\nInstall the optional Metal packages:\n\n```sh\nuv sync --package omp-coding --extra metal\n```\n\nExport successful Prime traces:\n\n```sh\nuv run --package omp-coding --extra metal omp-coding-train export \\\n  outputs/TRAIN_RUN outputs/VALIDATION_RUN \\\n  --output dataset/v1\n```\n\nThe exporter accepts only completed traces with a `tests` reward of `1.0`. It\nwrites one sample for each successful trajectory. Each sample has all assistant\naction turns and the final assistant turn.\n\nRun MLX LoRA training:\n\n```sh\nuv run --package omp-coding --extra metal omp-coding-train run \\\n  --data dataset/v1 \\\n  --model mlx-community/Qwen3-4B-Instruct-2507-4bit \\\n  --adapter adapters/omp-coding-v1 \\\n  --iters 300 \\\n  --checkpoint-interval 50 \\\n  --max-seq-length 8192 \\\n  --num-layers 8\n```\n\nThe command prints the Metal backend, `gpu:0`, device name, architecture,\nmemory, MLX version, and checked result before training. It uses the selected\nmodel tokenizer. It keeps only complete samples that fit the sequence limit,\nand it reports the kept and removed sample counts. It calculates loss on all\nassistant turns. It masks the system prompt, user turns, and tool results.\nEach reported loss must be\nfinite. The command installs the adapter files only after all checks pass.\n\nCompare the base model with a saved checkpoint:\n\n```sh\nuv run --package omp-coding --extra metal omp-coding-evaluate \\\n  --model mlx-community/Qwen3-4B-Instruct-2507-4bit \\\n  --data dataset/v1 \\\n  --adapter adapters/omp-coding-v1 \\\n  --weights adapters/omp-coding-v1/0000300_adapters.safetensors \\\n  --workspace . \\\n  --split validation \\\n  --max-tokens 1024 \\\n  --num-rollouts 1 \\\n  --output adapters/omp-coding-v1/comparison.json\n```\n\nThe comparison uses the same tasks, parser, prompt, sampling values, and token\nlimits for the base model and the fused adapter. It reports the sealed reward\nand the OMP tool-protocol rates.\n\n## Release\n\nBuild the wheel:\n\n```sh\nuv build --wheel environments/omp_coding --out-dir dist\n```\n\nInspect the wheel before publication. It must contain `omp_coding/tasks`, the\nfive OMP RPC files, the three runtime workers, and the training and evaluation\nmodules.\n","encoding":"utf-8","truncated":false,"total_bytes":5377},"status":null}