{"data":{"kind":"file","path":"README.md","version_id":"zg89wmww3y66dnt5c2uy7pac","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2438,"modified_at":"2026-08-21T01:14:13.659000","content_hash":"66fc59d41a2a2c16530c9d8be4a33df81159f9ddea75c6d622f32057c9631437"},"entries":[],"content":"# sregym-env\n\nSREGym as a verifiers taskset: on-call incident response with procedural generation\nand deterministic scoring. No LLM judge anywhere.\n\nEach task is one incident. A seed builds a full production stack on disk: a FastAPI\nservice, SQLite databases, a git repo with months of history, nginx/systemd/cron\nconfig, and hours of logs and metrics that agree with each other. The world picks one\nof 8 stack identities (service name, package name, database files, route prefixes) so\nnothing carries over between episodes, then injects a fault from a library of 10\ntemplates: config typos, silent ledger divergence, unapplied migrations, cron\nwrite-lock storms, permission clamps, crash-looping dependency pins, rate limit\nmisconfigurations, db quota exhaustion, rotated shared secrets, and torn config\nwrites. There are also two-fault compositions.\n\nThe agent works through a sandboxed on-call toolset over MCP: paginated log reading,\nmetrics queries, file read/edit, an allow-listed shell, and service control. Each\nrollout gets its own live world. The broken service really runs, synthetic traffic\nkeeps flowing, and cron keeps firing.\n\nScoring is a deterministic three-part verifier that runs against the live world: did\nthe symptom actually resolve (HTTP probes), was the real root cause fixed (workarounds\nand code patches around the problem are caught), and was anything else damaged (file\nhashes, db rows, logs, git history, destructive commands). Reward is 1.0 only when all\nthree pass, otherwise 0.3*symptom + 0.7*root_cause, halved on collateral damage. The\nrubrics were calibrated against measured runs. Claude Sonnet 5 baselines range from 55\nto 100% depending on the template, and the standard/hard profiles plus composition\nland in the 25 to 60% band.\n\n## Usage\n\n```bash\nvalidate sregym-env -n 2          # model-free wiring check\neval sregym-env -n 10 -m <model>  # the taskset is infinite, so bound it with -n\n```\n\nConfig knobs under `--env.taskset.*`: `faults` (comma list or \"all\"), `difficulty`\n(baseline|standard|hard), `stack` (auto|classic|variant), `seed-start`, `max-steps`,\n`history-minutes`.\n\nThe default harness is a plain tool loop. The model only gets the sandboxed MCP tools,\nno bash and no host filesystem. Don't run this taskset with a shell-bearing harness,\nsince a raw shell bypasses the sandbox the scoring depends on.\n\nSource, measured baselines, and methodology: https://github.com/ericxu88/SREGym\n","encoding":"utf-8","truncated":false,"total_bytes":2438},"status":null}