{"data":{"kind":"file","path":"README.md","version_id":"alv2di76u247en2fh9lj15hi","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4439,"modified_at":"2026-09-02T16:00:27.375000","content_hash":"03ecfdc1b9fd87f36067f8cc47bb0592f74f3f0043f2399bb6cae4f761853144"},"entries":[],"content":"# Long-Horizon Medical Agent Benchmark — Veterinary Clinic Ops\n\n> **Published by [Praesidium Compliance Systems Corporation](https://praesidiumsystems.ai)**\n> · Built on **[Stackhouse](https://github.com/ArjavDesa912/stackhouse)** ([stackhousedb.com](https://stackhousedb.com))\n> · **No LLM-as-judge anywhere** — all 100 tasks are graded by deterministic,\n> fail-closed Python verifiers asserting exact database state, never by a\n> model scoring the transcript.\n>\n> This is the full 100-task environment — one of **100+ RL environments, each\n> with its own 500+-task suite, already in our build pipeline and shipping\n> over the coming weeks**, all engineered to this same reward-hacking-resistant\n> standard. A 10-task open sample (mission + gold + verifier, publicly\n> readable) is at\n> [github.com/ArjavDesa912/Long-Horizon-Medical-Agent-Benchmark-Veterinary-Clinic-Ops](https://github.com/ArjavDesa912/Long-Horizon-Medical-Agent-Benchmark-Veterinary-Clinic-Ops).\n>\n> **Contact us:** `arjav.desai@praesidiumsystems.ai` · `sam.heidler@praesidiumsystems.ai`\n\nA 100-task veterinary-clinic operations benchmark running against a **live\nStackhouse BaaS** — packaged as one fully self-contained Docker container\n(Postgres 15 + Stackhouse BaaS + prebuilt Vite frontend + OpenEnv env server).\nNo external services, no volumes, no API keys.\n\n- **Contract**: OpenEnv gym (`/reset`, `/step`, `/state` on port 8000)\n- **Image**: built from `proj/server/Dockerfile` (the unified bundle Dockerfile)\n- **Action space**: one Stackhouse REST call per step\n  (`{method, endpoint, payload, as_user}`)\n- **Reward**: 1.0 iff the task's standalone fail-closed verifier exits 0\n- **Episode isolation**: `reset()` restores a pristine template database\n  (frozen at image build time) in ~1-2s and injects a fresh per-episode nonce\n  (`EP-XXXXXXXX`), so memorized answers go stale every episode\n- **Task discovery**: `GET /tasks` (or the TaskProvider routes) lists all 100\n  tasks; select one via `reset(task_id=...)`\n- **Difficulty mix**: 20 easy / 48 medium / 32 hard across create, update,\n  delete, workflow, aggregation, repair, idempotency; backend, frontend and\n  websearch surfaces\n\n## Using the environment\n\n```python\nimport asyncio\nfrom openenv import EnvClient\n\nasync def main():\n    client = await EnvClient.from_docker_image(\n        \"registry.primeintellect.ai/<owner>/veterinary_clinic_system:latest\"\n    )\n    async with client:\n        result = await client.reset(task_id=\"021_overdue_invoice_flagging\")\n        print(result.observation.instruction)  # the mission text\n        print(result.observation.nonce)        # EP-XXXXXXXX episode batch code\n        # agent works by emitting Stackhouse REST calls:\n        result = await client.step({\n            \"method\": \"GET\",\n            \"endpoint\": \"/v1/query/veterinary_clinic_system_billing_invoices?limit=5\",\n            \"payload\": None,\n            \"as_user\": \"verifier\",\n        })\n        print(result.reward, result.observation.reward_reason)\n\nasyncio.run(main())\n```\n\nPorts: `8000` env server (OpenEnv API), `8080` Stackhouse BaaS REST,\n`5110` vet-clinic frontend.\n\n## Credentials (baked into the image, intentionally not secret)\n\n- `verifier` login: `rl-admin@rl.local` / `RLVerifier2025!` (read/write API user)\n- `app_admin` login: `admin@pawsclinic.com` / `Demo123!`\n\n## Anti-reward-hacking design\n\n- Every task fails on pristine state (no-op/random baselines proven per task)\n- Fresh nonce + randomized per-episode parameters defeat hard-coded answers\n- Canary sha256 checks prove out-of-blast-radius collections are untouched\n- Verifiers are read-only, fail-closed, and re-hash task files at grade time\n- Grading code lives under `/app/env/grading`, read-only to non-root users;\n  the agent's only surface is the HTTP API\n- Postgres local-socket auth is `peer` and TCP requires scram-sha-256, so a\n  shell user inside the container cannot bypass the BaaS\n\n## How the image is built\n\n`proj/server/Dockerfile` is a unified single-container build: it compiles the\nStackhouse BaaS from source, builds the frontend, seeds Postgres, freezes the\npristine template DB, and installs the env server — no base image prebuild, no\nexternal services, no volumes. You don't need to build it yourself: this\npackage runs as a Prime Sandbox from the image already registered in\n`proj/.build.json`, so `EnvClient.from_docker_image(...)` (above) is all you\nneed — no local Docker required.\n","encoding":"utf-8","truncated":false,"total_bytes":4439},"status":null}