{"data":{"kind":"file","path":"README.md","version_id":"jcj6183yi8258bi9tbz33170","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2723,"modified_at":"2026-09-10T03:27:26.605000","content_hash":"51ad8b9d7fc4c6cd5410ee3eb96558ce193570f3f7cc2ce2f687f81ba5f85529"},"entries":[],"content":"# Law Firm Software — Practice Management Benchmark\n\n> **Published by [Praesidium Compliance Systems Corporation](https://praesidiumsystems.ai)**\n> · Built on **[Stackhouse](https://github.com/ArjavDesa912/stackhouse)** ([stackhousedb.com](https://stackhousedb.com))\n> · **No LLM-as-judge anywhere** — all 26 tasks are graded by deterministic,\n> fail-closed Python verifiers asserting exact database state, never by a\n> model scoring the transcript.\n\nA 26-task law-firm practice-management benchmark (matters, clients,\nbilling/trust accounting, e-discovery, grant management) running against a\n**live Stackhouse BaaS** — packaged as one fully self-contained Docker\ncontainer (Postgres 15 + Stackhouse BaaS + prebuilt Vite frontend + OpenEnv\nenv server). No external services, no volumes, no API keys.\n\n- **Contract**: OpenEnv gym (`/reset`, `/step`, `/state` on port 8000)\n- **Image**: built from `proj/server/Dockerfile` (the unified bundle Dockerfile)\n- **Action space**: one Stackhouse REST call per step\n  (`{method, endpoint, payload, as_user}`)\n- **Reward**: 1.0 iff the task's standalone fail-closed verifier exits 0\n- **Episode isolation**: `reset()` restores a pristine template database\n  (frozen at image build time) in ~1-3s and injects a fresh per-episode nonce\n  (`EP-XXXXXXXX`) plus a +/-1-day-jittered episode date, so memorized answers\n  go stale every episode\n- **Task discovery**: `GET /tasks` (or the TaskProvider routes) lists all 26\n  tasks; select one via `reset(task_id=...)`\n- **Dependent-stage chains**: every task carries `horizon_tier >= 2` — a\n  second stage reads the state the first stage actually mutated or wrote\n  (never an independent recompute), so a wrong or skipped first stage\n  silently corrupts the second stage's output instead of erroring. One task\n  (`026_firmwide_month_end_close_pack`) chains 5 such stages plus a\n  6-historical-month-end point-in-time reconstruction (`horizon_tier: 6`).\n  Every aggregate is verified two independent ways (Python filter + SQL)\n  before being compared to the agent's report.\n- Seed: ~50,250 rows across 17 collections, correct-by-construction by\n  default with a small, counted, taxonomy-tagged set of deliberate defects\n  planted on top (see the public sample repo's `README.md`/`BUILD_NOTES.md`/\n  `QC_REPORT.md` for the full defect table, QC transcripts, and every task's\n  own `REDTEAM.md` for its specific dependency mechanism and a live\n  adversarial attack transcript proving it isn't cosmetic).\n\n**Difficulty is author-asserted and unmeasured** — no calibration run against\na frontier-model pool has been performed, so there is no claimed measured\npass rate for any task ahead of the eval results attached to this listing.\n","encoding":"utf-8","truncated":false,"total_bytes":2723},"status":null}