{"data":{"kind":"file","path":"README.md","version_id":"uk6wjpwwa696kdz6u1xx562a","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3862,"modified_at":"2026-07-12T15:12:59.935000","content_hash":"c4977573f6dbf8ae4fe29683a054da22a73e9a4e5bb6fbe019c91866531e04cc"},"entries":[],"content":"# crypto-law-howey\n\nA leakage-resistant Verifiers v1 environment for training and evaluating small\nmodels on closed-universe U.S. crypto-offering issue spotting. Each scenario\nrequires factor-by-factor Howey analysis, exact quoted evidence, a public legal\nconnection, a benchmark conclusion, authority IDs, and targeted missing facts.\nIt is an educational benchmark, not legal advice.\n\nThe scope is deliberately narrow: a synthetic federal investment-contract triage\nrubric, not a comprehensive statement of U.S. crypto law as of the card date and\nnot a prediction of any court's result. The common-enterprise convention uses\nhorizontal pooling/linkage. District-court token cases are record-specific\nillustrations; the Supreme Court decisions supply the controlling closed-universe\nprinciples. Before any real-world deployment, qualified counsel should review the\nauthority summaries, operational rules, and representative outputs.\n\n## Contract\n\n- Task: one coherent scenario and one response containing exactly one structured JSON object.\n- Train/dev/test: 480/120/80 reserved coherent feature graphs with zero graph overlap. Atomic fact clauses, question templates, and entity vocabularies are split-owned.\n- Grounding: `factor_analyses` contains one object per factor with a disposition matching the top-level label, an exact 2–40 word fact-record quote, a factor-specific public connection enum, and one concise evidence-bound reasoning sentence. Private scoring aligns quotes to generator-owned factor spans; it does not reward lexical cue lists or use an LLM judge.\n- Reward: deterministic weighted score over four factor labels (40%), conclusion (22%), authority precision/recall (12%), missing-fact F1 (13%), per-factor grounding (10%), and disclaimer (3%). Invalid schema is gated to zero.\n- Harness: every comparator receives the same closed-universe authority card and exact output schema. A public-contract canonicalizer treats the model's exact evidence span and connection enum as its substantive choices, derives the compatible factor disposition/conclusion/missing-fact fields, and replaces only the short reasoning form. It never reads private gold, rejects nonexact quotes or unknown routes, and retains the raw response plus canonical response in `harness_trace`. The schema-only repair user remains available for a frozen two-turn protocol and contains no gold or reward details.\n- Holdout policy: any observed blind set is consumed diagnostic evidence only. A new deploy-time blind set is independently authored only after the data, scorer, authority card, one-turn harness, selected checkpoint, deployment behavior, and competition thresholds are frozen. It is not shipped in this package.\n\n## Data provenance and limits\n\nAll scenarios are deterministic synthetic combinations of hand-authored fact\nclauses and coherence constraints. No client records, scraped legal questions,\nbenchmark answers, or model-generated labels are used. Train/dev/test reserve\ndifferent semantic graphs and wording families, but synthetic success does not\nestablish performance on live matters, new jurisdictions, adversarial facts, or\nlaw outside the supplied authority card.\n\n## Quickstart\n\n```bash\nprime env install crypto-law-howey\nprime eval run crypto-law-howey -m Qwen/Qwen3.5-2B -n 20 -r 1 -s -C task\nprime eval run crypto-law-howey -m gpt-5.5 -p openai --api-client-type openai_responses -n 20 -r 1 -s -C task\n```\n\nPublic diagnostic split:\n\n```bash\nprime eval run crypto-law-howey \\\n  -a '{\"config\":{\"taskset\":{\"split\":\"test\",\"eval_size\":80},\"harness\":{\"max_turns\":1}}}' \\\n  -n 80 -r 1 -s -C task\n```\n\nThe final parity verifier compares runs with identical environment source, task\nIDs, prompt hash, sampling, and harness configuration. It fails closed on missing\ntask payloads, run errors, protocol drift, or a trained-model schema rate below\n99%.\n","encoding":"utf-8","truncated":false,"total_bytes":3862},"status":null}