{"data":{"kind":"file","path":"README.md","version_id":"owxjy7m0v4yapthe1cn2du2c","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":21201,"modified_at":"2026-07-22T16:34:47.007000","content_hash":"fd22fcea8d5e57b62f83faa00387a1864230ac3469aa955bcec28849f983f2da"},"entries":[],"content":"# valid-action\n\nA Prime Intellect `verifiers.v1` environment that measures whether a model can complete one corporate-governance action end-to-end through state-changing tool calls. The agent must read fictional corporate records, identify the operative approval rules, and execute the action through the available tools. The reward is deterministic: `min(1.0, oracle_minimum_process_cost / actual_process_cost)`. An optional LLM judge gate (v0.2.0) can re-check the executed action's economics; it is **off by default** and, per the honest assessment below, adds little on top of the deterministic check it re-verifies. See \"Judge gate\" below.\n\n### Overview\n- **Environment ID**: `valid-action`\n- **Short description**: Tool-using corporate-governance environment. Agent terminates a synthetic corporate action (contract, equity grant, security issuance, related-party transaction, token sale, or subsidiary financing) by orchestrating create + authorize + execute calls under a typed rule graph.\n- **Tags**: `legal`, `tool-use`, `multi-turn`, `corporate-governance`, `deterministic`, `synthetic`\n\n### Datasets\n- **Primary dataset(s)**: `valid-action-train`, `valid-action-eval`\n- **Source**: procedurally generated from 9 hand-authored golden templates (G1-G9) covering all six action families\n- **Split sizes**: default 64 train / 32 eval; configurable via `num_train` and `num_eval`\n\n### Task\n- **Type**: multi-turn tool use (14-18 turns depending on difficulty)\n- **Output format**: tool calls only — prose does not change the world and receives no credit\n- **Rubric**: single reward `final_validity_efficiency = min(1.0, oracle / max(1, actual))` where `oracle` is the deterministic DP-computed minimum number of authorization steps and `actual` is the agent's attempt count (only valid steps counted, redundant attempts penalized via `actual > oracle`)\n\n### Quickstart\n\n```bash\n# Run an evaluation against any model tier\nprime eval run valid-action -m openai/gpt-4.1-mini -n 4 -r 1\n\n# Or evaluate the environment offline via oracle-replay\ncd environments/valid_action\npython scripts/baselines.py --examples 32 --difficulty medium\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `difficulty` | `easy` \\| `medium` \\| `hard` | `medium` | Affects max_turns (10/14/18) and number of distractor records (0/2/4) |\n| `seed` | int | `17` | Master seed; train uses `seed`, eval uses `seed + split_stride()` |\n| `num_train` | int | `64` | Number of train rows |\n| `num_eval` | int | `32` | Number of eval rows |\n| `preflight_budget` | int | `1` | How many preflight_action calls the agent may make per rollout |\n| `max_search_results` | int | `5` | Per-call cap on search_records results |\n| `include_distractors` | bool | `true` | Whether to inject distractor records into the world |\n| `system_prompt` | object \\| null | `null` | Override the default agent prompt |\n| `max_turns_override` | int \\| null | `null` | Force a specific max_turns value |\n| `judge_model` | str \\| null | `null` | Judge model name/route. **Setting this is the only thing that turns the judge gate on** — every other default reproduces r-mode byte-identically. See \"Judge gate\" below. |\n| `judge_base_url` | str \\| null | `null` | Optional OpenAI-compatible base URL for the judge client |\n| `judge_api_key_var` | str | `\"JUDGE_API_KEY\"` | Env var name holding the judge API key; checked eagerly at `load_environment()` time when `judge_model` is set |\n| `judge_gate` | bool | `true` | Master on/off for the gate once `judge_model` is set (set `false` to configure a judge model without activating the gate) |\n| `judge_strict` | bool | `true` | On judge failure (majority of votes unreachable/unparseable after retries): `true` withholds the rollout's reward (collapses to 0.0); `false` leaves the deterministic reward standing. Either way `judge_error=1.0` is set — never a silent ungated score |\n| `judge_votes` | int | `3` | Parallel judge calls per gated rollout; confirmation requires a majority |\n| `judge_max_retries` | int | `2` | Retries per vote on parse failure or transport error |\n\n### Tools\n\nThe agent has access to eight tools organized into three toolsets. The `action_create` toolset is per-task filtered so only the create tool matching the action family is visible.\n\n**Search / read toolset** (always visible):\n- `search_records(query, entity_id?, record_type?)` — deterministic BM25 over rendered prose\n- `read_record(record_id)` — returns sections with `source_rule_ids` stripped\n- `inspect_register(entity_id, register_type, as_of_date)` — returns directors / officers / committee members / cap table / security capacity / equity plan capacity / subsidiaries\n\n**Authorization toolset** (always visible):\n- `record_authorization(action_id, authorizer_id, method, participant_ids, recused_ids?, disclosure_record_ids?)` — records one vote or consent step\n- `preflight_action(action_id)` — deterministic dry-run of the full requirement graph; consumes one preflight credit\n\n**Per-task create toolset** (one of six, filtered by action family):\n- `create_material_contract(...)`, `create_equity_grant(...)`, `create_security_issuance(...)`, `create_related_party_transaction(...)`, `create_token_treasury_transaction(...)`, `create_subsidiary_financing(...)`\n\n**Terminal tool**:\n- `execute_action(action_id, signatory_person_id)` — irreversible; sets `state.done=True` and stops the rollout\n\n### State model\n\nThe hidden state is a `ValidActionWorld`: an entity (e.g. `ent_lumen`), people, role appointments, governance bodies, security classes, holder positions, equity plan capacity, legal records (bylaws, board minutes, contracts, conflict disclosures, etc.), a typed requirement graph (AuthorizationRequirement, ConsentRequirement, CapacityRequirement, ConflictRequirement, SequenceRequirement, SignatoryRequirement, TermsRequirement, ProhibitionRequirement, composed via AllOf/AnyOf), and one action request. The agent-visible projection strips `source_rule_ids` and never reveals the oracle solution or the requirement graph structure.\n\n### Reward\n\nSingle reward `final_validity_efficiency` (still exactly one reward signal — the optional judge gate lives *inside* it, never as a second additive reward):\n- 0.0 if execution is missing, the action_id doesn't match, or any required step is missing\n- `min(1.0, oracle_minimum_process_cost / max(1, actual_process_cost))` otherwise\n- when the judge gate is on (see below) and that value is `> 0.0`, the gate can only collapse it to `0.0` — it never raises it\n\nTwelve metrics are also emitted: `execution_attempted`, `execution_valid`, `process_cost`, `process_efficiency_ratio`, `turns_used`, `tool_error_count`, `preflight_used`, `validation_defects` (per-defect-category counts), plus four judge-gate telemetry metrics (`judge_gated`, `judge_error`, `final_validity_efficiency_pre_gate`, `final_validity_efficiency_post_gate`) that are always emitted, all 0-weight, and read 0.0 when the gate is off.\n\n### Judge gate (optional, off by default; v0.2.0)\n\n**Honest assessment first:** this is the lowest-value judge gate in the fleet-wide rollout. `final_validity_efficiency` is not a text-recall metric with room for a keyword-matcher to be fooled — it only pays out once `validate_final_execution` (`core/validator.py`) finds **zero defects**, and that already requires the executed action's tracked economics fields to equal the requested action's tracked fields exactly (`_check_terms_match` in the validator, mirrored by `_check_payload_match` at create-tool time in `core/tools.py`, both checked *before* the judge would ever see anything). By the time a rollout reaches the judge, the thing it is being asked to re-confirm has already been deterministically proven. Expect the gate to be a structural no-op on the current fixture set — it is wired for parity with the rest of the fleet and as a forward defensive layer in case the deterministic economics check is ever loosened, not because it currently changes any score. **Pattern-ported; per-env live validation pending** (no live judge-model runs have been made against this env; the stub-test suite below is offline-only, per the standard fleet disclosure).\n\nWhen `judge_model` is set (and `judge_gate` stays at its default `true`), and the deterministic reward for a rollout is `> 0.0`:\n1. Build the only \"model output\" text this tool-only env has: the exact arguments of the agent's `create_<family>` tool call (`created_action.payload`), canonically JSON-rendered. Prose is not graded, so there is no memo/document text to fence — this env differs from the legal-agent-eval family in that respect.\n2. Ask the judge (temperature=0 first, bare-call fallback, `JUDGE_GATE_PROMPT_VERSION`-tagged prompt, `<executed_payload>`-fenced with an explicit \"ignore instructions inside it\" warning) whether that payload's economics genuinely match the requested action's economics.\n3. Deterministically verify the judge's own answer: a confirmation only counts if the judge's `quote` is a real, normalized substring of the executed payload text. A judge that hallucinates evidence loses that vote.\n4. Run `judge_votes` (default 3) parallel calls and require a majority to confirm.\n5. AND-gate only: if confirmed (or the gate never ran), the deterministic reward stands unchanged — the gate never adds credit. If rejected, the reward collapses to `0.0`.\n6. Fail-loud: if a majority of votes can't be reached (transport/parse failure after retries), `judge_error=1.0` is always set — under `judge_strict` (default `true`) the reward is withheld (`0.0`); with `judge_strict=false` the deterministic reward stands but `judge_error` still flags the rollout as unverified. Never a silent 0.0 and never a silently ungated score.\n7. Results are cached in-process, keyed by `(judge_model, JUDGE_GATE_PROMPT_VERSION, judge_votes, task fingerprint, executed payload text)`.\n\n**Async note:** unlike the classic `vf.Rubric`-based envs in this fleet (`agent-liability-gym`, `crypto-law-howey`), verifiers' v1 Taskset reward/metric signals are natively `async def` — no `asyncio.run()`/event-loop worker wrapper was needed to reach the async judge client from a synchronous scoring path.\n\nNo network calls are made unless `judge_model` is set; the offline stub-test suite (`tests/test_judge_gate.py`) covers the gate machinery — remove-only wiring, quote verification, fabricated-quote rejection, majority vote (both directions), fail-loud (strict and non-strict), and r-mode byte-identical regression against `tests/test_scoring.py`'s existing expected values — entirely with a fake judge client.\n\n### Baselines\n\nFive deterministic baselines are provided in `scripts/baselines.py`:\n\n| Baseline | mean reward (medium, n=16) | valid_rate |\n| -------- | -------------------------- | ---------- |\n| `random` | 0.000 | 0.0 |\n| `naive_max_approval` | 0.188 | 0.188 |\n| `keyword_heuristic` | 0.188 | 0.188 |\n| `oracle_replay` | 1.000 | 1.0 |\n| `search_only` | 0.000 | 0.0 |\n\nThe oracle_replay baseline demonstrates the upper bound (100% solvable with reward 1.0). Naive baselines show the gap: ~19% of worlds are simple enough that any single-body meeting approval is sufficient. Random and search-only baselines score 0, validating that prose-only and tool-error-only paths do not accidentally score reward.\n\n### Example trajectory\n\nFor a single G3 (employee RSU) world on seed 17:\n\n```\nturn 1: search_records(query=\"committee\", entity_id=\"ent_lumen\")\nturn 2: read_record(record_id=\"rec_bylaws_g3\")\nturn 3: inspect_register(entity_id=\"ent_lumen\", register_type=\"directors\", as_of_date=\"2028-09-15\")\nturn 4: create_equity_grant(entity_id=\"ent_lumen\", recipient_person_id=\"prs_maya_g3\",\n          award_type=\"rsu\", units=42000, vesting_months=48, cliff_months=12,\n          effective_date=\"2028-09-15\")\nturn 5: record_authorization(action_id=\"act_...\", authorizer_id=\"body_comp_g3\",\n          method=\"meeting\", participant_ids=[\"role_comp_a_g3\", \"role_comp_b_g3\"])\nturn 6: execute_action(action_id=\"act_...\", signatory_person_id=\"prs_ceo_g3\")\n→ score: 1.0 (oracle=1, actual=1)\n```\n\n### Reproducibility\n\nAll randomness is seeded. The generator seeds train and eval splits from a single `seed` plus `split_stride()` and uses a salted SHA256 retag so seeded worlds share a structural fingerprint but have unique IDs. To reproduce the eval set used by CI:\n\n```bash\ncd environments/valid_action\npython -m pytest tests/ -q   # 145 tests, all offline\npython scripts/baselines.py --examples 32 --difficulty medium --seed 17\n```\n\n### Limitations\n\n- Synthetic record prose is template-driven (16 wording variants) and may not reflect real-world document variation.\n- The rule graph covers six action families common to US-style private-company governance; public-company rules (SEC filings, 14A disclosures, NYSE/Nasdaq listing standards) are out of scope.\n- The oracle is a deterministic DP over a typed AND/OR graph; it does not simulate negotiation, amendment, or post-execution events.\n- All six action families are covered, but the world is single-action-per-rollout by design; multi-action sequencing across roles is not in scope.\n\n### Version history\n\n- `v0.1.0` — initial scaffold via `prime env init valid-action --multi-file`\n- `v0.2.0` — world checkpoint: 12 core modules, G1-G9 fixtures, generator, validator, oracle\n- `v0.3.0` — environment checkpoint: Taskset wiring, toolsets, reward/metrics, 8 tool functions\n- `v0.4.0` — release checkpoint: 5 baselines, 103 tests, public README, trace renderer stub\n\n## Changelog\n\n- **0.2.1** (2026-07-22): renderer content-fidelity fix (retest-triggered; grok-4.5 hit the same\n  0.000-reward / 0% execution_attempted floor as gpt-5.4-mini with 12/15 rollouts never returning\n  a single non-empty search hit in 14 turns — traced to the env, not the model). Three defects,\n  all in the record-rendering / retagging path, none in the reward math:\n  1. `_fill_template` (`core/render.py`) read numeric substitution values (`units`/`reserved`/\n     `available`/`floor`/`threshold`/`amount`) off `RecordSection.model_dump()` — a model that has\n     never had those fields (`RecordSection` is just `section_id`/`heading`/`text`/\n     `source_rule_ids`). Every numeric placeholder silently rendered `0` regardless of the world's\n     real economics (e.g. \"reserves 0 units for awards; 0 remain available\" when the real plan\n     reserve was 500000/400000). Fixed: values are now resolved from the requirement-graph node a\n     section's `source_rule_ids` actually points at (`CapacityRequirement.min_available` +\n     `plan_capacities`/`security_classes` lookups by `target_id`, `SignatoryRequirement\n     .max_commitment` — falling back to a sibling `SignatoryRequirement` when the section's own\n     node is an `AuthorizationRequirement`, per G1 — `TermsRequirement.expected_payload`'s\n     `{\"op\",\"value\"}` constraints, `ConsentRequirement.consent_holder_id`, `ConflictRequirement\n     .related_person_id`), never fabricated. A template variant is only selected if every\n     placeholder it needs actually resolved; every numeric WORDING_VARIANTS key now also has a\n     qualitative, placeholder-free fallback so an unresolved value degrades to an honest sentence,\n     never a `0` or a raw `{placeholder}`. Also fixed: `body` was resolved via\n     `body_by_id(record.entity_id)` — entity_id is never a body_id, so this was always a miss;\n     now resolved via the requirement node's `authorizer_id`. Also fixed: the `plan_reserve`\n     template's default plan name (\"the equity plan\") collided with the template's own leading\n     \"The \", rendering \"The the equity plan reserves...\"; now uses the record's real title.\n  2. `_template_key` (`core/render.py`) matched on heading substrings (`\"delegation\" in heading\n     and \"committee\" in heading`, `\"...and 'borrow' in heading\"`, etc.) that don't appear in any\n     of the 15 actual section headings across G1-G9 (the real headings are e.g. \"Committee\n     authority\", \"Subsidiary board authority\", \"Facts disclosed\") — 9 of 15 headings (60%) fell\n     through to the bare `\"[record_type] heading: \"` stub with zero substantive content,\n     independent of the numeric bug above. Fixed: matcher rewritten against the fixture set's\n     actual headings; three new WORDING_VARIANTS keys added (`authorized_capacity`,\n     `conflict_disclosure`, `investor_consent`) for headings that had no template at all. Combined\n     with (1), every section across all 9 golden fixtures now renders real, correct prose (see\n     `tests/test_render_and_search.py`'s new pinning tests) — the search/read tools have\n     substantive, accurate content for an agent to find and reason over, which they effectively\n     did not before.\n  3. `tool_search_records` (`core/tools.py`) hard-errored on `record_type=\"\"` (\"unknown\n     record_type \") and silently zero-matched on `entity_id=\"\"` (treated as a literal,\n     unmatchable filter value) instead of treating an empty string as \"not provided\" the same as\n     `None`/omitted — a routine pattern for tool-calling models filling an optional arg they have\n     no value for. In the retest trace this alone accounted for roughly half of every rollout's\n     tool_error_count. Fixed: `entity_id`/`record_type` now normalize `\"\" -> None` at the top of\n     the function.\n  Independently while auditing the retag path: `_retag_requirement_graph` (`core/generator.py`)\n  had no `ConflictRequirement` branch, so `related_person_id`/`required_approval_body_id` were\n  never remapped off their pre-retag fixture IDs on any generated (non-raw-fixture) world —\n  confirmed by direct reproduction of the pre-fix logic (`tests/test_world_checkpoint.py\n  ::test_conflict_requirement_ids_retagged_for_generated_worlds`). Fixed by adding the missing\n  branch. Measured effect on `oracle_minimum_process_cost` across a 64-example G4/G9 sample:\n  **none observed** — the DP's step-count metric doesn't appear to price the conflict gate\n  separately from the authorization step it rides on, so this looks latent rather than\n  score-moving today; flagged and fixed anyway since it's a genuine correctness gap (any future\n  validator/oracle logic keying off those two fields directly would silently break on every\n  generated world) — not re-verified beyond the G4/G9 sample checked here.\n  13 new regression tests (`test_tools.py` x2, `test_render_and_search.py` x10,\n  `test_world_checkpoint.py` x1); 145 tests total, all pass. No public API/param changes; `info`\n  schema and reward math untouched.\n- **0.2.0** (2026-07-22): optional LLM judge gate on `final_validity_efficiency`, off by default\n  (fleet-wide judge-gate port; AK directive 2026-07-22). `judge_model`/`judge_base_url`/\n  `judge_api_key_var`/`judge_gate`/`judge_strict`/`judge_votes`/`judge_max_retries` params added\n  to `ValidActionTasksetConfig` and threaded into every dataset row's `info`; default\n  `judge_model=None` reproduces r-mode byte-identically (all 107 pre-existing tests pass\n  unchanged, unmodified). The gate is remove-only (AND-gate inside the single reward, never a\n  second additive reward — `len(taskset.rewards) == 1` still holds), quote-verified\n  (deterministically checked against the executed action's tool-call payload — the only \"model\n  output\" text this tool-only env has), majority-voted (default 3), and fail-loud\n  (`judge_error=1.0` always set on judge failure; `judge_strict` default `true` withholds reward,\n  `false` leaves the deterministic reward standing but still flags `judge_error`). New telemetry\n  metrics: `judge_gated`, `judge_error`, `final_validity_efficiency_pre_gate`,\n  `final_validity_efficiency_post_gate` (all 0-weight). Honest caveat (see README \"Judge gate\"):\n  the deterministic validator already proves the exact claim the judge re-checks before the judge\n  ever runs, so this gate is expected to be a structural no-op on the current fixture set — ported\n  for fleet parity and as a forward defensive layer, not because it currently changes any score.\n  Status: **pattern-ported, per-env live validation pending** (no live judge-model runs made\n  against this env yet). New regression suite: `tests/test_judge_gate.py` (25 tests, offline-only,\n  fake judge client — no network). 132 tests total.\n- **0.1.3** (2026-07-21): hosted-harness compatibility — rollout state is now JSON-serializable\n  end-to-end. Tools previously stored model objects (`CreatedAction`/`AuthorizationAttempt`/\n  `ExecutionAttempt`) and a `LexicalIndex` cache in state; this passed in-process eval but\n  failed Prime CI's zmq transport with `Task and State values must be JSON-serializable`.\n  Writers now store `model_dump(mode=\"json\")`; readers revalidate via `ensure_created_action`/\n  `ensure_execution_attempt`/`ensure_authorization_attempts` (same contract as `ensure_world`);\n  the search index moved to a bounded module-level cache. New regression test:\n  `tests/test_state_serializable.py`. 107 tests.\n- **0.1.2**: `ensure_world()` requirements round-trip fix (reward-path crash on hosted evals);\n  verifiers pinned `>=0.1.14,<0.2.0` (`vf.TasksetConfig` is absent from 0.2.x).\n","encoding":"utf-8","truncated":false,"total_bytes":21201},"status":null}