{"data":{"kind":"file","path":"README.md","version_id":"psdexjkjjqqovbbzownl1ipx","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":8146,"modified_at":"2026-07-01T03:46:45","content_hash":"ce4f80c06f66a97b5b8e34a4471431681c4b6eec772e7980e2865bd165372d53"},"entries":[],"content":"# valid-action\n\nA Prime Intellect `verifiers.v1` environment that measures whether a model can complete one corporate-governance action end-to-end through state-changing tool calls. The agent must read fictional corporate records, identify the operative approval rules, and execute the action through the available tools. The reward is deterministic: `min(1.0, oracle_minimum_process_cost / actual_process_cost)`. There is no LLM judge.\n\n### Overview\n- **Environment ID**: `valid-action`\n- **Short description**: Tool-using corporate-governance environment. Agent terminates a synthetic corporate action (contract, equity grant, security issuance, related-party transaction, token sale, or subsidiary financing) by orchestrating create + authorize + execute calls under a typed rule graph.\n- **Tags**: `legal`, `tool-use`, `multi-turn`, `corporate-governance`, `deterministic`, `synthetic`\n\n### Datasets\n- **Primary dataset(s)**: `valid-action-train`, `valid-action-eval`\n- **Source**: procedurally generated from 9 hand-authored golden templates (G1-G9) covering all six action families\n- **Split sizes**: default 64 train / 32 eval; configurable via `num_train` and `num_eval`\n\n### Task\n- **Type**: multi-turn tool use (14-18 turns depending on difficulty)\n- **Output format**: tool calls only — prose does not change the world and receives no credit\n- **Rubric**: single reward `final_validity_efficiency = min(1.0, oracle / max(1, actual))` where `oracle` is the deterministic DP-computed minimum number of authorization steps and `actual` is the agent's attempt count (only valid steps counted, redundant attempts penalized via `actual > oracle`)\n\n### Quickstart\n\n```bash\n# Run an evaluation against any model tier\nprime eval run valid-action -m openai/gpt-4.1-mini -n 4 -r 1\n\n# Or evaluate the environment offline via oracle-replay\ncd environments/valid_action\npython scripts/baselines.py --examples 32 --difficulty medium\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `difficulty` | `easy` \\| `medium` \\| `hard` | `medium` | Affects max_turns (10/14/18) and number of distractor records (0/2/4) |\n| `seed` | int | `17` | Master seed; train uses `seed`, eval uses `seed + split_stride()` |\n| `num_train` | int | `64` | Number of train rows |\n| `num_eval` | int | `32` | Number of eval rows |\n| `preflight_budget` | int | `1` | How many preflight_action calls the agent may make per rollout |\n| `max_search_results` | int | `5` | Per-call cap on search_records results |\n| `include_distractors` | bool | `true` | Whether to inject distractor records into the world |\n| `system_prompt` | object \\| null | `null` | Override the default agent prompt |\n| `max_turns_override` | int \\| null | `null` | Force a specific max_turns value |\n\n### Tools\n\nThe agent has access to eight tools organized into three toolsets. The `action_create` toolset is per-task filtered so only the create tool matching the action family is visible.\n\n**Search / read toolset** (always visible):\n- `search_records(query, entity_id?, record_type?)` — deterministic BM25 over rendered prose\n- `read_record(record_id)` — returns sections with `source_rule_ids` stripped\n- `inspect_register(entity_id, register_type, as_of_date)` — returns directors / officers / committee members / cap table / security capacity / equity plan capacity / subsidiaries\n\n**Authorization toolset** (always visible):\n- `record_authorization(action_id, authorizer_id, method, participant_ids, recused_ids?, disclosure_record_ids?)` — records one vote or consent step\n- `preflight_action(action_id)` — deterministic dry-run of the full requirement graph; consumes one preflight credit\n\n**Per-task create toolset** (one of six, filtered by action family):\n- `create_material_contract(...)`, `create_equity_grant(...)`, `create_security_issuance(...)`, `create_related_party_transaction(...)`, `create_token_treasury_transaction(...)`, `create_subsidiary_financing(...)`\n\n**Terminal tool**:\n- `execute_action(action_id, signatory_person_id)` — irreversible; sets `state.done=True` and stops the rollout\n\n### State model\n\nThe hidden state is a `ValidActionWorld`: an entity (e.g. `ent_lumen`), people, role appointments, governance bodies, security classes, holder positions, equity plan capacity, legal records (bylaws, board minutes, contracts, conflict disclosures, etc.), a typed requirement graph (AuthorizationRequirement, ConsentRequirement, CapacityRequirement, ConflictRequirement, SequenceRequirement, SignatoryRequirement, TermsRequirement, ProhibitionRequirement, composed via AllOf/AnyOf), and one action request. The agent-visible projection strips `source_rule_ids` and never reveals the oracle solution or the requirement graph structure.\n\n### Reward\n\nSingle reward `final_validity_efficiency`:\n- 0.0 if execution is missing, the action_id doesn't match, or any required step is missing\n- `min(1.0, oracle_minimum_process_cost / max(1, actual_process_cost))` otherwise\n\nEight metrics are also emitted: `execution_attempted`, `execution_valid`, `process_cost`, `process_efficiency_ratio`, `turns_used`, `tool_error_count`, `preflight_used`, `validation_defects` (per-defect-category counts).\n\n### Baselines\n\nFive deterministic baselines are provided in `scripts/baselines.py`:\n\n| Baseline | mean reward (medium, n=16) | valid_rate |\n| -------- | -------------------------- | ---------- |\n| `random` | 0.000 | 0.0 |\n| `naive_max_approval` | 0.188 | 0.188 |\n| `keyword_heuristic` | 0.188 | 0.188 |\n| `oracle_replay` | 1.000 | 1.0 |\n| `search_only` | 0.000 | 0.0 |\n\nThe oracle_replay baseline demonstrates the upper bound (100% solvable with reward 1.0). Naive baselines show the gap: ~19% of worlds are simple enough that any single-body meeting approval is sufficient. Random and search-only baselines score 0, validating that prose-only and tool-error-only paths do not accidentally score reward.\n\n### Example trajectory\n\nFor a single G3 (employee RSU) world on seed 17:\n\n```\nturn 1: search_records(query=\"committee\", entity_id=\"ent_lumen\")\nturn 2: read_record(record_id=\"rec_bylaws_g3\")\nturn 3: inspect_register(entity_id=\"ent_lumen\", register_type=\"directors\", as_of_date=\"2028-09-15\")\nturn 4: create_equity_grant(entity_id=\"ent_lumen\", recipient_person_id=\"prs_maya_g3\",\n          award_type=\"rsu\", units=42000, vesting_months=48, cliff_months=12,\n          effective_date=\"2028-09-15\")\nturn 5: record_authorization(action_id=\"act_...\", authorizer_id=\"body_comp_g3\",\n          method=\"meeting\", participant_ids=[\"role_comp_a_g3\", \"role_comp_b_g3\"])\nturn 6: execute_action(action_id=\"act_...\", signatory_person_id=\"prs_ceo_g3\")\n→ score: 1.0 (oracle=1, actual=1)\n```\n\n### Reproducibility\n\nAll randomness is seeded. The generator seeds train and eval splits from a single `seed` plus `split_stride()` and uses a salted SHA256 retag so seeded worlds share a structural fingerprint but have unique IDs. To reproduce the eval set used by CI:\n\n```bash\ncd environments/valid_action\npython -m pytest tests/ -q   # 103 tests, all offline\npython scripts/baselines.py --examples 32 --difficulty medium --seed 17\n```\n\n### Limitations\n\n- Synthetic record prose is template-driven (16 wording variants) and may not reflect real-world document variation.\n- The rule graph covers six action families common to US-style private-company governance; public-company rules (SEC filings, 14A disclosures, NYSE/Nasdaq listing standards) are out of scope.\n- The oracle is a deterministic DP over a typed AND/OR graph; it does not simulate negotiation, amendment, or post-execution events.\n- All six action families are covered, but the world is single-action-per-rollout by design; multi-action sequencing across roles is not in scope.\n\n### Version history\n\n- `v0.1.0` — initial scaffold via `prime env init valid-action --multi-file`\n- `v0.2.0` — world checkpoint: 12 core modules, G1-G9 fixtures, generator, validator, oracle\n- `v0.3.0` — environment checkpoint: Taskset wiring, toolsets, reward/metrics, 8 tool functions\n- `v0.4.0` — release checkpoint: 5 baselines, 103 tests, public README, trace renderer stub\n","encoding":"utf-8","truncated":false,"total_bytes":8146},"status":null}