{"data":{"kind":"file","path":"README.md","version_id":"kg7bqmlr5y17ydejg8c1j7qp","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3898,"modified_at":"2026-03-15T07:46:38.166000","content_hash":"d4ac0f34ac17cbb46948e2d233eeb866ce483c7270beaf9a30ae0dadf4556839"},"entries":[],"content":"# impact-agent-bt\n\nEnvironment for Bradley-Terry style pairwise comparison of blinded scientific papers.\n\n## Overview\n\n- Environment package name: `impact-agent-bt`\n- Local eval ID after install: `impact-agent-bt`\n- Task type: multi-turn tool-use (`vf.StatefulToolEnv`)\n- Domain: blinded dementia-research paper comparisons\n\n## Task\n\nThe model is given two blinded papers, `Paper A` and `Paper B`, and must inspect sections from both before submitting which paper should win the pairwise comparison.\n\nThe final tool call must submit a JSON object with exactly:\n\n- `predicted_winner`: `\"A\"` or `\"B\"`\n- `confidence_logit`: finite numeric confidence\n- `reasoning`: brief explanation\n\n## Tools\n\nThe environment exposes two tools to the model:\n\n1. `scan_paper(section_name: str, target_paper: str)`\n- Reads a section from `Paper A` or `Paper B`.\n- `target_paper` must be `\"A\"` or `\"B\"`.\n- Raises a tool error if the section does not exist.\n\n2. `submit_preference(prediction_json: str)`\n- Submits the final pairwise decision as a JSON string.\n- Required schema:\n  - `{\"predicted_winner\": \"A\" | \"B\", \"confidence_logit\": <number>, \"reasoning\": <string>}`\n- Extra fields are rejected.\n\nNotes:\n- Hidden tool args are injected from environment state via `StatefulToolEnv.update_tool_args(...)`.\n- The environment tracks viewed sections for both papers.\n- The rollout is marked complete after a valid `submit_preference(...)` call.\n\n## Reward / Metrics\n\nThe default rubric (`BradleyTerryRubric`) combines:\n\n- `order_reward`: positive reward for the correct winner, scaled by sigmoid-transformed confidence\n- `bilateral_scan_bonus`: rewards scanning evidence-bearing sections from both papers\n- `reasoning_bonus`: rewards valid submission JSON\n- `completion_bonus`: rewards finishing with a valid submission\n\nKey tunables:\n\n- `order_reward_weight`\n- `bilateral_scan_bonus_weight`\n- `reasoning_bonus_weight`\n- `completion_weight`\n- `logit_clip`\n- `max_turns`\n\n## Dataset\n\nThe runtime loader supports normalized pair/paper JSONL artifacts and legacy JSONL records.\n\nDefault behavior:\n\n- Loads from packaged dataset resources under `impact_agent_bt/data/`\n- Supports `train`, `val`, `test`, and `all` split selection when those files are available\n\nOptional override:\n\n- Pass `jsonl_path` to `load_environment()` to use a custom dataset base path\n\n## Environment Arguments\n\nSupported `load_environment(...)` args:\n\n| Arg | Type | Default | Description |\n| --- | --- | --- | --- |\n| `split` | `str` | `\"train\"` | Dataset split: `train`, `val`, `test`, `all` |\n| `max_turns` | `int` | `10` | Max tool interactions per rollout |\n| `jsonl_path` | `str \\| None` | `None` | Optional dataset path override |\n| `order_reward_weight` | `float` | `1.0` | Correct-order reward weight |\n| `bilateral_scan_bonus_weight` | `float` | `0.2` | Bonus for scanning both papers |\n| `reasoning_bonus_weight` | `float` | `0.05` | Bonus for valid JSON reasoning payload |\n| `completion_weight` | `float` | `0.1` | Bonus for successful completion |\n| `logit_clip` | `float` | `8.0` | Confidence clipping before sigmoid |\n\n## Quickstart\n\nInstall the environment package locally:\n\n```bash\nprime env install impact-agent-bt\n```\n\nRun local evaluation:\n\n```bash\nprime eval run impact-agent-bt -m gpt-4.1-mini\n```\n\nRun with explicit args:\n\n```bash\nprime eval run impact-agent-bt -m gpt-4.1-mini -a '{\"split\":\"train\",\"max_turns\":10}'\n```\n\n## Hosted Training\n\nExample hosted training config lives at:\n\n- `configs/hosted/impact-agent-bt.toml`\n\nRun:\n\n```bash\nprime rl run @ configs/hosted/impact-agent-bt.toml\n```\n\n## Development Notes\n\n- Source of truth implementation: `environments/impact_agent_bt/impact_agent_bt/env.py`\n- Tools: `environments/impact_agent_bt/impact_agent_bt/tools.py`\n- Rubric: `environments/impact_agent_bt/impact_agent_bt/rubrics/bradley_terry.py`\n- Dataset loader: `environments/impact_agent_bt/impact_agent_bt/dataset.py`\n","encoding":"utf-8","truncated":false,"total_bytes":3898},"status":null}