{"data":{"kind":"file","path":"README.md","version_id":"t687z3e5ci8s89p4hjyfe0ud","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4499,"modified_at":"2026-07-10T06:14:52.272000","content_hash":"ccf078608ce37cee0d5c63576dc71732d4a3878f1905b97b377f063b74f3ca86"},"entries":[],"content":"# uav-operator\n\n`uav-operator` is a multi-turn Prime Intellect / Verifiers environment where\nan LLM acts as the remote pilot in command for small UAS missions over the San\nFrancisco Bay Area.\n\nDrones already fly themselves. Autopilots hold trajectories, execute return to\nlaunch, and enforce simple failsafes. This environment tests the layer above\nthat: ops-center judgment under shifting wind, pop-up flight restrictions,\nbattery anomalies, lost-link events, and mission updates. The simulator owns\nthe aircraft and every reward is computed from logged simulator state, not from\nthe model's prose.\n\n### Overview\n- **Environment ID**: `uav-operator`\n- **Short description**: Multi-turn UAV operations environment with\n  physics-derived rewards and tool-based operator decisions.\n- **Tags**: `uav`, `drone-operations`, `multi-turn`, `tool-use`, `train`,\n  `eval`\n- **Status**: Day 4 implemented: seeded T0-T1 generator, event interrupts for\n  wind shift / pop-up TFR / battery degrade / site closure, full SPEC §5 reward\n  v0, rulebook and reckless baselines, deterministic sim logging, and the full\n  SPEC §3.2 console. T2/T3 generation and renderer remain next.\n\n### Datasets\n- **Primary dataset(s)**: Seeded scenario generator emitting T0-T1 examples.\n- **Source links**: Static Day 2 world data is generated in-repo with\n  `scripts/build_world.py` from simplified public-structure airspace and\n  synthetic obstacle assumptions.\n- **Current split sizes**: default Day 4 eval set is 20 mixed T0-T1 examples.\n  v0.1 target remains at least 300 train and 60 eval scenarios, stratified\n  across T0-T3 difficulty tiers.\n\n### Task\n- **Type**: Multi-turn tool use.\n- **Role**: Remote pilot in command / ops-center operator.\n- **Model decisions**: File or amend plans, query telemetry/weather/airspace,\n  hold, resume, return to launch, land, release payload, abort missions, and\n  override simulator-owned failsafes when justified by the scenario state.\n- **Rubric overview**: Mission value, hard safety violations, reserve and\n  margin policy, procedural compliance, and efficiency. Reward components read\n  only saved `sim_log` snapshots; model prose is inert.\n\n### Quickstart\nRun a small local smoke evaluation:\n\n```bash\nprime --plain eval run uav-operator -n 2 -r 1 --skip-upload --disable-tui\n```\n\nConfigure model, sampling, and saved simulator state:\n\n```bash\nprime --plain eval run uav-operator \\\n  -m poolside/laguna-m.1 \\\n  -n 5 \\\n  -r 1 \\\n  -t 512 \\\n  -T 0.2 \\\n  --skip-upload \\\n  --disable-tui \\\n  --save-results \\\n  --state-columns sim_state,sim_log\n```\n\nNotes:\n- Put task-owned settings under `[env.taskset]` and harness-owned settings\n  under `[env.harness]` in TOML configs.\n- The core sim is event-driven and analytic. It tests supervisory operator\n  judgment, not low-level flight control or 3D collision physics.\n- Inspect or rebuild the static Day 2 world JSON with\n  `uv run python scripts/build_world.py`.\n- Inspect the deterministic scripted rollout artifact at\n  `assets/rollouts/day3_scripted_rollout.json`.\n- Run local baselines with\n  `uv run python scripts/baselines.py --policy both --episodes 20`.\n\n### Taskset Config\nPlanned fields:\n\n| Field | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `tier` | string | `mixed_day4` | Scenario tier selection: `T0`, `T1`, or mixed Day 4 T0/T1. |\n| `seed` | int | `0` | Base seed for deterministic scenario generation. |\n| `max_examples` | int | `-1` | Limit on dataset size; use `-1` for all generated examples. |\n| `wind_enabled` | bool | `true` | Enable seeded Day 3 wind field and altitude shear. |\n| `gust_front_probability` | float | `0.5` | Probability that an episode includes a gust front. |\n\n### Harness Config\nPlanned fields:\n\n| Field | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `max_turns` | int | `40` | Maximum operator decision turns per episode. |\n| `sim_time_cap_min` | int | `90` | Maximum simulated episode duration. |\n\n### Metrics\nImplemented rubric metrics:\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Main scalar reward, computed from simulator state |\n| `mission_value` | Completed mission value after timeliness decay |\n| `hard_safety` | Airspace incursions, aircraft loss, and critical battery outcomes |\n| `margin_policy` | Reserve, override, and minimum-safe-altitude penalties |\n| `procedure` | Alert acknowledgement, conflict filing, and hold-expiry penalties |\n| `efficiency` | Energy and simulated-time cost relative to scenario par |\n","encoding":"utf-8","truncated":false,"total_bytes":4499},"status":null}