{"data":{"kind":"file","path":"README.md","version_id":"u0ozsae4gnyr7rt6rjkvlol1","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":8020,"modified_at":"2026-11-11T04:20:49.781000","content_hash":"e0081e5b0e691fb115c0b289471446db347326a53cef264542b878ac46c6d0d2"},"entries":[],"content":"# UAV Robust Control Hub v6 — Learning & Generalization Evidence\n\n**Purpose:** A reproducible **fictional civilian UAV** policy-synthesis benchmark for training/evaluating software agents. It is not a real-aircraft controller, robotics certification system or military navigation solution.\n\n**Evidence status (2026-11-11):** CPU reward-learning and offline simulation are executed. Publication validation with real Verifiers v0.4.1 passed all 62 tests, native task/toolset construction, SDK discovery and standard/OOD configuration dry-runs. Model-free validation found 8 valid noop checks and 8 unchecked gold checks (no gold validation hook). **No real LLM/GRPO training, paid model evaluation or native Verifiers/MCP end-to-end model rollout has been executed.**\n\n## What v6 actually changes\n\n1. **Real CPU reward learning:** `uav_robust_hub/policy_bandit.py` learns a small *contextual bandit* selecting one of five safe-schema policy templates via online REINFORCE-style softmax updates. Training data are `train` only; eval/holdout do not update weights. This is **not LLM training**.\n2. **Learning curve + untouched eval/holdout:** `scripts/train_cpu_policy.py` checkpoints score/success at fixed train steps, and evaluates public-generator holdout + OOD once after training. All configs and source are reproducible.\n3. **Six-cell robustness matrix:** `scripts/robustness_matrix.py` reports standard + five OOD shifts, safety violations and mission-clustered bootstrap CIs, including mean of each mission's *worst* stress reward.\n4. **Blinded challenge support:** `uav_robust_hub/blinded.py` uses HMAC-derived procedural worlds and a salt-protected final grader. `UAV_CHALLENGE_SECRET` is mandatory for Verifiers holdout task generation. Security **requires** agent/grader process and filesystem isolation.\n5. **Counterbalanced live LLM A/B runner:** `scripts/rigorous_live_ablation.py` alternates tools-first/no-tools-first, records token usage and failures, uses mission-level bootstrap, and omits raw model policy/telemetry from its output. **Dry-run by default; paid API calls only with `--run`.**\n6. **Dependency hygiene:** CPU modules do not import Verifiers eagerly. The Hub package declares Verifiers v0.4.1 as a default dependency so standard installation supports SDK discovery; the `verifiers` extra remains a compatibility alias.\n\n## Reproduce offline results\n\nFrom repository root, with Python >=3.11 and pytest + pydantic:\n\n```bash\npython -m pytest -q\nPYTHONPATH=. python scripts/train_cpu_policy.py --steps 360 --eval-tasks 40 --trials 2 --checkpoints 6 --output results/v6/cpu_learning_curve.json\nPYTHONPATH=. python scripts/robustness_matrix.py --tasks 40 --trials 3 --bootstrap 500 --output results/v6/robustness_matrix.json\nPYTHONPATH=. python scripts/rigorous_live_ablation.py --model EXAMPLE_MODEL --tasks 12\nPYTHONPATH=. python scripts/check_native_integration.py\npython scripts/verify_release.py\n```\n\n**Publication validation:** `62 passed` with the real pinned framework installed, including the native adapter smoke. A separate adapter test still uses an API-shaped *stub*, **not** the actual framework. The `rigorous_live_ablation` dry run makes zero external network calls.\n\n### CPU learning (360 train tasks, 40 public eval tasks, 2 trials, seed 2026)\n\n| Step | Learned policy eval reward | A* reference eval reward | Learned policy success |\n|---:|---:|---:|---:|\n| 0 | 0.406575 | 0.813731 | 42.50% |\n| 60 | 0.791075 | 0.813731 | 88.75% |\n| 120 | 0.810771 | 0.813731 | 91.25% |\n| 360 | **0.810771** | **0.813731** | **91.25%** |\n\nThe trained bandit learned to choose the **same slow-A* candidate for all 40 eval tasks**. This establishes online reward-based optimization from a weak initialization, **not** superior policy synthesis over A*, and not proof of contextual adaptation.\n\nOn public-generator holdout (40 tasks, 2 trials), trained bandit **0.804927** vs fixed A* **0.766170**. On public-generator holdout OOD, bandit **0.537338** vs fixed A* **0.543288** — slightly worse. These are one-seed descriptive measurements, not pre-registered statistically conclusive discoveries. The public generator is readable from source, so it is **not a secure third-party holdout**.\n\n### Robustness matrix (40 holdout-generator tasks, 3 trials per shift)\n\n| Shift | A* reward | A* success | A* unsafe trials | Direct reward |\n|---|---:|---:|---:|---:|\n| Standard | 0.749227 | 85.83% | 11.67% | 0.375423 |\n| Crosswind | 0.538495 | 62.50% | 34.17% | 0.264962 |\n| GPS outage extension | 0.378558 | 50.83% | 49.17% | 0.035000 |\n| Crowded airspace | 0.618346 | 75.00% | 24.17% | 0.333614 |\n| Low battery reserve | 0.738318 | 82.50% | 12.50% | 0.353277 |\n| Compound shift | 0.266751 | 35.00% | 63.33% | 0.177984 |\n\nA* mean of each mission's **worst of the five stress scores = 0.197178**, revealing a real robustness gap. Each cell is an offline hand-coded policy baseline, **not LLM performance**. See `results/v6/robustness_matrix.json` for paired clustered CIs and per-mission records.\n\n## Optional: run genuine model evaluation (makes API requests / may cost money)\n\n```bash\nexport OPENAI_API_KEY='YOUR_TOKEN'      # never add secrets to repository\nPYTHONPATH=. python scripts/rigorous_live_ablation.py \\\n  --model YOUR_MODEL_ID --endpoint https://YOUR_PROVIDER/v1 \\\n  --tasks 12 --split eval --distribution standard \\\n  --output results/v6/real_model_eval.json --run\n\n# Real blind holdout must run on an isolated trusted grader host:\nexport UAV_CHALLENGE_SECRET='STRONG_PRIVATE_RANDOM_SECRET_32_PLUS_BYTES'\nPYTHONPATH=. python scripts/rigorous_live_ablation.py \\\n  --model YOUR_MODEL_ID --tasks 12 --split holdout \\\n  --distribution stress --output results/v6/real_model_holdout.json --run\n```\n\n**The above commands have not been executed against a real model.** Tokens differ naturally between no-tools and FlightLab. The runner reports token counts and alternating condition order, but it does not prove causal effects without repeated runs, cost adjustment and model/provider control.\n\n## Prime Intellect / Verifiers integration\n\nThe v1 adapter in `uav_robust_hub/taskset.py` uses `Taskset[TaskT, ConfigT]`, `TaskData`, per-task `Toolset`, `@vf.reward` and `@vf.metric`. `uav_robust_control_hub` re-exports the taskset for distribution-name discovery. The framework dependency is included by default:\n\n```bash\npython -m pip install .\nPYTHONPATH=. python scripts/check_native_integration.py\n```\n\nA successful native import smoke is **not** evidence that model/tool MPC calls, concurrent rollouts or GRPO training work. Use official installed commands and a real MCP-capable harness to test those explicitly. The existing `configs/*` files are examples requiring reconciliation against the pinned local CLI. **Never run holdout on a machine/container the agent can inspect.**\n\nUseful references: [Prime Intellect v1 tasksets guide](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/tasksets.md), [v1 overview](https://github.com/PrimeIntellect-ai/verifiers/blob/main/docs/v1/overview.md), and [Prime Intellect docs](https://docs.primeintellect.ai/verifiers/environments).\n\n## Repo map and limitations\n\n- `uav_robust_hub/`: simulator, policy schema, flight calibration, evaluation, CPU learner, blind generator and model client.\n- `scripts/`: training, shift matrix, model A/B eval, audit and release verification.\n- `tests/`: protocol, correctness, learning, blind generation, mock-v1 adapter, regression.\n- `results/v6/`: **only offline CPU training, algorithmic robustness and no-network dry-run records**.\n- `docs/BAO_CAO_V6.md` and `docs/BOUNTY_SUBMISSION_V6.md`: evidence and acceptance gates.\n- `RELEASE_MANIFEST.json`: SHA-256 release manifest; verify before sharing.\n\n**Known limits:** finite fictional point-mass dynamics, hand-coded policy candidate set, no unseen real-world data, no GPU or LLM training, no real model run, no audited grader isolation, and no verified native Verifiers end-to-end integration. Benchmark winnings are not implied. MIT license.\n","encoding":"utf-8","truncated":false,"total_bytes":8020},"status":null}