{"data":{"kind":"file","path":"README.md","version_id":"ezi36zqgegr39xk51e6ej5ta","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":11130,"modified_at":"2026-06-05T00:35:07.848000","content_hash":"e3f23ff948facaeb80d0e99d271c4f3cd5b217af270b939c1f775621b0c69a37"},"entries":[],"content":"# Self-Driving Negotiator\n\nAn LLM training and evaluation environment for social negotiation in autonomous driving. Built for Prime Intellect's Verifiers library.\n\n## Features\n\n- **3 validated negotiation scenarios** with game-theoretic foundations\n- **Text-based scene descriptions** for LLM reasoning\n- **Multi-dimensional scoring** (safety, social, efficiency, comfort, reasoning)\n- **Curriculum learning** with difficulty progression\n- **Dataset generation** for SFT and RLHF\n\n## Installation\n\n```bash\n# With uv (recommended)\nuv pip install -e .\n\n# With pip\npip install -e .\n```\n\n## Quick Start\n\n### Evaluation with Prime Intellect\n\n```bash\n# Run a local evaluation after installing the environment\nprime eval run self-driving-negotiator -m openai/gpt-4.1-mini -n 10\n\n# Analyze results\npython analyze_results.py outputs/evals/*/results.jsonl\n```\n\n### Python API\n\n```python\nfrom self_driving_negotiator import load_environment\n\n# Multi-turn environment for LLM evaluation\nenv = load_environment(\"yield_standoff\")\n\nmessages = env.reset()\n# [{\"role\": \"system\", \"content\": \"...\"}, {\"role\": \"user\", \"content\": \"...\"}]\n\n# LLM generates response\nresponse = '{\"reasoning\": \"...\", \"acceleration\": 0.5, \"steering\": 0.0, \"maneuver\": \"PROCEED\"}'\nmessages, reward, done, info = env.step(response)\n\n# Raw environment access\nenv = load_environment(\"yield_standoff\", multi_turn=False)\nobs = env.reset()\naction = {\"acceleration\": 0.5, \"steering\": 0.0, \"maneuver\": \"PROCEED\"}\nobs, reward, done, info = env.step(action)\n```\n\n## Scenarios\n\nAll three scenarios pass the environment integrity invariants in\n[`tests/test_invariants.py`](tests/test_invariants.py): solvable by a reference\nexpert, not gamed by a constant-action policy, hidden intent that must be\ninferred from behaviour, and goal reachable within the turn budget.\n\n| Scenario | Challenge | Game Theory |\n|----------|-----------|-------------|\n| `yield_standoff` | T-intersection negotiation | Chicken Game |\n| `pedestrian_feint` | Intent inference with pedestrians | Signaling Game |\n| `contested_merge` | Highway merge with traffic | Coordination Game |\n\nAll three validated scenarios are **hidden-intent inference** tasks where a\nconstant action is not optimal:\n\n* `yield_standoff` — the other vehicle holds a hidden disposition (commit vs.\n  yield); the agent observes only its behaviour (braking vs. holding speed) and\n  must yield to a committed vehicle / take the gap from a yielding one. Constant\n  PROCEED collides with committed vehicles; constant STOP deadlocks.\n* `pedestrian_feint` — a pedestrian either genuinely crosses or feints (lurches\n  and stops short); the agent must brake for real crossers and not over-react to\n  feints. Constant PROCEED collides with crossers; constant STOP never clears.\n* `contested_merge` — a longitudinal merge: the agent controls speed to slot\n  into the gap around one lane vehicle with a hidden \"make room\" disposition\n  (slows to open a gap vs. holds speed). Constant PROCEED side-swipes committed\n  vehicles; constant STOP never completes the merge.\n\n### Live LLM leaderboard (via Prime Inference)\n\nGenerated with `scripts/run_llm_eval.py` + `scripts/consolidate_llm_results.py`\n(tier 1, n=20 rollouts/scenario, via Prime Inference). Success rate with\n**bootstrap 95% CI**; `Avg` is the mean. `always_proceed` is the \"guessing\" bar;\nthe scripted `expert` is the skill ceiling. Per-metric radars:\n[`outputs/radar_yield_standoff.png`](outputs/radar_yield_standoff.png) (and\n`_pedestrian_feint`, `_contested_merge`); full table with CIs in\n[`outputs/llm_results.md`](outputs/llm_results.md).\n\n| Rank | Model | Avg | yield_standoff | pedestrian_feint | contested_merge |\n|------|-------|-----|----------------|------------------|-----------------|\n| 1 | qwen3-30b-a3b-instruct | 0.62 | 0.75 (0.55–0.95) | 0.55 (0.30–0.75) | 0.55 (0.35–0.75) |\n| 2 | gemini-2.5-flash | 0.60 | 1.00 (1.00–1.00) | 0.25 (0.10–0.45) | 0.55 (0.35–0.75) |\n| 3 | deepseek-v3.2 | 0.58 | 1.00 (1.00–1.00) | 0.20 (0.05–0.40) | 0.55 (0.35–0.75) |\n| 4 | llama-3.3-70b-instruct | 0.57 | 1.00 (1.00–1.00) | 0.15 (0.00–0.30) | 0.55 (0.35–0.75) |\n| 5 | mistral-small-3.2-24b | 0.55 | 0.55 (0.30–0.75) | 0.60 (0.40–0.80) | 0.50 (0.30–0.70) |\n| 6 | Llama-3.2-3B-Instruct | 0.45 | 0.45 (0.25–0.65) | 0.55 (0.35–0.75) | 0.35 (0.15–0.60) |\n| — | **expert (reference)** | **0.96** | 0.89 | 1.00 | 1.00 |\n| — | always_proceed (guess) | 0.58 | 0.50 | 0.61 | 0.64 |\n\nWhat the benchmark reveals (read with the CIs — at n=20 they are wide, so treat\nsmall gaps as ties; only large separations are real):\n\n* **`yield_standoff` separates a clean top cluster** — gemini, deepseek and\n  llama-70b hit 1.00 [1.00–1.00] with `decision_correctness` 1.00 (genuine\n  inference), well above the 3B model's 0.45 [0.25–0.65]. (At tier 3 a\n  bluff/feint opponent then *de-saturates* this — see Difficulty tiers below.)\n* **`pedestrian_feint` is hard and counterintuitive** — no model beats the\n  expert (1.00); the strong \"reasoners\" (llama-70b 0.15, deepseek 0.20, gemini\n  0.25) **over-stop on feints and stall**, while less-cautious models (mistral\n  0.60, 3B 0.55) do better by proceeding. Over-caution is penalized.\n* **`contested_merge` is uniformly unsolved** — every model's CI sits around\n  0.50–0.55 with fully overlapping intervals (statistically indistinguishable),\n  far below the expert (1.00). The merge-behind-a-committed-vehicle timing is\n  open headroom for all models.\n* **No LLM beats the scripted expert overall** (0.96) — large headroom remains.\n\nVariance/significance: pass `--rollouts-per-example`/`--seed-offset` to\n`run_llm_eval.py` for more samples; the reported CI is a percentile bootstrap\nover the per-rollout outcomes.\n\n**Difficulty tiers (`yield_standoff`).** An escalating ladder that stresses\n*theory-of-mind*, not perception:\n\n* **Tier 1** — honest: intent is shown from the start.\n* **Tier 2** — **delayed reveal**: both dispositions hold a neutral cruise until\n  the ego is close, then the committed vehicle pushes through / the yielder\n  brakes. The cue is *absent* early (memory can't recover it), so the correct\n  play is to hedge (creep-and-peek) until intent resolves.\n* **Tier 3** — **bluff/feint**: the opponent first shows the *opposite* of its\n  true intent (a committed vehicle soft-brakes to look like it is yielding; a\n  yielder holds to look committed) and reverses the instant the ego takes the\n  bait. A single instantaneous cue is now unsafe — the agent must wait for a\n  *committed* signal (the other actually stopping), not a momentary brake.\n\nThis directly punishes acting on instantaneous cues: in the raw simulator a\nconstant `PROCEED` and a \"proceed-on-any-brake\" policy both collapse at tier 3\n(success ≈ 0.17–0.19, collision ≈ 0.8), while the hedge-and-wait reference\nexpert stays at **0.99**. Among LLMs it separates cue-followers from genuine\nhedgers: at the bluff tier **qwen3-30b drops 0.90 → 0.40** and **deepseek-v3.2\n0.90 → 0.60**, while **gemini-2.5-flash holds 1.00** (it already waits for a\ncommitted signal — the *correct* behaviour). (This replaced\nan earlier per-step observation-noise lever that multi-turn models simply\naveraged out — fully inert: deepseek was 20/20/20 across tiers.)\n\n**Difficulty tiers (other scenarios).** `pedestrian_feint` has a\n**delayed-step-out** tier (the pedestrian commits to crossing later, shrinking\nthe reaction window; expert stays 1.00/1.00/0.96 across tiers while a naive\nPROCEED collides more). `contested_merge` keeps an observation-noise tier — the\nbluff/feint does **not** port cleanly to a merge (its \"ahead vs. behind\"\nstructure lets a merge-ahead policy sidestep the feint); a bluff merge and a\npause-then-dart pedestrian bluff are documented future work.\n\n**Figures & analysis** (`scripts/`): `plot_trajectories.py` →\n[`outputs/trajectories.png`](outputs/trajectories.png) (top-down ego/other paths\nwith start/goal/collision/clear markers); `plot_radar.py` → per-metric radars;\n`generate_baseline_report.py` → non-LLM baseline report; `run_llm_eval.py` +\n`consolidate_llm_results.py` → the CI leaderboard above.\n\n## Scoring Dimensions\n\n| Dimension | Weight | Description |\n|-----------|--------|-------------|\n| Safety | 1.0 | Collision avoidance, TTC, proximity |\n| Social | 0.8 | Negotiation, Nash equilibrium, coordination |\n| Efficiency | 0.6 | Goal progress, time efficiency |\n| Comfort | 0.3 | Smooth driving, jerk minimization |\n| Reasoning | 0.2 | Quality of chain-of-thought |\n\n## Response Format\n\nLLMs respond with JSON:\n\n```json\n{\n    \"reasoning\": \"The other vehicle arrived first. I should yield to avoid collision.\",\n    \"acceleration\": -1.5,\n    \"steering\": 0.0,\n    \"maneuver\": \"YIELD\"\n}\n```\n\n**Maneuvers:** `PROCEED`, `YIELD`, `STOP`, `NUDGE_FORWARD`, `MAINTAIN_LANE`\n\n## Dataset Generation\n\n```python\nfrom integrations import DatasetGenerator, DatasetConfig\n\nconfig = DatasetConfig(\n    n_samples=1000,\n    scenarios=[\"yield_standoff\", \"contested_merge\"],\n    personalities=[\"aggressive\", \"cooperative\", \"timid\"]\n)\n\ngenerator = DatasetGenerator(config)\nsamples = generator.generate(policy=\"expert\")\ngenerator.save_dataset(samples, \"training_data.jsonl\")\n```\n\n## Configuration\n\n| Argument | Default | Description |\n|----------|---------|-------------|\n| `scenario_type` | `yield_standoff` | Scenario type |\n| `multi_turn` | `True` | MultiTurnEnv (LLM) or raw env |\n| `config_path` | `None` | Custom config.json path |\n| `other_agent_personality` | `cooperative` | Other agent behavior |\n\n**Personalities:** `aggressive`, `timid`, `cooperative`, `distracted`\n\n## Project Structure\n\n```\nself_driving_negotiator/\n├── self_driving_negotiator.py   # Main environment + load_environment entrypoint\n├── analyze_results.py           # Results analysis tool\n├── config.json                  # Scenario configuration\n├── core/                        # Types, dynamics, game theory, validation\n├── scenarios/                   # The 3 validated scenarios + base/reward utils\n├── integrations/                # Verifiers integration\n│   ├── verifiers_env.py         # vf.MultiTurnEnv wrapper + rubric\n│   ├── scene_descriptions.py    # LLM-friendly scene context\n│   ├── dataset_generator.py     # Reference experts + dataset/curriculum generation\n│   └── llm_agent.py             # response parsing utilities\n├── scripts/                     # baseline report + LLM eval/leaderboard runners\n├── tests/                       # contract + integrity-invariant tests\n└── pyproject.toml               # Package configuration\n```\n\n## Testing\n\n```bash\n# Contract + integrity-invariant tests, plus packaging guards that build the\n# wheel, install it into a clean venv, and load from an unrelated CWD (catches\n# missing-data-file regressions before they reach the Hub).\npython -m pytest\n\n# Quick non-LLM baseline sanity suite\npython -m integrations.baselines\n\n# Reproducible baseline report (expert vs constant policies)\npython scripts/generate_baseline_report.py --episodes 200\n```\n\n## License\n\nMIT\n","encoding":"utf-8","truncated":false,"total_bytes":11130},"status":null}