{"data":{"kind":"file","path":"README.md","version_id":"bx1twmn0ikxvare8702kgmam","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":7740,"modified_at":"2026-04-22T21:25:15.117000","content_hash":"c736426075d012ef541db8e71f9b11bbdf4b6c173180077984a9afc4fea99add"},"entries":[],"content":"# carla-env\n\nCARLA verifiers environment for autonomous-driving evaluation and RL.\n\n| Field | Value |\n|-------|-------|\n| **Environment ID** | `carla-env` |\n| **Version** | `0.3.0` |\n| **Type** | `StatefulToolEnv` |\n\n## Overview\n\nScenario families:\n\n- `trolley_micro_*`: benchmark trolley dilemmas\n- `action_bias_*`, `bias_*`: action-vs-inaction trolley variants\n- `maze`: hidden-goal text navigation\n- `navigation*`: text-first open navigation with optional NPC vehicles/pedestrians\n- `navigation_vision*`: vision-first navigation with RGB camera access\n- `free_roam*`: open-ended driving with optional traffic\n\nRenderer modes:\n\n- Native CARLA RGB cameras\n- NuRec neural rendering on CARLA 0.9.16\n- Cosmos Transfer2.5 stylized RGB via remote frame server\n\nObservation modes:\n\n- Text-first: trolley, action-bias, maze, `navigation*`\n- Vision-first: `navigation_vision*`\n\n## Quickstart\n\n```bash\n# Recommended today: run the validated Prime H100 flow.\nscripts/e2e_prime_h100_runtime.sh --stack both --runtime-image sinatras/carla-env-runtime:latest\n\n# Or run only NuRec / only Cosmos\nscripts/e2e_prime_h100_runtime.sh --stack nurec --runtime-image sinatras/carla-env-runtime:latest\nscripts/e2e_prime_h100_runtime.sh --stack cosmos --runtime-image sinatras/carla-env-runtime:latest\n```\n\n## Local Install\n\nThe old “install one extra and immediately run” path is no longer the right\ndefault. The validated package install today is:\n\n```bash\nuv venv --python 3.12\nsource .venv/bin/activate\nuv pip install --upgrade pip setuptools wheel\n\n# Validated local client path\nuv pip install -e \".[carla9]\"\n\n# If you need NuRec support\nuv pip install -e \".[carla9,nurec]\" \"imageio[ffmpeg]\" \"huggingface_hub\"\n\n# CARLA 0.10.0-only workflows still exist, but are not the main validated path\n# uv pip install -e \".[carla10]\"\n```\n\nFor Hugging Face-backed features like NuRec scene download and Cosmos gated\nweights, keep a repo-local `.env` with a valid token:\n\n```bash\ncat > .env <<'EOF'\nHF_TOKEN=hf_your_token_here\nEOF\n```\n\n## Local CARLA Server\n\nStart a local CARLA 0.9.16 server explicitly before running the env:\n\n```bash\ndocker run --rm \\\n  --gpus all \\\n  --runtime nvidia \\\n  --network host \\\n  --ipc=host \\\n  --shm-size=8g \\\n  -e XDG_RUNTIME_DIR=/tmp/runtime-carla \\\n  -e NVIDIA_DRIVER_CAPABILITIES=all \\\n  carlasim/carla:0.9.16 \\\n  /bin/sh -lc 'mkdir -p /tmp/runtime-carla && chmod 700 /tmp/runtime-carla && cd /workspace && exec ./CarlaUE4.sh -RenderOffScreen -nosound -carla-rpc-port=2000 -stdout -FullStdOutLogOutput -unattended'\n```\n\nThen run examples:\n\n```bash\nuv run vf-eval carla_env -m \"openai/gpt-4.1-mini\" \\\n  -a '{\"scenario\": \"maze\", \"sandbox\": {\"mode\": \"disabled\"}, \"host\": \"127.0.0.1\", \"port\": 2000, \"carla_version\": \"0.9.16\"}' \\\n  -n 1 -r 1\n\nuv run vf-eval carla_env -m \"openai/gpt-4.1-mini\" \\\n  -a '{\"scenario\": \"navigation_Town10HD_v1_p1\", \"sandbox\": {\"mode\": \"disabled\"}, \"host\": \"127.0.0.1\", \"port\": 2000, \"carla_version\": \"0.9.16\"}' \\\n  -n 1 -r 1\n\nuv run vf-eval carla_env -m \"qwen/qwen3-vl-8b-instruct\" \\\n  -a '{\"scenario\": \"navigation_vision_Town10HD_v1_p1\", \"sandbox\": {\"mode\": \"disabled\"}, \"host\": \"127.0.0.1\", \"port\": 2000, \"carla_version\": \"0.9.16\"}' \\\n  -n 1 -r 1\n```\n\n## Pod Local Server\n\nOn some GPU pods, nested Docker cannot unpack or launch the official `carlasim/carla`\nimages even though the node has a usable NVIDIA GPU. For that case, use\n[`scripts/start_local_carla_on_pod.sh`](scripts/start_local_carla_on_pod.sh), which:\n\n- installs the required host packages\n- reuses a preinstalled CARLA root when available, or unpacks the official CARLA image onto the host\n- launches CARLA directly from the extracted rootfs on the host as a non-root user\n- uses `/ephemeral` or `/workspace` for large CARLA cache/rootfs data\n- can apply a Vulkan ICD override when the provider image needs it\n\nExample:\n\n```bash\nsudo scripts/start_local_carla_on_pod.sh start --mode vision --port 4000\n\nuv run vf-eval carla_env -m \"qwen/qwen3-vl-8b-instruct\" \\\n  -a '{\"scenario\":\"navigation_vision_Town10HD_v1_p1\",\"sandbox\":{\"mode\":\"disabled\"},\"host\":\"127.0.0.1\",\"port\":4000,\"carla_version\":\"0.10.0\"}' \\\n  -n 1 -r 1\n```\n\n## Stage 3 Rendering\n\nNuRec:\n\n- enable with `enable_nurec=True` and `nurec_scene_path=...`\n- forces `carla_version=\"0.9.16\"`\n- supports `nurec_mode=\"replay\"` and `nurec_mode=\"drive\"`\n\nCosmos:\n\n- enable with `enable_cosmos=True` and `cosmos_server_url=\"http://host:8080\"`\n- keeps native CARLA depth capture when the depth sensor is available\n- degrades to raw CARLA RGB if the Cosmos server is unhealthy or stylization fails\n\nOperational helper:\n\n- [`scripts/cosmos_frame_server.py`](scripts/cosmos_frame_server.py) runs a long-lived FastAPI frame server for Cosmos stylization on a GPU host\n\n## Environment Args\n\n| Argument | Default | Description |\n|----------|---------|-------------|\n| `scenario` | `\"action_bias_saves\"` | Scenario identifier |\n| `host` | `$CARLA_HOST` or `127.0.0.1` | CARLA host |\n| `port` | `$CARLA_PORT` or `2000` | CARLA port |\n| `dataset_path` | `None` | Custom JSONL dataset |\n| `trolley_micro_scoring` | `\"expected\"` | `\"expected\"` or `\"actual\"` |\n| `sandbox` | `{\"mode\":\"prime\"}` | Prime sandbox config or local mode |\n| `traffic_manager_enabled` | `None` | Explicit TrafficManager opt-in/out |\n| `carla_version` | inherited | `0.10.0`, `0.9.16`, or `auto` (local only) |\n| `enable_nurec` | `False` | Enable NuRec neural rendering |\n| `nurec_scene_path` | `None` | Path to NuRec USDZ scene |\n| `nurec_mode` | `None` | `replay` or `drive` |\n| `enable_cosmos` | `False` | Enable Cosmos stylization |\n| `cosmos_server_url` | `None` | Cosmos frame server URL |\n\n## Scenario Names\n\n| Scenario | Description |\n|----------|-------------|\n| `maze` | Hidden-goal text navigation |\n| `navigation` | Text-first open navigation |\n| `navigation_<Map>_v<N>_p<M>` | Text-first navigation with map, vehicles, pedestrians |\n| `navigation_vision` | Vision-first open navigation |\n| `navigation_vision_<Map>_v<N>_p<M>` | Vision-first navigation with map, vehicles, pedestrians |\n| `action_bias_saves` | Swerve avoids all pedestrians |\n| `action_bias_less` | Swerve hits fewer |\n| `action_bias_equal` | Equal harm either way |\n| `bias_<C>v<S>` | Custom action-bias setup |\n| `trolley_micro_<id>` | Benchmark trolley scenario |\n\n## Tools\n\nCommon tools:\n\n- `control_vehicle(throttle, steer)`\n- `brake_vehicle(intensity)`\n- `emergency_stop()`\n- `lane_change(direction)`\n- `init_navigation_agent(behavior)`\n- `set_destination(x, y, z)`\n- `follow_route(steps)`\n- `get_goal_info()`\n\nText-first only:\n\n- `observe()`\n\nVision-first only:\n\n- `capture_image()`\n\nCosmos-enabled vision episodes only:\n\n- `capture_depth()`\n\n`get_goal_info()` behavior:\n\n- `maze` and `navigation*`: `distance_to_goal_m=... direction=... improving=...`\n- `navigation_vision*`: `distance_to_goal_m=... improving=...`\n\n## Notes\n\n- The validated install path today is Python 3.12 + `.[carla9]`.\n- `.[carla10]` is still available for CARLA 0.10.0-only workflows, but it is not the main validated renderer path.\n- NuRec requires `.[carla9,nurec]`, a valid `nurec_scene_path`, and Hugging Face auth.\n- The most reliable full deployment path today is the Prime H100 wrapper in `scripts/e2e_prime_h100_runtime.sh`.\n- `navigation*` defaults to `traffic_manager_enabled=False`. TrafficManager is opt-in only.\n- Sandbox startup defaults are mode-aware:\n  - text-only scenarios: `./CarlaUnreal.sh -nullrhi -nosound`\n  - vision scenarios: `./CarlaUnreal.sh -RenderOffScreen -nosound`\n- `navigation_vision*` does not expose normal text observations.\n- `navigation_vision*` does not include destination coordinates in the prompt.\n- `import carla_env` works without CARLA installed, but loading the environment still requires one of the CARLA client extras.\n","encoding":"utf-8","truncated":false,"total_bytes":7740},"status":null}