{"data":{"kind":"file","path":"README.md","version_id":"t00lqqolycd3wgrpedc78oza","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5550,"modified_at":"2026-08-12T06:21:10.366000","content_hash":"9ec9adc07f285bd682f51a69324593938785aa4ea80d41abbdf22980635ad6ab"},"entries":[],"content":"# my-android-env\n\nA Prime Intellect `verifiers` environment that runs tasks from the\nofficial Google AndroidWorld benchmark against a KVM-accelerated Android\nemulator.\n\nThe environment initializes a real AndroidWorld task, sends the model a PNG\nscreenshot plus Android accessibility elements, executes one AndroidWorld JSON\naction per model turn, and uses AndroidWorld's durable task verifier as reward.\n\n## Architecture\n\n```text\nPrime vision model\n       |\n       | screenshot + UI elements / JSON action\n       v\nmy-android-env (Verifiers multi-turn adapter)\n       |\n       | HTTP on 127.0.0.1:5000\n       v\nAndroidWorld FastAPI server + Android Emulator\n       |\n       v\n/dev/kvm (hardware acceleration)\n```\n\nOne emulator has one mutable phone state. Always use one evaluation worker and\n`--max-concurrent 1`. To run tasks concurrently, provision one\nAndroidWorld container and port per worker.\n\n## What is already configured on this host\n\n- Ubuntu x86_64 with Intel VT-x\n- readable and writable `/dev/kvm`\n- current user in the `kvm` and `docker` groups\n- Docker, ADB, `uv`, and Prime CLI\n- authenticated Prime Intellect account\n\nRecheck at any time:\n\n```bash\n./scripts/android_world.sh check\n```\n\n## First-time setup\n\n### 1. Install the Python environment\n\n```bash\nuv sync\n```\n\n### 2. Build AndroidWorld\n\n```bash\n./scripts/android_world.sh build\n```\n\nThis clones AndroidWorld at pinned revision\n`3e50888527ef9f29b9157ecd537e408008bb1c85`, adds a compact\n`/state` endpoint for PNG screenshots and indexed UI elements, replaces\nupstream's retired OpenJDK base tag with Eclipse Temurin 17, scopes Python\npackage discovery to `/workspace`, installs the Pillow dependency omitted by\nthe upstream Docker metadata, and builds image\n`my-android-world:3e508885`. The first build\ndownloads the Android SDK/API 33 image and compiles Python, so it can take a\nwhile and needs several gigabytes of disk.\n\n### 3. Start the KVM emulator and wait for health\n\n```bash\n./scripts/android_world.sh start\n./scripts/android_world.sh smoke\n```\n\nThe server binds only to `127.0.0.1:5000`. The first emulator\nboot also installs AndroidWorld's required apps and configures permissions.\n\n### 4. Run one inexpensive Prime evaluation\n\n```bash\nprime eval run my-android-env \\\n  -m qwen/qwen3-vl-30b-a3b-instruct \\\n  -n 1 -r 1 -c 1 --num-workers 1 \\\n  -t 512 --temperature 0 \\\n  --skip-upload --disable-tui \\\n  -a '{\"task_types\":[\"ClockStopWatchRunning\"],\"task_combinations\":1}'\n```\n\nThe model must support images. The command above uses a currently available\nPrime Inference vision model. It sends Android screenshots and accessibility\nmetadata to Prime Intellect and consumes paid inference. Remove\n`--skip-upload` if the result should also be uploaded to the Prime platform.\n\n## Task selection\n\nWith no `task_types` argument, the environment uses five quick\nbuilt-in-app tasks:\n\n- `ClockStopWatchRunning`\n- `ClockTimerEntry`\n- `ContactsAddContact`\n- `SystemWifiTurnOn`\n- `SystemBluetoothTurnOff`\n\nSelect named tasks:\n\n```bash\nprime eval run my-android-env \\\n  -m qwen/qwen3-vl-30b-a3b-instruct \\\n  -n 2 -r 1 -c 1 --num-workers 1 \\\n  -a '{\"task_types\":[\"ContactsAddContact\",\"ClockTimerEntry\"]}'\n```\n\nUse the complete AndroidWorld registry:\n\n```bash\nprime eval run my-android-env \\\n  -m qwen/qwen3-vl-30b-a3b-instruct \\\n  -n 116 -r 1 -c 1 --num-workers 1 \\\n  -a '{\"task_types\":\"all\",\"task_combinations\":1,\"suite_seed\":42}'\n```\n\nIncrease `task_combinations` to generate multiple deterministic\nparameter variations per task. A full run can make many paid model calls, so\nvalidate one task first.\n\n## Environment arguments\n\n| Argument | Default | Meaning |\n| --- | --- | --- |\n| `server_url` | `http://127.0.0.1:5000` | AndroidWorld server |\n| `task_types` | five-task smoke subset | list, comma-separated string, or `\"all\"` |\n| `task_combinations` | `1` | randomized instances per task |\n| `suite_seed` | `42` | reproducible AndroidWorld suite seed |\n| `keep_recent_screenshots` | `2` | screenshots retained in model context |\n| `http_timeout_seconds` | `45` | timeout per server request |\n| `max_turns` | `15` | maximum model action turns |\n| `timeout_seconds` | `900` | complete rollout timeout |\n\n## Operations\n\n```bash\n./scripts/android_world.sh status\n./scripts/android_world.sh logs\n./scripts/android_world.sh stop\n```\n\nThe `start` command is idempotent and never replaces an existing\ncontainer. The source checkout is stored in `.android-world-src/`\nand ignored by Git.\n\n## Local versus hosted Prime runs\n\nThis setup runs the evaluation harness locally while using Prime Inference for\nthe model. A Prime-hosted evaluation cannot reach a server bound to this\nmachine's loopback address, and its generic sandbox is not guaranteed to expose\n`/dev/kvm`. Hosted execution therefore needs a dedicated\nAndroidWorld service or custom instance/image with nested virtualization. The\nlocal KVM path here is the reliable starting point.\n\n## Troubleshooting\n\n- `/dev/kvm` missing: enable virtualization in BIOS/UEFI and load\n  `kvm_intel` or `kvm_amd`.\n- permission denied on KVM: add the user to `kvm` and start a new\n  login session.\n- port 5000 already in use: set `ANDROID_WORLD_PORT=5001` for the\n  script and pass\n  `-a '{\"server_url\":\"http://127.0.0.1:5001\"}'` to Prime.\n- model rejects images: choose a VL/multimodal model, such as a Qwen3-VL model.\n- rollout interference: confirm `-c 1 --num-workers 1`.\n\nUpstream references:\n[AndroidWorld](https://github.com/google-research/android_world) and\n[Prime Intellect Verifiers](https://github.com/PrimeIntellect-ai/verifiers).\n","encoding":"utf-8","truncated":false,"total_bytes":5550},"status":null}