{"data":{"kind":"file","path":"README.md","version_id":"iyf6yst11gc9y7fzaa43iphd","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1831,"modified_at":"2026-11-13T17:06:14.173000","content_hash":"6f91716be3f1f5da917de673c5e3df74d824f0b83109726a044e1e65c9c59f86"},"entries":[],"content":"# splatbench-prime\n\n### Overview\n- **Environment ID**: `splatbench-prime`\n- **Short description**: SplatBench procedural scenes: rebuild or reassemble a 3D object from reference photos with 3D Gaussians, using editing and rendering tools.\n- **Tags**: splatbench, vision, 3d, multi-turn, tool-use\n\n### Datasets\n- **Primary dataset(s)**: SplatBench procedural scenes (robot, mushroom, rocket, snowman, table, tree)\n- **Source links**: [perpetual-research/splatbench](https://huggingface.co/datasets/perpetual-research/splatbench)\n- **Split sizes**: 12 tasks (6 scenes x `assemble` / `sculpt`)\n\n### Task\n- **Type**: multi-turn tool use with image observations\n- **Rubric overview**: Final scene is scored on held-out views with PSNR, SSIM and LPIPS (SplatScore). Reward is `clip(SplatScore / 100, 0, 1)`. Budgets: 300 Gaussians, 40 tool calls, 16 renders.\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nuv run vf-eval splatbench-prime -m <vision-model> -s\n```\n\nConfigure scenes and mode:\n\n```bash\nuv run vf-eval splatbench-prime -m <vision-model> -n 5 -r 3 -s -a '{\"scene_names\": [\"robot\", \"mushroom\"], \"kind\": \"sculpt\"}'\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `scene_names` | list[str] | all six | Scenes to load |\n| `kind` | str | `\"assemble\"` | `assemble`, `sculpt` or `both` |\n| `device` | str | `\"cpu\"` | PyTorch device |\n| `reward_mode` | str | `\"benchmark\"` | `benchmark` = `clip(score/100)`, `shifted` = `clip((score+50)/150)` |\n| `data_root` | str | HF cache | Local folder with `data/bench/scenes/...` |\n| `training` | bool | `false` | Also expose tasks as the training dataset |\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Main scalar reward (`splat_reward`) |\n| `raw_splat_score` | Unclipped SplatScore |\n","encoding":"utf-8","truncated":false,"total_bytes":1831},"status":null}