{"data":{"kind":"file","path":"README.md","version_id":"mlnyzn9es6va41eibnowiowc","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":958,"modified_at":"2025-09-09T22:10:03.589000","content_hash":"d7ba59aa325742cdab3010559c5fe306584c1ffd2bcfb50c6799ec834d008312"},"entries":[],"content":"# colorfill\r\n\r\n### Overview\r\n- **Environment ID**: `colorfill`\r\n- **Short description**: Tests the models' ability to discern the percentage of each color in an image.\r\n- **Tags**: multimodal,train\r\n\r\n### Datasets\r\n- **Primary dataset(s)**: colorfill\r\n- **Source links**: https://huggingface.co/datasets/camelCase12/colorfill\r\n- **Split sizes**: 100\r\n\r\n### Task\r\n- **Type**: single-turn\r\n- **Parser**: extract_boxed_answer\r\n- **Rubric overview**: Rewarded by distance from correct fill values.\r\n\r\n### Quickstart\r\nRun an evaluation with default settings:\r\n\r\n```bash\r\nuv run vf-eval colorfill\r\n```\r\n\r\nConfigure model and sampling:\r\n\r\n```bash\r\nuv run vf-eval colorfill   -m gpt-4.1-mini   -n 20 -r 3 -t 1024 -T 0.7\r\n```\r\n\r\n### Metrics\r\nSummarize key metrics your rubric emits and how they’re interpreted.\r\n\r\n| Metric | Meaning |\r\n| ------ | ------- |\r\n| `reward` | Main scalar reward (weighted sum of criteria) |\r\n| `accuracy` | Exact match on target answer |","encoding":"utf-8","truncated":false,"total_bytes":958},"status":null}