{"data":{"kind":"file","path":"README.md","version_id":"byeczvo9odb2volody2i5zt0","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3800,"modified_at":"2026-02-22T01:06:21.275000","content_hash":"31e1d478913bde08f55144a81263d297e5763b94f0078cf7002908d52d6b7c92"},"entries":[],"content":"# czech-beer-brand-name\n\n### Overview\n- **Environment ID**: `czech-beer-brand-name`\n- **Short description**: Predict a Czech beer brand from bottle appearance descriptions.\n- **Tags**: `czech`, `beer`, `classification`, `vision`, `ocr`\n\n### Task\n- **Type**: single-turn classification\n- **Output format expectations**: reasoning allowed, final brand must be inside `<guess>...</guess>`\n- **Rubric**:\n1. `brand_exact_match_weighted` (weight `1.0`): exact canonical match using aliases, scaled by description level difficulty.\n2. `brand_exact_match` (metric only): unweighted exact match for plain accuracy tracking.\n3. `format_reward` (weight `0.05`): requires a valid XML answer tag format (default `<guess>...</guess>`).\n\n### No Hardcoded Brands\n- Brand names are not defined in code.\n- The environment derives the full brand set from `dataset_path` (and `eval_dataset_path` if provided).\n- Optional aliases can be provided per row via `aliases`.\n\n### Image Dataset Workflow\nPut bottle images in:\n- `environments/czech_beer_brand_name/data/images/`\n\nUse either:\n1. **Filename labels** (quick start):\n- `pilsner_urquell_01.jpg`\n- `kozel_12.png`\n\n2. **Explicit labels file** (`labels.jsonl`):\n- one row per image with `image_path` and `brand`\n- copy template from `environments/czech_beer_brand_name/data/labels.template.jsonl`\n\nGenerate OCR descriptions (bottle/logo/distinct features, no taste):\n```bash\nexport OPENROUTER_API_KEY=your_key_here\n\npython environments/czech_beer_brand_name/scripts/build_image_ocr_dataset.py \\\n  --images-dir environments/czech_beer_brand_name/data/images \\\n  --labels-jsonl environments/czech_beer_brand_name/data/labels.jsonl \\\n  --out environments/czech_beer_brand_name/data/beer_bottles_ocr.jsonl \\\n  --model minimax/minimax-m2\n```\n\nOutput row format:\n- required: `brand`, `description`\n- optional: `description_short`, `description_brief`, `description_minimal`\n- optional: `aliases` (list), `image_path`, `source`, `label_method`, `id`\n\nGenerate 3 or 4 description levels from existing full descriptions:\n```bash\nexport OPENROUTER_API_KEY=your_key_here\n\npython environments/czech_beer_brand_name/scripts/generate_description_levels.py \\\n  --in environments/czech_beer_brand_name/data/beer_bottles_ocr.jsonl \\\n  --out environments/czech_beer_brand_name/data/beer_bottles_ocr_levels.jsonl \\\n  --model openai/gpt-4.1-mini \\\n  --num-levels 3\n```\n\n### Environment Arguments\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `dataset_path` | str | required | JSONL with labeled OCR rows |\n| `eval_dataset_path` | str | `\"\"` | optional separate eval JSONL |\n| `eval_size` | int | `40` | split size if no `eval_dataset_path` |\n| `include_candidates` | bool | `true` | include candidate brand list in prompt |\n| `seed` | int | `7` | deterministic split/shuffle seed |\n| `description_levels` | str | `\"detailed,short,brief\"` | comma-separated levels to include (`detailed,short,brief,minimal`) |\n| `detailed_level_reward` | float | `1.0` | reward multiplier for detailed descriptions |\n| `short_level_reward` | float | `1.0` | reward multiplier for short descriptions |\n| `brief_level_reward` | float | `1.0` | reward multiplier for brief descriptions |\n| `minimal_level_reward` | float | `1.0` | reward multiplier for minimal descriptions |\n| `wrong_answer_penalty` | float | `-0.1` | reward returned when predicted brand is wrong or unparsable |\n| `format_reward_weight` | float | `0.0` | rubric weight for XML format reward metric |\n| `answer_tag` | str | `\"guess\"` | XML tag used for final answer (model must answer in `<tag>...</tag>`) |\n\n### Quickstart\n```bash\nprime eval run czech-beer-brand-name -m gpt-4.1-mini -n 20 -r 1 \\\n  -a '{\"dataset_path\":\"environments/czech_beer_brand_name/data/beer_bottles_ocr.jsonl\",\"eval_size\":8}' -s\n```\n","encoding":"utf-8","truncated":false,"total_bytes":3800},"status":null}