{"data":{"kind":"file","path":"README.md","version_id":"bdef3dchdr553pbrwmu10601","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2685,"modified_at":"2026-01-14T02:04:15.799000","content_hash":"902c64c79991cb9360b6dc4adccf7a460cf3bc8c83c4ce299d880defd7e457d3"},"entries":[],"content":"# deepscaler-multireward\n\n### Overview\n- **Environment ID**: `deepscaler-multireward`\n- **Short description**: DeepScaleR with multi-reward support (correctness + length)\n- **Tags**: math, single-turn, think, boxed-answer, multi-reward\n\n### Datasets\n- **Primary dataset(s)**: [agentica-org/DeepScaleR-Preview-Dataset](https://huggingface.co/datasets/agentica-org/DeepScaleR-Preview-Dataset)\n- **Source links**: [HuggingFace](https://huggingface.co/datasets/agentica-org/DeepScaleR-Preview-Dataset)\n- **Split sizes**: train split\n\n### Task\n- **Type**: single-turn\n- **Parser**: MaybeThinkParser (handles `<think>...</think>` blocks, extracts `\\boxed{}` answers)\n- **Rubric overview**:\n  - `correct_answer`: Binary (0/1) from math-verify rule-based grading\n  - `length_reward`: Binary (0/1) based on completion length threshold\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nuv run vf-eval deepscaler-multireward\n```\n\nConfigure model and sampling:\n\n```bash\nuv run vf-eval deepscaler-multireward \\\n  -m gpt-4.1-mini \\\n  -n 20 -r 3 -t 1024 -T 0.7 \\\n  -a '{\"length_threshold_tokens\": 4096}'\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `dataset_name` | str | `\"agentica-org/DeepScaleR-Preview-Dataset\"` | HuggingFace dataset name |\n| `dataset_subset` | str | `\"default\"` | Dataset subset/config |\n| `dataset_split` | str | `\"train\"` | Dataset split |\n| `system_prompt` | str \\| None | `None` | Optional system prompt |\n| `length_threshold_tokens` | int \\| None | `4096` | Token threshold for length reward (None to disable) |\n| `length_reward_weight` | float | `1.0` | Weight for length reward in final score |\n| `chars_per_token` | float | `4.0` | Estimated characters per token |\n| `math_verify_max_workers` | int | `10` | Thread pool size for math verification |\n| `math_verify_timeout` | float | `5.0` | Timeout for math verification (seconds) |\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Weighted sum of all reward functions |\n| `correct_answer` | Binary correctness from math verification (0 or 1) |\n| `length_reward` | Binary length reward (1 if under threshold, 0 otherwise) |\n\n### Example Output\n\nAfter scoring, `state[\"metrics\"]` contains the individual rewards:\n\n```python\nstate[\"metrics\"] = {\n    \"correct_answer\": 1.0,  # Math verification passed\n    \"length_reward\": 1.0,   # Under token threshold\n}\nstate[\"reward\"] = 2.0  # Weighted sum (1.0 * 1 + 1.0 * 1)\n```\n\n### References\n\n- [DeepScaleR](https://pretty-radio-b75.notion.site/DeepScaleR-Surpassing-O1-Preview-with-a-1-5B-Model-by-Scaling-RL-19681902c1468005bed8ca303013a4e2) - Original DeepScaleR experiments\n","encoding":"utf-8","truncated":false,"total_bytes":2685},"status":null}