{"data":{"kind":"file","path":"README.md","version_id":"m4lccrbmw2m016qif9s75a0w","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1535,"modified_at":"2025-08-22T03:30:56","content_hash":"5897e528513f2e5cdb508494ace3d4b17527959a4031aae22817839aab2695bc"},"entries":[],"content":"# summarize-text\n\n### Overview\n- **Environment ID**: `summarize-text`\n- **Short description**: Summarize a paragraph into three sentences using a specified XML response format.\n- **Tags**: summarization, single-turn, xml, lcs\n\n### Datasets\n- **Primary dataset(s)**: `agentlans/wikipedia-paragraphs` mapped to `question`=`text`, `answer`=`text`\n- **Source links**: Hugging Face Datasets\n- **Split sizes**: Uses the `train` split for evaluation\n\n### Task\n- **Type**: single-turn\n- **Parser**: `XMLParser([\"think\",\"answer\"])`\n- **Rubric overview**: (1) Exactly 3 sentences; (2) LCS similarity to source; (3) format check\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nuv run vf-eval summarize-text\n```\n\nConfigure model and sampling:\n\n```bash\nuv run vf-eval summarize-text \\\n  -m gpt-4.1-mini \\\n  -n 20 -r 3 -t 1024 -T 0.7\n```\n\nNotes:\n- Reports are written under `./environments/summarize_text/reports/` and auto-embedded below.\n\n### Environment Arguments\nThis loader does not expose custom arguments.\n\n### Metrics\n| Metric | Meaning |\n| ------ | ------- |\n| `sentence_reward_func` | 1.0 if exactly three sentences, else 0.0 |\n| `lcs_reward_func` | LCS similarity between source and parsed summary |\n| `format_reward` | Adherence to `<think>`/`<answer>` XML format |\n\n## Evaluation Reports\n\n<!-- Do not edit below this line. Content is auto-generated. -->\n<!-- vf:begin:reports -->\n<p>No reports found. Run <code>uv run vf-eval vf-summarize-text -a '{\"key\": \"value\"}'</code> to generate one.</p>\n<!-- vf:end:reports -->\n","encoding":"utf-8","truncated":false,"total_bytes":1535},"status":null}