{"data":{"kind":"file","path":"README.md","version_id":"ug16r9ymp1fxhqol8yg6hvjt","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1419,"modified_at":"2026-08-25T09:34:21.548000","content_hash":"0eb93d73c8e08cd72fa0434bc13fd943d3d17fb9105531bb28d08869c28f86a4"},"entries":[],"content":"# xlam-qwencoder-env\n\n### Overview\n- **Environment ID**: `xlam-qwencoder-env`\n- **Short description**: A function calling environment evaluating the XLAM function-calling dataset using the Qwen3 coder tool format.\n- **Tags**: function-calling, xlam, qwen3, tool-use\n\n### Datasets\n- **Primary dataset(s)**: `Salesforce/xlam-function-calling-60k`\n- **Source links**: [Hugging Face](https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k)\n- **Split sizes**: Evaluates on the `train` split.\n\n### Task\n- **Type**: single-turn | tool use\n- **Output format expectations**: Native Qwen3 XML format. Models are prompted to think step-by-step inside `<think>...</think>` tags, and call tools using:\n  ```xml\n  <tool_call>\n  <function=tool_name>\n  <parameter=arg_name>arg_value</parameter>\n  </function>\n  </tool_call>\n  ```\n- **Rubric overview**: Exact match on predicted vs. target tool calls. The custom `Qwen3ToolParser` is used to extract tools and their arguments.\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nprime eval run xlam-qwencoder-env\n```\n\nConfigure model and sampling:\n\n```bash\nprime eval run xlam-qwencoder-env -m openai/gpt-4o-mini -n 20 -r 3 -t 1024 -T 0.7\n```\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Binary reward (1.0 or 0.0). Returns 1.0 if all predicted tool names and arguments exactly match the target tools for the prompt, otherwise 0.0. |\n","encoding":"utf-8","truncated":false,"total_bytes":1419},"status":null}