{"data":{"kind":"file","path":"README.md","version_id":"rohs73kgy4e0qbaiy5syn5v3","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1824,"modified_at":"2026-02-19T05:36:23.472000","content_hash":"1df4c34076e24e663e2e776daa236bdd8b3940e19075e4a3d5c5ef77ebbfbcc1"},"entries":[],"content":"# traverse-tasks\n\n### Overview\n- **Environment ID**: `traverse-tasks`\n- **Short description**: Multi module reasoning environment testing LLM ability to maintain state across library systems, game inventory optimization, and priority job queues.\n- **Tags**: logic-reasoning, knapsack-problem, state-management, python-execution\n\n### Datasets\n- **Primary dataset(s)**: Custom synthetic dataset consisting of three distinct architectural challenges.\n- **Source links**: Defined internally within traverse_tasks.py.\n- **Split sizes**: 3 evaluation tasks (Library Renewal, Game Loadout, Job Queue)\n\n### Task\n- **Type**: single-turn\n- **Output format expectations (optional)**: A single Python code block containing multiple merged modules.\n- **Rubric overview**: Automated execution-based rubric that verifies logical correctness, state transitions, and optimization accuracy via unit tests in a sandboxed exec() environment.\n\n### Quickstart\nRun an evaluation with default settings:\n\n```bash\nprime eval run traverse-tasks\n```\n\nConfigure model and sampling:\n\n```bash\nprime eval run traverse-tasks   -m gpt-4.1-mini   -n 20 -r 3 -t 1024 -T 0.7   -a '{\"key\": \"value\"}'  # env-specific args as JSON\n```\n\nNotes:\n- Use `-a` / `--env-args` to pass environment-specific configuration as a JSON object.\n\n### Environment Arguments\nDocument any supported environment arguments and their meaning. Example:\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `foo` | str | `\"bar\"` | What this controls |\n| `max_examples` | int | `-1` | Limit on dataset size (use -1 for all) |\n\n### Metrics\nSummarize key metrics your rubric emits and how they’re interpreted.\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Main scalar reward (weighted sum of criteria) |\n| `accuracy` | Exact match on target answer |\n\n","encoding":"utf-8","truncated":false,"total_bytes":1824},"status":null}