{"data":{"kind":"file","path":"README.md","version_id":"bxol4inbman528lerbe0l3py","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2231,"modified_at":"2026-05-27T23:30:01.309000","content_hash":"ebce62ddb433c9ab97bcbb6f83bb596f5e362de6d2f6328ffe114f14c82b6c35"},"entries":[],"content":"# swebench-pro\n\nSWE-bench Pro as a Verifiers v1 environment. Tasks are loaded through the\nHarbor taskset from `scale-ai/swe-bench-pro`.\n\n## Overview\n\n- **Environment ID**: `swebench-pro`\n- **Type**: v1 Harbor taskset\n- **Dataset**: `scale-ai/swe-bench-pro`\n- **Verifiers dependency**: `verifiers==0.1.15.dev11`\n\nSWE-bench Pro task metadata does not set `environment.docker_image` in\n`task.toml`. Each task instead names its matching prebuilt SWEAP image in\n`tests/config.json` as `dockerhub_tag`, so this environment copies that tag into\nthe Harbor sandbox image fields after Harbor loads the task rows.\n\n## Install\n\n```bash\nprime env install swebench-pro\n```\n\n## Run\n\n```bash\nprime eval run swebench-pro -n 1 -r 1\n```\n\n## Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `tasks` | str \\| None | `scale-ai/swe-bench-pro` | Harbor dataset ID. |\n| `task_names` | list[str] \\| None | `None` | Optional task-name allowlist. |\n| `cache_dir` | str \\| Path \\| None | `None` | Override the Harbor task cache directory. |\n| `refresh` | bool \\| None | `None` | Refresh the cached Harbor dataset. |\n| `harness` | vf.Harness \\| str | `verifiers.v1.packages.harnesses.mini_swe_agent` | Custom v1 harness instance or harness module string. |\n| `harness_config` | dict \\| None | `None` | Optional config passed to string-loaded harness modules. |\n| `max_turns` | int \\| None | `None` | Convenience override passed to the selected harness config. |\n\nTyped Verifiers configs can also supply `taskset` and generic `harness` sections\nthrough `SwebenchProEnvConfig`. Use `[eval.args]` for loader-level knobs such as\n`harness` and `harness_config`.\n\nTo use a packaged v1 harness from TOML, pass its module as an environment arg.\nFor example:\n\n```toml\n[eval.args]\nharness = \"verifiers.v1.packages.harnesses.rlm\"\n\n[eval.args.harness_config]\nrlm_tools = [\"bash\", \"edit\"]\n```\n\n## Changelog\n\n### 0.1.0\n\n- Initial SWE-bench Pro environment using Harbor tasks from `scale-ai/swe-bench-pro`.\n- Copies each task's SWEAP `dockerhub_tag` metadata into Harbor's sandbox image fields.\n- Supports packaged v1 harness modules via string config.\n- Requires `verifiers==0.1.15.dev11` and `prime-sandboxes>=0.2.19`.\n","encoding":"utf-8","truncated":false,"total_bytes":2231},"status":null}