{"data":{"kind":"file","path":"README.md","version_id":"fcs7mwec172c1ph1m8ynnbum","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3755,"modified_at":"2026-11-13T17:37:44.913000","content_hash":"650f14fbfdd6f3e64ee9bcc9c97ac1086845b296c89410b78d5ce18c4f7b4068"},"entries":[],"content":"![FrontierBioML by Valen: evaluating AI agents on machine learning tasks in computational biology](https://www.frontierbioml.com/assets/brand/fbml-header.png?v=2)\n\n**[Leaderboard and traces](https://www.frontierbioml.com)** · **[Blog post](https://valenlabs.com/blog/frontierbioml)** · **[Valen Labs](https://valenlabs.com)**\n\nMachine learning tasks in computational biology, for coding agents. An agent reads the task statement, works\nin `/workspace`, and writes a **prediction script** to `/app/submission`. Grading re-runs that\nscript twice on a sealed held-out split in a fresh, network-disabled sandbox and scores its\noutput. The hidden split and the scorer never enter the agent's sandbox.\n\nScores are rebased so that the task's reference method scores 0 and a perfect submission scores\n100. A submission that fails any validity check scores 0.\n\n## Scope of this leaderboard\n\nThis environment ships three of the 25 FrontierBioML tasks, so its leaderboard ranks models on\nthose three only, and the order differs from the full benchmark. The full leaderboard, 11 models\nacross all 25 tasks with a trace for every run, is at [frontierbioml.com](https://www.frontierbioml.com), and the\n[blog post](https://valenlabs.com/blog/frontierbioml) describes the benchmark and the results.\n\n## Grader access\n\nScoring sends the submission to a hosted grader and needs a grader token in\n`FRONTIERBIOML_GRADER_TOKEN`. Tokens are available on request from\n[Valen Labs](https://valenlabs.com). The held-out split is never distributed.\n\n## Tasks\n\n| Task | Title | What the agent predicts |\n|---|---|---|\n| `t01` | Nanobody binder triage | Binding probability for every VHH-IL-6 pair, and a family-diverse candidate panel |\n| `t09` | Docking pose scoring for antibody complexes | A score for every docking pose, so the top-scored pose in each panel is near-native |\n| `t22` | Partner-selective KRAS variants | KRAS folding and partner binding, and variants that keep one partner and lose another |\n\n## Data sources and licences\n\nEach task redistributes third-party data. The full notice, including the changes made, ships with\nthe task at `environment/input/licenses/DATA-SOURCE.md`.\n\n| Task | Dataset | Source | Licence |\n|---|---|---|---|\n| `t01` | AVIDa-hIL6 | COGNANO — [dataset](https://huggingface.co/datasets/COGNANO/AVIDa-hIL6) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |\n| `t09` | DeepRank-Ab | [Zenodo 17911452](https://zenodo.org/records/17911452) | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) |\n| `t22` | KRAS deep mutational scan | Weng, Faure, Escobedo and Lehner, *Nature* **626**, 643-652 (2024) — [doi](https://doi.org/10.1038/s41586-023-06954-0) | [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) |\n\n**`t01` is licensed CC BY-NC 4.0: noncommercial use only.** Running this task, and any use of its\ndata or results, must be noncommercial. `t22` additionally redistributes six PDB entries from the\nRCSB Protein Data Bank, unchanged; please cite those entries individually.\n\n## Changelog\n\n- **0.1.7** — Describe the benchmark as machine learning tasks in computational biology; updated header image.\n- **0.1.6** — README: FrontierBioML header image instead of the title and leaderboard graphic.\n- **0.1.5** — README: updated leaderboard graphic.\n- **0.1.4** — README: leaderboard graphic, links to the website and the blog post.\n- **0.1.3** — the hosted grader moves to its new address (the `frontierbioml-grader` app).\n- **0.1.2** — README: scope of the leaderboard, and how to get a grader token.\n- **0.1.1** — fix the taskset export: `frontierbioml/__init__.py` imported from the old package name.\n- **0.1.0** — first release: `t01`, `t09`, `t22`, each with its data-source notice.\n","encoding":"utf-8","truncated":false,"total_bytes":3755},"status":null}