{"data":{"kind":"file","path":"README.md","version_id":"p42wut6r02c9kkg561ml1tyz","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3726,"modified_at":"2026-07-08T23:16:44.906000","content_hash":"3748e7df477eb0cabe44282bc7a93def3f3e84bb57fe82678280df96ce4317c4"},"entries":[],"content":"# drug-tool-rl\n\nA multi-turn, tool-using RL environment for drug-discovery agents. The model\nanswers biomedical questions about drug targets, disease mechanisms, protein\nfunction, molecular interactions, and candidate molecules by calling biomedical\nretrieval tools, then synthesizing a grounded final answer.\n\n## Task\n\nEach example is a biomedical prompt paired with a reference tool-use trace from\n`lokahq/drug-tool-sft`. Rollouts run an evidence-first loop: call tools, inspect\nresults, reformulate empty searches, and end with one concise synthesis.\n\n## Tools\n\nSeven retrieval tools are always available:\n\n- `search_pubmed` - PubMed literature via NCBI eUtils.\n- `search_geo_datasets` - NCBI GEO expression datasets.\n- `search_kegg_by_keyword` / `search_kegg_by_id` - KEGG pathways, diseases,\n  drugs, and compounds.\n- `search_uniprot_by_name` / `search_uniprot_by_id` - UniProt protein records\n  and sequences.\n- `search_string_interactions` - STRING protein-protein interaction networks.\n\nThree NVIDIA NIM structural-biology tools are optional when\n`include_nvidia_tools=True`:\n\n- `fold_protein` - ESMFold structure prediction with rollout-local PDB storage.\n- `dock_ligand` - DiffDock protein-ligand docking from raw PDB or a stored fold\n  artifact.\n- `molmim_generate` - MolMIM molecule generation or optimization from a seed\n  SMILES.\n\nThe environment caches identical tool calls within a rollout and shares\nrollout-local structural artifacts across stateful tools.\n\n## Reward\n\nThe only supported reward profile is `grounded_tool_synthesis`.\n\n| signal | weight | description |\n| --- | ---: | --- |\n| `key_fact_grounding_reward` | 0.33 | Fraction of final-answer key facts grounded in tool evidence, down-weighted when the model ignores tool failures. |\n| `tool_result_success_reward` | 0.19 | Penalizes tool errors, warnings, no-result hits, invalid stateful-tool arguments, duplicate calls, and UniProt entity mismatches. |\n| `tool_selection_reward` | 0.14 | Tool multiset F1 plus call-order LCS similarity against the reference trace. |\n| `tool_argument_reward` | 0.12 | Best-match argument-key/value similarity for selected tools against the reference. |\n| `tool_activation_reward` | 0.05 | Rewards emitting at least one parser-compatible tool call. |\n| `synthesis_completion_reward` | 0.05 | Rewards a non-truncated final synthesis after tool use. |\n| `efficiency_reward` | 0.04 | Matches the reference trace's tool-call budget, capped at 8. |\n| `concise_final_answer_reward` | 0.04 | Gives full credit to concise final answers, decaying by length. |\n| `non_truncated_reward` | 0.04 | Rewards rollouts that do not truncate. |\n\nAdditional parser, grounding, hallucination, error, warning, and rate-limit\nmetrics are logged at weight 0 for analysis.\n\n## Contract\n\n```python\nfrom drug_tool_rl import load_environment\n\nenv = load_environment(\n    dataset_name=\"lokahq/drug-tool-sft\",\n    dataset_config=\"openrouter_trinity_v1\",\n    split=\"train\",\n    num_examples=-1,\n    max_turns=8,\n    include_nvidia_tools=False,\n    nvidia_api_key_var=\"NVIDIA_API_KEY\",\n    reward_profile=\"grounded_tool_synthesis\",\n)\n```\n\nUnsupported `reward_profile` values raise `ValueError`.\n\n## Environment Variables\n\n- `NVIDIA_API_KEY` or `NVCF_RUN_KEY` - required only when\n  `include_nvidia_tools=True`.\n- `NCBI_API_KEY` - optional; raises NCBI eUtils rate limits for PubMed/GEO.\n\nRetrieval tools hit public NCBI, KEGG, UniProt, and STRING endpoints with bounded\nretries. NVIDIA NIM calls are capped by `NVIDIA_CALL_DEADLINE` so a slow\nstreaming call surfaces as a tool error instead of hanging the eval.\n\n## Dependencies\n\n`datasets`, `httpx`, `rdkit`, `tenacity`, and `verifiers`.\n\nPublished environment handle: `lokahq/drug-tool-rl@1`.\n","encoding":"utf-8","truncated":false,"total_bytes":3726},"status":null}