{"data":{"kind":"file","path":"README.md","version_id":"u00nqni3quefen492m1fvl1i","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2942,"modified_at":"2026-02-01T14:06:42.365000","content_hash":"2b5ebf238bf3c4947b47b4afdb4335759e6dfb9dc853e3dcbccf76b1fff4288a"},"entries":[],"content":"# OpenMed MedSTS Environment\n\nMedical Semantic Textual Similarity environment for RL fine-tuning - scoring similarity between clinical and biomedical sentence pairs.\n\n## Task Description\n\nGiven two medical/clinical sentences, predict their semantic similarity on a 0-4 scale:\n\n| Class | Score | Description |\n|-------|-------|-------------|\n| A | 0 | Unrelated - No meaningful similarity |\n| B | 1 | Low - Vaguely related or share a topic |\n| C | 2 | Moderate - Related concepts but different meanings |\n| D | 3 | High - Similar meaning with some differences |\n| E | 4 | Equivalent - Mean essentially the same thing |\n\n## Dataset\n\nThis environment combines two sources:\n\n- **BIOSSES**: [tabilab/biosses](https://huggingface.co/datasets/tabilab/biosses)\n  - 100 PubMed sentence pairs with expert similarity scores (0-4)\n  - 5 annotators per pair, mean score as ground truth\n\n- **MedNLI-derived**: [presencesw/mednli](https://huggingface.co/datasets/presencesw/mednli)\n  - 11K+ clinical sentence pairs from MIMIC-III\n  - NLI labels mapped to similarity: entailment→3.5, neutral→2.0, contradiction→0.5\n\n| Split | Examples |\n|-------|----------|\n| Train | ~11,300 (BIOSSES + MedNLI train) |\n| Validation | ~1,400 (MedNLI validation) |\n\n## Reward Structure\n\n| Component | Weight | Description |\n|-----------|--------|-------------|\n| Accuracy | 50% | Exact match on similarity bucket (A-E) |\n| Proximity | 25% | Partial credit for adjacent buckets |\n| Thinking | 20% | Quality of reasoning in `<think>` tags |\n| Format | 5% | Proper `\\boxed{}` answer format |\n\n## Example\n\n**Input:**\n```\nSentence 1: \"The patient has a history of type 2 diabetes mellitus managed with metformin.\"\n\nSentence 2: \"Diabetic patient currently on oral hypoglycemic therapy.\"\n```\n\n**Expected Output:**\n```\n<think>\nBoth sentences describe a diabetic patient on medication. Sentence 1 specifies\ntype 2 diabetes and names metformin specifically. Sentence 2 uses the broader\nterm \"oral hypoglycemic therapy\" which includes metformin. The core meaning is\nvery similar - both describe a diabetic patient on oral medication - with slight\ndifferences in specificity.\n</think>\n\\boxed{D}\n```\n\n## Usage\n\n```python\nfrom OpenMed_MedSTS import load_environment\n\nenv = load_environment()\n```\n\n## Citation\n\n```bibtex\n@article{souganciouglu2017biosses,\n  title={BIOSSES: a semantic sentence similarity estimation system for the biomedical domain},\n  author={So{\\u{g}}anc{\\i}o{\\u{g}}lu, Gizem and {\\\"O}zt{\\\"u}rk, Hakime and {\\\"O}zg{\\\"u}r, Arzucan},\n  journal={Bioinformatics},\n  volume={33},\n  number={14},\n  pages={i49--i58},\n  year={2017},\n  publisher={Oxford University Press}\n}\n\n@inproceedings{romanov2018lessons,\n  title={Lessons from Natural Language Inference in the Clinical Domain},\n  author={Romanov, Alexey and Shivade, Chaitanya},\n  booktitle={EMNLP},\n  pages={1586--1596},\n  year={2018}\n}\n```\n\n## License\n\nGPL-3.0 (BIOSSES), PhysioNet License (MedNLI/MIMIC-III derived)\n","encoding":"utf-8","truncated":false,"total_bytes":2942},"status":null}