{"data":{"kind":"file","path":"README.md","version_id":"eenawz1cd5t4axyi7wwrq5nz","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":683,"modified_at":"2025-09-16T21:32:31.193000","content_hash":"560421f1e3d1230afef51e07ee5a9f8329f91ef28e10300c3e795106f0637a98"},"entries":[],"content":"# med-lm-eval\nAutomated LLM evaluation suite for medical tasks\n# MetaMedQA Eval\n\nThis repository provides an evaluation environment for the [MetaMedQA](https://huggingface.co/datasets/maximegmd/MetaMedQA).\n\n## Usage\n\nTo run an evaluation using [vf-eval](https://github.com/EleutherAI/vf-eval) with the Mistral API, use:\n\n```sh\nuv run vf-eval \\\n\t-m mistral-small-latest \\\n\t-b https://api.mistral.ai/v1 \\\n\t-k MISTRAL_API_KEY \\\n\t--env-args '{\"split\":\"test\"}' \\\n\t--num-examples 200 \\\n\t-s \\\n\tmetamedqa\n```\n\nReplace `MISTRAL_API_KEY` with your actual API key.\n\n## Environment\n\nThe evaluation environment is defined in `metamedqa.py` and uses the HuggingFace `maximegmd/MetaMedQA` dataset.\n","encoding":"utf-8","truncated":false,"total_bytes":683},"status":null}