{"data":{"kind":"file","path":"README.md","version_id":"ofiemv6rb4s3wjbfum7ue84e","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":6913,"modified_at":"2026-05-18T10:11:40.420000","content_hash":"7fda82f94661d3b0c300019221950d9da31598b14d293f9d33f1d4fcbc544d6c"},"entries":[],"content":"# openfarm-bioacustics\n\n<p>\n  <a href=\"https://github.com/ob1-s/happy-farm/tree/main/environments/openfarm_bioacustics\">\n    <img align=\"left\" hspace=\"4\" src=\"https://img.shields.io/badge/GitHub-181717?style=for-the-badge&logo=github&logoColor=white\" alt=\"GitHub\">\n  </a>\n  <a href=\"https://app.primeintellect.ai/dashboard/environments/openfarm/openfarm-bioacustics\">\n    <img align=\"left\" hspace=\"4\" src=\"https://img.shields.io/badge/Prime%20Intellect-Envs%20Hub-181717?style=for-the-badge&labelColor=181717&logoColor=white\" alt=\"Prime Intellect Environments Hub\">\n  </a>\n</p>\n<br clear=\"all\" />\n\n## Overview\n\nOpenFARM Bioacoustics is a task-registry environment for animal vocalization\nbenchmarks. It keeps each source dataset/task explicit, while exposing a single\nSoundWel-like interface for audio, spectrogram, multimodal, and text/acoustic\nmetadata ablations.\n\n- **Environment ID**: `openfarm-bioacustics`\n- **Type**: single-turn classification / EnvGroup when multiple tasks are selected\n- **Modalities**: `audio`, `vision`, `multimodal`, `text`\n- **Output format**: XML answer, with optional explanation\n- **Primary metric**: exact normalized answer reward\n\nVision mode uses a source spectrogram column when one exists. For audio-only\ndatasets, the environment generates a compact spectrogram from the audio at load\ntime.\n\n## Tasks\n\n| Task | Dataset | Label |\n| --- | --- | --- |\n| `soundwel_valence` | `oliveirabruno01/soundwel-pig-vocalizations` | `Positive` / `Negative` |\n| `soundwel_context` | `oliveirabruno01/soundwel-pig-vocalizations` | pig vocalization context |\n| `soundwel_call_type` | `oliveirabruno01/soundwel-pig-vocalizations` | `HF` / `LF` |\n| `soundwel_age` | `oliveirabruno01/soundwel-pig-vocalizations` | pig age category |\n| `catmeows_context` | `oliveirabruno01/openfarm-catmeows` | feline source context |\n| `laying_hen_stress_context` | `oliveirabruno01/openfarm-laying-hen-stress` | experimental stress context |\n| `laying_hen_stress_binary` | `oliveirabruno01/openfarm-laying-hen-stress` | stress-response binary |\n| `ungulate_valence` | `oliveirabruno01/openfarm-ungulate-valence` | `Positive` / `Negative` |\n\n## Source Attribution\n\n| Dataset | Prepared HF Dataset | Source Dataset / Paper |\n| --- | --- | --- |\n| SoundWel pig vocalizations | [`oliveirabruno01/soundwel-pig-vocalizations`](https://huggingface.co/datasets/oliveirabruno01/soundwel-pig-vocalizations) | The Soundwel Database: a labeled pig vocalization repository, Zenodo record [`8252482`](https://zenodo.org/records/8252482), DOI [`10.1038/s41598-022-07174-8`](https://doi.org/10.1038/s41598-022-07174-8), CC-BY-4.0 |\n| CatMeows | [`oliveirabruno01/openfarm-catmeows`](https://huggingface.co/datasets/oliveirabruno01/openfarm-catmeows) | CatMeows: A Publicly-Available Dataset of Cat Vocalizations, Zenodo record [`4008297`](https://zenodo.org/records/4008297), DOI [`10.5281/zenodo.4008297`](https://doi.org/10.5281/zenodo.4008297), CC-BY-4.0 |\n| Laying hen stress vocalizations | [`oliveirabruno01/openfarm-laying-hen-stress`](https://huggingface.co/datasets/oliveirabruno01/openfarm-laying-hen-stress) | Vocalization Patterns in Laying Hens - An Analysis of Stress-Induced Audio Responses, Zenodo record [`10433023`](https://zenodo.org/records/10433023), DOI [`10.5281/zenodo.10433023`](https://doi.org/10.5281/zenodo.10433023), CC-BY-4.0 |\n| Ungulate vocalization valence | [`oliveirabruno01/openfarm-ungulate-valence`](https://huggingface.co/datasets/oliveirabruno01/openfarm-ungulate-valence) | Machine Learning Algorithms Can Predict Emotional Valence Across Ungulate Vocalizations, Zenodo record [`14636641`](https://zenodo.org/records/14636641), DOI [`10.5281/zenodo.14636641`](https://doi.org/10.5281/zenodo.14636641), CC-BY-4.0 |\n\n## Quickstart\n\n```bash\nprime eval run openfarm-bioacustics \\\n  -a '{\"task\": \"soundwel_valence\", \"modality\": \"audio\", \"max_examples_per_task\": 20}'\n```\n\nRun several tasks as one EnvGroup:\n\n```bash\nprime eval run openfarm-bioacustics \\\n  -a '{\"task\": [\"soundwel_valence\", \"catmeows_context\"], \"modality\": \"multimodal\", \"max_examples_per_task\": 20}'\n```\n\nUse generated spectrograms for an audio-only dataset:\n\n```bash\nprime eval run openfarm-bioacustics \\\n  -a '{\"task\": \"catmeows_context\", \"modality\": \"vision\", \"max_examples_per_task\": 20}'\n```\n\nRun the ungulate valence benchmark on its balanced, non-pig animal-heldout test\nsplit:\n\n```bash\nprime eval run openfarm-bioacustics \\\n  -a '{\"task\": \"ungulate_valence\", \"modality\": \"audio\", \"max_examples_per_task\": 20}'\n```\n\n## Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | --- | --- | --- |\n| `task` | str/list | `\"soundwel_valence\"` | Task name, list of task names, or `\"all\"`. |\n| `modality` | str | `\"audio\"` | `audio`, `vision`, `multimodal`, or `text`. |\n| `include_tabular_data` | bool | `false` | Adds task-specific leakage-safe metadata. Forced on for `text`. |\n| `use_escape_hatch` | bool | `false` | Adds `UNINTELLIGIBLE` as an allowed answer. |\n| `max_examples_per_task` | int | `-1` | Optional per-task subsampling budget. |\n| `balancing_strategy` | str | `\"proportional\"` | `proportional` or `balanced` when subsampling. |\n| `target_pad_seconds` | float | `3.592` | Center-pads short audio before encoding. |\n| `max_audio_seconds` | float/null | `null` | Clips long audio before encoding or spectrogram generation. |\n| `audio_clip_strategy` | str | `\"start\"` | `start`, `center`, or `even_windows` for clipping long recordings. |\n| `require_explanation` | bool | `false` | Requires an `<explanation>` field before `<answer>`. |\n| `format_reward_weight` | float | `0.0` | Optional XML format reward weight. |\n\n## Dataset Notes\n\n### Laying Hen Stress\n\n- Public audio artifacts are bounded 15-second mono 16 kHz excerpts.\n- Excerpts use `even_windows` over the longer Zenodo recordings because the\n  source data does not provide stress-event timestamps.\n- The task still defaults to `max_audio_seconds=15.0` and\n  `audio_clip_strategy=\"even_windows\"` as a safety net for older dataset\n  revisions or local experiments pointed at full source recordings.\n\n### Ungulate Valence\n\n- Headline evals should use the cleaned, balanced `train`/`test` split pair.\n- `train_raw` and `test_raw` preserve the natural non-pig class distribution for\n  explicit diagnostics.\n- `pig_family_heldout` isolates domestic pig and wild boar rows for optional\n  transfer checks against SoundWel/PVWB-adjacent work.\n- Default tabular prompts use only acoustic features. Context, source reference,\n  and animal ID can be predictive of the valence label.\n\n## Metrics\n\nThe live rubric is intentionally small:\n\n| Metric | Meaning |\n| --- | --- |\n| `accuracy_reward` | 1.0 when the parsed answer matches the task label after normalization. |\n| `format_reward` | Optional XML-format reward when `format_reward_weight > 0`. |\n\nUse post-eval reporting for macro accuracy, balanced accuracy, macro F1, and\nper-class recall.\n","encoding":"utf-8","truncated":false,"total_bytes":6913},"status":null}