{"data":{"kind":"file","path":"README.md","version_id":"tq7rui1dwm53js0zk384fc5u","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2744,"modified_at":"2026-03-24T21:54:03.071000","content_hash":"17de5cff50e2e2be40b935b65a093040b966bca61f5e3f98481fe09b1590eca1"},"entries":[],"content":"# learn-coval-pointwise-sdpo\n\n### Overview\n- **Environment ID**: `learn-coval-pointwise-sdpo`\n- **Short description**: SDPO-inspired 2-phase rubric learning with judge feedback between generation phases\n- **Tags**: pointwise, sdpo, multi-turn, verifiers, xml, coval\n\n### Datasets\n- **Primary dataset(s)**: `sumuks/coval-world-prefs` — conversation contexts with chosen/rejected response pairs\n- **Split sizes**: 5,762 train / 637 eval\n\n### Task\n- **Type**: multi-turn (2-phase)\n- **Output format**: XML `<rubric>...</rubric>` tags\n- **Rollout structure**:\n  1. Phase 1: Model generates rubric from conversation context\n  2. Environment: Judge scores chosen/rejected with Phase 1 rubric, returns feedback\n  3. Phase 2: Model generates improved rubric conditioned on feedback\n  4. Reward: Computed on Phase 2 rubric\n\n### Quickstart\n\n```bash\nprime env install learn-coval-pointwise-sdpo\nprime eval run learn-coval-pointwise-sdpo -m gpt-4.1-mini -n 5 -r 1\n```\n\n### Environment Arguments\n\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `dataset_name` | str | `sumuks/coval-world-prefs` | HuggingFace dataset |\n| `train_split` | str | `train` | Training split name |\n| `test_split` | str | `test` | Evaluation split name |\n| `run_setting` | str | `rupo_pointwise_advantage` | `rupo_pointwise` or `rupo_pointwise_advantage` |\n| `judge_model` | str | env `JUDGE_MODEL` | Model for pointwise scoring |\n| `judge_base_url` | str | env `JUDGE_BASE_URL` | OpenAI-compatible judge endpoint |\n| `judge_api_key_var` | str | env `JUDGE_API_KEY_VAR` or `PRIME_API_KEY` | Env var name whose value is the judge API key (same pattern as `ss-rupo`) |\n| `judge_max_concurrent_requests` | int | `1024` | Semaphore limit for judge concurrency |\n| `baseline_model_name` | str | `None` | Key for dataset baseline scores |\n| `hard_only` | bool | `False` | Filter to examples where baseline margin <= 0 |\n\n### Metrics\n\n| Metric | Meaning |\n| ------ | ------- |\n| `reward` | Main scalar (Phase 2 pointwise margin or advantage) |\n| `rupo_pointwise_reward` | (chosen - rejected) / 100 from Phase 2 rubric |\n| `rupo_chosen_pointwise_score` | Judge score on chosen with Phase 2 rubric |\n| `rupo_rejected_pointwise_score` | Judge score on rejected with Phase 2 rubric |\n| `rubric_xml_format_score` | XML format adherence |\n| `phase1_chosen_score` | Judge score on chosen with Phase 1 rubric |\n| `phase1_rejected_score` | Judge score on rejected with Phase 1 rubric |\n| `phase1_margin` | Phase 1 chosen - rejected (raw) |\n| `phase2_minus_phase1_margin` | Improvement from feedback (positive = feedback helped) |\n| `baseline_pointwise_margin` | Dataset baseline margin |\n| `baseline_pointwise_binary_reward` | 1.0 if baseline chosen > rejected |\n","encoding":"utf-8","truncated":false,"total_bytes":2744},"status":null}