{"data":{"kind":"file","path":"README.md","version_id":"ozskdt0pxs83x7rx5r1az8b3","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":4014,"modified_at":"2026-06-05T03:54:40.675000","content_hash":"894ef10c4f6e06ece3672c87a77282a06d841f40f03dff7f4e5fc4dfd54c97f3"},"entries":[],"content":"# voice-match\n\nAn RL / evaluation environment for ghostwriting in a specific person's voice. A\npolicy is given a four-bucket intake and writes a post; the reward measures how\nwell the draft matches the target voice.\n\n### Overview\n- **Environment ID**: `voice-match`\n- **Short description**: Score a generated post on how well it matches a client's voice, using deterministic rule checks plus an LLM judge anchored to a human-written reference.\n- **Tags**: writing, style-transfer, judge, single-turn\n\n### Task\n- **Type**: single-turn\n- **Input**: a four-bucket intake (Topic / What happened / What you think / Takeaway), built into a user message. The client's voice + rules go in the system prompt (toggleable).\n- **Output**: a finished post, plain text.\n- **Reward**: weighted sum (normalized to [0, 1]) of one LLM-judge score and four deterministic checks.\n\n### Datasets\nEach row is one post's edit pair. Drop your real pairs into\n`voice_match/data/edit_pairs.jsonl` (one JSON object per line):\n\n```json\n{\n  \"client\": \"faizan\",\n  \"topic\": \"...\",\n  \"what_happened\": \"...\",\n  \"what_you_think\": \"...\",\n  \"takeaway\": \"...\",\n  \"final\": \"the human-edited post you actually published\"\n}\n```\n\n`final` is the reference. It anchors the judge on voice and length but is **not**\na target to copy. The seed file ships three example rows so the env runs out of\nthe box; replace them with your own.\n\n### Rubric\n| Function | Weight | What it measures |\n| --- | --- | --- |\n| `voice_judge` | 0.45 | LLM judge, 0-100, graded against the voice layer + style rules. Carries the real signal. |\n| `no_em_dashes` | 0.20 | Hard rule. 1.0 unless an em/en dash appears. |\n| `no_banned_phrases` | 0.15 | Fraction of banned-phrase/regex checks passed (intrigue-bait, filler transitions). |\n| `length_match` | 0.15 | Triangular credit for matching the reference word count. Kills \"inflate a 2-sentence take into 3 paragraphs\". |\n| `not_slogan_ending` | 0.05 | Soft heuristic against ending on a short abstract slogan. |\n\nDeterministic checks and the judge are split on purpose: regexes catch the\nmechanical tells (these are floors), and the judge handles everything that needs\ntaste. Don't try to push a subjective rule into a regex; it gets gamed.\n\n### Quickstart\n```bash\n# install + run an eval (needs OPENAI_API_KEY for the judge)\nprime eval run voice-match -m openai/gpt-4.1-mini -n 5 -r 3\n```\n\n### Environment Arguments\n| Arg | Type | Default | Description |\n| --- | ---- | ------- | ----------- |\n| `client` | str | `\"yasin\"` | Which voice profile to use (see `voice_match/profiles.py`). |\n| `data_path` | str | bundled seed | Path to your JSONL of edit pairs. |\n| `include_rules_in_prompt` | bool | `True` | If True the policy sees the full brief + rules (tests execution). Set False for RL where you want the voice learned into weights. |\n| `length_tolerance` | float | `0.5` | Relative band for full length-match credit. |\n| `judge_model` | str | `\"gpt-4.1-mini\"` | Judge model id. |\n| `judge_base_url` | str | `None` | Point at a local/self-hosted judge if you want it off-family from the policy. |\n| `eval_fraction` | float | `0.2` | Fraction held out as the eval split. |\n\n### Adding a client\nAdd a `VoiceProfile` to `PROFILES` in `voice_match/profiles.py` with the brief,\nvoice layer, style rules, and banned phrases. Nothing else changes.\n\n### Notes on reward hacking\n- **Run the judge off-family from the policy** (`judge_base_url`) so the policy can't exploit a shared prior.\n- **`length_match` is the anti-padding guard.** Without it a model learns to flood the judge with extra text.\n- **The judge is told not to reward content overlap and to penalize verbatim copying**, so a model can't win by parroting the reference.\n- Keep the deterministic weights moderate. They are floors, not the objective. If they dominate, the model satisfies regexes while writing nothing worth reading.\n- With a small dataset this is most useful as an **eval and a synthetic-data filter** first; scale the pairs before leaning on it for RL.\n\n","encoding":"utf-8","truncated":false,"total_bytes":4014},"status":null}