{"data":{"kind":"file","path":"README.md","version_id":"fditisv4df7x36dborbd0yb5","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":8920,"modified_at":"2026-06-26T23:12:08.514000","content_hash":"39c70795c7c6e65d755af938040570a59414605889d5d5ad17236ba4187b5084"},"entries":[],"content":"# humanize-rl-env\n\n**Prime Intellect Verifiers** single-turn environment for Humanize-RL.\n\nPublished at: `jayshah5696/humanize-rl-env`\n\n---\n\n## What this does\n\nEach episode is a single writing task — rewrite a corporate Slack update into\ncasual human prose, compress a meeting recap, or draft a direct workplace\nmessage. The model receives the task prompt, produces one completion, and the\nenvironment scores it.\n\n---\n\n## Reward Modes\n\nDefault training mode:\n\n```\nreward_mode=\"p50_50_no_penalty\"\nreward = 0.50 × ridge_rubric + 0.50 × deterministic\n         clipped to [-1.0, 1.0]\n```\n\nPenalties are still computed and logged as diagnostics, but they do not affect\nthe optimizer scalar in this mode.\n\nStrict eval mode:\n\n```\nreward_mode=\"strict\"\nreward = 0.50 × ridge_rubric + 0.50 × deterministic + penalties\n         clipped to [-1.0, 1.0]\n```\n\n`reward_mode=\"scalar_softened\"` is kept for legacy Modal/TRL comparison runs.\n\n### Ridge Rubric (50%)\n\nMean of 8 regression heads from a TF-IDF Logistic+Ridge scorer trained on\n10,000 Gemini-labelled writing samples (AUROC 0.9988, rubric MSE 0.041).\nThe scorer estimates the same humanness dimensions the LLM judge uses:\n\n| dim | what it penalizes |\n|---|---|\n| `structural_symmetry` | rigid intro/body/conclusion templates, formulaic lists |\n| `specificity` | vague claims without names, numbers, or concrete context |\n| `formality_gradient` | unnatural tone shifts |\n| `voice_consistency` | generic assistant voice instead of a specific writer |\n| `rhetorical_sophistication` | filler analysis, stakes inflation, shallow reasoning |\n| `padding_density` | repeated restatements, zero-information sentences |\n| `personality_presence` | lack of opinion, perspective, friction |\n| `copula_avoidance` | pompous substitutes for simple is/are/was |\n\nThe scorer pkl is **bundled inside the wheel** (`ridge_state.pkl`) — no\nexternal download or API call required.\n\n### Deterministic (50%)\n\nEqual-weight mean of constraint checks that fire on hard task rules:\n\n| component | what it checks |\n|---|---|\n| `faithfulness` | no invented details, missing numbers/entities, dropped/added facts |\n| `task_following` | no option menus, wrapper phrases, or refusals |\n| `length` | within `max_words` / `min_words` / `exact_sentences` |\n| `format` | no subject lines, signoffs, or forbidden markdown |\n| `placeholder` | placeholder usage matches task constraints |\n| `clarity` | avg sentence length (≤24 words → 1.0, ≤35 → 0.7, else 0.4) |\n| `repetition` | repeated n-grams and density caps |\n| `semantic_faithfulness` | hard semantic failures, including vague substitutions for concrete facts |\n| `recommendation_suitability` | task-specific recommendation suitability caps |\n| `surface_naturalness` | emoji/all-caps/AI tells plus fake-casual, broken-grammar, slang, and padding caps |\n| `hard_format` | direct-answer format, exact phrase, structure, and placeholder gates |\n\n### Penalties (diagnostics; additive only in strict mode)\n\n| penalty | value |\n|---|---|\n| `invented_detail` | −0.50 |\n| `forbidden_fact` | −0.50 |\n| `low_specificity_substitution` | −0.50 |\n| `option_menu` | −0.40 |\n| `missing_number` | −0.40 |\n| `missing_entity` | −0.40 |\n| `refusal` | −0.40 |\n| `placeholder_disallowed` | −0.35 |\n| `missing_required_fact` | −0.35 |\n| `fake_casual_phrase` | −0.35 |\n| `broken_informal_grammar` | −0.35 |\n| `register_mismatch` | −0.35 |\n| `too_long` | −0.30 |\n| `placeholder_required` | −0.30 |\n| `ai_tell_phrase` | −0.25 |\n| `thanks_padding` | −0.20 |\n| `wrapper_phrase` | −0.20 |\n| `sentence_count` | −0.20 |\n| `wrong_format_markdown` | −0.20 |\n| `subject_line` | −0.20 |\n| `too_short` | −0.15 |\n| `signoff` | −0.15 |\n\n---\n\n## Metrics logged (zero-weight, diagnostics only)\n\n```\nstyle_metric            — layer1 heuristics + ridge P(human) blend\ntask_following_metric   — option_menu / wrapper / refusal / length checks\nfaithfulness_metric     — invented detail / missing facts / forbidden facts\nlength_metric           — word count within constraints\nformat_metric           — subject / signoff / markdown checks\nclarity_metric          — avg sentence length\nplaceholder_metric      — placeholder constraint compliance\nrisk_penalty_metric     — sum of all triggered penalties\noption_menu_penalty_metric\nwrapper_phrase_penalty_metric\ninvented_detail_penalty_metric\n```\n\n---\n\n## Dataset\n\nDefault task set: `mix_v2_p5050_filtered` — 972 tasks rebuilt from the saved\nModal `mix_v2` rollouts using `p50_50_no_penalty`.\n\nSupported `task_set` values:\n\n| task_set | file | rows |\n|---|---:|---:|\n| `v02_smoke` | `humanize_tasks_v02_smoke.jsonl` | 99 |\n| `v03` | `humanize_tasks_v03_filtered.jsonl` | 492 |\n| `mix_v2` | `humanize_tasks_rl_mix_v2.jsonl` | 998 |\n| `mix_v2_p5050` | `humanize_tasks_rl_mix_v2_p5050_filtered.jsonl` | 972 |\n| `mix_v2_p5050_filtered` | alias for `mix_v2_p5050` | 972 |\n\nPass `task_path=` to `load_environment()` only when testing a custom JSONL.\n\nTask shape (key fields):\n\n```json\n{\n  \"id\": \"rl_v01_000001\",\n  \"family\": \"rewrite_repair\",\n  \"domain\": \"slack\",\n  \"mode\": \"rewrite\",\n  \"instruction\": \"Clean up this Slack update. Keep it casual and under 30 words.\",\n  \"input_text\": \"Please be advised that staging has been restored...\",\n  \"constraints\": { \"max_words\": 30, \"preserve_numbers\": true, ... },\n  \"reward_profile\": \"rewrite_faithful_concise\",\n  \"required_facts\": [\"staging restored\", \"missing STRIPE_WEBHOOK_SECRET\"],\n  \"forbidden_facts\": [\"database migration\"],\n  \"split\": \"train\"\n}\n```\n\n---\n\n## Usage\n\n```bash\n# Install\nprime env install jayshah5696/humanize-rl-env\n\n# Run evaluation (3 examples, 2 rollouts each)\nprime eval run jayshah5696/humanize-rl-env \\\n  --model google/gemma-3-4b-it \\\n  --api-base-url https://openrouter.ai/api/v1 \\\n  --api-key-var OPENROUTER_API_KEY \\\n  --num-examples 3 \\\n  --rollouts-per-example 2\n\n# Python\nfrom verifiers import load_environment\nenv = load_environment(\"humanize-rl-env\")\n\n# p50 default training set\nenv = load_environment(\n    \"humanize-rl-env\",\n    split=\"train\",\n    task_set=\"mix_v2_p5050\",\n    reward_mode=\"p50_50_no_penalty\",\n)\n\n# strict eval gate\nenv = load_environment(\n    \"humanize-rl-env\",\n    split=\"validation\",\n    task_set=\"v03\",\n    reward_mode=\"strict\",\n)\n```\n\n---\n\n## Package structure\n\n```\nhumanize_rl_env/\n├── __init__.py              # load_environment(), preview_dataset_row()\n├── ridge_state.pkl          # bundled TF-IDF+Ridge scorer (1.1 MB)\n├── humanize_tasks_v02_smoke.jsonl\n├── humanize_tasks_v03_filtered.jsonl\n├── humanize_tasks_rl_mix_v2.jsonl\n├── humanize_tasks_rl_mix_v2_p5050_filtered.jsonl\n├── reward/\n│   ├── reward.py            # score_response(), RewardResult, load_ridge_scorer()\n│   ├── checks.py            # deterministic penalty checks\n│   ├── tasks.py             # RLTask model, load_tasks()\n│   ├── profiles.py          # legacy reward profiles (reference)\n│   ├── verifiers_adapter.py # Prime Verifiers async reward + metric functions\n│   ├── grpo_rewards.py      # TRL/GRPO reward functions\n│   └── env.py               # HumanizeRLEnv, prime_dataset_row()\n└── scoring/\n    ├── aggregator.py        # layer1 score_text()\n    ├── layer1.py            # 8 regex/heuristic dimensions\n    └── patterns.py          # AI-tell regex patterns\n```\n\nAll scoring logic is **self-contained** — no external `humanize-rl` package\ninstall required. The wheel includes the ridge pkl and smoke dataset.\n\n---\n\n## Versions\n\n| version | change |\n|---|---|\n| 0.3.15 | Fake-casual, low-specificity substitution, broken informal grammar, slang/register mismatch, and thanks-padding deterministic caps |\n| 0.3.13 | Ignore subject-title and discourse false entities such as Compliance Review, Firstly, Secondly, and Understanding |\n| 0.3.12 | Scaffold fact filtering plus invented-number/time, unsupported-negation, low-overlap, em-dash, inline-closing, and AI-tell surface caps |\n| 0.3.10 | Core semantic-failure cap for p50 deterministic reward |\n| 0.3.9 | Formal salutation and inline/multiline signature caps for direct-output tasks |\n| 0.3.8 | Emoji, hashtag, all-caps, and expanded fake-social/recommendation diagnostics |\n| 0.3.7 | Deterministic semantic suitability cap for uplifting romance recommendation failures |\n| 0.3.6 | Target-word length and repetition diagnostics/caps for p50 training |\n| 0.3.5 | Hosted-training example_id recovery for reshaped rollout inputs |\n| 0.3.2 | Hosted-training prompt recovery; explicit question/answer/example_id fields |\n| 0.3.1 | p50_50_no_penalty mode; default mix_v2_p5050 task set; v03/mix_v2 bundles |\n| 0.1.7 | 50/50 ridge-rubric + deterministic formula; 8 rubric dims exposed |\n| 0.1.6 | ridge scorer bundled as state dict pkl |\n| 0.1.5 | self-contained bundle, passes prime eval run end-to-end |\n| 0.1.0 | initial push |\n","encoding":"utf-8","truncated":false,"total_bytes":8920},"status":null}