{"data":{"kind":"file","path":"README.md","version_id":"yy5ddxzesm3v0txwavq9kq1g","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5397,"modified_at":"2026-07-22T01:09:19.373000","content_hash":"5ea84be35269e63ce6142f56e63b266e70466228665f6770b5ec07a52a4b51d1"},"entries":[],"content":"# Paragraph Author Rewrite\n\nEnvironment ID: `paragraph-author-rewrite`\n\nA single-turn Verifiers environment for training a language model to rewrite complete prose paragraphs in a direct, idea-driven, Paul Graham-inspired style while preserving the paragraph's reader-level semantic impact.\n\n## Objective\n\nThe policy receives an essay title, the paragraph to rewrite, and—when available—the preceding source paragraph as context. It returns only the rewritten target paragraph. The environment is intended to improve style without rewarding deletion, copying, or semantic drift.\n\nBy default, the scalar reward is:\n\n```text\nstyle_reward\n  * semantic_probability\n  * output_validity\n  * edit_requirement_multiplier\n```\n\n- **Style reward:** probability from a frozen paragraph-level n-gram classifier trained to distinguish Paul Graham paragraphs from paired LLM rewrites.\n- **Semantic probability:** a dense, label-order-averaged judgment of whether the rewrite preserves the original's central understanding, explanatory force, scope, causal structure, and confidence.\n- **Output validity:** zero for empty output or leaked prompt instructions.\n- **Edit requirement:** zero for copies and punctuation-only changes, increasing smoothly from 1% source-word change to full credit at 10%.\n\nAn optional embedding-direction reward is also included. It compares the rewrite-minus-source embedding change with a frozen centroid constructed from Paul Graham paragraph / LLM rewrite pairs. Set `author_reward_weight` and `ngram_reward_weight` to choose either style signal or a mixture.\n\n## Data construction\n\nThe default training source corpus is `CK0607/qwen3.5-9b-blogprovider-traces`. The environment:\n\n1. removes exact duplicate source essays;\n2. creates a seeded 90/10 split at the essay level;\n3. extracts prose paragraphs containing at least four sentences;\n4. retains every eligible paragraph rather than capping the pool; and\n5. keeps the preceding paragraph as context without asking the model to rewrite it.\n\nThe essay-level split prevents paragraphs from one essay appearing in both training and validation.\n\n## Install\n\n```bash\nprime env install matthewagi/paragraph-author-rewrite\n```\n\nOr with `uv`:\n\n```bash\nuv add paragraph_author_rewrite \\\n  --index https://hub.primeintellect.ai/matthewagi/simple/\n```\n\n## Load\n\n```python\nimport verifiers as vf\n\nenv = vf.load_environment(\"paragraph-author-rewrite\")\n```\n\nThe zero-argument loader uses `dry_run=True`: it loads a bundled three-essay fixture, uses an offline approximate tokenizer, and gives the semantic factor full credit. This makes installation and interface testing independent of external model and dataset services. It is not the training configuration.\n\nFor training, set `dry_run=False`. The default semantic judge then expects a compatible Gemma inference endpoint at `http://127.0.0.1:8000`. For distributed training, `reward_service_url` can point to a persistent scoring service that returns the semantic and embedding metrics. The policy tokenizer and semantic judge default to `google/gemma-4-12B-it`.\n\n## Important arguments\n\n- `dataset_name`, `dataset_split`, `text_field`\n- `dry_run` (leave enabled only for installation and smoke testing)\n- `validation_fraction`, `dataset_partition_seed`\n- `paragraph_min_words`, `paragraph_min_sentences`, `paragraph_max_words`\n- `include_previous_context`, `previous_context_max_tokens`\n- `policy_tokenizer_name`\n- `enable_ngram_reward`, `ngram_reward_weight`\n- `author_reward_weight`, `author_centroid_path`, `embedding_model_name`\n- `semantic_base_url`, `semantic_model_name`, `semantic_threshold`\n- `reward_service_url`\n- `edit_change_floor`, `edit_change_full_credit`\n\nThe default style configuration is n-gram-only:\n\n```python\nenv = vf.load_environment(\n    \"paragraph-author-rewrite\",\n    dry_run=False,\n    enable_ngram_reward=True,\n    ngram_reward_weight=1.0,\n    author_reward_weight=0.0,\n)\n```\n\nIn n-gram-only mode, the environment does not instantiate the Jasper embedding\nmodel and does not call the combined embedding reward-service proxy. Semantic\ngrading uses `semantic_base_url` directly when `dry_run=False`; the zero-argument\nsmoke-test path remains entirely offline. The embedding model or combined proxy\nis activated only when `author_reward_weight` is greater than zero.\n\nTo use the frozen embedding-direction centroid instead:\n\n```python\nenv = vf.load_environment(\n    \"paragraph-author-rewrite\",\n    dry_run=False,\n    enable_ngram_reward=False,\n    ngram_reward_weight=0.0,\n    author_reward_weight=1.0,\n)\n```\n\n## Output metrics\n\nThe rubric exposes the scalar `rewrite_reward` plus component metrics for semantic probability and margin, n-gram probability, embedding-centroid cosine, rewrite-source embedding distance, source-change rate, length ratio, prompt leakage, and output validity. These components are diagnostic only; `rewrite_reward` is the sole optimized reward.\n\n## Limitations\n\n- The semantic judge requires a colocated compatible inference endpoint or reward service.\n- The style probes encode a narrow corpus-derived approximation of style and can be exploited; inspect saved rollouts, not only aggregate reward.\n- A high reward does not guarantee factual correctness. Rewrites should be reviewed against their source paragraphs.\n- Paul Graham-inspired means a learned stylistic direction, not authorship by or endorsement from Paul Graham.\n","encoding":"utf-8","truncated":false,"total_bytes":5397},"status":null}