{"data":{"kind":"file","path":"README.md","version_id":"c9198xspvdl3yxcsvaptgmvs","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3478,"modified_at":"2026-07-28T20:06:21.615000","content_hash":"e8862156e514ecf577124cc4b21d6eac4c9aee9eb3d28016255b10b2a4dc5566"},"entries":[],"content":"# trace-forge-xss\n\n**An execution-verified XSS exploitation environment for training and evaluating LLMs.**\n\nEvery task is a small, self-contained vulnerable web app with exactly one cross-site-scripting\n(XSS) flaw. The model reads the source and proposes a payload. The reward is **whether that\npayload actually executes JavaScript** when delivered to the app in a real headless browser.\n\nThe reward is ground truth by construction:\n\n- **not** string matching against an answer key,\n- **not** an LLM grading another LLM,\n- a real browser either fires the exploit, or it doesn't.\n\nThat makes this a true *verifiable-rewards* environment (RLVR) in a domain where verifiable data\nis scarce — offensive security — rather than one more benchmark scored by proxy.\n\n## What's inside\n\n15 tasks spanning the modern XSS surface, so a model has to reason about *technique*, not\nmemorize one payload:\n\n- **Injection contexts** — HTML comment, `<textarea>` (RCDATA), `<style>`, `<title>`, unquoted\n  attribute, single-quoted attribute\n- **DOM sinks** — `innerHTML`, `document.write`, `insertAdjacentHTML`, `outerHTML`, `DOMParser`\n  + node adoption, `<iframe srcdoc>`\n- **Code / URL sinks** — the `Function` constructor, JS template-literal `eval`, `javascript:`\n  URLs\n- **A filter bypass** — defeating a naive, case-sensitive handler filter\n\nEach app delivers input through a real channel (URL query string or fragment); the grader uses\nthat same channel, so a \"solve\" is a genuine end-to-end exploit.\n\n## How it works\n\n```\nmodel reads the app source ─▶ proposes a payload (PAYLOAD: ...)\n                                        │\n                     grader serves the app over a local HTTP origin,\n                     delivers the payload through its real input channel,\n                     and watches for a JavaScript dialog / beacon\n                                        │\n                     fired ▶ reward 1.0        did not fire ▶ reward 0.0\n```\n\n## Usage\n\n```python\nimport verifiers as vf\n\nenv = vf.load_environment(\"trace-forge-xss\")\n# env.dataset  -> the tasks (prompt + metadata)\n# env.rubric   -> single reward: xss_fired_reward (1.0 fired / 0.0 not)\n```\n\nReward contract: the model should end its answer with a line `PAYLOAD: <injection string>`.\nThe grader extracts that line, runs it, and scores execution.\n\n## Requirements\n\n- Python ≥ 3.10, `verifiers`, `datasets`, `playwright`\n- A Chromium runtime: `playwright install chromium`\n\n## Verify locally\n\n```bash\npip install -e .\nplaywright install chromium\npython test_environment.py     # confirms each lab fires on a real exploit and stays safe on inert input\n```\n\nVerified against `verifiers 0.2.1`: `load_environment()` builds a 15-task `SingleTurnEnv`,\nand the reward returns `1.0` on a payload that executes and `0.0` otherwise (inert input or no\npayload). The grader is tested standalone at 15/15.\n\n## Notes\n\n- This is a curated public subset of a larger private corpus of verified offensive-reasoning\n  tasks. The apps here are original and were published nowhere before this environment, so no\n  model has memorized them.\n- Scope is deliberately narrow (reflected/DOM XSS via query and fragment) to keep the reward\n  bulletproof. Harder channels (stored, `postMessage`, blind/out-of-band) exist but are not\n  part of this open set.\n\n## License\n\nApache-2.0. The vulnerable apps are intentionally insecure teaching targets — run them only in\na sandbox, never expose them on a network.\n","encoding":"utf-8","truncated":false,"total_bytes":3478},"status":null}