{"data":{"kind":"file","path":"README.md","version_id":"uzqdtqx1ych1uuf9e7f1k6b4","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1051,"modified_at":"2026-08-05T13:36:47.421000","content_hash":"645d554eca3fc4c8f3bad10f7864010448ca8748a50d0ebb738555c15a763f07"},"entries":[],"content":"# Regex Craft\n\nEvaluate an LLM's ability to work with regular expressions across four task types:\n\n1. **Generate** — Write a regex that matches a described pattern\n2. **Explain** — Describe what a given regex does in plain language\n3. **Debug** — Find and fix errors in a broken regex\n4. **Match** — Determine whether specific strings match a regex\n\n## Difficulty Levels\n\n- **Level 0**: Basic literals, character classes, quantifiers (`[a-z]+`, `\\d{3}`)\n- **Level 1**: Groups, alternation, anchors, lookaheads (`^(?=.*\\d)[a-z]+$`)\n- **Level 2**: Backreferences, lookahead/lookbehind, non-greedy, nested groups\n- **Level 3**: Complex patterns — email validation, URL parsing, recursive patterns\n\n## Rubric\n\n- **regex_correctness** (weight 1.0): Does the generated/debugged regex actually work?\n- **explanation_quality** (weight 0.8): Is the explanation accurate and complete?\n- **match_accuracy** (weight 0.9): Are match/no-match judgments correct?\n\n## Usage\n\n```bash\nprime eval run liuliu/regex-craft -m meta-llama/Llama-3.2-1B-Instruct\n```\n","encoding":"utf-8","truncated":false,"total_bytes":1051},"status":null}