{"data":{"kind":"file","path":"README.md","version_id":"p3uatdvax6vou0ry0kkhrbf1","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3172,"modified_at":"2026-08-09T04:01:21.901000","content_hash":"e9c152c9e9ffd67295206b38a3d682fcfacca62bab89440176b647654fa9c9d5"},"entries":[],"content":"# Nonogram / Picross Solver\n\nA verifiers environment that evaluates an LLM's ability to solve **Nonogram (Picross)** puzzles — classic Japanese logic puzzles where the agent must reconstruct a hidden binary grid from row and column clue sequences.\n\n## What is a Nonogram?\n\nA Nonogram is a grid-based logic puzzle. Each row and column has a sequence of numbers indicating consecutive runs of filled cells. The solver must deduce which cells are filled (1) and which are empty (0) to reveal a hidden pattern.\n\n**Example (5x5):**\n- Row clues: `[2,1]`, `[1,3]`, `[5]`, `[1,1]`, `[2]`\n- Column clues: `[1,2]`, `[3]`, `[1,3]`, `[3]`, `[1,2]`\n\nThe solver must reason step-by-step about overlaps, forced placements, and eliminations.\n\n## Environment Design\n\n### Difficulty Levels\n\n| Level | Grid Size | Fill Ratio | Description |\n|-------|-----------|------------|-------------|\n| 0     | 5×5       | 45%        | Simple patterns, easy deduction |\n| 1     | 8×8       | 42%        | Moderate complexity |\n| 2     | 10×10     | 40%        | Complex clue interactions |\n| 3     | 12×12     | 38%        | Challenging patterns |\n| 4     | 15×15     | 35%        | Dense, requires deep reasoning |\n\n### Scoring (Rubric)\n\n| Metric              | Weight | Description |\n|---------------------|--------|-------------|\n| Grid Accuracy       | 0.45   | Exact cell-by-cell match with ground truth |\n| Clue Satisfaction   | 0.30   | Whether the output grid satisfies all row/column clues |\n| Format Validity     | 0.10   | Whether output is a valid nested list of 0s and 1s |\n| Reasoning Quality   | 0.15   | Quality of step-by-step deduction process |\n\n### Task Format\n\nEach problem presents:\n1. Row clues (list of consecutive filled-cell counts per row)\n2. Column clues (list of consecutive filled-cell counts per column)\n3. Expected output: a nested list of 0s and 1s\n\nThe model must reason about:\n- **Forced placements**: cells that must be filled based on clue overlap\n- **Elimination**: cells that must be empty\n- **Cross-referencing**: using row clues to constrain column placements and vice versa\n\n## Usage\n\n```bash\n# Quick eval with a small model\nprime eval run sachi/nonogram-solver -m meta-llama/Llama-3.2-1B-Instruct\n\n# With custom difficulty\nprime eval run sachi/nonogram-solver -m Qwen/Qwen3-1.7B --env-args '{\"max_level\": 1, \"num_problems\": 10}'\n```\n\n## Configuration\n\n| Parameter      | Default | Description |\n|----------------|---------|-------------|\n| `max_level`    | 2       | Maximum difficulty level (0-4) |\n| `num_problems` | 20      | Total number of puzzles to generate |\n| `seed`         | 42      | Random seed for reproducibility |\n\n## Why Nonograms?\n\nNonograms are an excellent benchmark for evaluating LLM reasoning because:\n\n1. **Pure logic**: No world knowledge required — only deduction\n2. **Verifiable**: Solutions are objectively correct or incorrect\n3. **Scalable difficulty**: Grid size and clue density scale naturally\n4. **Multi-step reasoning**: Requires chains of logical inferences\n5. **Constraint satisfaction**: Tests ability to handle interacting constraints\n6. **Structured output**: Requires precise grid formatting\n\n## License\n\nMIT\n","encoding":"utf-8","truncated":false,"total_bytes":3172},"status":null}