{"data":{"kind":"file","path":"README.md","version_id":"pi5kf2vxk9glcoyxd25m4st0","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5582,"modified_at":"2025-11-14T04:38:22.161000","content_hash":"6a7a33523836700b60eb33434e96e535944b4709c0d425b7e97695b830e2af88"},"entries":[],"content":"# Pacman Environment for Verifiers\n\nA multimodal multi-turn Pacman game environment for evaluating LLM agents using the verifiers framework.\n\n## Overview\n\nThis environment allows Language Models to play Pacman by:\n1. Viewing the game state (as **images**, **ASCII text**, or **2D arrays**)\n2. Analyzing game information (score, positions, etc.)\n3. Outputting actions (0-3) in XML format\n4. Receiving visual or textual feedback\n\n## Features\n\n- **Three Render Modes**: Choose between IMAGE (visual), ASCII (text), or ARRAY (2D array) rendering\n- **Multimodal**: LLM receives both images and text (IMAGE mode)\n- **Text-Only Compatible**: Pure text representation for text-based LLMs (ASCII/ARRAY modes)\n- **Structured Data**: 2D array representation for programmatic game state analysis (ARRAY mode)\n- **Multi-turn**: Up to 200 decision turns per game\n- **Action Repeat**: Each LLM action repeats for game frames (configurable)\n- **Rich Rewards**: Multiple reward metrics (win rate, score, survival, efficiency)\n- **Headless Compatible**: ASCII and ARRAY modes work without display/GPU\n\n## Project Structure\n\n```\npacman/\n├── __init__.py          # Package exports (load_environment, PacmanVerifiersEnv)\n├── config.py            # Configuration constants (render mode, rewards, actions)\n├── game.py              # Core Pacman game engine (540 lines)\n├── llm_game.py          # LLM integration wrapper (screenshot capture, state export)\n└── pacman.py            # Verifiers environment integration\n\nplay_game.py             # Manual game player for testing (IMAGE or ASCII mode)\npyproject.toml           # Project metadata, dependencies, and build config\nRENDER_MODES_GUIDE.md    # Detailed documentation on IMAGE vs ASCII modes\n```\n\n### Key Files\n\n- **`game.py`**: Complete Pacman game implementation with pygame rendering and ASCII output\n- **`pacman.py`**: Verifiers environment adapter that handles LLM interactions, rewards, and multi-turn gameplay\n- **`llm_game.py`**: Utility wrapper for capturing screenshots/ASCII and managing game state for custom LLM integrations\n- **`config.py`**: All configurable parameters (render mode, action constants, reward values)\n- **`play_game.py`**: Standalone script to play Pacman manually (useful for testing and demos)\n\n## Installation\n\n```bash\npip install pacman\n```\n\n## Usage\n\n### Basic Usage with IMAGE Mode (default)\n\n```python\nimport verifiers as vf\n\n# Load the environment with visual screenshots\nfrom pacman import load_environment\nenv = load_environment(\n    num_examples=10, \n    max_turns=200, \n    action_repeat=3,\n    render_mode=\"image\"  # Visual mode\n)\n\n# Run evaluation with a vision-capable model\nresults = await vf.evaluate(\n    env=env,\n    model=\"gpt-4o\",\n    num_examples=10\n)\n```\n\n### Usage with ASCII Mode (text-based, default)\n\n```python\nimport verifiers as vf\n\n# Load the environment with ASCII text rendering\nfrom pacman import load_environment\nenv = load_environment(\n    num_examples=10, \n    max_turns=200, \n    action_repeat=3,\n    render_mode=\"ascii\"  # Text mode (default)\n)\n\n# Run evaluation with any text-based model\nresults = await vf.evaluate(\n    env=env,\n    model=\"gpt-4o\",  # Works with any model\n    num_examples=10\n)\n```\n\n## Action Space\n\n- `0`: Up\n- `1`: Down\n- `2`: Left\n- `3`: Right\n\n## Render Modes\n\nThis environment supports three render modes that can be switched using the `render_mode` parameter:\n\n### IMAGE Mode\n- **Format**: PNG screenshots (base64-encoded for LLM)\n- **Use Case**: Visual analysis, vision-capable models (GPT-4V, Claude 3, Gemini Pro Vision)\n- **Requirements**: Display/GPU (pygame)\n- **File Size**: ~50-100 KB per frame\n- **Token Cost**: Higher (vision tokens)\n\n### ASCII Mode (default)\n- **Format**: Text-based ASCII art representation\n- **Use Case**: Text-only models, headless servers, cost optimization\n- **Requirements**: None (headless compatible)\n- **File Size**: ~2-3 KB per frame\n- **Token Cost**: Lower (text tokens only)\n\n#### ASCII Legend\n```\n█  Wall\n·  Small dot (10 points)\n●  Power pellet (50 points)\nC  Pacman (you)\nϾ  Pacman powered up\nM  Ghost (dangerous)\nW  Ghost scared (edible)\n```\n\n### ARRAY Mode\n- **Format**: 2D array (16x16) with numeric values\n- **Use Case**: Structured data analysis, programmatic reasoning, grid-based LLMs\n- **Requirements**: None (headless compatible)\n- **File Size**: ~2-4 KB per frame\n- **Token Cost**: Lower (text tokens only)\n\n#### Array Legend\n```\n0 = Empty space\n1 = Wall\n2 = Small dot (10 points)\n3 = Power pellet (50 points)\n4 = Pacman (you)\n5 = Ghost (dangerous)\n```\n\n### Switching Modes\n\n```python\n# In config.py (global setting)\nRENDER_MODE = \"ascii\"  # or \"image\" or \"array\"\n\n# Or per-instance\nenv = load_environment(render_mode=\"ascii\")  # or \"image\" or \"array\"\n```\n\n**📖 For detailed documentation, see [RENDER_MODES_GUIDE.md](RENDER_MODES_GUIDE.md)**\n\n## Configuration\n\nConfigure the environment in your `pyproject.toml`:\n\n```toml\n[tool.verifiers.eval]\nnum_examples = 10\nmax_turns = 200\naction_repeat = 3\n```\n\n## Reward Metrics\n\n- **Win Reward** (weight: 1.0): Binary reward for completing the game\n- **Score Reward** (weight: 0.3): Normalized game score\n- **Survival Reward** (weight: 0.1): Reward for lasting longer\n- **Efficiency Reward** (weight: 0.2): Reward for winning quickly\n- **Total Reward** (weight: 0.5): Normalized cumulative game reward\n- **Format Reward** (weight: 0.1): Correct XML action format\n\n## Development\n\n```bash\n# Install locally for development\npip install -e .\n\n# Run tests\npytest\n\n# Test evaluation locally\nvf-eval pacman\n```\n\n## License\n\nMIT\n\n","encoding":"utf-8","truncated":false,"total_bytes":5582},"status":null}