{"data":{"kind":"file","path":"README.md","version_id":"jgcxsz2swn3uwr3zrn5dopha","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3069,"modified_at":"2026-09-03T14:06:21.688000","content_hash":"6c832eb526aa57ea55e90381d7d865a8ae21a7ca66c7eb62e8976313591c5a6b"},"entries":[],"content":"# Geospatial Path Planning (GIS-based) Environment\n\nA comprehensive evaluation environment for testing LLM reasoning capabilities on GIS-based geospatial path planning problems.\n\n## Overview\n\nThis environment evaluates an LLM's ability to solve path planning problems that require understanding of:\n\n- **Grid-based pathfinding** with obstacles and weighted terrain\n- **Multi-directional movement** (4-directional and 8-directional)\n- **Terrain-aware routing** with elevation, risk, and cost models\n- **Multi-objective optimization** balancing distance, safety, time, and cost\n- **Algorithm selection** and complexity analysis (Dijkstra, A*, PRM, etc.)\n- **Geodesic calculations** using Haversine formula and coordinate systems\n- **Real-world scenarios** including logistics, search-and-rescue, and autonomous navigation\n\n## Task Categories\n\n| Tier | Category | Problems | Description |\n|------|----------|----------|-------------|\n| 1 | Basic Grid Pathfinding | 3 | Simple shortest path on unweighted grids |\n| 2 | Weighted Terrain | 3 | Variable-cost cells, terrain penalties |\n| 3 | Diagonal Movement | 3 | 8-directional movement with sqrt(2) costs |\n| 4 | Elevation-Aware Routing | 3 | Height-based cost models, drone altitude |\n| 5 | Multi-Objective | 3 | Weighted scoring, Pareto-optimal solutions |\n| 6 | Coordinate Systems | 3 | Haversine, UTM, GPS waypoints |\n| 7 | Algorithm Selection | 3 | Algorithm choice, complexity, heuristics |\n| 8 | Complex Scenarios | 19 | Real-world problems combining multiple concepts |\n\n## Scoring Rubric\n\nThe environment uses 7 weighted scoring functions:\n\n| Function | Weight | Description |\n|----------|--------|-------------|\n| `path_cost_accuracy` | 0.25 | Numeric cost/distance accuracy with fuzzy matching |\n| `path_sequence_accuracy` | 0.20 | Coordinate sequence correctness (Jaccard similarity) |\n| `algorithm_reasoning` | 0.15 | Algorithm selection and complexity analysis |\n| `geospatial_concepts` | 0.12 | GIS domain knowledge and terminology |\n| `answer_format_compliance` | 0.10 | Adherence to requested output format |\n| `multi_objective_reasoning` | 0.10 | Multi-criteria optimization reasoning |\n| `domain_depth` | 0.08 | Domain-specific keyword depth (anti-gaming) |\n\n## Anti-Gaming Design\n\n- **Fuzzy numeric matching**: Tolerates rounding differences (1%, 5%, 10%, 20% thresholds)\n- **Set-based path comparison**: Jaccard similarity instead of exact string match\n- **Context-aware domain scoring**: Terms must appear in explanatory context, not just listed\n- **Rotating keyword pool**: 25 domain terms prevent memorization\n- **Sigmoid-scaled rewards**: Smooth scoring curves instead of step functions\n\n## Usage\n\n```python\nimport verifiers as vf\n\nenv = vf.load_environment(\"geospatial-path-planning\")\n# or\nfrom geospatial_path_planning import load_environment\nenv = load_environment()\n```\n\n## Evaluation\n\n```bash\nprime eval run OWNER/geospatial-path-planning -m meta-llama/Llama-3.2-1B-Instruct\n```\n\n## Version History\n\n- **v1.0.0** — Initial release with 40 problems across 8 difficulty tiers\n\n## License\n\nMIT\n","encoding":"utf-8","truncated":false,"total_bytes":3069},"status":null}