{"data":{"kind":"file","path":"README.md","version_id":"a36j3w60jz367z829hwhsd46","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":3098,"modified_at":"2026-08-26T13:42:36.588000","content_hash":"1d28fc58fa9131eaaeb5314badc3cb84d6540df82ad16d31743781ae4874fdd0"},"entries":[],"content":"# GraphQL Schema Designer & Query Optimizer\n\nEvaluates LLM ability to design production-grade GraphQL schemas, optimize queries for performance, and implement advanced API patterns.\n\n## What It Tests\n\nThis environment covers the full spectrum of GraphQL engineering:\n\n### Schema Design (4 tasks)\n- **Blog Platform**: Relay-style pagination, nested types, enums, input types\n- **E-Commerce**: Product variants, hierarchical categories, cart-to-checkout flow\n- **Multi-Tenant SaaS**: Custom fields, workflows, audit logs, bulk mutations\n- **Schema Migration**: Deprecation strategy, backward-compatible evolution\n\n### Query Optimization (4 tasks)\n- **N+1 Detection**: Identify and fix N+1 problems with DataLoader\n- **Query Complexity**: Calculate cost, implement depth limiting, rate limiting\n- **Resolver Optimization**: SQL injection fixes, query batching, caching\n- **Monitoring**: Field-level tracing, Prometheus metrics, query allowlisting\n\n### Resolver Patterns (2 tasks)\n- **Subscriptions**: Real-time collaborative editing with PubSub\n- **Apollo Federation**: Multi-service architecture with entity resolution\n\n### Security (1 task)\n- **Authorization**: RBAC, field-level auth, row-level security, introspection control\n\n## Scoring Rubric (7 metrics)\n\n| Metric | Weight | What It Measures |\n|--------|--------|-----------------|\n| Required Elements | 20% | Key GraphQL concepts present (types, directives, patterns) |\n| Task-Specific | 25% | Structural requirements met per task category |\n| Structural Quality | 15% | Code blocks, explanations, proper formatting |\n| GraphQL Syntax | 15% | Valid SDL with balanced braces, proper declarations |\n| Code Quality | 10% | Async patterns, error handling, documentation |\n| Depth & Breadth | 10% | Response detail, multiple sections, concrete examples |\n| Excluded Elements | 5% | No anti-patterns (REST endpoints, etc.) |\n\n## Difficulty Levels\n\n- **Level 1**: Basic schema design with standard types\n- **Level 2**: Complex schemas + N+1 detection\n- **Level 3**: Subscriptions, monitoring, query complexity analysis\n- **Level 4**: Federation, authorization, schema migration\n- **Level 5**: Multi-tenant SaaS with custom fields and workflows\n\n## Usage\n\n```python\nimport verifiers as vf\n\n# Default (all tasks, single-turn)\nenv = vf.load_environment(\"roger/graphql-schema-optimizer\")\n\n# Filter by difficulty\nenv = vf.load_environment(\"roger/graphql-schema-optimizer\", difficulty=3)\n\n# Filter by category\nenv = vf.load_environment(\"roger/graphql-schema-optimizer\", category=\"query_optimization\")\n\n# Multi-turn mode (adds follow-up questions)\nenv = vf.load_environment(\"roger/graphql-schema-optimizer\", max_turns=2)\n```\n\n## Anti-Gaming Design\n\n- **Multi-signal scoring**: 7 independent metrics prevent single-trick gaming\n- **Structural checks**: Task-specific patterns require genuine understanding\n- **Syntax validation**: Actual GraphQL SDL parsing, not just keyword matching\n- **Code quality gates**: Async patterns, error handling, not just presence of code\n- **Excluded element penalties**: Anti-patterns actively reduce score\n\n## Version\n\n0.1.0\n","encoding":"utf-8","truncated":false,"total_bytes":3098},"status":null}