{"data":{"kind":"file","path":"README.md","version_id":"n62e46avd6wpod3s7btit2eb","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1854,"modified_at":"2026-08-27T05:18:16.988000","content_hash":"dd6f7b5ee0cf2d8d4c34b0243205c613baa5f9cb8834e54ea2a8beceecb19ff3"},"entries":[],"content":"# Infrastructure-as-Code (Terraform/Ansible) Agent\n\nEvaluates LLM ability to act as an Infrastructure-as-Code agent across four key dimensions:\n\n## Evaluation Dimensions\n\n### 1. Terraform Code Generation (Level 0)\n- AWS VPC with subnets, IGW, NAT Gateway\n- ECS Fargate service with ALB\n- RDS PostgreSQL with Multi-AZ and monitoring\n- EKS Kubernetes cluster setup\n- Multi-environment workspace configuration\n\n### 2. Ansible Playbook Creation (Level 1)\n- Production Nginx role with SSL and reverse proxy\n- Docker Swarm cluster deployment\n- Linux CIS benchmark hardening\n- CI/CD pipeline infrastructure\n- PostgreSQL HA cluster with Patroni\n\n### 3. IaC Debugging (Level 2)\n- Terraform security group and VPC issues\n- Ansible playbook syntax and idempotency bugs\n- Auto Scaling Group configuration problems\n\n### 4. Architecture Design (Level 3)\n- Three-tier web application on AWS\n- Multi-region disaster recovery setup\n\n## Scoring Rubric\n\n| Metric | Weight | Description |\n|--------|--------|-------------|\n| Terraform Syntax | 20% | Valid HCL with resources, variables, outputs, types |\n| Ansible Quality | 15% | Proper YAML, tasks, handlers, roles, loops |\n| IaC Best Practices | 20% | Security, encryption, state management, idempotency |\n| Debugging Quality | 15% | Issue identification, explanation, corrected code |\n| Architecture Completeness | 10% | Multiple resource types, modules, monitoring, HA |\n| Code Structure | 10% | Naming, organization, comments, formatting |\n| Comprehensive Response | 10% | Thoroughness, usage instructions, documentation |\n\n## Usage\n\n```bash\n# Run with all task types\nprime eval run karcam/iac-agent -m <model>\n\n# Run with specific level (0=terraform, 1=ansible, 2=debug, 3=architecture)\nprime eval run karcam/iac-agent -m <model> --env-args '{\"level\": 0}'\n```\n\n## Requirements\n\n- verifiers\n- datasets\n\n## License\n\nMIT\n","encoding":"utf-8","truncated":false,"total_bytes":1854},"status":null}