{"data":{"kind":"file","path":"README.md","version_id":"nqsiw23kdpsas424qeiu2xtp","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2169,"modified_at":"2026-08-12T10:49:08.221000","content_hash":"5d3ba0f2ea7a3dced7b8dfa687d1fe6569670d50641841aea1afb0c30f611574"},"entries":[],"content":"# Causal Inference & Experiment Design\n\nEvaluates LLM reasoning across core causal inference methods and experimental design principles.\n\n## Coverage\n\n- **Randomized Controlled Trials**: ATE estimation, standard errors, confidence intervals\n- **Confounding & DAGs**: backdoor criterion, d-separation, collider bias\n- **Propensity Score Matching**: ATT estimation, dimensionality reduction\n- **Instrumental Variables**: Wald estimator, LATE, relevance and exclusion\n- **Difference-in-Differences**: parallel trends, placebo tests\n- **Regression Discontinuity**: sharp/fuzzy RDD, running variables\n- **A/B Testing**: power analysis, sample size, sequential testing\n- **Simpson's Paradox**: stratification vs marginalization\n- **Mediation Analysis**: direct/indirect effects, proportion mediated\n- **SUTVA Violations**: spillover, general equilibrium, treatment versions\n- **Selection Bias**: collider bias (Berkson's paradox)\n- **Bayesian Causal Inference**: priors, posteriors, skeptical priors\n- **Multiple Testing**: Bonferroni, Benjamini-Hochberg, FWER vs FDR\n- **Causal Discovery**: Markov equivalence, functional form methods\n- **Sensitivity Analysis**: Rosenbaum framework for unmeasured confounding\n- **Heterogeneous Treatment Effects**: Causal Forests, CATE estimation\n- **Natural Experiments**: compulsory schooling laws, IV-RDD hybrids\n- **Transportability**: cross-population causal generalization\n- **Ethics in Experimentation**: utilitarian vs deontological frameworks\n\n## Scoring\n\nThree rubric functions (weighted):\n\n| Metric | Weight | Description |\n|--------|--------|-------------|\n| Numerical Accuracy | 0.35 | Correctness of calculations and numerical results |\n| Conceptual Reasoning | 0.40 | Use of causal concepts, assumptions, structured explanation |\n| Answer Completeness | 0.25 | Addresses all sub-questions with sufficient detail |\n\n## Dataset\n\n20 evaluation tasks covering all topics above. Each task includes a system prompt setting expert context and a multi-part question requiring both calculation and conceptual explanation.\n\n## Usage\n\n```python\nimport verifiers as vf\nenv = vf.load_environment(\"causal-inference-experiment-design\")\n```\n","encoding":"utf-8","truncated":false,"total_bytes":2169},"status":null}