{"data":{"kind":"file","path":"README.md","version_id":"oyvzd7hh19q1z2msfaqy8vh0","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2382,"modified_at":"2026-09-01T15:37:44.902000","content_hash":"f5b3f1d823fd1606a10d76dd5ce9808996d3102167ab59a102b992e40df2100c"},"entries":[],"content":"# Bayesian Inference & Probabilistic Programming\n\nA comprehensive evaluation environment for testing deep understanding of Bayesian statistics, probabilistic programming, and applied statistical modeling.\n\n## Overview\n\nThis environment evaluates an LLM's ability to reason about and implement Bayesian methods across four difficulty levels, covering theory, computation, and real-world applications.\n\n## Evaluation Dimensions\n\nEach response is scored across five weighted dimensions:\n\n| Dimension | Weight | Description |\n|-----------|--------|-------------|\n| Mathematical Rigor | 25% | Correct notation, derivations, step-by-step proofs |\n| Code Quality | 25% | Working implementations, proper imports, diagnostics |\n| Conceptual Depth | 20% | Explanations, comparisons, trade-offs, practical insights |\n| Completeness | 15% | All parts of the question addressed |\n| PP Knowledge | 15% | Framework-specific knowledge (PyMC, Pyro, Stan, etc.) |\n\n## Task Categories\n\n### Level 0: Fundamentals (3 tasks)\n- Bayes' theorem application (medical testing)\n- Beta-Binomial conjugate derivation\n- Conjugate prior family comparison\n\n### Level 1: MCMC & Computation (3 tasks)\n- Metropolis-Hastings from scratch implementation\n- HMC/NUTS theory and leapfrog integrator\n- MCMC vs Variational Inference comparison\n\n### Level 2: Probabilistic Programming (3 tasks)\n- Hierarchical model in PyMC (Eight Schools)\n- Gaussian Mixture Model in Pyro/NumPyro\n- Bayesian logistic regression in Stan\n\n### Level 3: Advanced Topics (7 tasks)\n- Bayesian Neural Networks for uncertainty quantification\n- Gaussian Processes for non-parametric regression\n- Bayesian changepoint detection\n- Model comparison (Bayes Factors, WAIC, LOO-CV)\n- Bayesian A/B testing workflow\n- Bayesian optimization\n- Bayesian nonparametrics (DP, IBP)\n\n## Key Concepts Tested\n\n- **Theory**: Bayes' theorem, conjugate priors, posterior derivation, ELBO, KL divergence\n- **MCMC**: Metropolis-Hastings, Gibbs, HMC, NUTS, convergence diagnostics\n- **Variational Inference**: Mean-field, CAVI, ADVI, ELBO optimization\n- **Frameworks**: PyMC, Pyro/NumPyro, Stan, GPyTorch\n- **Models**: Hierarchical, mixture, BNN, GP, time series, nonparametric\n- **Diagnostics**: R-hat, ESS, trace plots, divergences, PPC, WAIC, LOO\n\n## Usage\n\n```python\nimport verifiers as vf\nenv = vf.load_environment(\"bayesian-inference-probprog\")\n```\n\n## Version\n\n1.0.1\n","encoding":"utf-8","truncated":false,"total_bytes":2382},"status":null}