{"data":{"kind":"file","path":"README.md","version_id":"t5m81bhh0om23vcar0r9seh0","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1316,"modified_at":"2026-08-11T07:04:08.485000","content_hash":"5632bc9f88919139072f1e96b3da309b60d01db7c052edee761e9ad8c288aa77"},"entries":[],"content":"# News Aggregator & Fact-Checker\n\nA verifiers environment that evaluates LLM capabilities in news aggregation and fact-checking tasks.\n\n## Overview\n\nThis environment tests the model's ability to:\n\n1. **Aggregate News** — Synthesize information from multiple source excerpts into coherent summaries\n2. **Fact-Check Claims** — Evaluate factual claims against provided evidence and rate their veracity\n3. **Detect Misinformation** — Identify logical fallacies, misleading framing, and fabricated details\n4. **Cross-Reference Sources** — Compare accounts across sources and identify inconsistencies\n\n## Tasks\n\nEach task presents the model with:\n- A claim or news scenario to evaluate\n- Multiple source excerpts (simulated news articles, reports, statements)\n- A required structured output format (verdict, confidence, explanation)\n\n## Scoring\n\n- **Accuracy** — Correctness of fact-check verdicts against ground truth\n- **Evidence Quality** — How well the model cites and uses provided sources\n- **Reasoning** — Logical coherence of the explanation chain\n- **Completeness** — Coverage of key points in aggregation tasks\n- **Calibration** — Confidence scores aligned with actual correctness\n\n## Usage\n\n```bash\nprime eval run hdbank/news-factcheck -m meta-llama/Llama-3.2-1B-Instruct\n```\n\n## License\n\nMIT\n","encoding":"utf-8","truncated":false,"total_bytes":1316},"status":null}