{"data":{"kind":"file","path":"README.md","version_id":"vige89zkal0zds4lhwb60nwa","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":2337,"modified_at":"2026-02-05T22:39:47.593000","content_hash":"a210a6dcdf09a89109f5c65b6829acb0f0af1feac8f36d080919105800fd71de"},"entries":[],"content":"# distributed-calculation\r\n\r\n### Overview\r\n- **Environment ID**: `distributed-calculation`\r\n- **Short description**: A multi-turn environment that tests active context management by forcing agents to identify, remember, and combine three numeric values scattered through a massive deterministic noise stream to solve a simple arithmetic task.\r\n- **Tags**: reasoning, memory, active-context, long-horizon, deterministic, train, eval\r\n\r\n### Datasets\r\n- **Primary dataset(s)**: Procedurally generated document streams. Each episode generates a unique sequence of text chunks from a fixed seed and configuration.\r\n- **Source links**: N/A (generated at runtime using deterministic noise)\r\n- **Split sizes**: Infinite/Configurable (default: 100 episodes per evaluation run)\r\n\r\n### Task\r\n- **Type**: multi-turn\r\n- **Parser**: Custom JSON Parser\r\n- **Rubric overview**: Sparse Reward: +1.0 for correct final answer, -1.0 for incorrect\r\n\r\n### Quickstart\r\nRun an evaluation with default settings:\r\n\r\n```bash\r\nprime eval run distributed-calculation\r\n```\r\n\r\nConfigure model and sampling:\r\n\r\n```bash\r\nprime eval run distributed-calculation \\\r\n  -m gpt-4o-mini \\\r\n  -n 50 -r 2 -t 1024 -T 0.7 \\\r\n  -a '{\"chunk_size\": 500, \"n_chunks\": 200, \"seed\": 42}'\r\n```\r\n\r\nNotes:\r\n- Use `-a` / `--env-args` to pass environment-specific configuration as a JSON object.\r\n\r\n### Environment Arguments\r\n\r\n| Arg | Type | Default | Description |\r\n| --- | ---- | ------- | ----------- |\r\n| `chunk_size` | int | `1000` | Number of words in each processing chunk |\r\n| `n_chunks` | int | `100` | Total number of chunks in the document (context length = chunk_size × n_chunks) |\r\n| `seed` | int | `42` | Random seed for deterministic generation |\r\n| `max_episodes` | int | `100` | Maximum number of episodes to evaluate |\r\n| `needle_format` | str | `\"[VAR_{key}: {value}]\"` | Format for embedding target values (guaranteed not to appear in noise) |\r\n\r\n### Metrics\r\n\r\n| Metric | Meaning |\r\n| ------ | ------- |\r\n| `reward` | Main scalar reward: +1.0 (correct) or -1.0 (incorrect) |\r\n| `accuracy` | Exact match on the target calculation: 1 if (A+B)*C is correct, else 0 |\r\n| `episode_length` | Number of turns taken (should equal n_chunks + 1 for the final question) |\r\n| `needle_positions` | Normalized positions (0-1) where A, B, C were injected (for correlation analysis) |\r\n","encoding":"utf-8","truncated":false,"total_bytes":2337},"status":null}