{"data":{"kind":"file","path":"README.md","version_id":"nwmvn2y6f8ixe16uuch69sa7","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":6604,"modified_at":"2026-07-10T05:10:47.039000","content_hash":"8d25626e86b7f6f80655f4706e607378099593eb37eeed37a5f23f7ea28259f7"},"entries":[],"content":"# Gate Runner\n\nGate Runner is a single-turn strategy-design environment whose grader penalizes\nthe usual backtest search game. A model receives a point-in-time market brief\nand must return one strict JSON strategy config. The environment then evaluates\nthat config on hidden sequential windows with trading costs, deflates Sharpe by\nthe number of sampled configs in the rollout group, and applies a CSCV/PBO\npenalty.\n\n## Environment contract\n\n- **Environment ID:** `gate-runner`\n- **Task:** single-turn JSON generation\n- **Default panel:** deterministic synthetic 22-asset daily panel, used so the\n  package is self-contained and redistributable\n- **Public real-data panel:** opt-in `ecb_fx` profile with 29 ECB daily euro\n  reference-rate series, 2009-2024\n- **Point-in-time rule:** prompt features use data strictly before the episode\n  cutoff; all grading windows begin at or after the cutoff\n- **Execution:** close-to-close signals, equal-weight long-only positions,\n  at most five concurrent positions\n- **Costs:** 10 bps per traded side plus a per-symbol spread proxy\n- **Walk-forward horizon:** eight sequential 42-session windows by default\n\nMalformed JSON, markdown-wrapped JSON, unknown keys, wrong primitive names, and\nout-of-range parameters receive exactly zero reward.\n\n## Reward\n\nFor every schema-valid config:\n\n```text\nreward = 0.70*tanh(DSR) - 0.20*PBO - 0.05*complexity + 0.10*validity\n```\n\n- **DSR** is the Deflated Sharpe Ratio probability. Its expected-max-Sharpe\n  benchmark uses every sampled rollout in the same episode group as a trial,\n  including invalid attempts, and the valid trials' Sharpe dispersion. A null\n  standard error is used when fewer than two valid Sharpe estimates exist.\n- **PBO** uses combinatorially symmetric cross-validation across the sequential\n  windows and the valid configs in that episode group. It is `0` when fewer\n  than two valid configs exist because cross-strategy selection risk is then\n  undefined.\n- **complexity** is the active numeric parameter count normalized by eight.\n- **passed** is a headline metric, not a separate reward: `DSR > 0.90` and\n  `PBO < 0.25`.\n\nThe rubric also logs `validity`, `raw_sharpe`, `dsr`, `pbo`, `complexity`,\n`parameter_count`, `trial_count`, `passed`, and `turnover`.\n\nVerifiers' generic `pass@k` display thresholds the continuous shaped reward.\nBecause Gate Runner's binary gate is a conjunction of DSR and PBO conditions,\nno single shaped-reward threshold is equivalent. Use the logged `passed` metric\nas the evaluation headline, not generic `pass@k`.\n\n## Quickstart\n\nInstall and evaluate the public Hub environment:\n\n```bash\nprime env install br-322/gate-runner --plain\nprime eval run br-322/gate-runner\n```\n\nRun the bundled real-data profile with enough held-out rows for the 200-example\nbaseline:\n\n```bash\nprime eval run br-322/gate-runner \\\n  -a '{\"dataset\":\"ecb_fx\",\"eval_examples\":200}' \\\n  -n 20 -r 3\n```\n\nKeep the 20-example run as a preflight. Once its reward distribution and samples\nlook healthy, raise `-n` to `200` for the baseline report.\n\nFor development from a Prime Lab workspace containing this source:\n\n```bash\nprime env install gate-runner --plain\nprime eval run gate-runner\n```\n\nThe environment defaults in `pyproject.toml` run five examples with three\nrollouts each. Group scoring is required for trial-count deflation and PBO, so\ndo not use independent scoring for representative results.\n\nRun the deterministic test suite with:\n\n```bash\nuv run --project environments/gate_runner --group dev pytest -q\n```\n\n## Environment arguments\n\n| Argument | Default | Meaning |\n| --- | ---: | --- |\n| `seed` | `17` | Task sampling and synthetic panel seed |\n| `train_examples` | `64` | Number of training cutoffs |\n| `eval_examples` | `24` | Number of held-out eval cutoffs |\n| `windows` | `8` | Even number of sequential CSCV windows, at least four |\n| `window_days` | `42` | Sessions per hidden window, at least 20 |\n| `dataset` | `synthetic` | Built-in panel: `synthetic` or `ecb_fx` |\n| `data_path` | `null` | Optional caller-owned long-form market CSV |\n\nExample with local data:\n\n```bash\nprime eval run gate-runner \\\n  -a '{\"data_path\":\"/absolute/path/market.csv\"}'\n```\n\n`data_path` is an alternative to the built-in profiles and cannot be combined\nwith `dataset=\"ecb_fx\"`.\n\nThe CSV must be a complete rectangular panel with at least 1,600 dates and these\ncolumns:\n\n```text\ndate,symbol,close\n```\n\nOptional `high`, `low`, and `volume` columns improve the spread proxy. Missing\nvalues and duplicate date/symbol rows fail early.\n\nTraining and evaluation cutoffs are sampled from chronological regions separated\nby a full grading-horizon embargo, so no training episode's hidden return window\noverlaps an evaluation episode's hidden return window.\n\n## Signature test\n\n`tests/test_gate_runner.py` fixes the benchmark seed and cutoff, then compares a\nparsimonious 120-day momentum config with a noisy 10-day single-name breakout.\nThe honest config must have both higher out-of-sample Sharpe and a shaped reward\nat least 0.10 higher. This is the environment's regression guard against losing\nits defining behavior.\n\n## Data status and limitations\n\nThe deterministic synthetic panel remains the default and the signature-test\noracle. The opt-in ECB profile is the public real-data baseline. Its package\ncontains the exact standard CSV returned by the documented ECB API query,\ngzip-compressed; runtime filtering and generated fields are listed in\n[DATA_PROVENANCE.md](DATA_PROVENANCE.md).\n\nECB rates are reference rates published for information, not executable dealing\nquotes. They provide close-like observations but no OHLCV or transaction-cost\ndata, so Gate Runner uses a disclosed constant spread proxy. The 29 pairs also\nshare the euro as their base currency; this is an FX cross-section, not the\nequity universe represented by the synthetic fixture.\n\nThis compact backtester intentionally omits shorts, corporate actions, tax\neffects, borrow, and intraday execution. Results are environment scores, not\ninvestment advice.\n\n## Method references\n\n- Bailey and López de Prado, [The Deflated Sharpe Ratio](https://www.davidhbailey.com/dhbpapers/deflated-sharpe.pdf)\n- Bailey, Borwein, López de Prado, and Zhu, [The Probability of Backtest Overfitting](https://escholarship.org/uc/item/4w1110bb)\n\n## License\n\nGate Runner's code is licensed under the\n[Apache License 2.0](https://github.com/BR-322/gate-runner/blob/main/LICENSE).\nThe bundled ECB source snapshot remains subject to the ECB reuse terms recorded\nin [DATA_PROVENANCE.md](DATA_PROVENANCE.md); it is not relicensed under Apache-2.0.\n","encoding":"utf-8","truncated":false,"total_bytes":6604},"status":null}