{"data":{"kind":"file","path":"README.md","version_id":"a6soeqdg7uwfwpn45wf23mes","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":5809,"modified_at":"2026-09-20T02:29:03.249000","content_hash":"f043d98c0c88ee6da9e5bf4ea7edd05e9e73dc42ab91368fc8d414e892baa87b"},"entries":[],"content":"# Bazaar Env\r\n\r\nStatus: **v1 and v2 trainer-verified.** v1: a hosted GRPO run (Qwen3.5-9B\r\non `micro`, $10.88) raised the frozen-split eval from the passive baseline\r\nto the honest-dealer anchor. v2: a curriculum run warm-started from the v1\r\ncheckpoint ($8.12) evals **above the vol-aware maker anchor** on\r\n`maker_micro` with zero forgetting of the v1 skill — and plateaus fast,\r\nwhich is stated plainly in `results/results.md`. Scope: one run per tier,\r\none model, warm-started v2.\r\n\r\nBazaar is a short-horizon dealer-market environment for LLM agents. A model\r\ntrades one commodity against scripted quotes while the commodity's fair value\r\nfollows a seeded exogenous index. The terminal score marks inventory at that\r\nindex, so the agent cannot move the reward by trading against itself or pumping\r\nits own book.\r\n\r\nThis repository currently implements **v1** only: solo taker, one commodity, no\r\ncredit, no adversary, 10 turns. The v2/v3 mechanics are pre-registered in\r\n`DESIGN.md`, not implemented yet.\r\n\r\n## Why This Exists\r\n\r\nMost public agent environments cover coding, browser use, or enterprise\r\nworkflow. Bazaar targets a thinner shelf: economic environments with deterministic\r\nmarket mechanics, rewards that cannot be moved by the agent's own trades, and a\r\ndocumented exploit pass.\r\n\r\nThe first build is deliberately small. A trainable LLM environment is useful only\r\nif rollouts are cheap enough to sample in groups, so v1 keeps the task to 10\r\nturns and a compact text protocol.\r\n\r\n## Install\r\n\r\n```powershell\r\nuv sync --extra dev\r\n```\r\n\r\nThe published wheel deliberately does **not** depend on `verifiers` or\r\n`prime-sandboxes`. Hosted training images provide that stack. This package is a\r\npure plugin over the platform image; local development installs the dev extra.\r\n\r\n## Run Tests\r\n\r\n```powershell\r\nuv run pytest\r\n```\r\n\r\n## Scripted Smoke Runs\r\n\r\n```powershell\r\nuv run python -m bazaar_env.core --tier micro --seed 0 --policy honest\r\nuv run python -m bazaar_env.exploits --tier micro --seeds 20\r\nuv run python -m bazaar_env.exploits --tier hard --seeds 20\r\n```\r\n\r\n## Model Probes\r\n\r\nAfter tests pass, the next step is an Ollama band probe. The first target is\r\n`qwen2.5:7b-instruct`, matching the Magic Sort training-thread protocol.\r\n\r\n```powershell\r\nvf-eval bazaar -m qwen2.5:7b-instruct -b http://localhost:11434/v1 -k OPENAI_API_KEY -n 5 -r 2 -a '{\\\"tier\\\":\\\"micro\\\"}' --disable-env-server\r\n```\r\n\r\nOn Windows, avoid `--save-results` when the model name contains a colon.\r\n\r\n## Implemented v1 Loop\r\n\r\nEach turn:\r\n\r\n1. Scripted counterparties post a bid/ask around the visible index.\r\n2. The agent replies with a command: `buy Q`, `sell Q`, or `pass`. The parser\r\n   extracts the last command found in the message (Magic Sort protocol), so\r\n   surrounding reasoning text does not fail the turn.\r\n3. The environment executes at the quote or returns `illegal: <reason>`.\r\n4. The index advances.\r\n5. State is re-marked and returned as compact JSON.\r\n\r\nTerminal reward:\r\n\r\n```text\r\nnet_worth_at_index / starting_net_worth\r\n+ once-only first-valid-trade bonus\r\n+ small format bonus\r\n```\r\n\r\nStarting net worth = starting cash + starting inventory marked at the first\r\nindex. `micro` starts the dealer with a small book so both edge directions\r\nare tradable from turn 1.\r\n\r\nThe once-only trade bonus prevents the no-trade collapse case without becoming a\r\nfarm: once every rollout trades, GRPO's group baseline cancels it.\r\n\r\n## Files\r\n\r\n- `src/bazaar_env/core.py` — deterministic market engine and scripted policies.\r\n- `src/bazaar_env/env.py` — `verifiers` adapter, datasets, replay scoring.\r\n- `src/bazaar_env/exploits.py` — deterministic exploit pass.\r\n- `DESIGN.md` — reward rationale, roadmap, and pre-registered farms.\r\n- `results/results.md` — commands and a place for first eval numbers.\r\n\r\n## V2: Maker + Adversary\r\n\r\nStatus: **v1 trainer-verified; v2 engine-tested**. v1 has one hosted GRPO run. v2 adds maker quoting and scripted adversary flow, with model preflights still pending.\r\n\r\nOn maker tiers (`maker_micro`, `maker_easy`) the agent can post a two-sided quote:\r\n\r\n```text\r\nquote BP BQ AP AQ\r\n```\r\n\r\nThat means bid price/size and ask price/size. `pass` leaves the standing quote in place. After the quote is posted, the index steps forward and scripted flow trades against the standing quote:\r\n\r\n- **Noise flow** fills attractive quotes.\r\n- **Pickoff flow** sees the stepped index and takes stale quotes.\r\n- **Bait depth** can make displayed size larger than executable size on juicy taker quotes.\r\n- **Trigger hunting** can move displayed taker quotes just inside a threshold the agent has revealed in prior trades.\r\n\r\nScripted anchors confirm the mechanic: a tight quoter gets many fills and loses to pickoff; a wide quoter does nothing; a vol-aware quoter earns spread with near-zero pickoff on `maker_micro`.\r\n\r\n## V3: Credit (automatic margin)\r\n\r\nStatus: **engine-tested; model preflight below.** On credit tiers\r\n(`credit_micro`) the agent has an automatic margin loan: buy beyond your cash\r\nand the loan is drawn for you (up to 1x equity); surplus cash repays it each\r\nturn; interest accrues at 0.5%/turn on outstanding debt. If equity falls below\r\n30% of debt: margin call, one turn to cure, then default (episode over,\r\nterminal score 0). The terminal score is final equity over starting net worth.\r\n\r\n`credit_micro` is deliberately capital-poor with rare, fat edges, and\r\ntaker-only (an outside review found a maker degeneracy; see DESIGN.md).\r\nMeasured anchors: sizing real edges to full buying power beats staying\r\nunlevered (1.050 vs 1.043); max-size gambling loses to doing nothing (0.981\r\nvs 0.998). `DESIGN.md` carries the leverage sweep, the four measured design\r\ndead-ends that led to automatic margin, and the two outside-review findings.\r\n","encoding":"utf-8","truncated":false,"total_bytes":5809},"status":null}