{"data":{"kind":"file","path":"README.md","version_id":"gvogss8y7752tmeu622608lk","entry":{"name":"README.md","path":"README.md","is_directory":false,"size":1766,"modified_at":"2026-05-12T16:56:27.637000","content_hash":"781b32b1a52b4abe7dbabf5d41505daa0ccec4774f73e0e7c21648eedadcda74"},"entries":[],"content":"# analog_instrument\n\nverifiers env for analog instrument reading from photos. single-turn, deterministic rubric, real images.\n\n## install\n\n```\nprime env install kartikkapoor/analog_instrument\n```\n\n## what the model sees\n\n- system prompt instructing strict json output\n- one image of an analog instrument\n- \"Read this instrument.\"\n\nexpected output:\n```json\n{\n  \"instrument_type\": \"<type>\",\n  \"reading\": <number or \"HH:MM\">,\n  \"unit\": \"<unit>\",\n  \"scale_max\": <number or null>,\n  \"confidence\": \"<high|medium|low>\"\n}\n```\n\n## scoring\n\n- instrument type must canonicalize to the gt type (alias table built in)\n- unit must match (alias table covers psi/bar, kg/lb/oz, mph/kph, °c/°f, etc)\n- clocks: pass if predicted hh:mm within 1 minute of gt (12-hour mod)\n- numeric: pass if predicted reading within 5% of `scale_max`\n\ntracked-only metrics: format compliance, confidence_correct, confidence_wrong (for calibration analysis).\n\n## known failure modes (from v0.1 opus run)\n\n- clocks: 1 of 9 passed. zero variance at T=1.0 — model commits to one wrong answer\n- \"snap to round number\" on off-tick gauges\n- unit mismatch when scale has multiple units printed (e.g. postal scale lb vs oz)\n- \"high confidence\" reported when wrong more often than when right\n\n## dataset\n\n17 verified public images (clockbench public sample + wikimedia commons). personal photos folder exists but empty in v0.1 — user is shooting workplace instruments tomorrow.\n\nsee `data/manifest.json` for the master index. each image has a matching `data/ground_truth/<name>.json` with verification reasoning.\n\n## tests\n\n```\npython -m pytest test_rubric.py -v\n```\n\n29 tests covering type canonicalization, unit aliases, clock tolerance, numeric tolerance, malformed json, parenthetical type names, etc.\n","encoding":"utf-8","truncated":false,"total_bytes":1766},"status":null}